A cross-border live broadcast system based on cloud server
Through a cross-border live broadcast system based on cloud servers, key comments are extracted and translated, the information missed problem caused by too many comments in the live broadcast system is solved, and international subtitles support is provided, improving the information transmission efficiency and user experience of live broadcasts.
Patent Information
- Application Number
- CN202110835343.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-23
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-07-23
AI Technical Summary
When there are too many comments in the existing live broadcast system, the anchor is prone to missing important information, especially when the audience is large, the narrow display area cannot meet the need to display a large number of comments in a short period of time.
Design a cross-border live broadcast system based on cloud servers, obtain video data through the anchor terminal and transmit it to the cloud server. The cloud server processes and extracts key comments, sends them to the anchor terminal, and the client terminal performs natural language processing and translation, displays and transmits videos and subtitle information.
It effectively solves the problem that the anchor misses important comments, and understands the live broadcast content to users in different countries through subtitles, improving the information transmission efficiency and international support of live broadcasts.
Smart Images

Figure CN113596491B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of live broadcasting, and in particular to a cross-border live broadcasting system based on a cloud server. Background Art
[0002] The existing live broadcast system often causes the host to miss some important information when there are too many comments. For example, in a live broadcast for the purpose of selling goods, the audience's purchase requests and consultation on discounts are relatively important information. However, when the number of viewers is large, the comments are refreshed quickly, but the comment display area is relatively small. The small display area obviously cannot meet the needs of displaying a large number of comments in a short period of time, so some important comment information will be missed. Summary of the invention
[0003] In view of the above problems, the object of the present invention is to provide a cross-border live broadcast system based on a cloud server, including an anchor terminal, a cloud server and a client terminal;
[0004] The anchor terminal is used to obtain video data of the live broadcast site and transmit the live broadcast data to the cloud server;
[0005] The cloud server is used to process the video data, obtain subtitle information of a preset language type, and respond to a viewing request from the client terminal to transmit the video data and the subtitle information to the client terminal;
[0006] The client terminal is used to display the video data and subtitle information, and to obtain comment information input by the user and send the comment information to the client terminal;
[0007] The client terminal is also used to perform natural language processing on the comment information from different client terminals, obtain key comments within a preset time period, and send the key comments to the anchor terminal;
[0008] The anchor terminal is used to display the key comments.
[0009] Preferably, the anchor terminal includes an identity authentication module, a data acquisition module, a first communication module and a first display module;
[0010] The identity authentication module is used to authenticate the identity of the anchor, and to grant the anchor terminal access rights to the anchor who has passed the identity authentication;
[0011] The data acquisition module is used to acquire video data of the live broadcast site, and transmit the video data to the cloud server through the first communication module;
[0012] The first communication module is used to receive the key comments sent by the cloud server and transmit the key comments to the first display module;
[0013] The first display module is used to display the key comments.
[0014] Preferably, the video data includes video pictures and video sounds.
[0015] Preferably, the processing of the video data to obtain subtitles in a preset language type includes:
[0016] The preset language types include native language types and foreign language types;
[0017] Performing speech recognition processing on the video sound and translating the video sound into subtitle information of the native language;
[0018] The text information in the native language type is translated into subtitle information in a foreign language type.
[0019] Preferably, in response to the viewing request of the client terminal, transmitting the video data and the subtitle information to the client terminal comprises:
[0020] Obtaining the country where the client terminal is located according to the IP address of the viewing request;
[0021] Acquire corresponding subtitle information according to the country where the client terminal is located;
[0022] The subtitle information and the video data are transmitted to the client terminal.
[0023] Preferably, the client terminal comprises an input module, a second display module and a second communication module;
[0024] The input module is used to obtain comment information input by the user;
[0025] The second display module is used to display the video data and subtitle information;
[0026] The second communication module is used to receive video data and subtitle information sent from the cloud server, and to send the comment information to the cloud server.
[0027] Preferably, the step of performing natural language processing on the comment information from different client terminals to obtain key comments within a preset time period includes:
[0028] Store the comment information in the same time period into set S;
[0029] Translate the comment information contained in the set S into the comment information of the native language and store it in the set S';
[0030] Get the similar comment set of the comment information in the set S', and record the similar comment set of comment information i in S' as U i ;
[0031] The comment information in the set S' is sorted from most to least according to the number of elements contained in the similar comment set, and the comment information ranked in the top thre% is taken as the key comment, where thre represents a preset threshold parameter.
[0032] The present invention can utilize the powerful performance of the cloud server to process the comment information from the client terminal, and output the key comments obtained after processing to the anchor terminal for display, thereby effectively solving the problem in the prior art that the anchor is prone to miss important comments due to too many people watching the live broadcast. At the same time, the present invention can also identify and process the live video data, obtain the subtitle information of the native language type, and translate the subtitle information of the native language type into the subtitle information of the foreign language type, so as to facilitate users in different countries to understand the content of the live broadcast. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.
[0034] Figure 1 , which is a diagram of an exemplary embodiment of a cross-border live broadcast system based on a cloud server of the present invention. DETAILED DESCRIPTION
[0035] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.
[0036] like Figure 1 In one embodiment shown, the present invention provides a cross-border live broadcast system based on a cloud server, including an anchor terminal, a cloud server and a client terminal;
[0037] The anchor terminal is used to obtain video data of the live broadcast site and transmit the live broadcast data to the cloud server;
[0038] The cloud server is used to process the video data, obtain subtitle information of a preset language type, and respond to a viewing request from the client terminal to transmit the video data and the subtitle information to the client terminal;
[0039] The client terminal is used to display the video data and subtitle information, and to obtain comment information input by the user and send the comment information to the client terminal;
[0040] The client terminal is also used to perform natural language processing on the comment information from different client terminals, obtain key comments within a preset time period, and send the key comments to the anchor terminal;
[0041] The anchor terminal is used to display the key comments.
[0042] Specifically, the cloud server is used to send the key comments to the anchor terminal, and also to send the comment information to the anchor terminal. The anchor terminal displays the key comments and the comment information at the same time.
[0043] A special key comment display area may be set in the anchor terminal or key comments may be highlighted in other ways, such as bolding the text.
[0044] Preferably, the anchor terminal includes an identity authentication module, a data acquisition module, a first communication module and a first display module;
[0045] The identity authentication module is used to authenticate the identity of the anchor, and to grant the anchor terminal access rights to the anchor who has passed the identity authentication;
[0046] The data acquisition module is used to acquire video data of the live broadcast site, and transmit the video data to the cloud server through the first communication module;
[0047] The first communication module is used to receive the key comments sent by the cloud server and transmit the key comments to the first display module;
[0048] The first display module is used to display the key comments.
[0049] The method of setting up identity authentication is mainly used to improve the security of the live broadcast process. Only users who have passed the registration review can use the live broadcast terminal, thereby preventing the live broadcast terminal from being used for illegal activities. If the identity authentication is not passed, the anchor cannot use the live broadcast terminal for live broadcast.
[0050] Specifically, the identity verification module includes a camera unit, an image processing unit, and an identification unit;
[0051] The camera unit is used to obtain a facial image of the anchor and transmit the facial image to the image processing unit;
[0052] The image processing unit is used to obtain feature data contained in the facial image;
[0053] The recognition unit is used to match the feature data obtained by the image processing unit with the feature data pre-stored in the feature database. If the match is successful, it means that the host has passed the identity authentication, otherwise, it means that the host has not passed the identity authentication.
[0054] Specifically, the feature database stores feature data of facial images of users who have passed the registration review.
[0055] The recognition unit can determine whether the match is successful based on the similarity. If there is feature data in the feature database whose similarity with the feature data obtained by the image processing unit is greater than a preset similarity threshold, it means that the match is successful.
[0056] Specifically, the photographing unit includes a camera and a judging subunit;
[0057] The camera user obtains the anchor's facial image;
[0058] The judging subunit is used to calculate the effective information content parameter of the facial image, and if the effective information content parameter of the facial image is greater than a preset parameter threshold, transmit the facial image to the image processing unit; otherwise, notify the camera to re-acquire the facial image of the user;
[0059] The effective information content parameter is calculated as follows:
[0060]
[0061] Wherein, eicp represents the effective information content parameter, w1, w2, w3 represent the preset weight parameters, numUf represents the total number of pixels in the face image, Uf represents the set of pixels in the face image, and U k represents the set of pixels in a window of size H×H centered at pixel k in the face image, fd(k) and fd(m) represent the gradient magnitudes of pixels k and m respectively, vfd represents the average value of the gradient magnitudes of the pixels in Uf, numU k Indicates U k The total number of pixels contained in , numfcp represents the number of skin pixels in Uf, and litva represents the variance of the L component of the face image in the Lab color model.
[0062] The above-mentioned embodiment of the present invention calculates from aspects such as gradient amplitude, number of skin pixels, variance of L component, etc. to obtain effective information content parameter, so that the effective information content parameter of the facial image with larger gradient amplitude difference, more skin pixels, and smaller variance of L component is larger, which is conducive to selecting facial images with high effective information content, thereby improving the accuracy of the processing process of the image processing unit. It is conducive to improving the security of the live broadcast system of the present invention.
[0063] Specifically, the acquiring of feature data contained in the facial image includes:
[0064] Converting the facial image into a grayscale image;
[0065] Performing noise reduction processing on the grayscale image to obtain a noise reduction processed image;
[0066] Performing image segmentation processing on the noise reduction processed image to divide the noise reduction processed image into a foreground image and a background image;
[0067] Performing recognition processing on the facial image to obtain a set of skin pixel points imgfcU contained in the facial image;
[0068] Process the foreground image according to imgfcU to obtain a processed foreground image;
[0069] A preset feature extraction algorithm is used to obtain feature data contained in the processed foreground image.
[0070] Converting to a grayscale image is beneficial to reducing the dimension of information to be processed, and is beneficial to improving the processing speed of the present invention. Performing noise reduction processing can effectively avoid the influence of noise pixels on feature data extraction.
[0071] In the present invention, an image segmentation algorithm such as the Otsu algorithm can be used to perform image segmentation processing on the denoised image, and the denoised image can be divided into a foreground image and a background image.
[0072] Since image segmentation cannot completely and correctly divide all pixels, the present invention supplements the foreground image with skin pixels so that the processed foreground image contains as many pixels belonging to the facial skin area as possible, which is beneficial to increase the content of effective information contained in the image for the feature data extraction step, thereby improving the security of the identity authentication of the present invention.
[0073] Preset feature extraction algorithms, such as the HOG algorithm, the LBP algorithm, etc., can be used to obtain feature data contained in the processed foreground image.
[0074] Specifically, converting the facial image into a grayscale image includes:
[0075] The face image is converted into a grayscale image using a weighted average method.
[0076] Specifically, performing noise reduction processing on the grayscale image to obtain a noise reduction processed image includes:
[0077] Suspected noise points are detected for the pixels in the grayscale image, and the pixels in the facial image are divided into a suspected noise point set conp and a non-suspected noise point set nconp:
[0078] For a pixel point xc in a facial image, if its pixel value is the maximum value in the 8-neighborhood, or its pixel value is greater than 255×T1, or its pixel value is less than 255×T2, then xc is a suspected noise point;
[0079] The pixel point p in conp is processed as follows:
[0080]
[0081] Where cf(p) represents the value of pixel p after denoising; δ1 and δ2 represent the preset weight parameters, numa represents the total number of filter windows of different sizes, and w a Indicates the weight value of the filter window of size a, mid(U p,a ) represents taking the middle value of the pixel values of the pixels in the filter window of the ath size centered at the pixel p; Wherein, numsc represents the total number of non-suspected pixels in the filter window of the ath size centered on pixel point p, f(sc) represents the pixel value of the scth non-suspected pixel in the filter window of the ath size centered on pixel point p, bwf(p,a) represents the average pixel value of the non-suspected pixels in the filter window of the ath size centered on pixel point p, and va(p,a) represents the standard deviation of the pixel values of the non-suspected pixels in the filter window of the ath size centered on pixel point p;
[0082] The pixel point q in nconp is processed in the following way:
[0083]
[0084] Wherein, cf(q) represents the value of pixel q after denoising, α1 and α2 represent weight parameters, f(q) represents the pixel value of pixel q, (i,j) represents the coordinates of the pixel in the Q×Q neighborhood of pixel q, f(i,j) represents the pixel value of the pixel with coordinates (i,j), hc represents the standard deviation of Gaussian filtering of pixel q, tf(i,j) represents the gradient amplitude of the pixel with coordinates (i,j), tf(q) represents the gradient amplitude of pixel q, and gc represents the standard deviation of the gradient amplitude between the pixel in the Q×Q neighborhood of pixel q and pixel q;
[0085] In the grayscale image, the pixels in conp and nconp are filtered respectively to obtain a denoised image.
[0086] In the above embodiment, the present invention uses different noise reduction methods to process different types of pixels by dividing the pixels into suspected noise points and non-suspected noise points, which is beneficial to improving the accuracy of the noise reduction result. Specifically, when processing the pixels in conp, filtering is performed through filter windows of different sizes, and then a weighted fusion method is used to obtain the noise-reduced values of the pixels in conp. At the same time, two different noise reduction methods are used in filter windows of different sizes, and then the noise reduction results of the two different noise reduction methods are weighted to finally obtain the noise-reduced values. This setting method is beneficial to improving the adaptability of the filtering function. Different types of noise reduction methods focus on different types of pixels and are more targeted. In addition, in the setting of the function, the influence of the pixels in conp on the noise reduction results is eliminated, so that the results obtained by the noise reduction function are more accurate. When performing denoising on the pixels in nconp, the denoised pixel values are comprehensively obtained by obtaining information on the pixel value differences, gradient value differences, etc. between the pixels in the neighborhood and the pixels currently being filtered. This is beneficial in making the weights of the pixels in the neighborhood with smaller differences from the pixels currently being filtered larger, thereby obtaining accurate denoising results.
[0087] Specifically, the foreground image is processed according to imgfcU to obtain the processed foreground image, including:
[0088] Store the pixels contained in the foreground image into the set imgfrU;
[0089] Delete the pixels in imgfcU that belong to imgfrU from imgfcU to obtain the set imgfcU';
[0090] Get the union of imgfrU and imgfcU', frtlU;
[0091] The pixels in frtlU are combined into the processed foreground image.
[0092] In another embodiment, the identity authentication module includes a fingerprint recognition unit, and the fingerprint recognition unit is used to determine whether the anchor has passed the identity authentication by fingerprint recognition.
[0093] In yet another embodiment, the identity verification module determines whether the host has passed the identity verification by means of an account number and a password.
[0094] Preferably, the video data includes video pictures and video sounds.
[0095] Preferably, the processing of the video data to obtain subtitles in a preset language type includes:
[0096] The preset language types include native language types and foreign language types;
[0097] Performing speech recognition processing on the video sound and translating the video sound into subtitle information of the native language;
[0098] The text information in the native language type is translated into subtitle information in a foreign language type.
[0099] Specifically, the native language type is the language type of the country where the anchor terminal is located or the language type of the country where the anchor's nationality is located, and the anchor can set it according to actual needs. For example, when an anchor of Chinese nationality broadcasts live in China, the native language type is Chinese.
[0100] Foreign language types refer to language types of other countries except the country corresponding to the native language type. For example, when the native language type is Chinese, the foreign language types may include English, French, German, Spanish, etc.
[0101] Preferably, in response to the viewing request of the client terminal, transmitting the video data and the subtitle information to the client terminal comprises:
[0102] Obtaining the country where the client terminal is located according to the IP address of the viewing request;
[0103] Acquire corresponding subtitle information according to the country where the client terminal is located;
[0104] The subtitle information and the video data are transmitted to the client terminal.
[0105] Specifically, the country where the client terminal is located can be obtained by using the IP address contained in the viewing request sent by the client terminal to the cloud server, and then the subtitle information of the language type corresponding to the country can be obtained from the subtitle information of the foreign language type that has been translated according to the name of the country.
[0106] Preferably, the client terminal comprises an input module, a second display module and a second communication module;
[0107] The input module is used to obtain comment information input by the user;
[0108] The second display module is used to display the video data and subtitle information;
[0109] The second communication module is used to receive video data and subtitle information sent from the cloud server, and to send the comment information to the cloud server.
[0110] Specifically, the input module may include a touch screen, a microphone, an external input device, etc. These devices can all realize the input of comment information.
[0111] Preferably, the step of performing natural language processing on the comment information from different client terminals to obtain key comments within a preset time period includes:
[0112] Store the comment information in the same time period into set S;
[0113] Translate the comment information contained in the set S into the comment information of the native language and store it in the set S';
[0114] Get the similar comment set of the comment information in the set S', and record the similar comment set of comment information i in S' as U i ;
[0115] The comment information in the set S' is sorted from most to least according to the number of elements contained in the similar comment set, and the comment information ranked in the top thre% is taken as the key comment, where thre represents a preset threshold parameter.
[0116] Specifically, obtaining a similar comment set of the comment information in the set S' includes:
[0117] For comment information i in S', calculate the text similarity between comment information i and comment information j in S', i≠j, and store comment information j with text similarity greater than the preset threshold in U i Then delete the comment information j from the set S' to obtain the updated set S';
[0118] Then, a comment information is selected from the updated set S' to calculate its similar comment set, and so on, until the set S' is an empty set, and the calculation of the similar comment set is terminated.
[0119] The present invention can utilize the powerful performance of the cloud server to process the comment information from the client terminal, and output the key comments obtained after processing to the anchor terminal for display, thereby effectively solving the problem in the prior art that the anchor is prone to miss important comments due to too many people watching the live broadcast. At the same time, the present invention can also identify and process the live video data, obtain the subtitle information of the native language type, and translate the subtitle information of the native language type into the subtitle information of the foreign language type, so as to facilitate users in different countries to understand the content of the live broadcast.
[0120] Although embodiments of the present invention have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.
Claims
1. A cross-border live broadcast system based on a cloud server, characterized in that: Including anchor terminal, cloud server and client terminal; The anchor terminal is used to obtain video data of the live broadcast site and transmit the live broadcast data to the cloud server; The cloud server is used to process the video data, obtain subtitle information of a preset language type, and respond to a viewing request from the client terminal to transmit the video data and the subtitle information to the client terminal; The client terminal is used to display the video data and subtitle information, and to obtain comment information input by the user and send the comment information to the client terminal; The client terminal is also used to perform natural language processing on the comment information from different client terminals, obtain key comments within a preset time period, and send the key comments to the anchor terminal; The anchor terminal is used to display the key comments; The anchor terminal includes an identity authentication module, a data acquisition module, a first communication module and a first display module; The identity authentication module is used to authenticate the identity of the anchor, and to grant the anchor terminal access rights to the anchor who has passed the identity authentication; The data acquisition module is used to acquire video data of the live broadcast site, and transmit the video data to the cloud server through the first communication module; The first communication module is used to receive the key comments sent by the cloud server and transmit the key comments to the first display module; The first display module is used to display the key comments; The identity verification module includes a camera unit, an image processing unit and an identification unit; The camera unit is used to obtain a facial image of the anchor and transmit the facial image to the image processing unit; The image processing unit is used to obtain feature data contained in the facial image; The recognition unit is used to match the feature data acquired by the image processing unit with the feature data pre-stored in the feature database. If the match is successful, it means that the anchor has passed the identity authentication, otherwise, it means that the anchor has not passed the identity authentication; The photographing unit includes a camera and a judging subunit; The camera user obtains the anchor's facial image; The judging subunit is used to calculate the effective information content parameter of the facial image, and if the effective information content parameter of the facial image is greater than a preset parameter threshold, transmit the facial image to the image processing unit; otherwise, notify the camera to re-acquire the facial image of the user; The effective information content parameter is calculated as follows: Wherein, eicp represents the effective information content parameter, w1, w2, w3 represent the preset weight parameters, numUf represents the total number of pixels in the face image, Uf represents the set of pixels in the face image, and U k represents the set of pixels in a window of size H×H centered at pixel k in the face image, fd(k) and fd(m) represent the gradient magnitudes of pixels k and m respectively, vfd represents the average value of the gradient magnitudes of the pixels in Uf, numU k Indicates U k The total number of pixels contained in , numfcp represents the number of skin pixels in Uf, and litva represents the variance of the L component of the face image in the Lab color model.
2. According to the cloud server-based cross-border live broadcast system of claim 1, it is characterized in that: The video data includes video images and video sounds.
3. A cross-border live broadcast system based on a cloud server according to claim 2, characterized in that: The processing of the video data to obtain subtitles of a preset language type includes: The preset language types include native language types and foreign language types; Performing speech recognition processing on the video sound and translating the video sound into subtitle information of the native language; The text information in the native language type is translated into subtitle information in a foreign language type.
4. According to claim 3, a cross-border live broadcast system based on a cloud server is characterized in that: The step of responding to the viewing request of the client terminal and transmitting the video data and the subtitle information to the client terminal comprises: Obtaining the country where the client terminal is located according to the IP address of the viewing request; Acquire corresponding subtitle information according to the country where the client terminal is located; The subtitle information and the video data are transmitted to the client terminal.
5. According to the cross-border live broadcast system based on cloud server in claim 1, it is characterized in that: The client terminal includes an input module, a second display module and a second communication module; The input module is used to obtain comment information input by the user; The second display module is used to display the video data and subtitle information; The second communication module is used to receive the video data and subtitle information sent from the cloud server, and to send the comment information to the cloud server.
6. A cross-border live broadcast system based on a cloud server according to claim 3, characterized in that: The natural language processing of the comment information from different client terminals to obtain key comments within a preset time period includes: Store the comment information in the same time period into set S; Translate the comment information contained in the set S into the comment information of the native language and store it in the set S'; Get the similar comment set of the comment information in the set S', and record the similar comment set of comment information i in S' as U i ; The comment information in the set S' is sorted from most to least according to the number of elements contained in the similar comment set, and the comment information ranked in the top thre% is taken as the key comment, where thre represents a preset threshold parameter.
