Live broadcast content auditing method and device, electronic equipment and readable storage medium

Abstract

The invention provides a live broadcast content auditing method and device, electronic equipment and a readable storage medium. The method comprises the following steps: extracting a to-be-audited frame from a live broadcast video stream; calling a pre-trained cascaded intelligent character service model group to determine a character recognition result of the to-be-audited frame; performing multi-modal sensitive semantic analysis on the character recognition result to determine whether illegal content exists or not; if yes, marking the to-be-audited frame and reporting the to-be-audited frame and frame information; otherwise, not processing. According to the invention, the illegal content can be accurately and efficiently identified in the live scene, and the security of the live environment is ensured.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a live content review method, device, electronic device and readable storage medium. Background Art

[0002] With the rapid development of Internet technology, live streaming, as an emerging form of interaction, has been widely used in multiple fields such as video, social networking, and news, and can significantly improve user stickiness and activity.

[0003] Currently, content review is a crucial step in ensuring compliance and maintaining a positive user experience in live streaming scenarios. However, the real-time nature and diversity of live content present unprecedented challenges for review. In particular, identifying text violations in live streams presents multiple technical challenges, including the strong heterogeneity of text forms, the complexity of environmental interference, and strict review timelines. While traditional manual review methods can guarantee accuracy to a certain extent, they are slow, costly, subject to significant subjective errors, and struggle to cope with high concurrency, making them unable to meet the demands of the rapidly developing live streaming industry.

[0004] Therefore, there is an urgent need for an efficient, accurate and stable intelligent review solution to improve the efficiency and accuracy of live content review. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a live content review method, device, electronic device and readable storage medium for efficiently and accurately identifying illegal content in live broadcast scenarios. To achieve the above purpose, the technical solutions adopted in the embodiments of the present invention are as follows:

[0006] In a first aspect, the present invention provides a live content review method, which includes: extracting frames to be reviewed from a live video stream; calling a pre-trained cascaded intelligent text service model group to determine the text recognition results of the frames to be reviewed; performing multimodal sensitive semantic analysis on the text recognition results to determine whether there is any illegal content; if so, marking the frames to be reviewed and reporting the frames to be reviewed and frame information; otherwise, no processing is performed.

[0007] In the second aspect, the present invention provides a live content review device, including: a frame extraction module, used to extract frames to be reviewed from a live video stream; a text detection and recognition module, used to call a pre-trained cascaded intelligent text service model group to determine the text recognition result of the frame to be reviewed; a sensitive semantic discrimination module, used to perform multimodal sensitive semantic analysis on the text recognition result to determine whether there is any illegal content; an audit reporting module, used to mark the frame to be reviewed and report the frame to be reviewed and frame information if the judgment result of the sensitive semantic discrimination module is yes; otherwise, no processing is performed.

[0008] In a third aspect, the present invention provides an electronic device comprising a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the live content review method described in any of the aforementioned embodiments.

[0009] In a fourth aspect, the present invention provides a readable storage medium having machine-executable instructions stored thereon, and the machine-executable instructions can be executed by a processor to implement the live content review method as described in any of the aforementioned embodiments.

[0010] The live content review method, device, electronic device and readable storage medium provided by the embodiments of the present invention include: first, extracting frames to be reviewed from the live video stream to perform content review, avoiding frame-by-frame review, and ensuring system performance, and then calling a pre-trained cascaded intelligent text service model group to determine the text recognition results of the frames to be reviewed, and using multi-model information to perform text detection and recognition, which can improve the efficiency and accuracy of text recognition, and then performing multimodal sensitive semantic analysis on the text recognition results to determine whether there is any illegal content, which can improve the recognition accuracy of illegal content; finally, after identifying the illegal content, marking the frames to be reviewed and reporting the frames to be reviewed and frame information for reviewers to review. The entire process identifies illegal content through multi-model and multimodal sensitive semantic analysis mechanisms, thereby improving the review speed and accuracy.

[0011] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 This is a schematic diagram of a live broadcast application scenario;

[0014] Figure 2 A schematic flow chart of a live content review method provided by an embodiment of the present invention;

[0015] Figure 3 A functional module diagram of a text positioning model provided by an embodiment of the present invention;

[0016] Figure 4A functional module diagram of a content review device provided by an embodiment of the present invention;

[0017] Figure 5 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0019] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely intended to represent selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0020] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0021] See Figure 1 , Figure 1 This is a schematic diagram of a live broadcast application scenario. The technical solution provided by the embodiment of the present invention can be applied to Figure 1 In the live broadcast application scenario, the live broadcast application scenario includes a live broadcast server and a client; the client is divided into an anchor end and an audience end, and the anchor end and the audience end communicate data with the live broadcast server through a communication network, so that the anchor on the anchor end and the audience users on the audience end can conduct real-time network live broadcast; among them, for the anchor end and the audience end, their terminal devices can be but are not limited to various personal computers, laptops, smart phones and tablets, and the live broadcast server can be implemented with an independent server or a server cluster composed of multiple servers.

[0022] The live content review method provided by the embodiment of the present invention can be applied on a live broadcast server. By obtaining video stream data during the live broadcast process, sensitive information can be identified and reported in real time so that relevant personnel can take corresponding measures to ensure the compliance of the live broadcast environment.

[0023] During their research, the inventors discovered the following difficulties in identifying illegal information in live broadcasts: 1. The text is complex and may be miniaturized, use a variety of fonts, have low resolution, or even be motion blurred. 2. The background may be chaotic, and the text may be partially obscured or deformed due to dynamic changes. 3. The system needs to complete recognition and judgment in an extremely short time (hundreds of milliseconds) and process hundreds of millions of data points daily. These difficulties lead to low efficiency and accuracy in existing audit solutions.

[0024] To solve the above problem, see Figure 2 , Figure 2 A schematic flow chart of a live content review method provided by an embodiment of the present invention, the execution subject of the method may be Figure 1 The live broadcast server shown in , includes steps S201 to S204, which are described as follows:

[0025] S201: extracting frames to be reviewed from the live video stream;

[0026] S202: calling a pre-trained cascade intelligent text service model group to determine the text recognition result of the frame to be reviewed;

[0027] S203: Perform multimodal sensitive semantic analysis on the text recognition results to determine whether there is any illegal content;

[0028] S204: If yes, mark the frame to be reviewed and report the frame to be reviewed and frame information; otherwise, do nothing.

[0029] The live content review method provided by the embodiment of the present invention first extracts the frames to be reviewed from the live video stream to perform content review, avoiding frame-by-frame review, which can ensure system performance, and then calls a pre-trained cascaded intelligent text service model group to determine the text recognition results of the frames to be reviewed, and uses multi-model information to perform text detection and recognition, which can improve the efficiency and accuracy of text recognition. Then, by performing multi-modal sensitive semantic analysis on the text recognition results, it is determined whether there is any illegal content, which can improve the recognition accuracy of illegal content; finally, after identifying the illegal content, the frames to be reviewed are marked and the frames to be reviewed and frame information are reported to the reviewers for review. The entire process identifies illegal content through multi-model and multi-modal sensitive semantic analysis mechanisms, thereby improving the review speed and accuracy.

[0030] Next, the embodiment of the present invention will introduce the above-mentioned live content review process in detail in combination with relevant drawings.

[0031] In step S201, considering that there will be a huge amount of video stream data during the live broadcast process, if content review is performed on each frame, it will take up too many resources and may even result in unnecessary detection, affecting system performance. Therefore, the live broadcast server can first extract the frames to be reviewed from the live video stream for review.

[0032] In an embodiment of the present invention, the live broadcast server can adopt a task-driven frame extraction method to obtain frames to be reviewed. Specifically, the live broadcast tasks that need to be performed in different business scenarios are different, so different frame extraction intervals can be configured according to different business scenarios. For example, in certain high-risk or high-sensitivity live broadcast scenarios, the frame extraction interval can be shortened to increase the detection frequency. Therefore, in the actual implementation process, a benchmark frame extraction interval (such as 10s) can be set first to extract frames to be reviewed from the video stream. When encountering video stream data in a certain business scenario, the frame extraction interval corresponding to the business scenario is changed to obtain frames to be reviewed.

[0033] Next, the embodiment of the present invention may input the obtained frame to be reviewed into the cascaded intelligent text service model group to obtain the text information in the picture.

[0034] In the embodiment of the present invention, the cascaded intelligent text service model group can be deployed on the cloud server through the remote HTTP interface, ensuring that the system has good scalability and high concurrent processing capabilities.

[0035] The cascaded intelligent text service model group consists of a text location model and a text recognition model. The text location model can locate the text area in the image and generate a text detection frame. The text recognition model then extracts the specific text content from the text area where the text detection frame is located. Therefore, the implementation process of step S202 includes the following steps a1 to a3:

[0036] Step a1: Determine the location of the text in the frame to be reviewed by the text localization model;

[0037] Step a2: Obtain the image of the location from the frame to be reviewed;

[0038] Step a3: Input the image into the text recognition model of the cascaded intelligent text service model group to determine the text to be recognized results.

[0039] Next, the embodiments of the present invention will introduce in detail the structural composition and working process of the above-mentioned text positioning model and text recognition model.

[0040] In step a1, the text localization model is a key part of the entire review process, responsible for accurately locating and extracting text areas from the image. Considering the complex text forms and strong background interference in live scenes, the text localization model in the embodiment of the present invention can be functionally divided into the following: Figure 3 The structure shown is shown in Figure 3 , Figure 3 The functional module diagram of the text localization model provided by the embodiment of the present invention includes: a feature extraction network, a feature fusion module and a detection head.

[0041] It should be understood that Figure 3 This is only presented as an example, in which some basic model structures such as the pooling module are omitted for simplicity, but this is by no means a fixed limitation on the structure of the text localization model.

[0042] based on Figure 3 The model structure shown in the figure, the implementation process of step a1 can be understood as:

[0043] In the first step, the feature extraction network extracts the initial feature map from the frame to be reviewed, and then extracts multi-scale spatial information and spatial context information from the initial feature map;

[0044] Continue to see Figure 3 In the embodiments of the present invention, the feature extraction network is used to extract features from the frames to be reviewed, and its performance directly affects the accuracy of subsequent detection results. To maintain high precision while reducing computational overhead, the feature extraction network can also include a convolution module. By adjusting the channel distribution of the convolution module to a preset empirical value, such as [3, 6, 6, 3], computational efficiency can be optimized, enabling the network to extract high-quality features even under resource constraints.

[0045] Furthermore, to effectively enhance the ability to capture spatial contextual information, the feature extraction network also includes a multi-layer pooling module and a multi-gradient feature combination module, which are used to capture multi-scale spatial information and improve feature extraction capabilities, respectively. Specifically, the multi-layer pooling module enhances the fusion of global and local features by combining pooling windows of different sizes; the multi-gradient feature combination module further strengthens feature representation capabilities by running multiple gradient flow branches in parallel, effectively capturing small and hidden text features.

[0046] It can be understood that the text localization model is based on the above-mentioned feature extraction network, feature fusion mechanism and detection head design to accurately locate the position of text in the image. The output results include the bounding box coordinates (x_min, y_min, x_max, y_max) of each text area and the corresponding confidence score, providing reliable input data for the subsequent text recognition model.

[0047] Optionally, the feature extraction network may, but is not limited to, adopt an improved CrossStagePartialNetworks (CSPDarknet) architecture, such as CSPDarknet-53.

[0048] In the second step, the feature fusion module is used to fuse the final feature map input by the feature extraction network.

[0049] In the embodiment of the present invention, considering that text regions may appear at different scales in an image and have different aspect ratios and shapes, in order to better adapt to the diversity of text shapes (such as large differences in aspect ratios, font deformation, etc.), the feature fusion module can combine high-level semantic features with low-level spatial features through a top-down path during the feature fusion process to generate a multi-scale feature map, which helps capture text regions of different sizes. The feature fusion is then further strengthened through a bottom-up path to obtain the final fused feature.

[0050] For example, the feature fusion module can use a lightweight multi-scale feature pyramid network, which includes a PAN (Path Aggregation Network) module and an FPN (Feature Pyramid Network) module. Through the coordinated optimization of the two modules, efficient detection of multi-scale and complex-shaped text areas can be achieved.

[0051] When applying this feature fusion module, in order to significantly improve the feature fusion efficiency without increasing the amount of computation, the 1×1 convolution compression layer before upsampling can also be eliminated. At the same time, a short connection path is established between FPN and PAN to ensure that feature information can be efficiently transmitted.

[0052] In the third step, the detection head is used to detect the bounding box coordinates of the text area based on the fusion features to determine the location of the text.

[0053] In this embodiment of the present invention, the detection head is responsible for predicting the coordinate position and confidence level of the text area. Considering that in text detection tasks, traditional detection heads typically combine coordinate regression and confidence level prediction, this approach can cause conflicts and affect detection accuracy. This embodiment of the present invention utilizes a decoupled detection head to avoid the conflicts that can arise from traditional shared detection heads.

[0054] Specifically, the decoupled detection head can separate the coordinate regression task and the confidence prediction task and optimize them independently. Among them, the coordinate regression task focuses on predicting the bounding box coordinates (x_min, y_min, x_max, y_max) of the text area. The confidence prediction task focuses on evaluating the probability value of whether the current prediction result is a real text area. In this way, the coordinate regression and confidence prediction tasks no longer interfere with each other, and the model can focus more on its respective goals, avoiding the conflict problems that may be caused by traditional shared detection heads, thereby improving detection accuracy.

[0055] In order to further improve the detection accuracy, an embodiment of the present invention also provides a joint loss function to train the decoupled detection head. Specifically, the CIOU loss function is used to optimize the coordinate regression task. The CIOU (Complete Intersection over Union) loss function not only takes into account the overlapping area of ​​the bounding box, but also introduces the center point distance and aspect ratio penalty terms, which can better fit the true shape of the target. The DFL (Distributed Focal Loss) loss function is used to optimize the classification task. The DFL loss function further improves the learning ability of difficult samples by adjusting the sample weights. Through the design of the joint loss function, the overall detection accuracy is significantly improved.

[0056] Furthermore, in traditional detection heads, the aspect ratio of the anchor frame is usually fixed (such as 1:1, 2:1, 1:2, etc.). However, the text in the live broadcast scene may have a larger aspect ratio (such as longer horizontal text and narrower vertical text). Therefore, considering the complex form of text in the live broadcast scene (such as large aspect ratio differences, font deformation, etc.), the embodiment of the present invention also designs a dynamic anchor frame mechanism in the detection head, which can more flexibly adapt to the different shapes and aspect ratio differences of the text area in the live broadcast scene, effectively improving the detection ability of text with large aspect ratio.

[0057] Among them, the dynamic anchor frame mechanism expands the preset anchor frame aspect ratio and cooperates with the convolution kernel size in the feature extraction network to enable the detection head to cover text areas of more shapes.

[0058] In an embodiment of the present invention, the convolution kernel size design in the feature extraction network can adopt an asymmetric strip convolution kernel strategy, that is, for the long strip text area, special asymmetric convolution kernels (such as 3×9, 9×3, etc.) are designed to enhance the extraction capability of such shape features. These convolution kernels can capture more slender directional information on the feature map, thereby improving the detection effect of long strip text. Based on this design strategy, the dynamic anchor frame mechanism in the embodiment of the present invention will automatically select the appropriate anchor frame size and shape according to the specific content of the input image, better adapting to the diversity of text shapes in live broadcast scenes, and significantly improving the detection performance of text with large aspect ratios.

[0059] Therefore, during the implementation of the third step above, an initial positioning frame can be generated based on the convolution kernel size in the convolution module; the aspect ratio range of the initial positioning frame can be dynamically adjusted according to the aspect ratio characteristics of the text area in the fused features; and the adjusted positioning frame can be used to determine the location of the text from the fused features.

[0060] It can be understood that the embodiments of the present invention can use the decoupled detection head to separate the coordinate regression and confidence prediction tasks, avoid conflicts between tasks, and further improve the detection accuracy by combining the CIOU and DFL loss functions. By introducing the dynamic anchor frame mechanism and expanding the aspect ratio range of the anchor frame, the combination of these two technical means enables the text detection model to accurately and efficiently locate text areas of various forms in highly complex live broadcast scenes, providing reliable input data for subsequent text recognition.

[0061] Through the text localization model provided above, the embodiment of the present invention can optimize core components such as feature extraction, feature fusion and detection head, significantly improving the detection capability of small, hidden, low-resolution, occluded or fast-moving text.

[0062] Next, we will introduce the process of extracting text content using the text recognition model in step a3. The text recognition model may include a feature encoder and a decoder.

[0063] In embodiments of the present invention, the feature encoder can be, but is not limited to, a depthwise separable convolutional network, which effectively reduces the number of model parameters compared to traditional convolutional networks. The feature encoder takes as input an image of the area where the text detection box is located. Convolution operations are performed on these images to extract image features, such as texture, shape, and edge information. The extracted features are then predicted to obtain a probability distribution of characters, the predicted height or width of text lines, and the number of recognized character categories.

[0064] As an implementation method, the working process of the feature encoder can be: feature extraction is performed on the input image, and the extracted features are spliced ​​into a feature matrix, the feature matrix is ​​expanded into a sequence form, each time step corresponds to a feature vector, and these feature vectors are learned to capture the time dependency in the sequence, and the probability distribution of the characters at each time step is output. The predicted size and number of categories can be decoded through the probability distribution at each time step.

[0065] Optionally, the feature encoder may use a bidirectional long short-term memory network (BiLSTM) to predict feature sequences and obtain the probability distribution of characters at each time step.

[0066] In embodiments of the present invention, the decoder can combine multiple decoding methods to form a hybrid decoding architecture to improve model performance and efficiency. The decoder can further process the output of the feature encoder to ultimately produce prediction results. These predictions include predicted text size, such as the height or width of a text line, and the predicted number of character categories.

[0067] Optionally, in the decoder training stage, an embodiment of the present invention can simultaneously use a joint optimization strategy of connectionist temporal classification (CTC) loss and attention decoding loss, wherein the CTC loss function can handle sequence data and label alignment problems; the attention decoding loss can better focus on the key information in the sequence. At the same time, focalloss (focal loss) and centerloss (center loss) can also be used for auxiliary training. Focalloss is used to solve the problem of category imbalance, and centerloss is used to enhance the separability of features, thereby improving the recognition effect. Through the combination of the above-mentioned multiple methods, the generalization ability of the model is improved, so that it can better adapt to different input data.

[0068] In one embodiment of the present invention, for the above-mentioned text recognition model, in order to improve the text recognition model's ability to recognize diverse texts, the embodiment of the present invention further provides a training data generation method for training, including steps b1 to b3, as described below:

[0069] Step b1: Build a text style transfer network;

[0070] Step b2: Perform adversarial training on the text style transfer network to generate image text data;

[0071] Step b3: Use the image text data as training data to train the text recognition model.

[0072] In this embodiment of the present invention, the goal of the text style transfer network is to convert input text from one style to another while preserving the semantic content of the text. The goal of adversarial training is to improve the quality and diversity of generated text while generating text data related to the image.

[0073] In the above implementation, the embodiment of the present invention can generate image-text synthetic data containing different font types, different lighting conditions, and different perspective deformations through adversarial training, and at the same time combine it with Chinese, English and other ecological corpora in live broadcast scenes for training, supporting more than 7,000 characters in various Chinese, English, Japanese, etc., and has excellent robustness for text in complex live broadcast scenes.

[0074] It is understandable that the text localization model provided by the embodiment of the present invention can improve the efficiency of feature extraction and solve the difficulties faced by text detection in live broadcast scenarios. The provided text recognition model enhances the adaptability to multi-scale text, and the decoupled detection head and dynamic anchor frame mechanism further improve the detection accuracy. Through the combined effect of these implementation methods, the cascaded intelligent text service model can accurately and efficiently locate text areas in complex backgrounds, providing a basis for subsequent text violation judgment.

[0075] In one embodiment of the present invention, in order to effectively connect the text localization model and the text recognition model, an end-to-end cascaded intelligent text service model group can also be constructed to effectively utilize different model information, and at the same time, a high-performance model deployment solution is used to accelerate model reasoning, ultimately effectively ensuring the recognition accuracy and recall rate of massive image text content.

[0076] Based on the text recognition results obtained by the cascaded intelligent text service model group in step S202, it can be determined whether there are sensitive words, see step S203.

[0077] In step S203, an embodiment of the present invention provides a multimodal sensitive semantic analysis strategy. The implementation process of the strategy is shown in steps c1 to c4, which are described as follows:

[0078] Step c1: Use the pre-built search tree to search for illegal content in the text recognition results.

[0079] In the embodiment of the present invention, step c1 is essentially a literal level determination method, that is, at the literal level, a pre-built search tree, such as a prefix tree (Trie tree), is used to quickly search for sensitive words in the input text.

[0080] For ease of understanding, let's take a Trie tree as an example. We use it to store all predefined sensitive word lists. This tree structure supports incremental updates to adapt to changing regulatory requirements. For the input text, we extract all possible substrings (including single words and combinations), then match the substrings character by character starting from the root node of the Trie tree. If the "end flag" of a matching node is True, it means that a violating word has been found, and the location and specific content of the violating content are recorded.

[0081] Step c2: Perform morphological transformation and phoneme transformation on the text recognition results, and determine whether the text before the transformation is illegal content based on the transformed text recognition results;

[0082] In an embodiment of the present invention, since users may evade detection by using similar characters or homophones, the embodiment of the present invention provides step c2, which is essentially a morphological layer discrimination method. This discrimination method introduces similarity conversion rules based on Unicode character mapping and phoneme conversion rules based on the "Modern Chinese Pinyin Scheme" to transform the input text, realize text expansion, and then identify whether there are sensitive words in the expanded text.

[0083] In actual implementation, this can be achieved in the following manner: first, morphological transformation is performed to establish a mapping relationship between each character and its possible similar characters. Second, phoneme conversion is performed to convert the input text into a corresponding phoneme sequence based on the pinyin rule library. All variant texts generated by morphological transformation and phoneme conversion are collected and checked one by one for the presence of sensitive words. For each variant text, an exact match or fuzzy match algorithm (such as regular expressions, AC automata, etc.) is used to detect whether it contains sensitive words. If a sensitive word is detected in any variant text, the original input text is determined to be illegal content.

[0084] Through the above two methods, the coverage of variant sensitive words can be significantly improved.

[0085] Step c3: Predict the probabilities of various text combination patterns in the text recognition results, and identify text combination patterns that hit predefined sensitive words and text combination patterns that do not hit predefined sensitive words but have a frequency exceeding a preset frequency threshold as illegal content;

[0086] In this embodiment of the present invention, considering that many illegal contents are not composed of a single word, but rather a combination of multiple words, or convey illegal meanings through cryptic expressions, this embodiment of the present invention provides step c3, which essentially uses a language model to predict the combination possibilities of the input text and assess whether it has potential violations.

[0087] During step c3, the probability distribution of each word combination in the input text can be predicted based on a pre-trained language model. Combinations that fall outside the predefined sensitive word range but have a high probability of being considered sensitive words are then identified as sensitive words. Predefined sensitive words are sensitive words or phrases that are explicitly listed in the system and can be directly matched. These words are typically built based on a rule base, and the system can quickly detect these predefined sensitive words through simple string matching.

[0088] Furthermore, users may circumvent this by using variant forms (such as similar characters, homophones), obscure expressions (such as metaphors or allusions), or other methods. These contents, while not directly matching predefined sensitive words, may still have a violation connotation. Therefore, in step c3, sensitive word determination is also performed based on probability. That is, if a word combination, while not matching predefined sensitive words, has a high probability of appearing in a specific violation category, it can be inferred that it may have a potential violation.

[0089] Optionally, the language model in the embodiment of the present invention may be, but is not limited to, a statistical language model (such as an N-gram model) or a deep learning language model (such as a Transformer architecture).

[0090] To improve the language model's accuracy in identifying sensitive words in specific categories, the model can be pre-trained with sensitive words from a specific category to capture high-frequency word combinations within that category. These patterns include not only direct sensitive word combinations but also potentially subtle expressions or variations. For example, high-frequency words such as "sensitive events" and "key figures," along with their common combinations, might be extracted. This ensures that the system comprehensively covers a wide range of word combinations while efficiently detecting specific violation categories.

[0091] This identification method overcomes the shortcomings of relying solely on predefined sensitive word libraries and can identify more potentially illegal content. Furthermore, probabilistic assessment can filter out content that requires special attention, avoiding misjudgments or missed detections caused by over-reliance on rule libraries.

[0092] Step c4: Extract the context representation vector of the text recognition result, and identify the illegal content based on the context representation vector.

[0093] In this embodiment of the present invention, the meaning of individual words or phrases often depends on context. Some words may be harmless when appearing alone, but may violate the law in a specific context. Step c4 effectively distinguishes these situations. Step c4 is essentially a context-based discrimination method that allows for in-depth analysis of the contextual information of the input text, further enhancing semantic understanding capabilities and identifying words with sensitive language.

[0094] During step c4, a fine-tuned language model (e.g., a BERT model) can be used to encode the input text, which may have already been processed through at least one of steps c1 through c3. Contextual representation vectors are then extracted from the input text. These vectors capture the specific meaning of words in a specific context. The contextual representation vectors are then input into the classification layer of the language model to calculate the probability of violation. Based on the output probability of the classification layer, the text is judged to be in violation. For example, a threshold (e.g., 0.5) can be set, and a violation is determined when the probability is greater than the threshold.

[0095] This method can effectively solve the problem of a single word or phrase changing in meaning due to different contexts, distinguish the differences between input text in normal contexts and sensitive contexts, and significantly improve the system's semantic understanding and violation detection capabilities.

[0096] It should be noted that there is no specific order in which step c1, step c2, step c3 and step c4 are to be executed. They may be executed simultaneously or in sequence, and this is not limited in the embodiment of the present invention.

[0097] The multimodal sensitive semantic analysis strategy from steps c1 to c4 provided in this embodiment of the present invention combines multiple technical approaches, including literal-level matching, morphological-level conversion, composition-level prediction, and context-level analysis. Each layer complements the others, ensuring efficient detection of common sensitive words while also being adaptable to complex variants and cryptic expressions. This multi-layered design enables the engine to maintain high accuracy and low false positive rates in highly concurrent and complex live broadcast scenarios, providing strong technical support for the security of the live broadcast environment.

[0098] In one embodiment of the present invention, different categories of illegal content can also be classified and marked. The classification results are not only used to guide subsequent manual review, but can also be used as feedback data to continuously optimize model performance.

[0099] Next, for the frames to be reviewed that have been identified as violating content in step S203, step S204 may be executed to achieve timely reporting for further review by reviewers.

[0100] In step S204, frames identified as violating content can be marked, for example, with a "high-risk" label, for subsequent manual review. Marked frames can be pushed to the manual review office along with the frame information (such as the frame's corresponding timestamp, user ID, and other information) for final violation penalty processing.

[0101] In one embodiment of the present invention, it is considered that some of the frames marked for review may actually contain illegal content, while some frames may be misjudged, and therefore, invalid illegal content may be pushed. In addition, the system has to process hundreds of millions of images every day. The more frames pushed for review, the greater the pressure on manual review. Therefore, on the basis of ensuring quality and effectiveness, the embodiment of the present invention also provides a frequency control mechanism, including a frequency reduction mechanism and a frequency reduction suppression mechanism, which are described as follows:

[0102] Frequency reduction mechanism: If the number of marked pending review frames that have not been submitted within a preset first time period is greater than a preset first number threshold, the reporting frequency of marked pending review frames is reduced within a first effective time period;

[0103] Frequency reduction suppression mechanism: If the number of submitted marked frames for review within the preset second time period is greater than or equal to the preset second number threshold, the reporting frequency will be maintained within the second valid time period, and the reporting frequency that has been reduced within the valid period will be set to invalid.

[0104] Understandably, the number of frames submitted for review is determined by the reviewer during the review process. Submission indicates a valid violation, while non-submission indicates an invalid violation. The frequency reduction mechanism reduces the number of invalid violation frames reported to the reviewer, while the frequency reduction suppression mechanism ensures that valid violation frames reach the reviewer in a timely manner for review. This entire process effectively eliminates invalid reports, reduces manual review workload, and ensures the efficiency of the entire review system.

[0105] By dynamically adjusting the reporting frequency of frames to be reviewed, the frequency of invalid and illegal push notifications can be effectively reduced, and optimal control of the number of work orders can be achieved while ensuring the capture rate of illegal content.

[0106] In order to execute the corresponding steps in the above embodiments and various possible methods, a method for implementing the live content review device 40 is given below. Figure 4 , Figure 4 This is a functional module diagram of a live content review device provided by an embodiment of the present invention. The live content review device 40 includes: a frame extraction module 401, a text detection and recognition module 402, a sensitive semantics discrimination module 403 and a violation reporting module 404.

[0107] The frame extraction module 401 is used to extract the frames to be reviewed from the live video stream;

[0108] The text detection and recognition module 402 is used to call the pre-trained cascade intelligent text service model group to determine the text recognition result of the frame to be reviewed;

[0109] Sensitive semantics identification module 403, used to perform multimodal sensitive semantics analysis on the text recognition results to determine whether there is any illegal content;

[0110] The violation reporting module 404 is used to mark the frame to be reviewed and report the frame to be reviewed and frame information if the judgment result of the sensitive semantic judgment module is yes; otherwise, no processing is performed.

[0111] It is understandable that the frame extraction module 401, the text detection and recognition module 402, the sensitive semantic identification module 403 and the violation reporting module 404 can be executed in a coordinated manner. Figure 2 Each step in the process is performed to achieve the corresponding technical effects.

[0112] It should be noted that the basic principles and technical effects of the live content review device 40 provided in this embodiment are the same as those in the above embodiments. For the sake of brief description, for matters not mentioned in the live content review device 40 in the embodiment of the present invention, reference may be made to the corresponding contents in the above embodiments.

[0113] Optionally, the above modules can be stored in the form of software or firmware. Figure 5 The memory 501 shown in FIG. 1 may be solidified in the operating system (OS) of the electronic device 50 and may be Figure 5 Meanwhile, the data, program codes, etc. required to execute the above modules may be stored in the memory 501.

[0114] See Figure 5 , Figure 5 This is a block diagram of the structure of an electronic device provided in an embodiment of the present invention. The device includes a memory 501, a processor 502, and a communication interface 503. The memory 501, processor 502, and communication interface 503 are electrically connected to each other, directly or indirectly, to enable data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines.

[0115] Optionally, the bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0116] In an embodiment of the present invention, the processor 502 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor. The software module may be located in the memory 501, and the processor 502 reads the program instructions in the memory 501 and, in combination with its hardware, completes the steps of the above-mentioned method.

[0117] In an embodiment of the present invention, the memory 501 may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or a volatile memory (Volatile Memory), such as RAM. The memory may also be any other medium that can be used to carry or store the desired program executable code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in an embodiment of the present invention may also be a circuit or any other device that can implement a storage function, for storing instructions and / or data.

[0118] The memory 501 can be used to store software programs and modules, such as the instructions / modules of the live content review device 40 provided in the embodiment of the present invention, which can be stored in the memory 501 in the form of software or firmware (Firmware) or in the operating system (OS) of the solidified electronic device 50. The processor 502 executes the software programs and modules stored in the memory 501 to perform various functional applications and data processing. The communication interface 503 can be used to communicate signaling or data with other node devices.

[0119] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0120] I understand. Figure 5 The structure shown is for illustration only. The electronic device 50 may also include Figure 5 More or fewer components than shown, or with Figure 5 Different configurations shown. Figure 5 The components shown may be implemented in hardware, software, or a combination thereof.

[0121] Based on the above embodiments, the present application also provides a storage medium, in which a computer program is stored. When the computer program is executed by a computer, the computer executes the live content review method provided in the above embodiments.

[0122] Based on the above embodiments, an embodiment of the present invention further provides a computer program, which, when running on a computer, enables the computer to execute the live content review method provided in the above embodiments.

[0123] Based on the above embodiments, an embodiment of the present invention further provides a chip, which is used to read a computer program stored in a memory and to execute the live content review method provided in the above embodiments.

[0124] A computer program product is also provided in an embodiment of the present invention, comprising instructions, which, when executed on a computer, enable the computer to execute the live content review method provided in the above embodiment.

[0125] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by instructions. These instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0126] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0128] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A live content review method, characterized in that: The method comprises: Extract the frames to be reviewed from the live video stream; Calling a pre-trained cascade intelligent text service model group to determine a text recognition result of the frame to be reviewed; Performing multimodal sensitive semantic analysis on the text recognition results to determine whether there is any illegal content; If so, mark the frame to be reviewed and report the frame to be reviewed and frame information; otherwise, do not process it.

2. The live content review method according to claim 1, characterized in that: Calling a pre-trained cascaded intelligent text service model group to determine the text recognition result of the frame to be reviewed, including: Determining the location of the text in the frame to be reviewed by a text positioning model in the cascaded intelligent text service model group; Obtaining an image of the location from the frame to be reviewed; The image is input into the text recognition model of the cascaded intelligent text service model group to determine the text to be recognized result.

3. The live content review method according to claim 2, characterized in that: The text positioning model includes a feature extraction network, a feature fusion module, and a detection head; the text positioning model in the cascaded intelligent text service model group determines the location of the text in the frame to be reviewed, including: Extracting an initial feature map from the frame to be reviewed by the feature extraction network, and then extracting multi-scale spatial information and spatial context information from the initial feature map; Performing feature fusion on the final feature map input by the feature extraction network through the feature fusion module; The detection head is used to detect the coordinates of the bounding box of the text area based on the fusion features to determine the location of the text.

4. The live content review method according to claim 3, characterized in that: Utilizing the detection head to detect the bounding box coordinates of the text area based on the fused features to determine the location of the text, including: Generate an initial positioning frame based on the convolution kernel size in the feature extraction network; Dynamically adjusting the aspect ratio range of the initial positioning frame according to the aspect ratio feature of the text area in the fusion feature; The adjusted positioning frame is used to determine the location of the text from the fused features.

5. The live broadcast content review method according to claim 2, characterized in that: The method further comprises: Build a text style transfer network; Performing adversarial training on the text style transfer network to generate image text data; The text recognition model is trained using the image text data as training data.

6. The live broadcast content review method according to claim 1, characterized in that: Perform multimodal sensitive semantic analysis on the text recognition results to determine whether there is any illegal content, including: Using a pre-built search tree to search for illegal content in the text recognition results; Performing morphological transformation and phoneme transformation on the text recognition results, and identifying whether the text before the transformation is illegal content based on the transformed text recognition results; Predicting the probabilities of various text combination patterns in the text recognition results, and identifying text combination patterns that hit predefined sensitive words and text combination patterns that do not hit predefined sensitive words but have a frequency exceeding a preset frequency threshold as illegal content; A context representation vector of the text recognition result is extracted, and illegal content is identified based on the context representation vector.

7. The live broadcast content review method according to any one of claims 1 to 6, characterized in that: The method further comprises: If the number of marked pending review frames that are not submitted within the preset first time period is greater than a preset first number threshold, reducing the reporting frequency of the marked pending review frames within the first effective time period; If the number of submitted marked frames for review within the preset second time period is greater than or equal to the preset second number threshold, the reporting frequency is maintained within the second valid time period, and the reporting frequency that has been reduced within the valid period is set to invalid.

8. A live content review device, characterized in that: include: Frame extraction module, used to extract frames to be reviewed from live video streams; A text detection and recognition module, configured to call a pre-trained cascade intelligent text service model group to determine a text recognition result of the frame to be reviewed; A sensitive semantics discrimination module is used to perform multimodal sensitive semantics analysis on the text recognition results to determine whether there is any illegal content; The violation reporting module is used to mark the frame to be reviewed and report the frame to be reviewed and frame information if the judgment result of the sensitive semantic judgment module is yes; otherwise, no processing is performed.

9. An electronic device, characterized in that: It includes a processor and a memory, the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the live content review method described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: Machine executable instructions are stored thereon, and the machine executable instructions can be executed by a processor to implement the live content review method as described in any one of claims 1 to 7.

Images