Live broadcast content auditing method, apparatus and device, and computer medium

Abstract

The invention provides a live broadcast content auditing method, device and equipment and a computer readable medium, and the method comprises the steps: extracting a cut frame from a live broadcast video stream, taking the extracted cut frame as a to-be-audited image, calling a text detection model to carry out text detection on the to-be-audited image, and cutting the to-be-audited image according to the detected text. The method comprises the following steps: acquiring a to-be-audited image to obtain at least one character image, calling a lightweight text recognition model to perform text recognition on the character image for each character image to obtain a text recognition result, and performing violation judgment according to all the text recognition results corresponding to the to-be-audited image to obtain a violation judgment result. And then processing the live broadcast video stream which is judged to be illegal and / or the live broadcast room of the live broadcast video stream according to a preset illegal live broadcast processing rule, thereby realizing automatic auditing and processing of text content in a live broadcast picture, and effectively preventing spreading of illegal content.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a live content review method, device, equipment and computer-readable medium. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the application that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.

[0003] In the current information society, the explosive growth of content on live streaming platforms has posed unprecedented challenges to the review of live streaming content. Current live streaming review technology mainly focuses on reviewing live streaming images. Its core is to monitor and analyze live video streams in real time to ensure that they comply with community guidelines and laws and regulations. Live streaming review technology uses computer vision algorithms to quickly identify and analyze images and actions in the screen, such as violent behavior, nudity or pornographic content, dangerous activities, etc., so as to take corresponding measures immediately, such as warning, suspending or banning live streaming.

[0004] In actual applications, text information in live broadcasts, including background text, subtitles, etc., may contain illegal content, such as sensitive remarks, advertising promotions, or insulting language. Traditional live broadcast review systems are often unable to effectively identify and determine whether these text contents comply with regulations.

[0005] In view of the above problems, there is an urgent need to provide a technical solution for security review of text content in live broadcast images. Summary of the invention

[0006] Multiple aspects of the present application provide a live content review method, device, equipment and computer-readable storage medium for reviewing text content in a live broadcast screen.

[0007] In one aspect of the present application, a live broadcast content review method is provided, wherein the method comprises:

[0008] Extracting frames from a live video stream and using the extracted frames as images to be reviewed;

[0009] Calling a text detection model to perform text detection on the image to be reviewed, and cropping the image to be reviewed according to the detected text to obtain at least one text image;

[0010] For each of the text images, calling a lightweight text recognition model to perform text recognition on the text image to obtain a text recognition result;

[0011] Performing violation judgment according to all the text recognition results corresponding to the image to be reviewed to obtain a violation judgment result;

[0012] The live video stream and / or the live broadcast room of the live video stream that are judged to be in violation are processed according to the preset illegal live broadcast processing rules.

[0013] Another aspect of the present application provides a live content review device, wherein the device includes:

[0014] An image extraction module is used to extract frames from a live video stream and use the extracted frames as images to be reviewed;

[0015] A text detection module, used to call a text detection model to perform text detection on the image to be reviewed, and to crop the image to be reviewed according to the detected text to obtain at least one text image;

[0016] A text recognition module, for calling a lightweight text recognition model to perform text recognition on each of the text images to obtain a text recognition result;

[0017] A violation judgment module, used to make a violation judgment based on all the text recognition results corresponding to the image to be reviewed, and obtain a violation judgment result;

[0018] The illegal image processing module is used to process the live video stream and / or the live broadcast room of the live video stream that are judged to be illegal according to the preset illegal live broadcast processing rules.

[0019] Another aspect of the present application provides an electronic device, the electronic device comprising:

[0020] at least one processor; and

[0021] a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the live content review method as described above.

[0023] In another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions can be executed by a processor to implement the live content review method as described above.

[0024] In the solution provided by the embodiment of the present application, by extracting cut frames from the live video stream and using the extracted cut frames as images to be reviewed, then calling a text detection model to perform text detection on the images to be reviewed, and cropping the images to be reviewed according to the detected text to obtain at least one text image, and then for each of the text images, calling a lightweight text recognition model to perform text recognition on the text image to obtain a text recognition result, and then performing a violation judgment based on all the text recognition results corresponding to the images to be reviewed to obtain a violation judgment result, and then processing the live video stream and / or the live broadcast room of the live video stream that are judged to be in violation according to the preset violation live broadcast processing rules, thereby realizing automated review of text content in the live broadcast screen, which can not only realize real-time monitoring of text information in the live broadcast screen, but also accurately identify and judge whether these text contents contain violation information. Therefore, this method can significantly improve the accuracy and efficiency of live content review, effectively prevent the spread of illegal content, and thus ensure the security and compliance of platform content. Moreover, the present application adopts a lightweight text recognition model to recognize text images. The design of the lightweight text recognition model can significantly reduce the consumption of computing resources and improve the recognition speed, thereby ensuring the efficiency of the entire review process. Specifically, the lightweight model has a smaller model size and lower computational complexity, and can run efficiently in a resource-constrained environment. It is particularly suitable for live broadcast scenarios that require real-time processing of large amounts of data. By adopting a lightweight text recognition model, the present application solution can not only process a large number of live video streams in real time, but also ensure the accuracy of recognition and processing speed, thereby improving the reliability and efficiency of the entire live content review. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0027] Figure 1 A flowchart of a live content review method provided in an embodiment of the present application;

[0028] Figure 2 A flowchart of a specific example of a live content review method provided in an embodiment of the present application;

[0029] Figure 3A schematic diagram of the structure of a live content review device provided in one embodiment of the present application;

[0030] Figure 4 A schematic diagram of the structure of an electronic device suitable for implementing the solution in the embodiments of the present application;

[0031] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPU), input / output interface, network interface and memory.

[0034] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0035] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer program instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0036] In view of the problem that the live broadcast review technology in the prior art lacks the review of the text content in the live broadcast screen, resulting in missed judgments, the embodiments of the present application provide a live broadcast content review method, device, equipment and computer-readable storage medium for reviewing the text content in the live broadcast screen, which extracts cut frames from the live video stream and uses the extracted cut frames as images to be reviewed, then calls a text detection model to perform text detection on the images to be reviewed, and crops the images to be reviewed according to the detected text to obtain at least one text image, and then for each of the text images, calls a lightweight text recognition model to perform text recognition on the text image to obtain a text recognition result, and then performs a violation judgment based on all the text recognition results corresponding to the images to be reviewed to obtain a violation judgment result, and then processes the live video stream and / or the live broadcast room of the live video stream that are judged to be in violation according to the preset violation live broadcast processing rules, thereby realizing automatic review of the text content in the live broadcast screen, which can not only realize real-time monitoring of the text information in the live broadcast screen, but also accurately identify and judge whether these text contents contain violation information. Therefore, this method can significantly improve the accuracy and efficiency of live content review, effectively prevent the spread of illegal content, and thus ensure the security and compliance of platform content. Moreover, the present application adopts a lightweight text recognition model to recognize text images. The design of the lightweight text recognition model can significantly reduce the consumption of computing resources and improve the recognition speed, thereby ensuring the efficiency of the entire review process. Specifically, the lightweight model has a smaller model size and lower computational complexity, and can run efficiently in a resource-constrained environment. It is particularly suitable for live broadcast scenarios that require real-time processing of large amounts of data. In addition, the lightweight model can also reduce the misjudgment rate, because under conditions of sufficient resources, the model can make accurate recognition results faster. Therefore, by adopting a lightweight text recognition model, the present application solution can not only process a large number of live video streams in real time, but also ensure the accuracy of recognition and processing speed, thereby improving the reliability and efficiency of the entire live content review.

[0037] The following is an exemplary description of the live content review method, device, equipment and computer-readable storage medium provided in this application.

[0038] In actual scenarios, the execution subject of the live content review method provided in the embodiment of the present application can be a user device, or a device formed by integrating a user device and a network device through a network, or it can also be an application (such as a machine review system) running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablet computers, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer collections. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing (Cloud Computing), where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

[0039] Figure 1 The following is a processing flow of a live content review method provided by an embodiment of the present application, wherein the method at least includes the following processing steps:

[0040] Step S101, extracting cut frames from a live video stream, and using the extracted cut frames as images to be reviewed.

[0041] This step captures image frames from the live video stream in real time and uses these image frames as the basis for subsequent text detection and recognition.

[0042] The cut frame refers to a single image frame selected from a continuous video stream for subsequent processing. The image to be reviewed refers to the image frame extracted from the live video stream, which will be further processed to detect the text content therein.

[0043] In the specific implementation of this embodiment, image frames can be extracted from the video stream periodically or on demand during the live video stream transmission. The extraction frequency can be adjusted according to actual needs and computing resources, such as extracting one or more frames per second. The extracted image frames will be saved as separate image files and used as images to be reviewed for subsequent processing. For example, the system can be set to extract one frame of image from the video stream every second to ensure the representativeness of the image frame and reduce the workload of subsequent processing.

[0044] This step extracts frames from the live video stream and uses these frames as images to be reviewed, ensuring that the subsequent text detection and recognition process is always based on the latest live content. Therefore, it can not only monitor the text information in the live screen in real time, but also provide accurate basic data for subsequent text recognition and violation judgment. For example, when processing an ongoing game live broadcast, the system can extract a frame of image from the video stream every second and use these image frames as images to be reviewed, ensuring that the subsequent text recognition and violation judgment are always based on the latest live content, thereby improving the accuracy and timeliness of the entire live content review.

[0045] Step S102: calling a text detection model to perform text detection on the image to be reviewed, and cropping the image to be reviewed according to the detected text to obtain at least one text image.

[0046] Among them, the text detection model refers to an algorithm model trained through machine learning, which can recognize text areas in images.

[0047] In the specific implementation of this step, the image to be reviewed can be first input into the text detection model, which will scan the image, mark the areas in the image that may contain text, and annotate the location information of these areas. Next, the original image is cropped according to the text location information output by the model, so as to extract the image fragment containing text, that is, the text image. By removing irrelevant parts of the image, the amount of data for subsequent processing can be reduced, the processing speed can be improved, and at the same time, the processing focus can be ensured to be on the text part containing important information, thereby improving the accuracy of recognition.

[0048] For example, when processing an image containing a billboard for review, the text detection model can identify the text on the billboard and mark its bounding box. Then, based on the bounding box information, the algorithm will crop the original image to obtain an image segment containing only the billboard text for further feature analysis and text recognition. This method not only ensures that the valuable part of the image is processed, but also avoids non-text areas from interfering with the recognition results, thereby improving the overall review efficiency and accuracy.

[0049] It should be noted that the above-mentioned text detection model can be implemented using any text detection model provided by the prior art. After being trained with a large number of samples, it can be used in the present application to detect and extract text images from the images to be reviewed. The embodiments of the present application do not limit the specific content of the text detection model.

[0050] Step S103: for each of the text images, calling a lightweight text recognition model to perform text recognition on the text image to obtain a text recognition result.

[0051] One of the purposes of this step is to efficiently and accurately recognize the text content in the text image through a lightweight model.

[0052] Among them, the lightweight text recognition model refers to a text recognition model designed for real-time processing of a large number of live video streams, which is characterized by low computing resource consumption, fast execution speed, and can run efficiently in a resource-constrained environment. The lightweight text recognition model usually achieves efficient text recognition by optimizing the network structure, reducing the number of model parameters and computational complexity. The lightweight text recognition model can adopt a convolutional neural network (CNN) or other advanced deep learning architectures, but a lot of optimizations have been made in the design. For example, by using depthwise separable convolution, reducing the number of channels, introducing residual connections and other technologies, the model can greatly reduce the amount of calculation and memory usage while ensuring the recognition accuracy. In addition, the lightweight model may also convert floating-point operations into integer operations through quantization technology, further reducing the demand for computing resources. Those skilled in the art can use the existing lightweight model provided by the prior art directly or after fine-tuning training to apply to the present application scheme. The embodiment of the present application does not limit the specific content of the lightweight text recognition model.

[0053] In the specific implementation of this step, after the text detection model identifies the text area and crops the image to be reviewed, each text image can be input into the lightweight text recognition model. The model outputs the recognition result of the text content by analyzing the text features in the image. For example, when the system detects a text image containing "Welcome to watch", the lightweight text recognition model can quickly and accurately recognize the text content of "Welcome to watch".

[0054] This step uses a lightweight text recognition model to recognize text images. The system can not only significantly reduce the consumption of computing resources and improve the recognition speed, but also ensure the accuracy and real-time nature of the recognition. Therefore, it can monitor the text information in the live broadcast screen in real time and accurately identify the text content, thereby improving the reliability and efficiency of the entire live broadcast content review. For example, when processing an ongoing game live broadcast, the system can quickly and accurately identify the text content in the live broadcast screen, such as "Welcome to the game world", and ensure that subsequent text recognition and violation judgments are always based on the latest live broadcast content, thereby improving the accuracy and timeliness of the entire live broadcast content review.

[0055] Step S104: Perform violation judgment based on all the text recognition results corresponding to the image to be reviewed to obtain a violation judgment result.

[0056] One of the purposes of this step is to comprehensively analyze all the identified text content and evaluate whether the content meets the preset violation standards to determine whether the live video stream should be considered as a violation. Violation judgment can refer to the process of evaluating the identified text recognition results based on preset rules, with the purpose of identifying whether the text contains illegal information, such as sensitive words or other text content that does not comply with regulations.

[0057] When this step is implemented, after obtaining all the text recognition results, these text recognition results can be sent to the violation judgment module, which usually includes two main components: the violation keyword matching model (also known as the sensitive word matching model, where the sensitive words are the violation keywords) and the violation text recognition model. The sensitive word matching model can compare the identified text recognition results with the preset sensitive word list. If any sensitive words are included in the text recognition results, the live video stream will be marked as having violation information. On the other hand, the violation text recognition model uses a neural network model to identify whether the text recognition results are illegal or vulgar. If the evaluation result exceeds the preset risk threshold, the live video stream will also be marked as having violation information. Among them, the violation text recognition model is usually built based on a neural network, trained through a large amount of annotated text data, and learns to identify possible violations or vulgar content in the text. The model will analyze the content characteristics of the text and output a risk score. If this score exceeds the preset risk threshold, the system can determine that the text content contains violation information and mark the corresponding live video stream. For example, the model may identify sensitive words such as "illegal", "gambling", and "pornography" and score them based on their frequency of occurrence and context. By adopting the illegal text recognition model, the system can automatically evaluate the compliance of the text content, thereby improving the accuracy and efficiency of violation judgments.

[0058] For example, suppose there is an image to be reviewed that contains user comments. The optical character recognition (OCR) technology identifies the comment text as "There are some inappropriate remarks here". Then, the system feeds the text recognition result into the sensitive word matching model, and the model checks whether the text recognition result contains words in the preset sensitive word list. If the comment contains words in the sensitive word list, the system can immediately mark the live video stream as containing illegal information. In addition, the system can also feed the text recognition result into the text review model, which can evaluate whether the content of the text recognition result has a risk of violation. If the evaluation result shows that the risk level exceeds the set threshold, the system can also mark the live video stream as containing illegal information. In this way, not only can the illegal content be identified quickly and accurately, but also the false alarm rate can be effectively reduced, ensuring that only the live video streams that are truly illegal will be marked and processed, so that the accuracy and efficiency of live content review can be significantly improved, ensuring the security and compliance of platform content.

[0059] Step S105, processing the live video stream and / or the live broadcast room of the live video stream that are judged to be in violation according to the preset violation live broadcast processing rules.

[0060] One of the purposes of this step is to use a preset rule system to decide how to deal with live video streams and their live broadcast rooms that have been identified as containing illegal content. Among them, the illegal live broadcast handling rules refer to a series of guiding principles and specific operating guidelines for handling illegal live video streams and their live broadcast rooms formulated in accordance with platform policies and laws and regulations.

[0061] When this step is implemented, after completing the violation judgment of all text recognition results in the live video stream, the specific handling measures can be determined according to the violation judgment results and with reference to the preset violation live broadcast processing rules. This step can process a specific live video stream, and is suitable for situations where it is necessary to directly process the live video stream containing illegal content. For example, the transmission of the illegal live stream can be immediately interrupted, or the illegal video stream can be marked for further review. In addition, this step can also process the live broadcast room corresponding to the live video stream, and is suitable for the need to take more extensive measures against the live broadcast room that produces illegal content, for example, the entire live broadcast room can be paused or closed, or the live broadcast room can be warned. In addition, the system can also record relevant information of the illegal live video stream, such as the type of violation, processing time, and processing personnel, for later auditing and improvement.

[0062] For example, if the text in a live video stream is identified as containing the sensitive word "illegal", according to the preset rules for handling live broadcast violations, the system can immediately mark the live video stream as illegal and automatically trigger further manual review processes. In this process, the system can not only ensure that illegal content will not be directly released without processing, but also support subsequent audit work by recording the specific details of the violation. Therefore, it can not only effectively control the spread of illegal content, but also improve the transparency and traceability of the review process, ensuring the compliance of platform operations and the healthy environment of the user community.

[0063] The above-mentioned live content review method provided in the embodiment of the present application extracts cut frames from the live video stream and uses the extracted cut frames as the image to be reviewed, then calls the text detection model to perform text detection on the image to be reviewed, and crops the image to be reviewed according to the detected text to obtain at least one text image, and then calls a lightweight text recognition model for each of the text images to perform text recognition on the text image to obtain a text recognition result, and then performs a violation judgment based on all the text recognition results corresponding to the image to be reviewed to obtain a violation judgment result, and then processes the live video stream and / or the live broadcast room of the live video stream judged to be in violation according to the preset violation live broadcast processing rules, thereby realizing automatic review of the text content in the live broadcast screen, which can not only realize real-time monitoring of the text information in the live broadcast screen, but also accurately identify and judge whether these text contents contain violation information. Therefore, this method can significantly improve the accuracy and efficiency of live content review, effectively prevent the spread of illegal content, and thus ensure the security and compliance of platform content. Moreover, the present application adopts a lightweight text recognition model to recognize text images. The design of the lightweight text recognition model can significantly reduce the consumption of computing resources and improve the recognition speed, thereby ensuring the efficiency of the entire review process. Specifically, the lightweight model has a smaller model size and lower computational complexity, and can run efficiently in a resource-constrained environment. It is particularly suitable for live broadcast scenarios that require real-time processing of large amounts of data. In addition, the lightweight model can also reduce the misjudgment rate, because under conditions of sufficient resources, the model can make accurate recognition results faster. Therefore, by adopting a lightweight text recognition model, the present application solution can not only process a large number of live video streams in real time, but also ensure the accuracy of recognition and processing speed, thereby improving the reliability and efficiency of the entire live content review.

[0064] In some modified implementations, calling a lightweight text recognition model to perform text recognition on the text image may include:

[0065] Using a lightweight feature detection model to perform multi-dimensional feature detection on the text image;

[0066] If it is determined according to the feature detection result that the text image has the specified feature, calling the dedicated text recognition model corresponding to the specified feature to perform text recognition on the text image;

[0067] Otherwise, the lightweight universal text recognition model is called to perform text recognition on the text image.

[0068] One of the purposes of this embodiment is to improve the accuracy and efficiency of recognition by selecting the most suitable recognition model according to the multi-dimensional features of the text image.

[0069] Among them, the lightweight feature detection model refers to a model specifically used to detect multiple features in text images, including but not limited to the arrangement of text (such as vertical text), text length (such as long text) and text language (such as Chinese, English, etc.).

[0070] A lightweight general text recognition model refers to a model that can process various common text images and is suitable for text image recognition tasks without specific features (such as vertical text, long text or specific language). The model has made a lot of optimizations in design to ensure that while ensuring recognition accuracy, the amount of calculation and memory usage are greatly reduced. Lightweight general text recognition models usually use convolutional neural networks (CNN) or other advanced deep learning architectures, but have made a lot of optimizations in design, such as using depthwise separable convolution, reducing the number of channels, introducing residual connections and other technologies, so that the model can ensure recognition accuracy while significantly reducing the amount of calculation and memory usage. In addition, lightweight models may also convert floating-point operations into integer operations through quantization technology to further reduce the demand for computing resources.

[0071] During the specific implementation of this embodiment, after the text image is cropped, a lightweight feature detection model may be first called to perform multi-dimensional feature detection on the text image. These feature detections include but are not limited to calling a pre-trained vertical text detection model to detect whether the text image contains vertical text, calling a pre-trained long text detection model to detect whether the text image contains long text, and calling a pre-trained text language classification model to detect the text language of the text image. Through these detections, the system can determine the specific features of the text image, and then select the most suitable recognition model for text recognition. If it is determined based on the feature detection results that the text image does not have any of the above specified features, the lightweight general text recognition model is called to perform text recognition on the text image.

[0072] For example, if the text image is judged to have a specified feature according to the feature detection result, for example, if the text image is detected to be vertical text, the system calls a recognition model that specifically processes vertical text for recognition; if the text image is detected to be a long text, the recognition model that specifically processes the long text is called for recognition; if the text image is detected to be in a specific language, the recognition model that specifically processes the language is called for recognition. Otherwise, if any of the specified features are not detected, the system calls a lightweight general text recognition model to perform text recognition on the text image. In this way, by selectively calling different recognition models, the system can reduce unnecessary processing steps and improve recognition speed without affecting recognition accuracy. In the case of not having any specified feature, the system uses a lightweight general text recognition model to recognize text images that have not detected the specified features. The combination of multiple recognition methods can not only improve the accuracy and efficiency of recognition, but also reduce unnecessary detection steps in live broadcast scenarios with high real-time requirements. Further improve the processing speed and ensure the security and compliance of live broadcast content. Therefore, it can significantly improve the accuracy and efficiency of live broadcast content review and ensure the security and compliance of platform content.

[0073] For example, if the text in a text image is ordinary horizontal text, the text length is moderate, and there is no special language requirement, then after the system calls the lightweight feature detection model for detection, it finds that the text image does not have any specified features. At this time, the system will call the lightweight general text recognition model for recognition, thereby ensuring that the recognition process is both efficient and accurate. In this way, the system can not only quickly identify the text content, but also improve the processing speed without affecting the recognition accuracy, ensuring the security and compliance of the live content.

[0074] In this implementation, by using a lightweight universal text recognition model to recognize text images that do not detect specified features, the system can not only improve the accuracy and efficiency of recognition, but also reduce unnecessary detection steps in live broadcast scenarios with high real-time requirements, further improve processing speed, and ensure the security and compliance of live broadcast content. Therefore, this implementation can significantly improve the accuracy and efficiency of live broadcast content review and ensure the security and compliance of platform content.

[0075] In some modified implementations, the performing multi-dimensional feature detection on the text image using a lightweight feature detection model may include at least one of the following:

[0076] Calling a pre-trained vertical text detection model to detect whether the text image contains vertical text;

[0077] Calling a pre-trained long text detection model to detect whether the text image contains long text;

[0078] A pre-trained text language classification model is called to detect the text language of the text image.

[0079] One of the purposes of this embodiment is to improve the accuracy and efficiency of text recognition by performing multi-dimensional feature detection on text images and selecting the most suitable recognition model.

[0080] The lightweight feature detection model may be a model specifically used to detect multiple features in text images, including at least one of vertical text, long text, and text language. Multi-dimensional feature detection may refer to performing a multi-faceted analysis of text images before recognizing text to determine its specific format and language characteristics.

[0081] In the specific implementation of this embodiment, after the text image is cropped, a series of pre-trained models can be called to detect the features of the text image, first the vertical text detection model, used to detect whether the text in the text image is vertical, then the long text detection model, used to detect whether the text in the text image is a long text, and finally the text language classification model, used to detect which language the text in the text image belongs to. Through these detections, the system can determine the specific features of the text image, and then select the most suitable recognition model for text recognition.

[0082] For example, if the text in an image is detected as vertical text, the system can call the vertical text recognition model to process this part of the text. Similarly, if the text is recognized as long text, the system can use the long text recognition model for recognition. In addition, if the text is detected as Chinese, the Chinese recognition model is selected; if it is English, the English recognition model is selected. In this way, the system can not only perform personalized recognition according to the specific situation of the text, but also reduce recognition errors and improve recognition accuracy.

[0083] It should be noted that, due to the high real-time requirements for live content review, in actual implementation, one or part of them can be selected for feature detection to improve real-time performance. For example, in actual applications, considering that the live content review needs to be processed in a very short time to ensure that the audience can watch the live broadcast seamlessly, when performing multi-dimensional feature detection, part of the feature detection can be selectively performed instead of all detection items. For example, it is possible to only detect whether the text image is vertical text or whether it is a long text, while omitting the text language classification detection, or only retaining the text language classification detection and ignoring other types of detection. In this way, the system can reduce processing time and improve the overall processing speed without affecting the recognition accuracy, thereby ensuring the real-time performance of the review process.

[0084] In this embodiment, a lightweight feature detection model is used to perform multi-dimensional feature detection on text images, and feature detection is selectively performed according to real-time requirements. The system can not only select the most suitable recognition model for recognition based on the specific features of the text image to improve recognition accuracy and efficiency, but also in live broadcast scenarios with high real-time requirements, by reducing unnecessary detection steps, further improve processing speed and ensure the security and compliance of platform content.

[0085] Accordingly, based on the above implementation, in some modified implementations, if it is determined according to the feature detection result that the text image has a specified feature, calling a dedicated text recognition model corresponding to the specified feature to perform text recognition on the text image includes:

[0086] If it is determined according to the feature detection result that the text image is a vertical text image, calling a vertical text recognition model to perform text recognition on the text image;

[0087] If it is determined according to the feature detection result that the text image is a long text image, calling a long text recognition model to perform text recognition on the text image;

[0088] If it is determined according to the feature detection result that the text image is a text image of a specified language, a text recognition model of the specified language is called to perform text recognition on the text image.

[0089] One of the purposes of this embodiment is to improve the accuracy and efficiency of recognition by selecting the most appropriate dedicated text recognition model based on the feature detection results. The dedicated text recognition model may refer to a recognition model optimized for a specific type of text, such as a text model for recognizing vertical text, long text, or a specific language.

[0090] In the specific implementation of this embodiment, after completing the multi-dimensional feature detection of the text image, the system can select the most suitable dedicated text recognition model for recognition according to the detection results. For example, if the feature detection result shows that the text image is vertical text, the system calls the vertical text recognition model for recognition; if the text image is a long text, the long text text recognition model is called for recognition; if the text image is in a specific language, the text recognition model of the language is called for recognition.

[0091] For example, assuming that the text in a text image is vertical text, the system calls the vertical text detection model for detection. If the detection result shows that the text is vertical, the system selects a recognition model that specializes in vertical text for recognition. Similarly, if the text in the text image is a long text, the system calls the long text detection model for detection. If the detection result shows that the text is a long text, the system selects a recognition model that specializes in long text for recognition. In addition, if the text in the text image is in a specific language, the system calls the text language classification model for detection. If the detection result shows that the text belongs to a specific language, the system selects a recognition model that specializes in that language for recognition. These models can better handle the grammatical structure and writing habits of a specific language when recognizing text in that language, thereby improving recognition accuracy.

[0092] In this embodiment, by selecting a dedicated text recognition model based on the feature detection results, the machine review system can provide more accurate recognition results for different types of text images. This can not only reduce recognition errors and improve recognition accuracy and efficiency, but also ensure that different types of text images can receive optimal processing. Therefore, this embodiment can significantly improve the performance of the entire machine review system and ensure the accuracy and efficiency of live content review.

[0093] In some implementation variations, before calling the lightweight text recognition model to perform text recognition on the text image, the method further includes:

[0094] A pre-trained text rotation detection model is called to perform rotation detection on the text image, and the text image detected to be rotated is corrected.

[0095] One of the purposes of this embodiment is to detect the rotation of the text image in advance and correct it to ensure that subsequent feature detection and text recognition can be performed on the basis of the correct image.

[0096] The text rotation detection model refers to a model specifically used to detect whether text in a text image is rotated.

[0097] In the specific implementation of this embodiment, before performing multi-dimensional feature detection, the text rotation detection model can be called to perform rotation detection on the text image. If rotation is detected in the text image, the system can correct the image to restore the text to a normal horizontal or vertical state. The correction process usually involves image rotation and / or cropping to ensure that the text is in a standard position for subsequent processing.

[0098] For example, when processing a photo containing tilted text, the text rotation detection model can detect that the text has rotated and calculate the rotation angle. Then, the system can correct the image according to the rotation angle, restoring the text to its normal horizontal or vertical state. In this way, subsequent feature detection and text recognition can be performed on a corrected standard image, improving the accuracy and efficiency of recognition.

[0099] In this embodiment, by adding the steps of text rotation detection and correction before multi-dimensional feature detection, it can be ensured that all subsequent processing steps are performed on the basis of a standardized image. This can not only avoid recognition errors caused by text rotation, but also improve the accuracy and reliability of the entire live review process. Therefore, this embodiment can significantly improve the performance of the machine review system and ensure the accuracy and efficiency of content review.

[0100] In some modified embodiments, before making a violation determination based on all the text recognition results corresponding to the image to be reviewed, it further includes:

[0101] Invoking a pre-trained text error correction model to perform error correction processing on all the text recognition results corresponding to the image to be reviewed.

[0102] One of the purposes of this embodiment is to correct potential errors in the recognition results through a pre-trained text error correction model, thereby improving the accuracy of text recognition.

[0103] Among them, the text error correction model can refer to a model specifically used to identify and correct errors in text recognition results. This model is usually trained based on machine learning techniques and can identify and correct errors generated during the OCR recognition process, such as character misrecognition, spelling mistakes, etc.

[0104] In the specific implementation of this embodiment, after the text in the image is recognized, the system sends all the text recognition results into the pre-trained text error correction model. This model will analyze the text recognition results, identify possible errors therein, and attempt to correct them. For example, if OCR recognition misidentifies "example" as "exaple", the text error correction model can detect this error and correct it to the correct "example".

[0105] For example, assume that an image contains a piece of text "We are committed to providing high-quality content", but due to certain reasons, the OCR recognition result wrongly shows "We are commited to providing high-quality content". At this time, the text error correction model can recognize that "ted" is not a reasonable word in this context and correct it to the correct "mitted" according to the context. In this way, the system can ensure that the final text recognition result is accurate and error-free.

[0106] In this embodiment, by adding a text error correction process step before making a violation judgment, the system can improve the accuracy of the text recognition results and reduce misjudgments caused by recognition errors. Therefore, this embodiment can ensure the reliability of subsequent violation judgments and improve the accuracy and efficiency of the entire machine review system.

[0107] In some modified implementations, the performing violation judgment based on all the text recognition results corresponding to the image to be reviewed includes:

[0108] Input all the text recognition results corresponding to the image to be reviewed into a pre-trained illegal text recognition model, and use the illegal text recognition model to make a violation judgment; or

[0109] All the text recognition results corresponding to the image to be reviewed are matched with illegal keywords, and illegal judgment is made based on the keyword matching results.

[0110] One of the purposes of this embodiment is to ensure that it is possible to accurately identify whether illegal information is contained by performing illegal judgment on all identified text contents.

[0111] Among them, the illegal text recognition model can refer to a model trained by machine learning technology, which can evaluate whether the text content contains illegal information. In the specific implementation of this embodiment, after obtaining all text recognition results, these results can be passed as input to the illegal text recognition model. The model will analyze the text content and evaluate whether these texts contain illegal information based on preset violation standards. If the model evaluation result shows that the text has a risk of violation, the text is determined to be illegal.

[0112] In addition, in another embodiment, violation judgment can be made by matching illegal keywords, such as comparing all identified text recognition results with a list containing known illegal words. If any text recognition result is found to contain illegal words in the list, it is considered that the text recognition result contains illegal information.

[0113] For example, suppose the text recognition result in an image is "This video contains adult content". If the illegal text recognition model is used for judgment, the model will evaluate the content of this text and determine whether it is illegal based on the preset violation standards. If the model evaluation result shows that this text has a risk of violation, then the text is judged to be illegal. For another example, if the judgment is made by using the illegal keyword matching method, the system will compare the identified text recognition result with a list containing known illegal words. If it is found that the text recognition result contains the illegal word "adult" in the list, then the text recognition result is judged to contain illegal information.

[0114] In this implementation, by using the illegal text recognition model or illegal keyword matching to make illegal judgments, the system can effectively evaluate whether the text content contains illegal information. Therefore, this implementation can improve the accuracy and efficiency of live broadcast review and ensure the security and compliance of platform content.

[0115] In some modified implementations, the performing violation judgment based on all the text recognition results corresponding to the image to be reviewed includes:

[0116] Violation keyword matching is performed for all the text recognition results corresponding to the image to be reviewed, and violation judgment is performed based on the violation keyword matching results.

[0117] One of the purposes of this implementation is to automatically match all identified text content with illegal keywords, so as to ensure that it is possible to accurately determine whether the image to be reviewed contains illegal content. Among them, illegal keyword matching refers to comparing the identified text recognition results with a preset sensitive word list to determine whether there is illegal content.

[0118] When this embodiment is implemented, after obtaining all text recognition results, the system can send these results to the illegal keyword matching module. This module will compare each identified text recognition result with the preset sensitive word list one by one. If any text recognition result contains words in the sensitive word list, the system can immediately mark the text recognition result as containing illegal information. For example, the system will check whether the identified text recognition results contain preset sensitive words, such as "gambling", "pornography", "illegal", etc. If these words are detected, the system can determine that the image to be reviewed contains illegal content according to the preset rules.

[0119] In this implementation, by matching illegal keywords for all the text recognition results corresponding to the image to be reviewed, and making illegal judgments based on the illegal keyword matching results, the system can not only ensure a comprehensive evaluation of all recognized text content, but also improve the accuracy and efficiency of illegal judgments. Therefore, this implementation can significantly improve the accuracy and efficiency of live content review and ensure the security and compliance of platform content.

[0120] Based on the above implementation, in some modified implementations, the violation judgment based on the violation keyword matching result includes:

[0121] If the text recognition result contains illegal keywords, a word segmentation model is used to determine whether the text recognition result can be segmented;

[0122] If the image cannot be split, it is judged that the image to be reviewed is in violation of the rules;

[0123] If segmentation is possible, the text recognition result is input into a pre-trained illegal text recognition model, and whether the image to be reviewed is illegal is determined based on the recognition result output by the illegal text recognition model.

[0124] One of the purposes of this implementation is to ensure accurate violation judgment of the identified text content through a multi-level violation judgment mechanism.

[0125] Among them, the word segmentation model refers to the model used to segment the text recognition results into meaningful word sequences, and is mainly used to determine whether the text recognition results are composed of multiple independent words. The illegal text recognition model refers to a model specifically used to identify whether the text content is illegal, and determines whether there is illegal information by analyzing the overall content of the text recognition results.

[0126] When the present embodiment is implemented, after obtaining all text recognition results, the system first matches the illegal keywords. If the text recognition result contains illegal keywords, the system further uses a word segmentation model to determine whether the text recognition result can be segmented. If it cannot be segmented, it means that the original meaning of the illegal keywords is expressed, and the system can immediately determine that the image to be reviewed is illegal. This is because the text recognition result that cannot be segmented is likely to be a complete illegal word itself, and it can be determined as illegal without further analysis. If it can be segmented, it means that the expression may not be the original meaning of the illegal keyword, and further judgment is required. The system inputs the text recognition result into a pre-trained illegal text recognition model, and judges whether the image to be reviewed is illegal based on the recognition result output by the illegal text recognition model. This is because there may be multiple combinations of text recognition results that can be segmented, and further analysis is required to accurately determine whether it is illegal.

[0127] In this implementation, by adopting a multi-level violation judgment mechanism, the system can not only improve the accuracy and reliability of violation judgment, but also reduce the false alarm rate, ensuring that only images that truly violate the rules will be marked and processed. Therefore, this implementation can significantly improve the accuracy and efficiency of live content review and ensure the security and compliance of platform content.

[0128] Based on the above implementation, in other modified implementations, the violation judgment based on the violation keyword matching result includes:

[0129] If the text recognition result does not contain the illegal keywords, the fuzzy matching model is called to perform fuzzy matching on the text recognition result to obtain a fuzzy matching result;

[0130] The fuzzy matching result is input into a pre-trained illegal text recognition model, and whether the image to be reviewed is illegal is determined according to the recognition result output by the illegal text recognition model.

[0131] One of the purposes of this implementation is to ensure accurate violation judgment of the identified text content through a multi-level violation judgment mechanism, and to identify potential violation content even in the absence of obvious violation keywords.

[0132] The fuzzy matching model refers to a model used to detect whether there are similar or deformed illegal words in the text recognition results.

[0133] In the specific implementation of this embodiment, after obtaining all text recognition results, the system can first perform illegal keyword matching. If the text recognition result does not contain the illegal keyword, the system will further call the fuzzy matching model to perform fuzzy matching on the text recognition result. The fuzzy matching model can detect whether there are approximate or deformed illegal words in the text recognition result. If the fuzzy matching result shows that there is potential illegal content, the system can input the fuzzy matching result into a pre-trained illegal text recognition model, and judge whether the image to be reviewed is illegal based on the recognition result output by the illegal text recognition model. This is because fuzzy matching can capture deformed or misspelled illegal words, and the illegal text recognition model can further confirm whether these words constitute illegal content.

[0134] In this implementation, by adopting a multi-level violation judgment mechanism, the system can not only improve the accuracy and reliability of violation judgment, but also reduce the false alarm rate, ensuring that only images that truly violate the rules will be marked and processed. Therefore, this implementation can significantly improve the accuracy and efficiency of live content review and ensure the security and compliance of platform content.

[0135] In some modified implementations, before performing violation judgment based on all the text recognition results corresponding to the image to be reviewed, the method further includes:

[0136] Calling a pre-trained computer vision recognition model to identify scene element information in the image to be reviewed;

[0137] The making violation judgment according to all the text recognition results corresponding to the image to be reviewed includes:

[0138] A violation judgment is performed based on the overall information consisting of all the text recognition results and the scene element information corresponding to the image to be reviewed to obtain a violation judgment result.

[0139] One of the purposes of this embodiment is to comprehensively evaluate whether an image contains illegal content by combining the text content and non-text scene element information in the image, thereby improving the accuracy and comprehensiveness of violation judgment.

[0140] Among them, the computer vision recognition (CVR) model can refer to an algorithm model trained by machine learning, which is used to identify various scene elements in an image, including people, animals, environments, objects, actions, colors, etc., and convert them into understandable and processable information. The CVR model is usually based on deep learning technology, such as convolutional neural networks (CNNs), which can accurately identify various scene elements in an image. In this embodiment, after text recognition, the computer vision recognition model can be called to analyze the scene elements in the image, and then the recognized text content is combined with the scene element information to form a complete image description, and then the overall information is used to make a violation judgment to determine whether the image contains illegal content.

[0141] For example, suppose an image contains a text "Illegal items for sale" and there is a scene in the image that displays prohibited items. First, the text "Illegal items for sale" in the image is identified through OCR technology. Then, a computer vision recognition model is used to analyze the scene elements in the image and identify the prohibited items displayed in the image. Then, the text content "Illegal items for sale" is combined with the scene element information "Displaying prohibited items" to make a comprehensive violation judgment. In this way, the system can not only identify the illegal content in the text, but also further confirm whether the image contains illegal information by analyzing the scene in the image.

[0142] In this embodiment, by combining text content and scene element information to make violation judgments, the system can more comprehensively evaluate the overall content of the image, thereby improving the accuracy and reliability of violation judgments. Therefore, this embodiment can significantly improve the performance of the machine review system and ensure the accuracy and efficiency of content review.

[0143] To facilitate understanding, the live content review method of the present application is further explained with reference to a specific example.

[0144] In a specific example, given that the characteristic of live machine review is high timeliness, in view of this characteristic, it is necessary to shorten the entire processing flow as much as possible. In the optical character recognition (OCR) link, traditional detection and recognition methods can be used. The image is directly input into a very lightweight text detection model and a very lightweight recognition model. This model only needs to slightly determine whether it is a vertical layout, and then it can output the OCR result. However, due to the lightweight design of the OCR model, its misjudgment rate is relatively high, and there is a lack of an error correction mechanism. Therefore, the design of the subsequent text processing link needs to be more complex.

[0145] Specifically, the recognized text will undergo sensitive word matching. Those that match need to be further analyzed through a word segmentation model to determine whether the sensitive word can be reasonably segmented. For example, if there are other words before and after a sensitive word, such as "x steps down x", the words before and after "steps down" can reasonably separate it, indicating that it does not really express the meaning of "steps down". In this case, after the sensitive word is segmented, it can be further recognized through a text classification model. If the sensitive word cannot be segmented, it means that it expresses the original meaning of the sensitive word. Therefore, it needs to be sent for review. In addition, considering that live broadcasts are mostly in game areas, there is no semantic context, so there is no need for text splicing logic. However, because there are many game areas, there will be an additional fuzzy matching part in the live broadcast solution.形近字等匹配 is performed on the text recognition results that do not match the sensitive words to avoid missed judgments.

[0146] Specifically, please refer to Figure 2 , which shows a flowchart of a specific example of the live content review method provided by an embodiment of the present application. The live content review method shown in the figure can include the following steps:

[0147] S1. Extract cut frames from the live video stream and use the extracted cut frames as images to be reviewed;

[0148] S2. Call a text detection model to detect the text in the image to be reviewed, and crop the image to be reviewed based on the detected text to obtain a text image;

[0149] S3. Call a vertical text detection model to detect whether the text in the text image is vertical text. If so, jump to step S402; if not, jump to S4;

[0150] S4. Call a text recognition model to recognize the text in the text image to obtain a text recognition result; among them, different solutions are adopted for different situations:

[0151] S401: For the non-vertical text image detected in S3, call a general text recognition model for recognition.

[0152] S402: for the vertical text image detected in S3, calling a vertical text recognition model to perform recognition;

[0153] S5, outputting the text recognition results corresponding to each of the text images;

[0154] S6, based on the sensitive word platform (sensitive word matching model), the sensitive word matching model is called to match the text recognition result with sensitive words. If a sensitive word (i.e., an illegal keyword) is matched, it is determined that there is illegal information in the text recognition result, and the result is output to S7; if no sensitive word is matched, further fuzzy matching is required, and the process jumps to S9;

[0155] S7, calling the word segmentation model to determine whether the text recognition result can be segmented. If it cannot be segmented, it means that the original meaning of the sensitive word is expressed and needs to be submitted for review;

[0156] If it can be segmented, it means that the expression may not be the original meaning of the sensitive word and needs to be further processed and output to S8;

[0157] S8, calling the sensitive word segmentation model to segment the text recognition result, and inputting the segmentation result into a text classification model (i.e., a violation text recognition model) to determine its violation risk level information;

[0158] S9, calling the fuzzy matching model to perform fuzzy matching on the text recognition results that do not hit the sensitive words, so as to further identify whether there is potential illegal information. If the match is successful, the text classification model (i.e., illegal text recognition model) is further input for judgment;

[0159] S10. Perform corresponding processing according to the violation judgment result of the frame cutting, including releasing, submitting for review, etc.

[0160] It should be noted that in the above embodiments of the present application, a large number of models are introduced, including text detection model, text rotation detection model, vertical text detection model, long text detection model, text language classification model, special text recognition model, vertical text recognition model, long text recognition model, text error correction model, illegal text recognition model and computer vision recognition model. In specific implementation, the above models can be applied to the present application scheme directly or after fine-tuning training using existing models provided by the prior art. The embodiments of the present application do not limit the specific structure of the above models.

[0161] Based on the same inventive concept, a live content review device is also provided in an embodiment of the present application. The method corresponding to the device may be the live content review method in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The live content review device provided in an embodiment of the present application can implement the above-mentioned live content review method, and the live content review device can be implemented by software, hardware, or a combination of software and hardware. For example, the live content review device may include integrated or separate functional modules or units to perform the corresponding steps in the above-mentioned methods. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is merely illustrative. Please refer to Figure 3 , the live content review device may include:

[0162] The text detection module 101 is used to call the text detection model to perform text detection on the image to be reviewed, and to crop the image to be reviewed according to the detected text to obtain at least one text image;

[0163] The text recognition module 102 is used to call a lightweight text recognition model to perform text recognition on each of the text images to obtain a text recognition result;

[0164] A violation judgment module 103 is used to perform violation judgment according to all the text recognition results corresponding to the image to be reviewed to obtain a violation judgment result;

[0165] The illegal image processing module 104 is used to process the live video stream and / or the live broadcast room of the live video stream that are judged to be illegal according to the preset illegal live broadcast processing rules.

[0166] In some modified implementations, the text recognition module 102 includes at least one of the following feature detection units:

[0167] A vertical feature detection unit, used to call a pre-trained vertical text detection model to detect whether the text image contains vertical text;

[0168] A long text feature detection unit, used to call a pre-trained long text detection model to detect whether the text image contains a long text;

[0169] The language feature detection unit is used to call a pre-trained text language classification model to detect the text language of the text image.

[0170] In some modified implementations, the text recognition module 102 includes at least one of the following text recognition units:

[0171] A vertical text recognition unit, configured to call a vertical text recognition model to perform text recognition on the text image if the text image is determined to be a vertical text image according to the feature detection result;

[0172] A long text recognition unit, configured to call a long text recognition model to perform text recognition on the text image if the text image is determined to be a long text image according to the feature detection result;

[0173] The designated language text recognition unit is used to call the designated language text recognition model to perform text recognition on the text image if it is determined that the text image is a designated language text image according to the feature detection result.

[0174] In some modified implementations, the text recognition module 102 further includes:

[0175] The rotation correction unit is used to call a pre-trained text rotation detection model to perform rotation detection on the text image, and correct the text image that is detected to be rotated.

[0176] In some variations, the device further comprises:

[0177] The text error correction model is used to call a pre-trained text error correction model to perform error correction processing on all the text recognition results corresponding to the image to be reviewed.

[0178] In some modified implementations, the violation determination module 103 includes:

[0179] A model judgment unit is used to input all the text recognition results corresponding to the image to be reviewed into a pre-trained illegal text recognition model, and use the illegal text recognition model to perform illegal judgment; or

[0180] The keyword judgment unit is used to match the illegal keywords of all the text recognition results corresponding to the image to be reviewed, and make illegal judgments based on the keyword matching results.

[0181] In some variations, the device further comprises:

[0182] A scene element recognition module, used to call a pre-trained computer vision recognition model to recognize scene element information in the image to be reviewed;

[0183] The violation judgment module 103 includes:

[0184] The overall judgment unit is used to make a violation judgment based on the overall information composed of all the text recognition results and the scene element information corresponding to the image to be reviewed, and obtain a violation judgment result.

[0185] The live content review device provided in the embodiment of the present application is based on the same inventive concept as the live content review method provided in the aforementioned embodiment of the present application and has the same beneficial effects.

[0186] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present application, and the method corresponding to the electronic device may be the live content review method in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The electronic device provided in an embodiment of the present application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the live content review methods and / or technical solutions of the aforementioned multiple embodiments of the present application.

[0187] The electronic device may be a user device, or a device formed by integrating a user device and a network device through a network, or may be an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablet computers, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, which can be used to implement some processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

[0188] Figure 4 The structure of an electronic device suitable for implementing the method and / or technical solution in the embodiment of the present application is shown, and the electronic device 1200 includes a central processing unit (CPU, Central Processing Unit) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM, Read Only Memory) 1202 or the program loaded from the storage part 1208 to the random access memory (RAM, Random Access Memory) 1203. In RAM 1203, various programs and data required for system operation are also stored. CPU 1201, ROM 1202 and RAM1203 are connected to each other through bus 1204. Input / output (I / O, Input / Output) interface 1205 is also connected to bus 1204.

[0189] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, etc., and a speaker, etc.; a storage section 1208 including one or more computer-readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet.

[0190] In particular, the methods and / or embodiments in the embodiments of the present application may be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 1201, the above functions defined in the method of the present application are executed.

[0191] Another embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of the present application described above.

[0192] Specifically, the present embodiment may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination therewith.

[0193] Computer readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than a computer readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0194] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0195] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0196] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0197] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0198] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0199] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0200] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0201] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0202] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0203] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in a device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

Claims

1. A method for reviewing live content, wherein: The method comprises: Extracting frames from a live video stream and using the extracted frames as images to be reviewed; Calling a text detection model to perform text detection on the image to be reviewed, and cropping the image to be reviewed according to the detected text to obtain at least one text image; For each of the text images, calling a lightweight text recognition model to perform text recognition on the text image to obtain a text recognition result; Performing violation judgment according to all the text recognition results corresponding to the image to be reviewed to obtain a violation judgment result; The live video stream and / or the live broadcast room of the live video stream that are judged to be in violation are processed according to the preset illegal live broadcast processing rules.

2. The live broadcast content review method according to claim 1, wherein: The calling of the lightweight text recognition model to perform text recognition on the text image includes: Using a lightweight feature detection model to perform multi-dimensional feature detection on the text image; If it is determined according to the feature detection result that the text image has the specified feature, calling the dedicated text recognition model corresponding to the specified feature to perform text recognition on the text image; Otherwise, the lightweight universal text recognition model is called to perform text recognition on the text image.

3. The live broadcast content review method according to claim 2, wherein: The adopting a lightweight feature detection model to perform multi-dimensional feature detection on the text image includes at least one of the following: Calling a pre-trained vertical text detection model to detect whether the text image contains vertical text; Calling a pre-trained long text detection model to detect whether the text image contains long text; A pre-trained text language classification model is called to detect the text language of the text image.

4. The live broadcast content review method according to claim 3, wherein: If it is determined according to the feature detection result that the text image has the specified feature, calling a dedicated text recognition model corresponding to the specified feature to perform text recognition on the text image, including: If it is determined according to the feature detection result that the text image is a vertical text image, calling a vertical text recognition model to perform text recognition on the text image; If it is determined according to the feature detection result that the text image is a long text image, calling a long text recognition model to perform text recognition on the text image; If it is determined according to the feature detection result that the text image is a text image of a specified language, a text recognition model of the specified language is called to perform text recognition on the text image.

5. The live broadcast content review method according to claim 1, wherein: The making violation judgment according to all the text recognition results corresponding to the image to be reviewed includes: Violation keyword matching is performed for all the text recognition results corresponding to the image to be reviewed, and violation judgment is performed based on the violation keyword matching results.

6. The live broadcast content review method according to claim 5, wherein: The violation judgment based on the violation keyword matching result includes: If the text recognition result contains illegal keywords, a word segmentation model is used to determine whether the text recognition result can be segmented; If the image cannot be split, it is judged that the image to be reviewed is in violation of the rules; If segmentation is possible, the text recognition result is input into a pre-trained illegal text recognition model, and whether the image to be reviewed is illegal is determined based on the recognition result output by the illegal text recognition model.

7. The live broadcast content review method according to claim 5, wherein: The violation judgment based on the violation keyword matching result includes: If the text recognition result does not contain the illegal keywords, the fuzzy matching model is called to perform fuzzy matching on the text recognition result to obtain a fuzzy matching result; The fuzzy matching result is input into a pre-trained illegal text recognition model, and whether the image to be reviewed is illegal is determined according to the recognition result output by the illegal text recognition model.

8. A live content review device, wherein: The device comprises: An image extraction module is used to extract frames from a live video stream and use the extracted frames as images to be reviewed; A text detection module, used to call a text detection model to perform text detection on the image to be reviewed, and to crop the image to be reviewed according to the detected text to obtain at least one text image; A text recognition module, for calling a lightweight text recognition model to perform text recognition on each of the text images to obtain a text recognition result; A violation judgment module, used to make a violation judgment based on all the text recognition results corresponding to the image to be reviewed, and obtain a violation judgment result; The illegal image processing module is used to process the live video stream and / or the live broadcast room of the live video stream that are judged to be illegal according to the preset illegal live broadcast processing rules.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method according to any one of claims 1 to 7.

Images