Video risk detection method and device

Abstract

One or more embodiments of the invention disclose a video risk detection method and device, and relates to the technical field of risk detection.The method comprises the steps that firstly, a risk detection request for a target video is received, segmentation processing is conducted on the target video based on the risk detection request, a plurality of video clips are obtained, each video clip comprises a plurality of video frames, and each video frame comprises a plurality of video frames; secondly, respectively extracting key frames of the target video from each video clip, respectively carrying out text information detection processing on video frames in each video clip to obtain a first video frame containing text information, then extracting the text information contained in the first video frame to obtain a target text in the target video, and finally, carrying out text information detection processing on the target text in the target video. And performing risk detection on the target video based on the target text in the target video and the video images in the key frame and the first video frame to obtain a risk detection result of the target video.

Description

Technical Field

[0001] This document relates to the field of risk detection technology, and in particular to a video risk detection method and device. Background Art

[0002] At present, with the development of Internet technology, more and more application content forms are presented to users. Among them, short videos, as a fast-paced, easy-to-consume, entertaining and creative emerging content form, occupy an important position in the layout of many application content. Due to the low production and sharing threshold of short videos, they can attract a large number of users, but this brings with it the possibility of non-compliant content in the video. In addition, as people pay more and more attention to their privacy data, risk detection of videos has become an important issue in the promotion of video content. Therefore, it is necessary to provide a video detection method that can improve the accuracy of video content risk detection while taking into account detection timeliness and detection capacity. Summary of the invention

[0003] On the one hand, one or more embodiments of the present specification provide a video risk detection method, including: receiving a risk detection request for a target video, and segmenting the target video based on the risk detection request to obtain multiple video clips, each video clip including multiple video frames; extracting key frames of the target video from each video clip, and performing text information detection processing on the video frames in each video clip to obtain a first video frame containing text information; extracting the text information contained in the first video frame to obtain a target text in the target video; performing risk detection on the target video based on the target text in the target video, and the key frames and the video images in the first video frames to obtain a risk detection result of the target video.

[0004] On the other hand, one or more embodiments of the present specification provide a video risk detection device, including: a preprocessing module, which receives a risk detection request for a target video, and segments the target video based on the risk detection request to obtain multiple video segments, each of which includes multiple video frames; a key frame and first video frame acquisition module, which extracts the key frames of the target video from each video segment, and performs text information detection processing on the video frames in each video segment to obtain a first video frame containing text information; a target text acquisition module, which extracts the text information contained in the first video frame to obtain a target text in the target video; a risk detection module, which performs risk detection on the target video based on the target text in the target video, and the video images in the key frames and the first video frames to obtain a risk detection result of the target video.

[0005] On the other hand, one or more embodiments of the present specification provide an electronic device, comprising: a processor; and a memory arranged to store computer-executable instructions, wherein when the executable instructions are executed, the processor is enabled to: receive a risk detection request for a target video, and segment the target video based on the risk detection request to obtain multiple video segments, each video segment including multiple video frames; extract key frames of the target video from each video segment, and perform text information detection on the video frames in each video segment to obtain a first video frame containing text information; extract the text information contained in the first video frame to obtain a target text in the target video; perform risk detection on the target video based on the target text in the target video, and the key frames and the video images in the first video frame to obtain a risk detection result of the target video.

[0006] On the other hand, one or more embodiments of the present specification provide a storage medium for storing a computer program, which can be executed by a processor to implement the following process: receiving a risk detection request for a target video, and segmenting the target video based on the risk detection request to obtain multiple video clips, each video clip including multiple video frames; extracting key frames of the target video from each video clip, and performing text information detection processing on the video frames in each video clip to obtain a first video frame containing text information; extracting the text information contained in the first video frame to obtain a target text in the target video; performing risk detection on the target video based on the target text in the target video, and the key frames and the video images in the first video frames to obtain a risk detection result of the target video.

[0007] On the other hand, one or more embodiments of the present specification also provide a computer program product, including a computer program, which implements the following process when executed by a processor: receiving a risk detection request for a target video, and segmenting the target video based on the risk detection request to obtain multiple video clips, each video clip including multiple video frames; extracting key frames of the target video from each video clip, and performing text information detection processing on the video frames in each video clip to obtain a first video frame containing text information; extracting the text information contained in the first video frame to obtain a target text in the target video; performing risk detection on the target video based on the target text in the target video, and the key frames and the video images in the first video frames to obtain a risk detection result for the target video. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0009] Figure 1 is a schematic flow chart of a video risk detection method according to an embodiment of this specification; Figure 2 is a schematic flow chart of a video risk detection method according to another embodiment of the present specification; Figure 3 is a schematic diagram of the implementation principle of a video risk detection method according to an embodiment of this specification; Figure 4 is a schematic block diagram of a video risk detection system according to an embodiment of this specification; Figure 5 It is a schematic block diagram of an electronic device according to an embodiment of the present specification. DETAILED DESCRIPTION

[0010] One or more embodiments of the present specification provide a video risk detection method and device.

[0011] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0012] like Figure 1As shown, an embodiment of this specification provides a video risk detection method, and the execution subject of the method can be a terminal device or a server, wherein the terminal device can be a certain terminal device such as a mobile phone, a tablet computer, or a computer device such as a laptop or a desktop computer, or an IoT device (specifically such as a smart watch, a car device, etc.). The server can be an independent server, or a server cluster composed of multiple servers, etc. The server can be a background server in the financial field or the online shopping field, or a background server of an application, etc. In this embodiment, the server is used as an example for detailed description. For the execution process of the terminal device, please refer to the following related content, which will not be repeated here. The method can specifically include the following steps: In step S102, a risk detection request for a target video is received, and the target video is segmented based on the risk detection request to obtain a plurality of video segments, each of which includes a plurality of video frames.

[0013] In the embodiments of this specification, the target video can be a short video, a live video, or a recorded video. In particular, short videos can attract a large number of users due to their low production and sharing thresholds, and the current level of short video releases is growing rapidly, which places high demands on the recognition accuracy and capacity guarantee of the content prevention and control system. In order to prevent and control risks in videos, one way is to uniformly cut frames of the video, extract key frames from the uniform frames based on the key frame model, and finally detect content risks of the identified key frames, which can take into account both detection timeliness and detection magnitude. However, for content such as text or special effect fonts in short videos, if the adjacent uniform frames are basically unchanged in the picture content and only the text changes, the key frame model often cannot output the video frame as a key frame, but the text content is also an important basis for locating whether the video content is risky. Therefore, this scheme proposes a video risk detection method that improves the accuracy of short video content risk detection on the basis of taking into account detection timeliness and detection capacity.

[0014] After receiving the risk detection request of the target video, the target video is segmented (intercepted) based on the risk detection request to obtain multiple video clips, each of which includes one or more video frames. In implementation, the method of segmenting the target video can be random segmentation to obtain multiple video clips, or even segmentation according to a preset time interval, or even segmentation according to the size of the video capacity.

[0015] In step S104, key frames of the target video are extracted from each video segment, and text information detection processing is performed on the video frames in each video segment to obtain a first video frame containing text information.

[0016] Among them, a key frame is a video frame that plays a decisive role among several video frames that constitute a video. In the embodiment of this specification, a key frame can be a video frame in each video segment whose video frame change exceeds a preset change amount. The video frame change amount can be a text content change amount and / or an image content change amount.

[0017] In implementation, the method of extracting the key frame of the target video from each video clip can be based on the video content by using the difference between video frames, for example, calculating the difference between consecutive video frames, and when the difference exceeds a preset threshold, the frame is considered to be a key frame. It is also possible to use a histogram comparison method based on the video content, for example, comparing the color histograms of consecutive video frames, and when the histogram has a significant change, extract the video frame as a key frame. It is also possible to use a motion-based method, for example, using the optical flow method to analyze the pixel motion between video frames, and when significant motion is detected, extract the video frame corresponding to the significant motion as a key frame.

[0018] The text information in the embodiments of this specification may be subtitle information in the target video, special effect font information in the target video, or may include subtitle information and special effect font information at the same time. In implementation, the method of performing text information detection processing on the video frames in each video clip to obtain the first video frame containing text information may use an OCR (Optical Character Recognition) tool, or a connected component method, that is, using connected components to roughly locate the area that may be the text, and then using texture analysis and feature vector extraction, using statistical graphs for screening and classification to identify the real text area.

[0019] In step S106, the text information contained in the first video frame is extracted to obtain the target text in the target video.

[0020] In implementation, the text information contained in the first video frame is extracted to obtain the target text in the target video. A feature matching method (such as SIFT, SURF, etc.) can be used to identify and match text features in the first video frame, or a deep learning neural network can be used to identify text features in the first video frame.

[0021] In step S108, risk detection is performed on the target video based on the target text in the target video and the video images in the key frame and the first video frame to obtain a risk detection result of the target video.

[0022] After determining the target text, risk detection is performed on the target video based on the target text and the video image. Based on the target text, risk detection can be performed on the text information in the target video. Based on the video images in the key frames and the first video frame, risk detection can be performed on the image information in the target video. Finally, based on the text information detection results and the image information detection results, the risk detection result of the target video is obtained.

[0023] In implementation, risk detection can be performed on the target video based on the target file and video image respectively, or risk detection can be performed on the target text and video image at the same time. For example, a graphic-text cross-modal model and a classification model are combined, wherein the graphic-text cross-modal model is used to extract the target text in the target video, as well as the video images in the key frames and the first video frame, and convert multiple modalities into the same modality, and then the risk detection result of the target video is obtained by comparing the classification model with a preset risk threshold.

[0024] The embodiment of the specification provides a video risk detection method, which first receives a risk detection request for a target video, and based on the risk detection request, performs segmentation processing on the target video to obtain multiple video clips, each of which includes multiple video frames, and then extracts the key frames of the target video from each video clip, and performs text information detection processing on the video frames in each video clip to obtain a first video frame containing text information, and then extracts the text information contained in the first video frame to obtain a target text in the target video, and finally, based on the target text in the target video, and the key frame and the video image in the first video frame, performs risk detection on the target video to obtain a risk detection result of the target video. In this way, by performing text information detection processing on each video frame to obtain a first video frame containing text information, and extracting the text information in the first video frame to obtain the target text, and finally, based on the target text and the key frame and the video image in the first video frame, the risk detection result obtained by performing risk detection on the target video includes both image risk and text content risk, and can cover a wider range of content risks while ensuring the timeliness and detection capacity of risk detection, thereby effectively improving the precision and accuracy of risk detection. The risk detection method in the embodiments of this specification can not only perform risk detection on videos with large changes in picture content, but also perform risk detection on videos with small changes in picture content, where only the text information has changed. In particular, risk detection of short videos can avoid missing detection of text information in the video, and effectively improve the detection precision and accuracy of video risk detection.

[0025] In the embodiment of the present specification, in the above step S102, the target video is segmented based on the risk detection request, and there are various processing methods for obtaining multiple video clips. An optional processing method is provided below, and the details can be referred to the processing of the following steps S1022-S1026.

[0026] In step S1022, video decoding processing is performed on the target video to obtain the target video after video decoding processing.

[0027] Videos usually need to be compressed or otherwise encoded during storage or transmission. By decoding the video, the target video can be restored to the original, displayable video frame, which is conducive to further improving the accuracy and reliability of video risk detection.

[0028] In step S1024, the target video after the video decoding process is evenly divided based on a preset time interval to obtain a plurality of video segments.

[0029] By setting a preset time interval, the target video after video decoding is evenly divided (i.e., evenly cut frame processing), the video processing method is simpler, which is beneficial to improving the efficiency of video processing, and the target video is evenly divided so that the number of video frames in each video segment is basically the same, which is beneficial to improving the efficiency of subsequent key frame extraction.

[0030] In implementation, the preset time interval can be set according to the video application scenario, video risk detection requirements and user needs, and is not limited here.

[0031] In the embodiment of the present specification, in the above step S102, the target video is segmented based on the risk detection request, and there are various processing methods for obtaining multiple video clips. An optional processing method is provided below, and the details can be referred to the processing of the following steps S1022-S1026.

[0032] like Figure 2 As shown, in the embodiment of the present specification, after step S102, step S110 is further included: deduplication processing is performed on the key frame and the first video frame to obtain the deduplication-processed key frame and the first video frame.

[0033] In implementation, the method for deduplicating the key frame and the first video frame can be performed by calculating the similarity between the key frame and the first video frame. Specifically, a perceptual hashing algorithm can be used to compare the visual features of the video and calculate the similarity between the two, thereby removing duplicate video frames. The similarity between the two can also be calculated based on the difference between the video frames, thereby deduplicating duplicate video frames. Complex patterns and repeated content in the video can also be identified based on a deep learning algorithm, thereby calculating the similarity between the two.

[0034] By deduplicating key frames and the first video frame, video data can be simplified and repeated judgments on image and video risks can be reduced, thereby saving risk detection time and improving the efficiency and accuracy of video risk detection.

[0035] Corresponding to step S110, the processing method of step S108 may specifically refer to the following step S1082: based on the target text in the target video, and the video images in the key frame and the first video frame, risk detection is performed on the target video to obtain a risk detection result of the target video.

[0036] In the embodiment of the present specification, in the above step S104, text information detection processing is performed on the video frames in each video clip respectively, and there are many processing methods to obtain the first video frame containing text information. An optional processing method is provided below. For details, please refer to the processing of the following steps S1042-S1044.

[0037] In step S1042, based on an optical character recognition tool, text information detection processing is performed on the video frames in each video clip to extract the video frames containing text information. The optical character recognition tool includes at least one or more of TesseractOCR and JavaOCR.

[0038] In step S1044, the video frame containing the text information is used as the first video frame. In implementation, the optical character recognition tool can be implemented using TesseractOCR or JavaOCR. Tesseract is an open source OCR engine that uses neural network technology to improve the accuracy of text recognition. JavaOCR is an OCR technology that can be implemented in Java, and can be any OCR library or tool developed using the Java language. In step S1042, it is only necessary to determine whether the video frame of each video clip contains text information. The accuracy requirements of the OCR tool are relatively low. The use of the above-mentioned open source OCR tool can save resources and quickly and effectively improve the efficiency of text detection.

[0039] In the embodiment of the present specification, there may be various processing methods for extracting the key frames of the target video from each video clip in the above step S104. An optional processing method is provided below. For details, please refer to the processing of the following step S1046.

[0040] In step S1046, each video frame segment is input into a pre-trained key frame model to obtain a key frame of the target video. The key frame model is a model obtained by model training based on historical video segments, historical key frames and a preset loss function.

[0041] Using a pre-trained key frame model to obtain the key frames of the target video can effectively improve the accuracy and efficiency of key frame extraction.

[0042] In the embodiments of this specification, the processing method of the above step S106 can be various. An optional processing method is provided below. For details, please refer to the processing of the following steps S1062-S1064.

[0043] In step S1062, an optical character recognition module is constructed based on a deep learning neural network.

[0044] In implementation, the OCR module can be constructed based on CNN (Convolutional Neural Network), or based on RNN (Recurrent Neural Network), or based on both CNN and RNN. Among them, CNN is mainly used for image feature extraction, and can automatically extract high-level features from images through structures such as convolutional layers, pooling layers, and fully connected layers for subsequent text recognition tasks. Through the image feature extraction capability of CNN, it is possible to handle problems such as noise, blur, and illumination changes in images, thereby improving the accuracy of text information recognition. RNN can effectively capture the temporal information in text sequences, and is particularly effective for recognizing continuous text lines or handwritten text. In the method of constructing OCR modules based on CNN and RNN at the same time, a CRNN model can be constructed, in which CNN is used for feature extraction, RNN is used for sequence modeling, and the model is obtained by end-to-end training through the CTC (Connectionist Temporal Classification) loss function.

[0045] In addition, the OCR module can also be implemented using a long short-term memory network (LSTM) and a gated recurrent unit (GRU). The OCR module constructed using the above deep learning network can be used to extract image features, recognize text patterns, and ultimately convert the text in the image into text.

[0046] In step S1064, the text information contained in the first video frame is extracted based on the constructed optical character recognition module to obtain the target text in the target video.

[0047] By constructing an optical character recognition module based on a deep learning neural network, the text information in the first video frame can be accurately extracted, thereby obtaining the target text. The accuracy of the optical character recognition module in step S1062 is higher than the accuracy of the optical character recognition tool in step S1042.

[0048] In the embodiments of this specification, the processing method of the above step S108 can be various. An optional processing method is provided below. For details, please refer to the processing of the following steps S1082-S1086.

[0049] In step S1082, based on the target text in the target video, a text content risk detection is performed on the target video to obtain a text risk detection result of the target video.

[0050] In step S1084, based on the key frame and the video image in the first video frame, image content risk detection is performed on the target video to obtain an image risk detection result of the target video.

[0051] In step S1086 , the risk detection result of the target video is determined based on the text risk detection result and the image risk detection result of the target video.

[0052] Through the above steps S1082-S1086, the target video is respectively subjected to text content risk detection and image content risk detection. The detection items are carried out separately, which can improve the accuracy of the risk detection results. Moreover, the text content risk detection and image content risk detection can be carried out at the same time, which is also conducive to improving the efficiency of risk detection.

[0053] The principle of video risk detection in the embodiments of this specification can be found in Figure 3 ,Depend on Figure 3 It can be seen that the target video in the video risk detection process takes a short video as an example. The OCR module is used to detect text information on the video frame to obtain the first video frame and to extract the text information in the first video frame to obtain the target text. The accuracy requirements of the OCR modules used in the two places are different. The OCR module for extracting the text information in the first video frame to obtain the target text has higher accuracy. At this time, the OCR module (that is, the OCR module with higher accuracy) can be the optical character recognition module constructed based on the deep learning neural network as described above, and the OCR module with lower accuracy can be the optical character recognition tool as described above. Figure 3 In the process, the recognition system can be understood as a functional module corresponding to step S108 that performs risk identification on the target text and the video images in the key frames and the first video frame; the decision system can be understood as a functional module that determines whether the target video has a preset risk based on the recognition result of the recognition system; the risk detection system can be understood as a functional module that collects and summarizes the risk detection results corresponding to the entire video risk detection method. Depend on Figure 3 It can be seen that the risk identification process mainly includes six stages: Stage 1: Figure 3 Step 1 in the above method is to perform video preprocessing (uniform segmentation) on the short video to obtain frames with an interval of n seconds; Stage 2: If Figure 3 Step 3, calling the key frame model to obtain the key frame from the screenshot; Phase 3: As shown in step 4 of step 3 in the figure, all uniformly cut frames are processed in parallel, the OCR module (OCR module with lower accuracy) is called, and the cut frame with text information (i.e., the first video frame) is returned; Stage 4: If Figure 3 Step 5, performing deduplication processing on key frames and frames with text information; Stage 5: Perform OCR processing (high-precision OCR module) on the deduplicated screenshots to extract text information, detect text content risks, and simultaneously detect other content risks (i.e., image content risks) of the screenshots; Stage 6: The detection results are returned, and the decision system determines whether the short video poses a risk.

[0054] In summary, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recorded in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing can be advantageous.

[0055] The above is a video risk detection method provided by one or more embodiments of this specification. Based on the same idea, one or more embodiments of this specification also provide a video risk detection device, such as Figure 4 shown.

[0056] The video risk detection device includes: a preprocessing module 210, a key frame and first video frame acquisition module 220, a target text acquisition module 230 and a risk detection module 240, wherein: The preprocessing module 210 receives a risk detection request for a target video, and performs segmentation processing on the target video based on the risk detection request to obtain multiple video segments, each of which includes multiple video frames; The key frame and first video frame acquisition module 220 extracts the key frame of the target video from each video segment, and performs text information detection processing on the video frames in each video segment to obtain the first video frame containing text information; The target text acquisition module 230 extracts the text information contained in the first video frame to obtain the target text in the target video; The risk detection module 240 performs risk detection on the target video based on the target text in the target video and the video images in the key frame and the first video frame to obtain a risk detection result of the target video.

[0057] In the embodiment of this specification, the preprocessing module 210 includes: A request receiving unit, receiving a risk detection request for a target video; The video decoding unit performs video decoding processing on the target video to obtain the target video after video decoding processing; The uniform segmentation unit uniformly segments the target video after the video decoding process based on a preset time interval to obtain a plurality of video segments.

[0058] In the embodiment of the present specification, the video risk detection device further includes: a deduplication processing module, which performs deduplication processing on the key frame and the first video frame to obtain the key frame and the first video frame after deduplication processing.

[0059] Correspondingly, the risk detection module 240 performs risk detection on the target video based on the target text in the target video, and the key frame after deduplication processing and the video image in the first video frame to obtain a risk detection result of the target video.

[0060] In an embodiment of the present specification, a deduplication processing module performs deduplication processing on the key frame and the first video frame based on a perceptual hash algorithm, a method based on differences between video frames, or a deep learning algorithm to obtain deduplicated key frames and first video frames.

[0061] In the embodiment of this specification, the key frame and first video frame acquisition module 220 includes: A key frame acquisition unit inputs each video frame segment into a pre-trained key frame model to obtain a key frame of a target video, wherein the key frame model is a model obtained by model training based on historical video segments, historical key frames, and a preset loss function; A text information detection processing unit performs text information detection processing on video frames in each video segment based on an optical character recognition tool to extract video frames containing text information, wherein the optical character recognition tool includes at least one or more of TesseractOCR and JavaOCR; The first video frame determining unit uses the video frame containing the text information as the first video frame.

[0062] In the embodiment of this specification, the target text acquisition module 230 includes: An optical character recognition module building unit, which builds an optical character recognition module based on a deep learning neural network; The text information extraction unit extracts the text information contained in the first video frame based on the constructed optical character recognition module to obtain the target text in the target video.

[0063] In the embodiment of this specification, the risk detection module 240 includes: A text content risk detection unit performs text content risk detection on a target video based on a target text in the target video to obtain a text risk detection result of the target video; A video content risk detection unit performs image content risk detection on a target video based on the key frame and the video image in the first video frame to obtain an image risk detection result of the target video; The risk detection result determination unit determines the risk detection result of the target video based on the text risk detection result and the image risk detection result of the target video.

[0064] An embodiment of the present specification provides a video risk detection device, which first receives a risk detection request for a target video through a preprocessing module, and segments the target video based on the risk detection request to obtain multiple video clips, each of which includes multiple video frames. Secondly, the key frames of the target video are extracted from each video clip through a key frame and a first video frame acquisition module, and text information detection is performed on the video frames in each video clip to obtain a first video frame containing text information. Then, the target text acquisition module is used to extract the text information contained in the first video frame to obtain the target text in the target video. Finally, the risk detection module performs risk detection on the target video based on the target text in the target video and the video images in the key frames and the first video frames to obtain a risk detection result for the target video. In this way, by performing text information detection processing on each video frame to obtain a first video frame containing text information, and extracting the text information in the first video frame to obtain the target text, and finally based on the target text and the key frame and the video image in the first video frame, the target video is subjected to risk detection. The risk detection result obtained includes both image risk and text content risk. Under the premise of ensuring the timeliness and detection capacity of risk detection, it can cover a wider range of content risks, thereby effectively improving the precision and accuracy of risk detection. The risk detection method in the embodiments of this specification can not only perform risk detection on videos with large changes in picture content, but also perform risk detection on videos with small changes in picture content and only changes in text information. In particular, risk detection of short videos can avoid missing detection of text information in the video, and effectively improve the detection precision and accuracy of video risk detection.

[0065] Those skilled in the art should understand that the above-mentioned video risk detection device can be used to implement the video risk detection method described above, and the detailed description thereof should be similar to the description of the method part above, and will not be further described here to avoid tediousness.

[0066] Based on the same idea, one or more embodiments of this specification also provide an electronic device, such as Figure 5As shown. The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 301 and memory 302, and the memory 302 may store one or more storage applications or data. Among them, the memory 302 may be a temporary storage or a permanent storage. The application stored in the memory 302 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 301 can be configured to communicate with the memory 302 to execute a series of computer executable instructions in the memory 302 on the electronic device. The electronic device may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more input and output interfaces 305, and one or more keyboards 306.

[0067] Specifically in this embodiment, the electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for the electronic device, and the one or more programs are configured to be executed by one or more processors, including the following computer executable instructions: Receiving a risk detection request for a target video, and segmenting the target video based on the risk detection request to obtain a plurality of video segments, each of which includes a plurality of video frames; Extracting the key frames of the target video from each video clip respectively, and performing text information detection processing on the video frames in each video clip respectively, to obtain a first video frame containing text information; Extracting text information contained in the first video frame to obtain target text in the target video; Based on the target text in the target video, and the video images in the key frame and the first video frame, risk detection is performed on the target video to obtain a risk detection result of the target video.

[0068] One or more embodiments of this specification provide a storage medium, the storage medium is used to store computer executable instructions, and the executable instructions implement the following process when executed by a processor: Receiving a risk detection request for a target video, and segmenting the target video based on the risk detection request to obtain a plurality of video segments, each of which includes a plurality of video frames; Extracting the key frames of the target video from each video clip respectively, and performing text information detection processing on the video frames in each video clip respectively, to obtain a first video frame containing text information; Extracting text information contained in the first video frame to obtain target text in the target video; Based on the target text in the target video, and the video images in the key frame and the first video frame, risk detection is performed on the target video to obtain a risk detection result of the target video.

[0069] One or more embodiments of the present specification further provide a computer program product, including a computer program, which implements the following process when executed by a processor: Receiving a risk detection request for a target video, and segmenting the target video based on the risk detection request to obtain a plurality of video segments, each of which includes a plurality of video frames; Extracting the key frames of the target video from each video clip respectively, and performing text information detection processing on the video frames in each video clip respectively, to obtain a first video frame containing text information; Extracting text information contained in the first video frame to obtain target text in the target video; Based on the target text in the target video, and the video images in the key frame and the first video frame, risk detection is performed on the target video to obtain a risk detection result of the target video.

[0070] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0071] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0072] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0073] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0074] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0075] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system, or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] One or more embodiments of this specification are described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0077] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0079] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0080] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0081] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0082] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0083] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0084] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0085] The above description is only one or more embodiments of this specification and is not intended to limit this document. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of one or more embodiments of this specification.

Claims

1. A video risk detection method, comprising: Receiving a risk detection request for a target video, and segmenting the target video based on the risk detection request to obtain a plurality of video segments, each of which includes a plurality of video frames; Extracting the key frames of the target video from each video clip respectively, and performing text information detection processing on the video frames in each video clip respectively, to obtain a first video frame containing text information; Extracting text information contained in the first video frame to obtain target text in the target video; Based on the target text in the target video, and the key frame and the video image in the first video frame, risk detection is performed on the target video to obtain a risk detection result of the target video.

2. According to the method of claim 1, after extracting the key frames of the target video from each video segment, and performing text information detection processing on the video frames in each video segment, and obtaining the first video frame containing text information, the method further comprises: Deduplication processing is performed on the key frame and the first video frame to obtain a deduplication-processed key frame and a first video frame; The step of performing risk detection on the target video based on the target text in the target video and the key frame and the video image in the first video frame to obtain a risk detection result of the target video includes: Based on the target text in the target video, the key frame after deduplication processing, and the video image in the first video frame, risk detection is performed on the target video to obtain a risk detection result of the target video.

3. According to the method of claim 2, the method of performing deduplication processing on the key frame and the first video frame comprises: Perceptual hashing algorithm, method based on the difference between video frames, or deep learning algorithm.

4. The method according to claim 1, wherein the step of segmenting the target video based on the risk detection request to obtain a plurality of video segments comprises: Performing video decoding processing on the target video to obtain the target video after video decoding processing; Based on a preset time interval, the target video after video decoding is evenly segmented to obtain multiple video segments.

5. The method according to claim 1, wherein the step of performing text information detection processing on the video frames in each video segment to obtain the first video frame containing the text information comprises: Based on an optical character recognition tool, performing text information detection processing on video frames in each video clip, respectively, to extract video frames containing text information, wherein the optical character recognition tool includes at least one or more of TesseractOCR and JavaOCR; The video frame containing the text information is used as the first video frame.

6. The method according to claim 5, wherein extracting the text information contained in the first video frame to obtain the target text in the target video comprises: Building an optical character recognition module based on a deep learning neural network; The text information contained in the first video frame is extracted based on the constructed optical character recognition module to obtain the target text in the target video.

7. The method according to claim 1, performing risk detection on the target video based on the target text in the target video and the video images in the key frame and the first video frame to obtain the risk detection result of the target video, comprising: Based on the target text in the target video, performing text content risk detection on the target video to obtain a text risk detection result of the target video; Based on the key frame and the video image in the first video frame, performing image content risk detection on the target video to obtain an image risk detection result of the target video; Based on the text risk detection result and the image risk detection result of the target video, a risk detection result of the target video is determined.

8. The method according to claim 1, wherein extracting the key frames of the target video from each video segment comprises: Each of the video frame segments is input into a pre-trained key frame model to obtain the key frame of the target video, wherein the key frame model is a model obtained by model training based on historical video segments, historical key frames and a preset loss function.

9. A video risk detection device, comprising: A preprocessing module receives a risk detection request for a target video, and performs segmentation processing on the target video based on the risk detection request to obtain a plurality of video segments, each of which includes a plurality of video frames; A key frame and first video frame acquisition module extracts the key frame of the target video from each video segment, and performs text information detection processing on the video frames in each video segment to obtain a first video frame containing text information; A target text acquisition module extracts text information contained in the first video frame to obtain a target text in the target video; The risk detection module performs risk detection on the target video based on the target text in the target video and the key frame and the video image in the first video frame to obtain a risk detection result of the target video.

10. An electronic device, comprising: processor; as well as a memory arranged to store computer executable instructions which, when executed, cause the processor to: Receiving a risk detection request for a target video, and segmenting the target video based on the risk detection request to obtain a plurality of video segments, each of which includes a plurality of video frames; Extracting the key frames of the target video from each video clip respectively, and performing text information detection processing on the video frames in each video clip respectively, to obtain a first video frame containing text information; Extracting text information contained in the first video frame to obtain target text in the target video; Based on the target text in the target video, and the key frame and the video image in the first video frame, risk detection is performed on the target video to obtain a risk detection result of the target video.

Images