Video detection processing method and device

Abstract

An embodiment of the present specification provides a video detection processing method and device, which includes: first receiving video detection parameters sent by a resource management platform, obtaining a video stream from a cloud application according to the video detection parameters, performing text recognition on each video frame contained in the video stream to obtain text data, and inputting each video frame into a convolutional neural network for feature extraction to obtain image features, performing anomaly detection on the text data to obtain a text detection result, and calculating feature similarity based on a reference image feature and an image feature, determining a detection result of the video stream according to the text detection result, feature similarity and a similarity threshold, and returning the detection result to the resource management platform, and removing the video stream from the shelves when the detection result is that the detection fails, thereby improving the efficiency of anomaly detection of videos played in cloud applications, ensuring the accuracy and smoothness of video playback in cloud applications, and improving user experience.

Description

Technical Field

[0001] This document relates to the field of data processing technology, and in particular to a video detection processing method and device. Background Art

[0002] With the improvement of people's demand for quality of life and the development of cloud technology, cloud HD technology has been vigorously promoted. In addition, with the trend of cloud business, diversified cloud services have also greatly enriched users' digital life and provided content creators with a wider display platform. As users' living standards improve, the user scale of cloud applications has gradually shown an exponential growth, which has led to a significant increase in the usage rate of cloud playback applications. In order to ensure the accuracy and smoothness of cloud application playback content, it is necessary to conduct a detailed check of the initial configuration and status of cloud playback videos, including but not limited to checking video quality, audio synchronization, content integrity, and whether it contains illegal or inappropriate information for public disclosure. If it is found that the video content is inconsistent with the playback content and configuration or the playback fails, the video removal measures are executed in real time to ensure the health of the platform content and the user experience. How to better check the video content has become a focus of increasing attention for service providers and users. Summary of the invention

[0003] An object of one embodiment of the present specification is to provide a video detection processing method and apparatus to solve the problem of low efficiency in detecting video played by cloud applications.

[0004] To solve the above technical problems, an embodiment of this specification is implemented as follows: In a first aspect, an embodiment of the present specification provides a video detection processing method, including: Receiving video detection parameters sent by the resource management platform, and obtaining a video stream from a cloud application according to the video detection parameters; Performing text recognition on each video frame included in the video stream to obtain text data, and inputting each video frame into a convolutional neural network to perform feature extraction to obtain image features; Performing anomaly detection on the text data to obtain a text detection result, and calculating feature similarity based on a reference image feature and the image feature; The detection result of the video stream is determined according to the text detection result, the feature similarity and the similarity threshold, and the detection result is returned to the resource management platform.

[0005] In a second aspect, another embodiment of the present specification provides a video detection processing device, including: A video stream acquisition module is configured to receive video detection parameters sent by the resource management platform, and acquire a video stream from a cloud application according to the video detection parameters; A feature extraction module is configured to perform text recognition on each video frame included in the video stream to obtain text data, and input each video frame into a convolutional neural network to perform feature extraction to obtain image features; a similarity calculation module configured to perform anomaly detection on the text data to obtain a text detection result, and calculate feature similarity based on a reference image feature and the image feature; The detection result determination module is configured to determine the detection result of the video stream according to the text detection result, the feature similarity and the similarity threshold, and return the detection result to the resource management platform.

[0006] In the third aspect, another embodiment of the present specification provides a video detection and processing device, comprising: a memory, a processor, and computer executable instructions stored on the memory and executable on the processor, wherein the computer executable instructions, when executed by the processor, implement the steps of the video detection and processing method as described in the first aspect above.

[0007] In a fourth aspect, another embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the video detection processing method described in the first aspect above are implemented.

[0008] The video detection processing method provided in this embodiment first receives the video detection parameters sent by the resource management platform, obtains the video stream from the cloud application according to the video detection parameters, performs text recognition on each video frame contained in the video stream to obtain text data, and inputs each video frame into the convolutional neural network for feature extraction to obtain image features, performs anomaly detection on the text data to obtain a text detection result, and calculates feature similarity based on the reference image features and the image features, determines the detection result of the video stream according to the text detection result, the feature similarity and the similarity threshold, and returns the detection result to the resource management platform, and removes the video stream from the shelves when the detection result is that the detection fails, thereby improving the efficiency of anomaly detection of videos played in cloud applications, ensuring the accuracy and smoothness of video playback in cloud applications, and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in one or more embodiments of the present specification, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0010] Figure 1 A processing flow chart of a video detection processing method provided by one embodiment of this specification; Figure 2 An interactive schematic diagram provided for one embodiment of this specification; Figure 3 A text recognition flow chart provided for one embodiment of this specification; Figure 4 A convolutional neural network architecture diagram provided for one embodiment of this specification; Figure 5 A detection flow chart of a video detection system provided in one embodiment of this specification; Figure 6 A processing flow chart of a video detection processing method applied to a cloud application playback video detection scenario provided by an embodiment of this specification; Figure 7 A schematic diagram of a video detection and processing device provided in one embodiment of this specification; Figure 8 A schematic diagram of the structure of a video detection and processing device provided in one embodiment of the present specification. DETAILED DESCRIPTION

[0011] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0012] This specification provides a video detection processing method embodiment: Reference Figure 1 , which shows a processing flow chart of a video detection processing method provided by this embodiment, referring to Figure 6 , which shows a processing flow chart of a video detection processing method provided by this embodiment and applied to a cloud application playback video detection scenario.

[0013] Reference Figure 1 The video detection processing method provided in this embodiment specifically includes the following steps S102 to S108.

[0014] Step S102: receiving video detection parameters sent by the resource management platform, and acquiring a video stream from a cloud application according to the video detection parameters.

[0015] In this embodiment, the cloud application refers to an application running in a cloud computing environment, and some or all of the functions and services of the cloud application are provided through the Internet rather than being installed on a local computer or device. The resource management platform can be a cloud application CP (CP, Content Provider), that is, a content provider of a cloud application. The cloud application CP is used as a content provider to provide the required video parameter data and to remove the video stream from the shelves according to the detection results. The video detection parameters include at least one of the following: video identification (ID, Identification), video name, video director, and video actor.

[0016] The video detection processing method provided in this embodiment can be applied to a video detection system, where the video detection system is used to receive video detection parameters uploaded by a cloud application CP and obtain a video stream sent by the cloud application. Optionally, the video stream is transmitted by transparent transmission. In addition, the video detection system can also perform video detection to notify the resource management platform to remove video streams whose detection results are failures (i.e., non-compliant).

[0017] In addition, this embodiment also includes a cloud client, which is on the user side and communicates with the cloud platform through end-cloud technology, and transmits video detection parameters and video streams; it can also include a cloud platform, which is a comprehensive platform for providing cloud services, integrating multiple cloud services, and providing users with video storage, transmission, processing and other services, as well as loading of video streams corresponding to cloud applications. It should be noted that the cloud client and cloud platform in this application can also be a cloud HD client and a cloud HD platform.

[0018] In actual applications, in order to ensure the accuracy and smoothness of the video stream played by the cloud application, it is necessary to detect the video stream played in the cloud application, including but not limited to detecting video quality, audio synchronization, content integrity, and whether it contains illegal or inappropriate information for public disclosure. If it is found that the playback content and playback rules are inconsistent or the playback fails in the video stream, the video removal measures are executed in real time to ensure the health of the platform content and the user experience. In view of this, in order to improve the efficiency of detecting the content played by the cloud application, this embodiment proposes a video detection processing method to automatically detect the video stream, thereby effectively improving the detection efficiency.

[0019] Specifically, in the process of performing video detection using the video detection system, the video detection parameters sent by the resource management platform are received, and the video stream is obtained from the cloud application according to the video detection parameters. In an optional implementation manner provided in this embodiment, the video stream is obtained from the cloud application according to the video detection parameters in the following manner: Sending the video detection parameters to a cloud client; The video detection parameters are forwarded to the cloud platform through the cloud client, the cloud platform sends the video detection parameters to the cloud application, the cloud application searches for the video stream corresponding to the video detection parameters in the video list, and sends the video stream to the cloud platform, and the cloud platform forwards the video stream to the cloud client; Receive the video stream sent by the cloud client.

[0020] Specifically, the cloud application CP uploads the video detection parameters in batches to the video detection system. The video detection system initiates a video stream acquisition request to the cloud HD client in a circular parameter-carrying manner. The cloud HD client transparently transmits the video detection parameters to the cloud HD platform through end-cloud communication. The cloud HD platform starts the relevant cloud application according to the received video detection parameters. The cloud application searches for the corresponding video stream according to the received video detection parameters and transmits it back to the cloud HD platform. The cloud HD platform captures the video stream and sends the video stream to the cloud HD client through end-cloud communication. The cloud HD client receives the video stream and sends it to the video detection system. Correspondingly, the video detection system receives the video stream sent by the cloud HD client.

[0021] In addition, in order to improve detection efficiency and avoid repeated detection, after receiving the video detection parameters sent by the resource management platform, the video detection parameters can also be pre-detected to determine whether the video detection parameters are detected parameters. In an optional implementation provided in this embodiment, the video detection parameters are pre-detected in the following manner: Query whether the detected list contains the video detection parameter or the video stream corresponding to the video detection parameter; If not, executing the operation of obtaining the video stream from the cloud application according to the video detection parameters; If so, no action is required.

[0022] Specifically, after receiving the video detection parameters sent by the resource management platform, query whether the detected list contains the video detection parameters or the video stream corresponding to the video detection parameters. If not, execute the operation of obtaining the video stream from the cloud application according to the video detection parameters. If so, no processing is performed.

[0023] In addition, when the video detection parameters include at least two parameters, the video detection parameters may be screened to obtain untested parameters, and the video stream may be acquired based on the untested parameters.

[0024] For example, Figure 2 As shown in the interactive diagram, the cloud application CP first uploads video detection parameters in batches. After the video detection system receives the video detection parameters, it deduplicates the video detection parameters and then starts the HD client based on the video detection parameters that have not been detected. The cloud HD client transparently transmits the video detection parameters to the cloud HD platform through end-cloud communication. The cloud HD platform starts the cloud application according to the received video detection parameters. The cloud application searches for the video stream corresponding to the video detection parameters according to the received video detection parameters and transmits it back to the cloud HD platform. The cloud HD platform captures the video stream and sends the video stream to the cloud HD client through end-cloud communication. The cloud HD client receives the video stream and sends it to the video detection system. Correspondingly, the video detection system receives the video stream sent by the cloud HD client and detects whether the video stream is abnormal. If the video stream is abnormal, the detection result is returned to the cloud application CP so that the cloud application CP removes the video stream.

[0025] Step S104, performing text recognition on each video frame included in the video stream to obtain text data, and inputting each video frame into a convolutional neural network to perform feature extraction to obtain image features.

[0026] After receiving the video detection parameters sent by the resource management platform and obtaining the video stream from the cloud application according to the video detection parameters, in this step, text recognition is performed on each video frame contained in the video stream to obtain text data, and each video frame is input into the convolutional neural network for feature extraction to obtain image features. The text data may be subtitles in the video frame. The convolutional neural network may be a ShuffleNetV2 network model. The video frame may be a video frame obtained by randomly extracting the video stream, or a video frame obtained by extracting according to a certain time interval, which is not limited in this embodiment.

[0027] In the specific process of text recognition, Tesseract image and text recognition technology can be used to obtain text data in the video stream. In an optional implementation provided in this embodiment, text recognition is performed on each video frame contained in the video stream to obtain text data in the following manner: Identifying character regions in each of the video frames; Detecting character contours contained in the character area, and splitting the character area based on the character contours to obtain character blocks containing the character contours; A text line is determined based on the character outline and the character block, and text recognition is performed according to the character spacing of each character in the text line to obtain the text data.

[0028] Specifically, in the process of text recognition using Tesseract image and text recognition technology, the character area of ​​the current image is first identified, and the character contours are detected and analyzed to form corresponding block areas. Then, the corresponding text lines are obtained based on the character contours and the obtained block areas, and the required words are identified based on the spacing between the characters.

[0029] For example, Figure 3 As shown in the text recognition flow chart shown, in the process of text recognition on video frames, the image data is firstly identified for text areas to obtain text wheels or corresponding block areas, and then the corresponding text lines are obtained according to the text contours and block areas. Based on the obtained text lines, the corresponding words are obtained according to the character intervals. Each word is analyzed and passed to the adaptive classifier to obtain the corresponding character recognition results. The recognized characters are analyzed for sensitive information to determine whether there is any non-standard text in the video content.

[0030] In the specific process of image feature extraction, due to error analysis and interference from the image background, errors often occur in the extracted image features. For example, in special scenes such as bodybuilding and talent shows, the difference between image features and sensitive features is small, which often leads to misjudgment, making the generalization of the video detection system weak. Therefore, this application optimizes the ShuffleNetV2 network model and performs feature extraction based on the optimized ShuffleNetV2 network model. In an optional implementation provided in this embodiment, each video frame is input into a convolutional neural network for feature extraction to obtain image features in the following manner: Inputting each video frame into the pooling convolution layer of the convolutional neural network for feature extraction to obtain initial image features; Performing global average pooling on the initial image features to obtain dimensionality reduction features, and performing activation processing on the dimensionality reduction features based on an activation function to obtain activation features; The initial image feature and the activation feature are spliced ​​to obtain a spliced ​​image feature, and the spliced ​​image feature is reorganized into image channels to obtain the image feature.

[0031] Furthermore, in an optional implementation manner provided by this embodiment, each video frame is input into the pooling convolution layer of the convolutional neural network for feature extraction to obtain initial image features in the following manner: Inputting each video frame into a first pooling convolution layer for feature extraction to obtain a first initial image feature, and inputting each video frame into a second pooling convolution layer for feature extraction to obtain a second initial image feature; The first initial image feature and the second initial image feature are spliced ​​to obtain the initial image feature.

[0032] Among them, the pooling convolution layer may include a pooling layer and a convolution layer. Specifically, in the process of inputting each video frame into the first pooling convolution layer for feature extraction to obtain the first initial image feature, it includes: inputting each video frame into the first pooling layer for dimensionality reduction processing to obtain the first dimensionality reduction feature; inputting the first dimensionality reduction feature into the first convolution layer for feature extraction to obtain the first initial image feature. In addition, inputting each video frame into the second pooling convolution layer for feature extraction to obtain the second initial image feature includes: inputting each video frame into the second convolution layer for feature extraction to obtain the intermediate image feature; inputting the intermediate image feature into the second pooling layer for dimensionality reduction processing to obtain the intermediate dimensionality reduction feature; inputting the intermediate dimensionality reduction feature into the third convolution layer for feature extraction to obtain the second initial image feature.

[0033] For example, as shown in the convolutional neural network architecture diagram shown in Figure 4, in the process of extracting image features, on the one hand, each video frame is input into the first pooling layer on the left for dimensionality reduction processing to obtain the first dimensionality reduction feature, and then the first dimensionality reduction feature is input into the first convolution layer for feature extraction to obtain the first initial image feature. On the other hand, each video frame is input into the second convolution layer on the right for feature extraction to obtain the intermediate image feature, and then the intermediate image feature is input into the second pooling layer for dimensionality reduction processing to obtain the intermediate dimensionality reduction feature, and the obtained intermediate dimensionality reduction feature is input into the third convolution layer for feature extraction to obtain the second initial image feature; the first initial image feature and the second initial image feature are subjected to feature splicing processing to obtain the initial image feature; the initial image feature is subjected to global average pooling to obtain the dimensionality reduction feature, the dimensionality reduction feature is activated based on the activation function to obtain the activation feature, the initial image feature and the activation feature are subjected to feature splicing processing to obtain the spliced ​​image feature, and the spliced ​​image feature is subjected to image channel reorganization to obtain the image feature.

[0034] It should be noted that before image feature extraction, the video frame may be subjected to region cropping processing to extract key areas in the image frame, such as the portrait area, so as to extract image features from the key areas and obtain image features with higher accuracy.

[0035] Step S106, performing anomaly detection on the text data to obtain a text detection result, and calculating feature similarity based on the reference image feature and the image feature.

[0036] After performing text recognition on each video frame contained in the video stream to obtain text data and inputting each video frame into the convolutional neural network for feature extraction to obtain image features, in this step, anomaly detection is performed on the text data to obtain text detection results, and feature similarity is calculated based on the baseline image features and image features.

[0037] In the specific implementation process, in order to improve the confidence of the text detection result, an adaptive classifier can be used to perform anomaly detection on the text data to obtain the text detection result. In an optional implementation provided by this embodiment, the following method is used to perform anomaly detection on the text data: Inputting the text data into a character recognizer to perform abnormal character recognition; If an abnormal character is identified, determining the text detection result is a detection abnormality; If no abnormal characters are identified, it is determined that the text detection result is normal.

[0038] Optionally, the character recognizer may be a classifier, or may also be a machine learning model, for identifying the category of characters (such as normal text, sensitive words, etc.).

[0039] In addition, based on the abnormal character recognition based on the character recognizer, the recognized characters or words can be further analyzed for sensitive information. The purpose of this step is to detect whether the video content contains non-standard text. Sensitive information analysis usually involves one or more rule sets to determine whether the text meets preset standards or regulatory requirements. For example, it may include filtering out profanity, prohibiting words that promote illegal activities, etc. Ultimately, the character recognizer will output a conclusion indicating whether there is non-standard text in the video content. If so, specific sensitive words or sentences may also be marked and the corresponding contextual information provided.

[0040] In the specific implementation process, the feature similarity between the extracted image features and the reference image features is calculated. Optionally, the Euclidean distance or cosine similarity between the reference image features and the image features can be calculated.

[0041] Step S108, determining the detection result of the video stream according to the text detection result, the feature similarity and the similarity threshold, and returning the detection result to the resource management platform.

[0042] After performing anomaly detection on the text data to obtain the text detection result and calculating the feature similarity based on the reference image features and the image features, in this step, the detection result of the video stream is determined based on the text detection result, the feature similarity and the similarity threshold, and the detection result is returned to the resource management platform.

[0043] In an optional implementation manner provided by this embodiment, the detection result of the video stream is determined based on the text detection result, the feature similarity and the similarity threshold, including: if the text detection result is a detection abnormality or if the feature similarity is less than the similarity threshold, determining that the detection result is a detection failure; if the text detection result is a detection normal, and if the feature similarity is greater than or equal to the similarity threshold, determining that the detection result is a detection pass.

[0044] Specifically, the detection result may include a text detection result and an image detection result. If the text detection result obtained by performing anomaly detection on the text data is a detection anomaly, then the detection result is determined to be a detection failure. Alternatively, if the feature similarity is less than a similarity threshold, indicating that the image detection result is a detection anomaly, then the detection result is determined to be a detection failure. If the text detection result is a normal detection, and if the feature similarity is greater than or equal to the similarity threshold, then the detection result is determined to be a detection pass.

[0045] In addition, when the above-mentioned detection result is that the detection fails, on the one hand, the detection result can be recorded, and on the other hand, abnormal information in the detection result, that is, abnormal text and / or abnormal image features, can be extracted, and abnormal information reminders are generated for the abnormal information and sent to the resource management platform, so that the resource management platform can remove the video stream according to the abnormal reminder information; When the detection result is that the detection is passed, the video stream or the video detection parameter corresponding to the video stream may be stored in a detected list to serve as a basis for avoiding repeated detection in the future.

[0046] The following Figure 5Taking the detection flow chart of the displayed video detection system as an example, the process of video detection processing by the video detection system is generally described: first, the video detection system receives the video detection parameters sent by the resource management platform, and obtains the video stream from the cloud application according to the video detection parameters. On the one hand, text recognition is performed on all video frames to obtain text data. In the process of text recognition, the text area is first detected to obtain the area contour and the character contour, and then the area contour and the character contour are merged to obtain the block area, and then the text line and the corresponding word, that is, the text data, are obtained from the block area, and the text data is classified to obtain the recognition result of each word in the text data; on the other hand, the video frame is randomly extracted as the image data, and the image enhancement is performed on the image data to obtain the portrait area in the image, and the image features of the portrait area are extracted. The feature similarity of the baseline image feature and the image feature is calculated, and the image detection result is determined based on the feature similarity and the similarity threshold; if the text detection result or the image detection result is a detection anomaly, it indicates that there is sensitive information in the video stream, the abnormal information in the detection result is extracted, and the abnormal information reminder is generated and recorded for the abnormal information, so that the resource management platform can remove the video stream according to the detection result.

[0047] To summarize, the video detection processing method provided in this embodiment first receives the video detection parameters sent by the resource management platform, obtains the video stream from the cloud application according to the video detection parameters, performs text recognition on each video frame contained in the video stream to obtain text data, and inputs each video frame into the convolutional neural network for feature extraction to obtain image features, performs anomaly detection on the text data to obtain text detection results, and calculates feature similarity based on the baseline image features and the image features, determines the detection result of the video stream according to the text detection result, feature similarity and similarity threshold, and returns the detection result to the resource management platform, and removes the video stream from the shelves when the detection result is that the detection fails, thereby improving the efficiency of anomaly detection of videos played in cloud applications, ensuring the accuracy and smoothness of video playback in cloud applications, and improving user experience.

[0048] The following combination Figure 6 , taking the application of the video detection processing method provided by this embodiment in the cloud application playing video detection scenario as an example, the video detection processing method provided by this embodiment is further described. Figure 6 The video detection processing method applied to the cloud application playback video detection scenario specifically includes steps S602 to S622.

[0049] Step S602: receiving video detection parameters sent by the resource management platform, and obtaining a video stream from a cloud application according to the video detection parameters.

[0050] Prior to this, the video detection parameters are sent to the cloud client; the video detection parameters are forwarded to the cloud platform through the cloud client, the cloud platform sends the video detection parameters to the cloud application, the cloud application searches for the video stream corresponding to the video detection parameters in the video list, and sends the video stream to the cloud platform, the cloud platform forwards the video stream to the cloud client; and the video stream sent by the cloud client is received.

[0051] Step S604, identifying the character area in each video frame, and detecting the character outline contained in the character area.

[0052] Step S606, splitting the character area based on the character outline to obtain character blocks containing the character outline.

[0053] Step S608, determining the text line based on the character outline and the character block, performing text recognition according to the character spacing of each character in the text line, and obtaining text data.

[0054] Step S610: Input each video frame into a convolutional neural network for feature extraction to obtain image features.

[0055] Step S612, performing anomaly detection on the text data to obtain a text detection result, and calculating feature similarity based on the reference image features and the image features.

[0056] Step S614, determining the detection result of the video stream according to the text detection result, feature similarity and similarity threshold, and returning the detection result to the resource management platform.

[0057] Figure 7 A schematic diagram of a video detection and processing device provided by an embodiment of the present invention is shown in FIG. Figure 7 As shown, the device comprises: The video stream acquisition module 702 is configured to receive the video detection parameters sent by the resource management platform, and acquire the video stream from the cloud application according to the video detection parameters; The feature extraction module 704 is configured to perform text recognition on each video frame included in the video stream to obtain text data, and input each video frame into a convolutional neural network to perform feature extraction to obtain image features; A similarity calculation module 706 is configured to perform anomaly detection on the text data to obtain a text detection result, and calculate feature similarity based on a reference image feature and the image feature; The detection result determination module 708 is configured to determine the detection result of the video stream according to the text detection result, the feature similarity and the similarity threshold, and return the detection result to the resource management platform.

[0058] The video detection processing device provided in this embodiment first receives the video detection parameters sent by the resource management platform by running the video stream acquisition module 702, obtains the video stream from the cloud application according to the video detection parameters, then runs the feature extraction module 704, performs text recognition on each video frame contained in the video stream to obtain text data, and inputs each video frame into the convolutional neural network for feature extraction to obtain image features, then runs the similarity calculation module 706, performs anomaly detection on the text data to obtain a text detection result, and calculates feature similarity based on the reference image feature and the image feature, and finally runs the detection result determination module 708 to determine the detection result of the video stream according to the text detection result, the feature similarity and the similarity threshold, and returns the detection result to the resource management platform, and removes the video stream from the shelves when the detection result is that the detection fails, thereby improving the efficiency of anomaly detection of video playback in cloud applications, ensuring the accuracy and smoothness of video playback in cloud applications, and improving user experience.

[0059] The video detection processing device provided in an embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0060] Furthermore, an embodiment of the present specification also provides a video detection processing device, Figure 8 A schematic diagram of the structure of a video detection and processing device provided in an embodiment of this specification is shown in FIG. Figure 8 As shown, the device includes: a memory 801, a processor 802, a bus 803 and a communication interface 804. The memory 801, the processor 802 and the communication interface 804 communicate through the bus 803, and the communication interface 804 may include an input and output interface, which includes but is not limited to a keyboard, a mouse, a display, a microphone, a loudspeaker, etc.

[0061] Figure 8 In the embodiment, the memory 801 stores computer executable instructions that can be run on the processor 802, and when the computer executable instructions are executed by the processor 802, the following process is implemented: Receiving video detection parameters sent by the resource management platform, and obtaining a video stream from a cloud application according to the video detection parameters; Performing text recognition on each video frame included in the video stream to obtain text data, and inputting each video frame into a convolutional neural network to perform feature extraction to obtain image features; Performing anomaly detection on the text data to obtain a text detection result, and calculating feature similarity based on a reference image feature and the image feature; The detection result of the video stream is determined according to the text detection result, the feature similarity and the similarity threshold, and the detection result is returned to the resource management platform.

[0062] The video detection processing device provided in this embodiment, through the cooperation of the memory 801, the processor 802, the bus 803 and the communication interface 804, first receives the video detection parameters sent by the resource management platform, obtains the video stream from the cloud application according to the video detection parameters, performs text recognition on each video frame contained in the video stream to obtain text data, and inputs each video frame into the convolutional neural network for feature extraction to obtain image features, performs anomaly detection on the text data to obtain text detection results, and calculates feature similarity based on the reference image features and the image features, determines the detection result of the video stream according to the text detection result, the feature similarity and the similarity threshold, and returns the detection result to the resource management platform, and removes the video stream from the shelves when the detection result is that the detection fails, thereby improving the efficiency of anomaly detection of videos played in cloud applications, ensuring the accuracy and smoothness of video playback in cloud applications, and improving user experience.

[0063] The video detection processing device provided in an embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0064] Furthermore, another embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the following process is implemented: Receiving video detection parameters sent by the resource management platform, and obtaining a video stream from a cloud application according to the video detection parameters; Performing text recognition on each video frame included in the video stream to obtain text data, and inputting each video frame into a convolutional neural network to perform feature extraction to obtain image features; Performing anomaly detection on the text data to obtain a text detection result, and calculating feature similarity based on a reference image feature and the image feature; The detection result of the video stream is determined according to the text detection result, the feature similarity and the similarity threshold, and the detection result is returned to the resource management platform.

[0065] The computer-readable storage medium provided in this embodiment first receives the video detection parameters sent by the resource management platform, obtains the video stream from the cloud application according to the video detection parameters, performs text recognition on each video frame contained in the video stream to obtain text data, and inputs each video frame into a convolutional neural network for feature extraction to obtain image features, performs anomaly detection on the text data to obtain a text detection result, and calculates feature similarity based on the reference image features and the image features, determines the detection result of the video stream according to the text detection result, the feature similarity and the similarity threshold, and returns the detection result to the resource management platform, and removes the video stream from the shelves when the detection result is that the detection fails, thereby improving the efficiency of anomaly detection of videos played in cloud applications, ensuring the accuracy and smoothness of video playback in cloud applications, and improving user experience.

[0066] The computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0067] The computer-readable storage medium provided in an embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0068] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0070] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0071] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0072] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0073] Memory may include non-permanent storage in a computer-readable storage medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable storage medium.

[0074] Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0075] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0076] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0077] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A video detection processing method, characterized in that: include: Receiving video detection parameters sent by the resource management platform, and obtaining a video stream from a cloud application according to the video detection parameters; Performing text recognition on each video frame included in the video stream to obtain text data, and inputting each video frame into a convolutional neural network to perform feature extraction to obtain image features; Performing anomaly detection on the text data to obtain a text detection result, and calculating feature similarity based on a reference image feature and the image feature; The detection result of the video stream is determined according to the text detection result, the feature similarity and the similarity threshold, and the detection result is returned to the resource management platform.

2. The video detection processing method according to claim 1, wherein obtaining a video stream from a cloud application according to the video detection parameter comprises: Sending the video detection parameters to a cloud client; The video detection parameters are forwarded to the cloud platform through the cloud client, the cloud platform sends the video detection parameters to the cloud application, the cloud application searches for the video stream corresponding to the video detection parameters in the video list, and sends the video stream to the cloud platform, and the cloud platform forwards the video stream to the cloud client; Receive the video stream sent by the cloud client.

3. The video detection processing method according to claim 1, after the operation of receiving the video detection parameters sent by the resource management platform is executed, and before the operation of obtaining the video stream from the cloud application according to the video detection parameters is executed, it also includes: Query whether the detected list contains the video detection parameter or the video stream corresponding to the video detection parameter; If not, execute the operation of obtaining the video stream from the cloud application according to the video detection parameters.

4. The video detection processing method according to claim 1, wherein the step of performing text recognition on each video frame included in the video stream to obtain text data comprises: Identifying character regions in each of the video frames; Detecting character contours contained in the character area, and splitting the character area based on the character contours to obtain character blocks containing the character contours; A text line is determined based on the character outline and the character block, and text recognition is performed according to the character spacing of each character in the text line to obtain the text data.

5. The video detection processing method according to claim 1, wherein the step of inputting each video frame into a convolutional neural network for feature extraction to obtain image features comprises: Inputting each video frame into the pooling convolution layer of the convolutional neural network for feature extraction to obtain initial image features; Performing global average pooling on the initial image features to obtain dimensionality reduction features, and performing activation processing on the dimensionality reduction features based on an activation function to obtain activation features; The initial image feature and the activation feature are spliced ​​to obtain a spliced ​​image feature, and the spliced ​​image feature is reorganized into image channels to obtain the image feature.

6. The video detection processing method according to claim 1, wherein the performing anomaly detection on the text data to obtain a text detection result comprises: Inputting the text data into a character recognizer to perform abnormal character recognition; If an abnormal character is identified, the text detection result is determined to be a detection abnormality.

7. The video detection processing method according to claim 1, wherein determining the detection result of the video stream according to the text detection result, the feature similarity and the similarity threshold comprises: If the text detection result is a detection anomaly or if the feature similarity is greater than or equal to the similarity threshold, it is determined that the detection result is a detection failure.

8. A video detection processing device, characterized in that: include: A video stream acquisition module is configured to receive video detection parameters sent by the resource management platform, and acquire a video stream from a cloud application according to the video detection parameters; A feature extraction module is configured to perform text recognition on each video frame included in the video stream to obtain text data, and input each video frame into a convolutional neural network to perform feature extraction to obtain image features; a similarity calculation module configured to perform anomaly detection on the text data to obtain a text detection result, and calculate feature similarity based on a reference image feature and the image feature; The detection result determination module is configured to determine the detection result of the video stream according to the text detection result, the feature similarity and the similarity threshold, and return the detection result to the resource management platform.

9. A video detection and processing device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer executable instructions, and when the computer executable instructions are executed on the processor, the steps of the method described in any one of claims 1 to 7 can be implemented.

10. A computer-readable storage medium having computer-executable instructions stored therein, characterized in that: When the computer executable instructions are executed by a processor, the steps of the method described in any one of claims 1 to 7 can be implemented.

Images