Identification and detection method and system for episode submission violation video ai

Abstract

The invention relates to the technical field of video identification, in particular to an episode submission violation video ai identification and detection method and system, and the method comprises the steps: determining the key frame validity degree of each video key frame through the characteristics that the features of effective key frames between adjacent frames are consistent, and the feature points of the effective key frames are densely distributed; effective key frames are screened out according to the key frame effectiveness degree; furthermore, in order to enhance the behavior region with higher information content in the effective key frame, determining each behavior characteristic region according to behavior characteristic points divided according to the characteristics that the contrast ratio of the characteristic points corresponding to the behavior actions is higher and the distribution of the characteristic points corresponding to the behavior actions is more concentrated; and adaptive filtering is carried out based on the overall behavior features and feature point distribution in the behavior feature region, so that adaptive image enhancement is carried out on important behavior information in each effective key frame, and the accuracy of identification and detection of the episode submission violation video ai according to the obtained enhanced key frame is higher.

Description

Technical Field

[0001] The present invention relates to the field of video recognition technology, and in particular to an AI recognition and detection method and system for illegal videos submitted for review. Background Art

[0002] With the continuous increase in the production and distribution of TV series, audiences can easily obtain rich film and television content through online platforms, meeting diverse cultural needs. However, the illegal content in some TV series may pose a threat to the growth of young people, so identifying and handling these illegal TV series has become a key task in content supervision; considering that the TV series library is usually large, and the traditional manual review method has low supervision efficiency and may miss the review. Therefore, AI is usually used to automatically identify and detect illegal videos to improve review efficiency and accuracy.

[0003] Existing technologies usually extract key frames from the videos submitted for review based on the inter-frame difference method to detect violations. However, when the exposure is too high or too low, it will cause image details to be lost or color to be distorted, so that the video frames that are not key frames affected by the exposure are identified as key frames, that is, invalid key frames. These invalid key frames cannot provide effective context analysis, and the existence of invalid key frames will introduce noise in the process of key frame behavior analysis and feature extraction, resulting in the information of valid key frames being covered up or misunderstood, thereby affecting the accuracy of behavior analysis and reducing the accuracy of AI recognition and detection of illegal videos submitted for review. Summary of the invention

[0004] The present application provides an AI recognition and detection method and system for illegal videos in TV series submitted for review. The key frame effectiveness of each video key frame is determined by taking advantage of the fact that the features of the effective key frames are consistent between adjacent frames and the feature points of the effective key frames are densely distributed, thereby screening out the effective key frames according to the key frame effectiveness. Furthermore, in order to enhance the behavior areas with higher information content in the effective key frames, the behavior feature points corresponding to the behavior actions are further divided according to the characteristics that the contrast of the feature points is high and the feature points corresponding to the behavior actions are more concentrated, thereby determining each behavior feature area. Adaptive filtering is performed based on the overall behavior features and the distribution of feature points in the behavior feature areas, thereby adaptively enhancing the image of important behavior information in each effective key frame, thereby solving the problem that the information of the effective key frames is concealed or misunderstood, thereby affecting the accuracy of the behavior analysis, and making the accuracy of AI recognition and detection of illegal videos in TV series submitted for review higher.

[0005] The first aspect of the present application provides an AI recognition and detection method for illegal videos submitted for review, including:

[0006] Extract all the video key frames from the video submitted for review;

[0007] In the video submitted for review, the initial feature points in each video frame are obtained based on the feature extraction method; the inter-frame stability of each video key frame is determined based on the similarity of the initial feature point distribution between each video key frame and the adjacent video frame; the key frame validity of each video key frame is determined based on the inter-frame stability and the density of the initial feature points in each video key frame; and the valid key frames are screened out based on the key frame validity;

[0008] In each valid key frame, according to the concentrated distribution of feature points and contrast in the local area where each initial feature point is located, a behavior evaluation index of each initial feature point in each valid key frame is determined; according to the behavior evaluation index, all behavior feature points in each valid key frame are screened out; according to the concentrated distribution of behavior feature points in each valid key frame, a behavior feature area in each valid key frame is divided;

[0009] According to the overall size of the behavior evaluation index of each behavior feature point in the behavior feature area and the density of the behavior feature points, the modified bilateral filtering spatial weight of each behavior feature area is determined; after filtering the behavior feature area according to the modified bilateral filtering spatial weight, the enhanced key frame corresponding to each valid key frame is obtained; according to the enhanced key frame, AI recognition and detection of illegal videos submitted for review are performed.

[0010] Furthermore, the process of acquiring the inter-frame stability includes:

[0011] In the video submitted for review, the average value between the initial number of feature points of each video key frame and the initial number of feature points of the corresponding previous video frame is used as the first reference number value; the first matching number of feature points between each video key frame and the corresponding previous video frame is obtained by the FLANN feature matching algorithm; the negative correlation mapping value of the difference between the first matching number of feature points and the first reference number value is used as the first stability level of each video key frame;

[0012] After replacing the previous video frame in the process of obtaining the first stability with the next video frame, determining the second stability between each video key frame and the next video frame; taking the minimum value between the first stability and the second stability as the stability of the feature point of each video key frame;

[0013] The difference between the grayscale value of each pixel in each video key frame and the grayscale value of the pixel at the same pixel position in the corresponding previous video frame is used as the first local grayscale difference of each pixel in each video key frame; the mean of the first local grayscale differences of all pixels in each video frame is negatively correlated and mapped to determine the first overall grayscale stability between each video key frame and the previous video frame;

[0014] After replacing the previous video frame in the process of obtaining the first overall grayscale stability with the next video frame, determining the second overall grayscale stability between each video key frame and the next video frame; taking the minimum value between the first overall grayscale stability and the second overall grayscale stability as the grayscale feature stability degree of each video key frame;

[0015] The inter-frame stability of each video key frame is determined according to the product of the stability of the feature point and the stability of the grayscale feature.

[0016] Furthermore, the process of obtaining the key frame validity level includes:

[0017] The product of the number of initial feature points in each video key frame and the inter-frame stability is normalized to determine the key frame effectiveness of each video key frame.

[0018] Furthermore, the process of acquiring the valid key frame includes:

[0019] The video key frames whose key frame validity degree is greater than a preset validity threshold are regarded as valid key frames.

[0020] Furthermore, the process of obtaining the behavior evaluation index includes:

[0021] Perform canny edge detection on each valid key frame, and obtain all edge connected domains corresponding to each valid key frame based on the image edges in the obtained edge image;

[0022] The average number of initial feature points of all edge connected domains in each valid key frame is used as the reference number average; the ratio between the number of initial feature points in the edge connected domain where each initial feature point in each valid key frame is located and the reference number average is used as the local feature point density of each initial feature point in each valid key frame;

[0023] In each valid key frame, the product of the contrast of each initial feature point and the density of the local feature points is normalized to determine the behavior evaluation index of each initial feature point in each valid key frame.

[0024] Furthermore, the process of acquiring the behavior feature points includes:

[0025] In each valid key frame, the initial feature point whose behavior evaluation index is greater than the preset behavior threshold is taken as the behavior feature point.

[0026] Furthermore, the process of acquiring the behavior feature area includes:

[0027] In each valid key frame, an edge connected domain whose number of behavior feature points is greater than a preset number threshold is used as the behavior feature region of each valid key frame.

[0028] Furthermore, the process of acquiring the modified bilateral filtering spatial weight includes:

[0029] In each valid key frame, the average of the behavior evaluation indexes of all behavior feature points in each behavior feature area is used as the regional behavior feature value of each behavior feature area;

[0030] In each valid key frame, the average of the number of initial feature points in all behavior feature areas is used as the corresponding reference feature point density; the ratio between the number of initial feature points in each behavior feature area and the reference feature point density is used as the feature point density weight of each behavior feature area in each valid key frame;

[0031] The product of the regional behavior feature value, the feature point density weight and the prior bilateral filtering spatial weight is used as the modified bilateral filtering spatial weight of each behavior feature region in each valid key frame.

[0032] Furthermore, the acquisition process of the enhanced key frame includes:

[0033] In each valid key frame, bilateral filtering is performed on the corresponding behavior feature region by using the modified bilateral filtering spatial weight to obtain a filter enhancement region of each behavior feature region;

[0034] After each behavior feature region in each valid key frame is replaced with the corresponding filter enhancement region, an enhanced key frame corresponding to each valid key frame is obtained.

[0035] In the second aspect, the present application provides an AI recognition and detection system for illegal videos submitted for review of dramas, the system comprising:

[0036] The data acquisition module is used to extract all the video key frames in the video submitted for review;

[0037] The effective key frame acquisition module is used to obtain the initial feature points in each video frame in the video submitted for review based on the feature extraction method; determine the inter-frame stability of each video key frame according to the similarity of the initial feature point distribution between each video key frame and the adjacent video frame; determine the key frame effectiveness of each video key frame according to the inter-frame stability and the initial feature point density in each video key frame; and screen out the effective key frames according to the key frame effectiveness;

[0038] The behavior feature area acquisition module is used to determine the behavior evaluation index of each initial feature point in each valid key frame according to the concentrated distribution of feature points in the local area where each initial feature point is located and the contrast; filter out all behavior feature points in each valid key frame according to the behavior evaluation index; and divide the behavior feature area in each valid key frame according to the concentrated distribution of behavior feature points in each valid key frame;

[0039] The AI ​​recognition and detection module is used to determine the modified bilateral filtering spatial weight of each behavior feature area according to the overall size of the behavior evaluation index of each behavior feature point in the behavior feature area and the density of the behavior feature points; after filtering the behavior feature area according to the modified bilateral filtering spatial weight, an enhanced key frame corresponding to each valid key frame is obtained; and AI recognition and detection of illegal videos submitted for review are performed according to the enhanced key frame.

[0040] In a third aspect, the present application provides a computer device, comprising a memory and a processor. The memory is used to store computer program code, and the processor is used to call and run the computer program code from the memory to execute the method of the first aspect or any embodiment of the first aspect of the present application.

[0041] In a fourth aspect, the present application provides a computer program product, comprising a computer program code, which, when executed, performs the method of the first aspect of the present application or any embodiment of the first aspect.

[0042] In a fifth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed, the method of the first aspect or any embodiment of the first aspect of the present application is performed.

[0043] This application has the following beneficial effects:

[0044] First, the inter-frame stability of each video key frame is determined based on the similarity of the initial feature point distribution between each video key frame and the adjacent video frames, since the feature points of the valid key frames are kept consistent between adjacent frames, while the invalid key frames generated by exposure have lower stability of feature points due to the static characteristics of the scene or the lack of obvious changes.

[0045] Valid key frames usually represent important scene changes and details in the video, so they usually contain more feature points; while the scene changes corresponding to the invalid key frames generated by exposure are not obvious, and the corresponding feature points are relatively sparsely distributed; therefore, the present application measures the effectiveness of the key frames based on the inter-frame stability combined with the initial feature point density in the video key frames, so as to screen out valid key frames according to the effectiveness of the key frames.

[0046] In a valid key frame, it usually contains a behavior area representing behavior information and a background area representing background information. The behavior area usually contains behavior information that is more important for violation detection, while the background information is less important for violation detection. Therefore, in order to improve the accuracy of subsequent violation detection, it is necessary to perform image enhancement on the behavior area.

[0047] In the valid keyframes, the area corresponding to the action behavior often contains more motion details, so it usually corresponds to a higher density of feature points, and dynamic actions will highlight edges and details, so the feature points in the action behavior area usually have a higher contrast; therefore, firstly, according to the concentrated distribution and contrast of the feature points in the local area where each initial feature point is located, the behavior evaluation index of each initial feature point in each valid keyframe is determined; thereby, according to the behavior evaluation index, more accurate behavior feature points that represent the behavior information are screened out; and based on the behavior feature points, the behavior feature area corresponding to the action behavior area is screened out.

[0048] Furthermore, it is necessary to perform local image enhancement on each behavior feature area to determine a more accurate enhanced key frame. Considering that bilateral filtering can effectively remove noise and can retain edge information to a certain extent, each behavior feature area is filtered and enhanced by bilateral filtering. For each behavior feature area, the density of feature points reflects the complexity and detail richness of the corresponding behavior feature area. High-density areas usually contain more information, so higher weights are required during filtering. Therefore, the modified bilateral filtering spatial weight of each behavior feature area is further determined based on the overall size of the behavior evaluation index of each behavior feature point in the behavior feature area and the density of the behavior feature points, and the behavior feature areas in the effective key frame are filtered and enhanced according to the obtained modified bilateral filtering spatial weight, so as to obtain an enhanced key frame with better filtering effect and clearer image information representation, so that the accuracy of AI recognition and detection of illegal videos submitted for review in the drama series based on the enhanced key frame is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A flow chart of an AI recognition and detection method for illegal videos in a TV series submitted for review provided by an embodiment of the present invention;

[0051] Figure 2 A structural diagram of an AI recognition and detection system for illegal videos submitted for review of TV series provided by an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the AI ​​identification and detection method and system for illegal videos submitted for review in accordance with the present invention, in combination with the accompanying drawings and preferred embodiments, as well as its specific implementation, structure, features and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment, and the specific features, structures or characteristics in one or more embodiments may be combined in any suitable form. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0054] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0055] The following is a detailed description of the specific scheme of the AI ​​recognition and detection method and system for illegal videos in TV series submitted for review provided by the present invention with reference to the accompanying drawings.

[0056] This application embodiment provides a method for AI recognition and detection of illegal videos in drama series submitted for review. Figure 1 , which shows a flow chart of an AI recognition and detection method for illegal videos submitted for review provided by an embodiment of the present invention, the method comprising:

[0057] Step S101: extract all video key frames in the video submitted for review.

[0058] First, obtain the video of the drama submitted for review, and extract the video key frames in the video of the drama submitted for review by using the frame difference method; it should be noted that the frame difference method is a technical means well known to those skilled in the art, and in other specific implementations of the embodiments of the present invention, other key frame extraction methods can also be used to extract video key frames, such as using a trained convolutional neural network to extract video key frames, which will not be further limited or elaborated here.

[0059] Step S102: In the video of the drama submitted for review, the initial feature points in each video frame are obtained based on the feature extraction method; the inter-frame stability of each video key frame is determined based on the similarity of the initial feature point distribution between each video key frame and the adjacent video frames; the key frame effectiveness of each video key frame is determined based on the inter-frame stability and the density of initial feature points in each video key frame; and the effective key frames are screened out based on the key frame effectiveness.

[0060] Since the feature points between adjacent frames are kept consistent through valid key frames, and the invalid key frames generated by exposure have low stability of feature points due to the static or unobvious characteristics of the scene, the inter-frame stability of each video key frame is first determined based on the similarity of the initial feature point distribution between each video key frame and the adjacent video frames; the larger the corresponding inter-frame stability, the more likely the video key frame is to be a valid key frame.

[0061] Valid key frames usually represent important scene changes and details in the video, so they usually contain more feature points; while the scene changes corresponding to the invalid key frames generated by exposure are not obvious, and the corresponding feature points are relatively sparsely distributed; therefore, the present application measures the effectiveness of the key frames based on the inter-frame stability combined with the initial feature point density in the video key frames, so as to screen out valid key frames according to the effectiveness of the key frames.

[0062] In a specific implementation of the embodiment of the present invention, the initial feature points in each video frame are extracted by using a SIFT feature extraction algorithm, and the SIFT feature extraction algorithm is a technical means well known to those skilled in the art and will not be further limited or elaborated herein.

[0063] Preferably, the process of acquiring inter-frame stability includes:

[0064] In the video of the drama submitted for review, the average value between the initial number of feature points of each video key frame and the initial number of feature points of the corresponding previous video frame is used as the first reference number value; the first matching number of feature points between each video key frame and the corresponding previous video frame is obtained by the FLANN feature matching algorithm; the negative correlation mapping value of the difference between the first matching number of feature points and the first reference number value is used as the first stability of each video key frame; after replacing the previous video frame with the next video frame in the process of obtaining the first stability, the second stability between each video key frame and the next video frame is determined; the minimum value between the first stability and the second stability is used as the feature point stability of each video key frame. The specific process of obtaining the second degree of stability includes: in the video of the drama submitted for review, taking the average of the number of initial feature points of each video key frame and the number of initial feature points of the corresponding subsequent video frame as the second reference quantity value; obtaining the second matching feature point number between each video key frame and the corresponding subsequent video frame through the FLANN feature matching algorithm; and taking the negative correlation mapping value of the difference between the second matching feature point number and the second reference quantity value as the second degree of stability of each video key frame.

[0065] First, considering that the feature points of the effective key frame remain consistent in multiple adjacent frames, the feature point matching degree is calculated for the previous video frame and the next video frame respectively based on each video key frame, so as to more accurately reflect the stability of the feature points of the video key frame in the two adjacent directions, and reduce the influence of accidental factors; and the mean number of feature points of the two adjacent video frames is introduced in the calculation process, so that the calculated stability of the feature points is more robust; so that the greater the stability of the obtained feature points, the more the corresponding video key frame conforms to the characteristics of the effective key frame. It should be noted that the FLANN feature matching algorithm is a technical means well known to those skilled in the art, and will not be further defined or elaborated here.

[0066] In a specific implementation of the embodiment of the present invention, the process of obtaining the stability of the feature points is expressed by the formula: k =Min((exp(-|n k,1 -n′ k |),exp(-|n k,2 -n″ k |)) Among them, T k is the stability of the feature points of the kth video key frame, n k，1 is the first reference quantity value of the kth video key frame; n′ k is the number of first matching feature points corresponding to the kth video key frame; n k，2 The second reference quantity value of the kth video key frame; n″k is the number of second matching feature points corresponding to the kth video key frame; exp(-|n k,1 -n′ k |) is the first stability level corresponding to the kth video key frame; exp(-|n k,2 -n″ k |) is the second stability level corresponding to the kth video key frame; Min() is the minimum value selection function, Min(exp(-|n k,1 -n′ k |), exp(-|n k,2 -n″ k |)) That is, in exp(-|n k,1 -n′ k |) and exp(-|n k,2 -n″ k |) selects a minimum value between them; exp() is an exponential function with a natural constant as the base. The implementer may adopt other negative correlation mapping methods according to the specific implementation environment, such as 1-Norm(), 1-tanh() and the inverse, where tanh() is a hyperbolic tangent function, which will not be further described here. It should be noted that for the video key frame of the first video frame or the last video frame of the video submitted for review, since there is no previous video frame or next video frame, the embodiment of the present invention directly analyzes the corresponding video key frame as a valid key frame, that is, the calculation of the inter-frame stability and the key frame effectiveness is no longer performed, making the embodiment of the present invention more complete.

[0067] The difference between the grayscale value of each pixel in each video key frame and the grayscale value of the pixel at the same pixel position in the corresponding previous video frame is taken as the first local grayscale difference of each pixel in each video key frame; the mean of the first local grayscale difference of all pixels in each video frame is negatively correlated to determine the first overall grayscale stability between each video key frame and the previous video frame. For invalid key frames caused by exposure effects, there is usually a large difference in the overall grayscale between the adjacent images. Therefore, the greater the change in the grayscale value at each pixel position, the more likely the corresponding video key frame is to be an invalid key frame; conversely, the smaller the change in the grayscale value at each pixel position, the more likely the corresponding video key frame is to be a valid key frame. Similarly, in order to weaken the influence of accidental factors, after further replacing the previous video frame with the next video frame in the process of obtaining the first overall grayscale stability, the second overall grayscale stability between each video key frame and the next video frame is determined. Specifically, the difference between the grayscale value of each pixel in each video key frame and the grayscale value of the pixel at the same pixel position in the corresponding next video frame is used as the second local grayscale difference of each pixel in each video key frame; the mean of the second local grayscale difference of all pixels in each video frame is negatively correlated to determine the second overall grayscale stability of each video key frame. Thus, the minimum value between the first overall grayscale stability and the second overall grayscale stability is further used as the grayscale feature stability of each video key frame. On the basis of each video key frame, the overall grayscale stability is calculated for the previous video frame and the next video frame respectively, so as to more accurately reflect the grayscale feature stability of the video key frame in the two adjacent directions, weaken the influence of accidental factors, so that the greater the grayscale feature stability, the more likely the corresponding video key frame is to belong to a valid key frame.

[0068] In a specific implementation of the embodiment of the present invention, the process of obtaining the grayscale feature stability is expressed by the formula: Among them, H k is the grayscale feature stability of the kth video key frame; N is the number of pixels in the video key frame; g k，i is the gray value of the i-th pixel in the k-th video key frame; g′ k，i is the gray value of the pixel point at the same pixel point position in the previous video frame of the i-th pixel point in the k-th video key frame; g″ k，i is the gray value of the pixel at the same pixel position in the next video frame of the i-th pixel in the k-th video key frame; |g k，i -g′ k，i | is the first local grayscale difference of the i-th pixel in the k-th video key frame; |gk，i -g″ k，i | is the second local grayscale difference of the i-th pixel in the k-th video key frame; is the first overall grayscale stability between the kth video key frame and the previous video frame; is the second overall grayscale stability between the kth video key frame and the next video frame; Min() is the minimum value selection function, That is and A minimum value is selected between; exp() is an exponential function with a natural constant as the base, and the implementer may adopt other negative correlation mapping methods according to the specific implementation environment, such as 1-Norm(), 1-tanh() and the inverse, where tanh() is a hyperbolic tangent function, which will not be further described here.

[0069] The possibility of valid key frames is further characterized in the stable dimension by combining the stability of feature points and the stability of grayscale features. The inter-frame stability of each video key frame is determined according to the product of the stability of feature points and the stability of grayscale features. The greater the inter-frame stability, the greater the possibility that the corresponding video key frame is a valid key frame.

[0070] In a specific implementation of the embodiment of the present invention, the process of obtaining the inter-frame stability is expressed by the formula: k =T k ×H k ; Among them, W k is the inter-frame stability of the kth video key frame; H k is the grayscale feature stability of the kth video key frame; T k is the stability of the feature points of the kth video key frame. It should be noted that, in addition to the product, the implementer can also calculate the inter-frame stability by other methods according to the correlation, such as the mean value, the normalized value of the sum value, etc., which will not be further elaborated here.

[0071] Valid key frames usually represent important scene changes and details in the video, so they usually contain more feature points; while the scene changes corresponding to the invalid key frames generated by exposure are not obvious, and the corresponding feature points are relatively sparsely distributed; therefore, the present application measures the effectiveness of the key frames based on the inter-frame stability combined with the initial feature point density in the video key frames, so as to screen out valid key frames according to the effectiveness of the key frames.

[0072] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the key frame validity level includes:

[0073] Since the sizes of video key frames are all the same, the more the corresponding initial feature points are, the greater the density of feature points is, and the greater the possibility of the corresponding valid key frame is; further combined with the characteristic that the greater the inter-frame stability is, the greater the possibility of the corresponding video key frame being a valid key frame is, the product of the number of initial feature points in each video key frame and the inter-frame stability is normalized to determine the key frame effectiveness of each video key frame; so that the greater the key frame effectiveness is, the more likely the corresponding video key frame is to be a valid video frame. In a specific implementation of an embodiment of the present invention, the key frame effectiveness acquisition process is expressed by the formula: E k =Norm(W k ×M k ), where E k is the key frame validity of the kth video key frame; W k is the inter-frame stability of the kth video key frame; M k is the number of initial feature points of the kth video key frame; Norm() is a linear normalization function, and the implementer can adjust the normalization method according to the specific implementation environment. It should be noted that in addition to the product, the implementer can also calculate the effectiveness of the key frame by other methods according to the correlation, such as the normalized value of the mean and the sum, etc., which will not be further elaborated here.

[0074] Preferably, in some possible implementations of the embodiments of the present invention, the process of acquiring a valid key frame includes:

[0075] Since the greater the key frame validity, the more likely the corresponding video key frame is a valid video frame, the video key frame whose key frame validity is greater than a preset validity threshold is further taken as a valid key frame. In a specific implementation of the embodiment of the present invention, the preset validity threshold is set to 0.5, which can be adjusted according to the specific implementation environment.

[0076] Step S103: In each valid key frame, determine the behavior evaluation index of each initial feature point in each valid key frame according to the concentrated distribution of feature points in the local area where each initial feature point is located and the contrast; filter out all behavior feature points in each valid key frame according to the behavior evaluation index; divide the behavior feature area in each valid key frame according to the concentrated distribution of behavior feature points in each valid key frame.

[0077] In a valid key frame, it usually contains a behavior area representing behavior information and a background area representing background information. The behavior area usually contains behavior information that is more important for violation detection, while the background information is less important for violation detection. Therefore, in order to improve the accuracy of subsequent violation detection, it is necessary to perform image enhancement on the behavior area.

[0078] In the valid keyframes, the area corresponding to the action behavior often contains more motion details, so it usually corresponds to a higher density of feature points, and dynamic actions will highlight edges and details, so the feature points in the action behavior area usually have a higher contrast; therefore, firstly, according to the concentrated distribution and contrast of the feature points in the local area where each initial feature point is located, the behavior evaluation index of each initial feature point in each valid keyframe is determined; thereby, according to the behavior evaluation index, more accurate behavior feature points that represent the behavior information are screened out; and based on the behavior feature points, the behavior feature area corresponding to the action behavior area is screened out.

[0079] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the behavior evaluation index includes:

[0080] Canny edge detection is performed on each valid key frame, and all edge connected domains corresponding to each valid key frame are obtained based on the image edges in the obtained edge image. In the valid key frame, the area corresponding to the action behavior feature usually has a large texture difference with the background area. Therefore, after dividing the valid key frame by the edges obtained by edge detection, the area corresponding to the more accurate action behavior feature can be screened out based on the obtained edge connected domains.

[0081] The mean of the number of initial feature points of all edge connected domains in each valid key frame is taken as the mean of the reference number; the ratio between the number of initial feature points in the edge connected domain where each initial feature point in each valid key frame is located and the mean of the reference number is taken as the local feature point density of each initial feature point in each valid key frame. For the edge connected domain where each initial feature point is located, the denser the distribution of the corresponding initial feature points is, the more it conforms to the characteristics of the action behavior area, that is, the corresponding edge connected domain is more likely to belong to the behavior feature area, and the initial feature points therein are more likely to belong to the behavior feature points; and the mean of the number of initial feature points of all edge connected domains in each valid key frame is calculated, that is, the mean of the reference number, and the reference number mean can be used as a reference to make the feature point density of each edge connected domain more accurately measured. Therefore, the larger the local feature point density obtained by the ratio between the number of initial feature points in the edge connected domain and the mean of the reference number is, the greater the possibility that the corresponding initial feature point belongs to the behavior feature point is. It should be noted that since the density of feature points has been taken into account in the screening process of valid key frames, the number of feature points in valid key frames is not 0, so the corresponding reference number mean cannot be 0, and there is no case where the denominator is 0.

[0082] In each valid key frame, the product of the contrast of each initial feature point and the density of local feature points is normalized to determine the behavior evaluation index of each initial feature point in each valid key frame. In addition, since the contrast of the initial feature points in the action behavior area is relatively higher, the contrast of the initial feature points is combined with the local feature point density to jointly characterize the behavior evaluation index, so that when the behavior evaluation index is larger, the corresponding initial feature point is more likely to represent the action behavior characteristics, that is, it is more likely to be a behavior feature point.

[0083] In a specific implementation of the embodiment of the present invention, the process of obtaining the behavior evaluation index is expressed by the formula: Z r，c is the behavior evaluation index of the cth initial feature point in the rth valid key frame; U r，c is the number of initial feature points in the edge connected domain where the cth initial feature point in the rth valid key frame is located; is the mean of the number of initial feature points of all edge connected domains in the rth valid key frame, that is, the mean of the reference number; is the local feature point density of the cth initial feature point in the rth valid key frame; D r，c is the contrast of the cth initial feature point in the rth valid key frame; Norm() is a linear normalization function, and the implementer can adjust the normalization method according to the specific implementation environment, and normalization can limit the value range of the behavior evaluation index to 0 to 1, which is convenient for subsequent screening. It should be noted that the process of obtaining the contrast of pixels in an image or video frame is a technical means well known to those skilled in the art, and will not be further elaborated here.

[0084] Preferably, the process of acquiring the behavior feature points includes:

[0085] Since the larger the behavior evaluation index is, the more likely the corresponding initial feature point is to represent the action behavior characteristics, that is, the more likely it is to be a behavior feature point; therefore, in each valid key frame, the initial feature point whose behavior evaluation index is greater than the preset behavior threshold is used as the behavior feature point. In a specific implementation of an embodiment of the present invention, the preset behavior threshold is set to 0.5, which can be adjusted according to the specific implementation environment.

[0086] Preferably, the process of acquiring the behavior feature area includes:

[0087] The more the number of behavior feature points in the edge connected domain is, the more likely the corresponding area is to belong to the area corresponding to the action behavior feature. Therefore, in each valid key frame, the edge connected domain with the number of behavior feature points greater than the preset number threshold is used as the behavior feature area of ​​each valid key frame. In a specific implementation of the embodiment of the present invention, the preset number threshold is set to 5, which can be adjusted according to the specific implementation environment.

[0088] Step S104: Determine the modified bilateral filtering spatial weight of each behavior feature area according to the overall size of the behavior evaluation index of each behavior feature point in the behavior feature area and the density of the behavior feature points; filter the behavior feature area according to the modified bilateral filtering spatial weight to obtain the enhanced key frame corresponding to each valid key frame; perform AI recognition detection of illegal videos submitted for review based on the enhanced key frame.

[0089] Furthermore, it is necessary to perform local image enhancement on each behavior feature area to determine a more accurate enhanced key frame. Considering that bilateral filtering can effectively remove noise and can retain edge information to a certain extent, each behavior feature area is filtered and enhanced through bilateral filtering. For each behavior feature area, the density of feature points reflects the complexity and detail richness of the corresponding behavior feature area. High-density areas usually contain more information, so higher weights are required during filtering. Therefore, according to the overall size of the behavior evaluation index of each behavior feature point in the behavior feature area and the density of the behavior feature points, the modified bilateral filtering spatial weight of each behavior feature area is determined, and the behavior feature areas in the effective key frame are filtered and enhanced according to the obtained modified bilateral filtering spatial weight, so as to obtain an enhanced key frame with better filtering effect and clearer image information representation, so that the accuracy of AI recognition and detection of illegal videos submitted for review in the drama series based on the enhanced key frame is higher.

[0090] Preferably, the process of obtaining the modified bilateral filtering spatial weight includes:

[0091] In each valid keyframe, the mean of the behavior evaluation indexes of all behavior feature points in each behavior feature area is used as the regional behavior feature value of each behavior feature area. For each behavior feature area, the larger the corresponding regional behavior feature value, the higher the behavior evaluation index of the behavior feature point in the behavior feature area, the higher the complexity and detail richness of the corresponding local area, and the more action behavior information it contains, so a larger filtering weight is needed for enhancement to highlight its action behavior information.

[0092] In each valid key frame, the mean of the number of initial feature points in all behavioral feature regions is used as the corresponding reference feature point density; the ratio between the number of initial feature points in each behavioral feature region and the reference feature point density is used as the feature point density weight of each behavioral feature region in each valid key frame. First, the more initial feature points there are in a behavioral feature region, the denser the feature distribution in that region is, and a higher spatial weight is required for image enhancement; and the mean of the number of initial feature points in all behavioral feature regions is used as a restriction, so that the feature point density weight represented by the dense feature distribution is more robust; and the larger the feature point density weight, the greater the spatial weight requirement for the corresponding bilateral filter.

[0093] Therefore, the product of the regional behavior feature value, the feature point density weight and the prior bilateral filtering spatial weight is further used as the modified bilateral filtering spatial weight of each behavior feature region in each valid key frame; the larger the regional behavior feature value and the feature point density weight, the larger the corresponding modified bilateral filtering spatial weight, and the more image details are retained after enhancement based on the spatial weight, that is, the clearer the action behavior information; the prior bilateral filtering spatial weight is used as the weighted basis, and in a specific implementation method of an embodiment of the present invention, the value is 3, which can be adjusted according to the specific implementation environment.

[0094] In a specific implementation of the embodiment of the present invention, the process of obtaining the modified bilateral filtering spatial weight is expressed by the formula: Among them, S′ r，x is the modified bilateral filtering spatial weight corresponding to the xth behavior feature area in the rth valid key frame; S is the prior bilateral filtering spatial weight; V r，x is the number of behavior feature points corresponding to the xth behavior feature area in the rth valid key frame; Z r，x，v is the behavior evaluation index of the uth behavior feature point in the xth behavior feature area in the rth valid key frame; r，x is the number of initial feature points in the xth behavior feature area in the rth valid keyframe; is the reference feature point density of the r-th valid key frame, that is, the average number of initial feature points of all behavior feature areas in the r-th valid key frame, and its value is not 0; is the regional behavior feature value of the xth behavior feature region in the rth valid key frame; is the feature point density weight of the xth behavior feature area in the rth valid key frame.

[0095] Preferably, in some possible implementations of the embodiments of the present invention, the process of acquiring the enhanced key frame includes:

[0096] In each valid key frame, the corresponding behavior feature area is bilaterally filtered by modifying the bilateral filtering spatial weights to obtain the filter enhancement area of ​​each behavior feature area; after replacing each behavior feature area in each valid key frame with the corresponding filter enhancement area, the enhanced key frame corresponding to each valid key frame is obtained. That is, in each valid key frame, different degrees of bilateral filtering enhancement are performed with the modified bilateral filtering spatial weights corresponding to each behavior feature area, so as to obtain the enhanced key frame after the action behavior information is enhanced, so that the subsequent AI recognition detection of illegal videos submitted for review is more accurate.

[0097] Finally, AI recognition and detection of illegal videos in the drama submitted for review are performed based on the enhanced key frames; in a specific implementation method of an embodiment of the present invention, each enhanced key frame is input into a trained convolutional neural network, and the enhanced key frames with violations are output, and the loss function selects the cross entropy loss function, which will not be further limited or elaborated here.

[0098] To summarize, an AI recognition and detection method for illegal videos in drama series submitted for review determines the key frame effectiveness of each video key frame through the characteristics that the features of the effective key frames are consistent between adjacent frames and the feature points of the effective key frames are densely distributed, thereby screening out the effective key frames according to the key frame effectiveness; and further, in order to enhance the behavior areas with higher information content in the effective key frames, the behavior feature points corresponding to the behavior actions are further divided according to the characteristics of high contrast and concentrated distribution of feature points corresponding to the behavior actions, to determine each behavior feature area; and adaptive filtering is performed based on the overall behavior characteristics and feature point distribution in the behavior feature area, so as to adaptively enhance the important behavior information in each effective key frame, thereby solving the problem that the information of the effective key frames is concealed or misunderstood, thereby affecting the accuracy of the behavior analysis, so that the accuracy of AI recognition and detection of illegal videos in drama series submitted for review is higher.

[0099] This application also provides an AI recognition and detection system for illegal videos submitted for review. Figure 2 , which shows a structural diagram of an AI recognition and detection system for illegal videos of dramas submitted for review provided by an embodiment of the present invention. The system includes: a data acquisition module 201, a valid key frame acquisition module 202, a behavior feature area acquisition module 203 and an AI recognition and detection module 204.

[0100] Data acquisition module 201, used to extract all video key frames in the video submitted for review;

[0101] The effective key frame acquisition module 202 is used to obtain the initial feature points in each video frame in the video submitted for review based on the feature extraction method; determine the inter-frame stability of each video key frame according to the similarity of the initial feature point distribution between each video key frame and the adjacent video frame; determine the key frame effectiveness of each video key frame according to the inter-frame stability and the density of the initial feature points in each video key frame; and select the effective key frames according to the key frame effectiveness;

[0102] The behavior feature region acquisition module 203 is used to determine the behavior evaluation index of each initial feature point in each valid key frame according to the distribution of the feature points in the local area where each initial feature point is located and the contrast in each valid key frame; select all the behavior feature points in each valid key frame according to the behavior evaluation index; and divide the behavior feature region in each valid key frame according to the distribution of the behavior feature points in each valid key frame;

[0103] The AI ​​recognition and detection module 204 is used to determine the modified bilateral filtering spatial weight of each behavior feature area according to the overall size of the behavior evaluation index of each behavior feature point in the behavior feature area and the density of the behavior feature points; after filtering the behavior feature area according to the modified bilateral filtering spatial weight, an enhanced key frame corresponding to each valid key frame is obtained; and AI recognition and detection of illegal videos submitted for review are performed based on the enhanced key frames.

[0104] It should be noted that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the above embodiment provides an AI recognition and detection system for illegal videos of dramas submitted for review and an embodiment of an AI recognition and detection method for illegal videos of dramas submitted for review, which belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0105] The present application also provides a computer device. Figure 3 , which shows a schematic diagram of the structure of a computer device provided by an embodiment of the present invention, wherein the computer device includes a memory 301, a processor 302, and a computer program 303 stored in the memory 301 and running on the processor 302, wherein when the processor 302 executes the computer program 303, the computer device can execute any of the aforementioned AI identification and detection methods for illegal videos submitted for review.

[0106] An embodiment of the present application also provides a computer program product. When the computer program product runs on a computer device, the computer device can execute any of the aforementioned AI identification and detection methods for illegal videos in TV series submitted for review.

[0107] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code runs on a computer device, the computer device can execute any of the aforementioned AI identification and detection methods for illegal videos submitted for review.

[0108] In the embodiments provided in the present application, it should be understood that the provided computer device, computer program product and computer-readable storage medium are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the methods provided above and will not be repeated here.

[0109] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0110] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A method for AI recognition and detection of illegal videos submitted for review, characterized in that: The method comprises: Extract all the video key frames from the video submitted for review; In the video submitted for review, the initial feature points in each video frame are obtained based on the feature extraction method; the inter-frame stability of each video key frame is determined based on the similarity of the initial feature point distribution between each video key frame and the adjacent video frame; the key frame validity of each video key frame is determined based on the inter-frame stability and the density of the initial feature points in each video key frame; and the valid key frames are screened out based on the key frame validity; In each valid key frame, according to the concentrated distribution of feature points and contrast in the local area where each initial feature point is located, a behavior evaluation index of each initial feature point in each valid key frame is determined; according to the behavior evaluation index, all behavior feature points in each valid key frame are screened out; according to the concentrated distribution of behavior feature points in each valid key frame, a behavior feature area in each valid key frame is divided; According to the overall size of the behavior evaluation index of each behavior feature point in the behavior feature area and the density of the behavior feature points, the modified bilateral filtering spatial weight of each behavior feature area is determined; after filtering the behavior feature area according to the modified bilateral filtering spatial weight, the enhanced key frame corresponding to each valid key frame is obtained; according to the enhanced key frame, AI recognition and detection of illegal videos submitted for review are performed.

2. According to claim 1, the method for AI identification and detection of illegal videos submitted for review is characterized in that: The process of obtaining the inter-frame stability includes: In the video submitted for review, the average value between the initial number of feature points of each video key frame and the initial number of feature points of the corresponding previous video frame is used as the first reference number value; the first matching number of feature points between each video key frame and the corresponding previous video frame is obtained by the FLANN feature matching algorithm; the negative correlation mapping value of the difference between the first matching number of feature points and the first reference number value is used as the first stability level of each video key frame; After replacing the previous video frame in the process of obtaining the first stability with the next video frame, determining the second stability between each video key frame and the next video frame; taking the minimum value between the first stability and the second stability as the stability of the feature point of each video key frame; The difference between the grayscale value of each pixel in each video key frame and the grayscale value of the pixel at the same pixel position in the corresponding previous video frame is used as the first local grayscale difference of each pixel in each video key frame; the mean of the first local grayscale differences of all pixels in each video frame is negatively correlated and mapped to determine the first overall grayscale stability between each video key frame and the previous video frame; After replacing the previous video frame in the process of obtaining the first overall grayscale stability with the next video frame, determining the second overall grayscale stability between each video key frame and the next video frame; taking the minimum value between the first overall grayscale stability and the second overall grayscale stability as the grayscale feature stability degree of each video key frame; The inter-frame stability of each video key frame is determined according to the product of the stability of the feature point and the stability of the grayscale feature.

3. According to claim 1, the method for detecting illegal videos submitted for review by AI is characterized in that: The process of obtaining the key frame validity degree includes: The product of the number of initial feature points in each video key frame and the inter-frame stability is normalized to determine the key frame effectiveness of each video key frame.

4. According to claim 1, the method for detecting illegal videos submitted for review by AI is characterized in that: The process of obtaining the valid key frame includes: The video key frames whose key frame validity degree is greater than a preset validity threshold are regarded as valid key frames.

5. According to claim 1, the method for AI identification and detection of illegal videos submitted for review is characterized in that: The process of obtaining the behavior evaluation index includes: Perform canny edge detection on each valid key frame, and obtain all edge connected domains corresponding to each valid key frame based on the image edges in the obtained edge image; The average number of initial feature points of all edge connected domains in each valid key frame is used as the reference number average; the ratio between the number of initial feature points in the edge connected domain where each initial feature point in each valid key frame is located and the reference number average is used as the local feature point density of each initial feature point in each valid key frame; In each valid key frame, the product of the contrast of each initial feature point and the density of the local feature points is normalized to determine the behavior evaluation index of each initial feature point in each valid key frame.

6. According to claim 1, the method for detecting illegal videos submitted for review by AI is characterized in that: The process of acquiring the behavior feature points includes: In each valid key frame, the initial feature point whose behavior evaluation index is greater than the preset behavior threshold is taken as the behavior feature point.

7. According to claim 5, the method for detecting illegal videos submitted for review by AI is characterized in that: The process of obtaining the behavior feature area includes: In each valid key frame, an edge connected region whose number of behavior feature points is greater than a preset number threshold is used as the behavior feature region of each valid key frame.

8. According to claim 1, the method for detecting illegal videos submitted for review by AI is characterized in that: The process of obtaining the modified bilateral filtering spatial weight includes: In each valid key frame, the average of the behavior evaluation indexes of all behavior feature points in each behavior feature area is used as the regional behavior feature value of each behavior feature area; In each valid key frame, the average of the number of initial feature points in all behavior feature areas is used as the corresponding reference feature point density; the ratio between the number of initial feature points in each behavior feature area and the reference feature point density is used as the feature point density weight of each behavior feature area in each valid key frame; The product of the regional behavior feature value, the feature point density weight and the prior bilateral filtering spatial weight is used as the modified bilateral filtering spatial weight of each behavior feature region in each valid key frame.

9. According to claim 1, the method for detecting illegal videos submitted for review by AI is characterized in that: The acquisition process of the enhanced key frame includes: In each valid key frame, bilateral filtering is performed on the corresponding behavior feature region by using the modified bilateral filtering spatial weight to obtain a filter enhancement region of each behavior feature region; After each behavior feature region in each valid key frame is replaced with the corresponding filter enhancement region, an enhanced key frame corresponding to each valid key frame is obtained.

10. An AI recognition and detection system for illegal videos submitted for review, characterized in that: The system comprises: The data acquisition module is used to extract all the video key frames in the video submitted for review; The effective key frame acquisition module is used to obtain the initial feature points in each video frame in the video submitted for review based on the feature extraction method; determine the inter-frame stability of each video key frame according to the similarity of the initial feature point distribution between each video key frame and the adjacent video frame; determine the key frame effectiveness of each video key frame according to the inter-frame stability and the initial feature point density in each video key frame; and screen out the effective key frames according to the key frame effectiveness; The behavior feature area acquisition module is used to determine the behavior evaluation index of each initial feature point in each valid key frame according to the concentrated distribution of feature points in the local area where each initial feature point is located and the contrast; filter out all behavior feature points in each valid key frame according to the behavior evaluation index; and divide the behavior feature area in each valid key frame according to the concentrated distribution of behavior feature points in each valid key frame; The AI ​​recognition and detection module is used to determine the modified bilateral filtering spatial weight of each behavior feature area according to the overall size of the behavior evaluation index of each behavior feature point in the behavior feature area and the density of the behavior feature points; after filtering the behavior feature area according to the modified bilateral filtering spatial weight, an enhanced key frame corresponding to each valid key frame is obtained; and AI recognition and detection of illegal videos submitted for review are performed according to the enhanced key frame.

Images