An improved pornographic video detection method and system based on key frames

Abstract

The application discloses an improved pornographic video detection method and system based on key frames, and adopts a pornographic video detection system to convert a video stream into a video frame set, adopts an image entropy extreme value method to obtain a candidate key frame set by using a key frame extraction module, and then adopts a local frame difference key frame extraction method to obtain a key video frame set from the key video frame images in the candidate key frame set; image preprocessing operations are performed on each key video frame extracted by using an image preprocessing module; local feature extraction is performed on the key video frame images after preprocessing by using a local sensitive feature detection module; global feature extraction is performed on the key video frame images after preprocessing by using a global feature extraction module; and the local features and the global features extracted are calculated by using a feature fusion module to obtain a final detection result through specific weighting, so as to determine whether the video stream is a pornographic video.

Description

Technical Field

[0001] This invention relates to the fields of network security technology, and more specifically, to an improved method and system for detecting pornographic videos based on keyframes. Background Technology

[0002] With the rapid development of the internet and social media, the internet has become an important platform for people to obtain and share information. However, this has also led to the spread of various harmful information online, among which the harm of pornography is becoming increasingly prominent. Therefore, it is imperative to regulate online pornography and accelerate the development of automatic detection and filtering technologies for pornographic videos.

[0003] In recent years, popular short videos and various live streams can be viewed as collections of multiple key video frames. By applying methods for detecting pornographic images to these key video frames or extracting information between frames, pornographic video streams can be effectively detected. Therefore, detecting pornographic videos can be seen as detecting collections of pornographic images. With the continuous breakthroughs in deep learning technology, researchers have conducted various studies in the field of pornographic content recognition, leading to a leap forward in image recognition technologies and a significant improvement in pornographic content detection technology. However, facing the rise of the short video and live streaming industries, current pornographic content recognition technology still struggles to meet the demands of detecting an explosive number of video streams. In the face of today's real-world detection tasks, research in pornographic video detection still faces many unresolved issues. Specifically, existing detection methods suffer from the following problems: 1) Too many key video frames lead to long detection times. Key video frames in a video refer to any image or shot that marks the beginning and end of a smooth transition. Most existing pornographic video detection methods determine the presence of pornographic content by detecting key video frames. However, with the widespread adoption of high-definition images and high-frame-rate video streams, the number of images contained in a video has increased several times, which has also greatly increased the number of key video frames that need to be detected, thus reducing the detection rate of pornographic videos. In the context of the rapid rise of the live streaming industry and increasingly stringent requirements for pornographic video detection speed, how to minimize the number of key video frames extracted without missing crucial information has become a difficult problem to solve in pornographic video detection. 2) Confused features lead to a high false positive rate. Existing methods have achieved high recognition rates in pornographic image detection tasks, but they are still prone to misclassification for normal images and videos with similar features to pornographic images. For example, images of bikinis on the beach, nude fighting, and infant breastfeeding may be detected as pornographic images. Images in these scenarios have similar features to pornographic images, namely a high proportion of skin color, making it very easy for trained detection models to misclassify such images. 3) Image distortion leads to low detection accuracy. Since most pornographic videos are filmed and recorded in low light conditions, the extracted key video frames are often blurry due to the light, and it is difficult to identify key pornographic information (such as skin and sensitive areas), which greatly affects the accuracy of pornographic image recognition models. Summary of the Invention

[0004] The purpose of this invention is to provide an improved method and system for detecting pornographic videos based on keyframes. Addressing the problem of long video detection times due to an excessive number of keyframes, the method employs a keyframe extraction approach combining image entropy extremum extraction and local frame differencing. This method discards redundant frames as much as possible while retaining keyframes and accelerating keyframe extraction. To fully utilize the pornographic features in the image, the method combines global and local feature-based pornographic image recognition methods to reduce model misjudgments of specific images. Furthermore, to avoid reduced model accuracy due to insufficient image lighting or unclear pornographic features, the method introduces a novel lighting enhancement model to enhance the image data. The system can be used to implement a pornographic video detection method based on keyframes and feature fusion.

[0005] This invention is achieved through the following technical solution: an improved method for detecting pornographic videos based on keyframes, comprising the following steps:

[0006] 1) After converting the video stream into a set of video frames, the keyframe extraction module is used to obtain a set of candidate keyframes by taking the extreme value of image entropy.

[0007] 2) The key video frame set is obtained by using the key frame extraction module to extract key video frame images from the candidate key frame set using the local frame difference key frame extraction method.

[0008] 3) The image preprocessing module performs image preprocessing operations on each extracted key video frame, including image data enhancement, light enhancement, and adaptive resizing.

[0009] 4) Local features are extracted from the preprocessed key video frame images using YOLOv5 technology through the local sensitive feature detection module;

[0010] 5) The global feature extraction module is used to extract global features from the preprocessed key video frame images using the MobileNetV3 neural network model;

[0011] 6) The feature fusion module is used to calculate the extracted local and global features by specific weighting to obtain the final detection result, thereby determining whether the key video frame contains pornographic information and whether the video stream is a pornographic video.

[0012] To further improve the keyframe-based improved pornographic video detection method described in this invention, the following settings are specifically adopted: the image entropy value is calculated using the following formula: ,in Represented as the first The probability density function of each frame is obtained by calculating its intensity histogram through normalization; the calculated image entropy value is then used. The images are mapped onto a coordinate axis with the key video frame sequence number as the x-axis. Finally, images with extreme entropy values ​​or significant differences in change within a segment are selected as key video frame images for that segment, forming a set of candidate key frames.

[0013] To further improve the keyframe-based improved pornographic video detection method described in this invention, the following settings are specifically adopted: Step 2) includes the following specific steps:

[0014] 2.1) Divide two adjacent key video frame images into a 26*26 grid;

[0015] 2.2) Local frame difference calculation is performed according to the corresponding grid positions before and after, and the absolute values ​​of the obtained frame difference values ​​are taken and summed to form a univariate value as the local frame difference value between adjacent frames;

[0016] 2.3) Select the local frame difference value with the largest value among all grids as the difference value of the adjacent frame, and determine whether the value exceeds a preset threshold. If it exceeds the threshold, proceed to step 2.4); otherwise, proceed to step 2.5.

[0017] 2.4) If the difference exceeds the limit, it is considered that the adjacent frames have a large change. The previous frame is saved to the key video frame set, and the current frame is used as the previous frame to perform local frame difference calculation with the next frame.

[0018] 2.5) If the set threshold is not exceeded, the two adjacent frames are considered to have little change. The previous frame is discarded, and the current frame and the next frame are used to continue the above local frame difference calculation until the last frame of the video stream is calculated and the final set of key video frames is obtained.

[0019] To further improve the keyframe-based improved pornographic video detection method described in this invention, the following settings are specifically adopted: Step 3) specifically includes:

[0020] Image enhancement: The Mosaic data augmentation method is used to randomly flip, crop, and arrange four images from the candidate keyframe set that are used as part of the training set. The four images that have undergone random operations are then stitched together to obtain a new image.

[0021] Light Enhancement: Introducing the SKNet mechanism into the data preprocessing process, a light enhancement method is proposed that relies solely on the specific image and does not depend on the training data. Specifically, the image is enhanced using a formula... To enhance the light intensity of an image and better preserve its colors, this invention preferably uses formulas to separately process the RGB channels of the image. Perform ray enhancement processing on the image, where x is the pixel coordinate of the input image, I(x) is the input image, LE((x);α) is the enhanced output of the original image, and the parameters are... [-1,1] is used to adjust the image and control the exposure effect;

[0022] Image adaptive resizing: The input image is standardized using LetterBox, which proportionally reduces the size of the input image and uniformly adjusts the original image size to 224×224.

[0023] The processing order of image enhancement, lighting enhancement, and image adaptive resizing is not important.

[0024] To further improve the keyframe-based improved pornographic video detection method described in this invention, the following settings are specifically adopted: Step 4) includes the following specific steps:

[0025] 4.1) Use YOLOv5’s Focus module to downsample the key video frame image obtained after step 3), that is, obtain a pixel value for every pixel in the input key video frame image to form a new channel, thereby expanding the feature channels of the original key video frame image to 4 times the original. Then, use convolution integral to obtain a new feature map from the newly obtained channel feature map.

[0026] 4.2) Use the CSP module in YOLOv5 to calculate features in the new feature map to avoid duplicate features;

[0027] 4.3) The output obtained in step 4.2) is fed into FPN (Feature Pyramid) and PAN (Path Aggregation Network) to extract local features of the image from multiple scales.

[0028] To further improve the keyframe-based improved pornographic video detection method described in this invention, the following settings are specifically adopted: Step 5) includes the following specific steps:

[0029] 5.1) Input the result from step 4) into the MobileNetV3 neural network model. Replace the traditional convolution calculation process with a combination of depthwise convolution and pointwise convolution. Perform convolution processing on each color channel. When performing convolution calculation on the output of depthwise convolution using pointwise convolution, calculate using the following formula: Where K is a number of size D K ×D K A convolution kernel of ×m, D KF represents the length or width of the input feature map (length and width are the same, i.e., the length and width of the input feature map are the same). m For the m-th channel of the input image, G m This is the m-th feature map output;

[0030] 5.2) After step 5.1), the feature map generated by each channel is globally pooled using a Squeeze-and-Excitation network structure. Then, a two-stage fully connected operation is performed. In the first fully connected operation, the number of nodes is 1 / 4 of the number of input channels, and then ReLU activation is applied. In the second fully connected operation, the number of nodes is the same as the number of input channels, and then Hard-sigmoid activation function is applied to obtain a weighted feature map.

[0031] The ReLU function is: ReLU = max(0, x); where x is the input data, and max() represents the maximum value function.

[0032] The Hard-sigmoid activation function is: x represents the input data, max() represents the maximum value function, and min() represents the minimum value function.

[0033] To further improve the keyframe-based improved pornographic video detection method described in this invention, the following settings are specifically adopted: the specific weighted calculation is performed using the following formula:

[0034] Where F represents the final fused feature, and w represents the weight corresponding to the global feature. As a global feature, Local features; where local features The calculation process is as follows:

[0035] , where F i is the predicted value for the i-th sensitive part, and N is the number of predicted sensitive part detection box categories.

[0036] The final fused feature F is fed into a sigmoid classifier to obtain a classification result that can determine whether key video frame images contain pornographic information. The sigmoid function is: .

[0037] An improved pornographic video detection system based on keyframes is disclosed, which implements an improved pornographic video detection method based on keyframes. The system includes a keyframe extraction module, an image preprocessing module, a local sensitive feature detection module, a global feature extraction module, and a feature fusion module.

[0038] The keyframe extraction module is used to extract key video frames from the video stream, thereby significantly improving the efficiency of video detection without affecting the system's detection accuracy.

[0039] The image preprocessing module performs image preprocessing operations, including image data enhancement, light enhancement, and adaptive sizing, on each extracted key video frame image.

[0040] The local sensitive feature detection module detects key sensitive parts in the preprocessed key video frame image based on YOLOv5 technology, forming local features;

[0041] The global feature extraction module uses the lightweight deep neural network model MobileNetV3 as the backbone. By removing redundant calculations through separable convolutional structures and lightening the model structure, it completes the global feature extraction of key video frame images. It can greatly accelerate the feature extraction calculation speed and improve the detection efficiency without reducing the model recognition accuracy.

[0042] The feature fusion module calculates the final detection result by weighting local and global features, thereby determining whether the key video frame contains pornographic information and whether the video stream is a pornographic video.

[0043] To further improve the keyframe-based improved pornographic video detection system described in this invention, the following settings are specifically adopted: the image data enhancement includes expanding the dataset and increasing the number of small targets in the image; the lighting enhancement is used to retain more pornographic target information; and the adaptive sizing is used to meet the model input requirements while retaining more image information.

[0044] To further improve the keyframe-based improved pornographic video detection system described in this invention, the following configuration is specifically adopted: the local sensitive feature detection module includes an Input module, a Backbone module, a Neck module, and a Prediction module. The Input module initializes anchor boxes and prediction boxes and performs grid division operations on the key video frame image. The Backbone module, as the neural network backbone of the local sensitive feature detection module, extracts feature information of key sensitive parts within the grid. The Neck module, composed of a Feature Pyramid (FPN) and a PAN, is mainly used to extract multi-scale information. The Prediction module generates three feature maps of different sizes to predict the category and anchor boxes of the key video frame image, ultimately obtaining the category and location of sensitive parts in the key video frame image, and merging and deduplicating overlapping anchor boxes in the key video frame image using non-maximum suppression.

[0045] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0046] To reduce redundant frames and improve model detection efficiency, this invention proposes a key video frame extraction method based on a combination of image entropy and local frame differencing. This method leverages image entropy's ability to effectively reflect the amount of information in an image, selecting local extrema on the image sequence number-image entropy coordinate axis as candidate key video frames for that video segment. Considering the characteristic that short videos and live video streams generally have relatively infrequent background changes, a frame extraction method based on local frame differencing is employed, further reducing the number of redundant frames. This method can improve the efficiency of key video frame extraction and reduce the number of redundant frames while extracting effective key video frames.

[0047] To reduce the false positive rate of the model for specific video images (such as bikini images on a beach), this invention proposes a method combining global and local features for key video frame detection. On one hand, this invention extracts sensitive local parts of the image and calculates a weighted average to obtain local features; on the other hand, it extracts global features from the entire image and finally fuses the two types of features for detection. This method reduces the model's false positive rate while also enhancing its robustness.

[0048] To address the issue of reduced model accuracy due to image distortion in low-light conditions, this invention employs a light-fitting curve method for image enhancement. Using a trained neural network model, a suitable light enhancement curve is applied to the input image. This enhances the image information in darker areas, ensuring no loss of pornographic information, while avoiding the loss of image features due to localized overexposure. Furthermore, this invention incorporates SKNet, adaptively adjusting convolutional kernel weights for images of different sizes. This enhances pornographic information lost due to underexposure while preserving as much existing image information as possible, thereby improving the accuracy of the pornographic video recognition model. Attached Figure Description

[0049] Figure 1 This is a flowchart of the pornographic video detection method based on key video frames and feature fusion as described in this invention.

[0050] Figure 2 This is a schematic diagram illustrating how the extracted local and global features are weighted and calculated to obtain the final detection result according to the present invention. Detailed Implementation

[0051] The present invention will be further described in detail below with reference to embodiments, but the implementation of the present invention is not limited thereto.

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] This invention aims to solve three problems in pornographic video detection: large video data volume leads to long detection latency, confusing features lead to high false alarm rate, and image distortion caused by video background, lighting and other factors leads to low detection accuracy.

[0054] Regarding the first question, the massive increase in video content and the long processing time of large amounts of video data have always been a difficult problem to solve. Therefore, filtering out similar redundant frames and retaining key video frames to reduce data processing time is crucial for handling large volumes of video data. The purpose of extracting key video frames is to reduce the number of subsequent video frames to be detected, filter redundant video frames, speed up the detection of the video stream, and achieve a near real-time pornographic video detection model. The key and challenge of extracting key video frames lies in how to remove duplicate redundant frames and how to extract the set of key video frames that best summarizes the video content.

[0055] Regarding the second question, pornographic image features are used to detect key video frames and determine whether the detected video is pornographic based on whether the key video frames are pornographic. The definition of a pornographic image is rather abstract; generally, an image containing sexual innuendo or exposed sensitive parts is considered pornographic. Furthermore, pornographic images exhibit variability in scenes and human poses, and often highly overlap with some normal everyday images, such as similar overall colors, backgrounds, and human poses. This presents a significant challenge to pornographic image detection systems. For global information, features such as scene or overall color can easily lead deep learning models to misclassify normal images. For local area information, factors such as shooting angle and object occlusion can also cause deep learning models to treat local areas in normal images as sensitive parts, resulting in misclassification.

[0056] Regarding the third issue, most video frames in pornographic videos are set in dimly lit backgrounds or scenes. Since pornographic image detection relies on pornographic feature information, the pornographic image detection module will have difficulty identifying the pornographic information in the image if the pornographic feature information is obscured by the lighting. This poses a significant challenge and difficulty to the detection performance of the pornographic image detection module.

[0057] Example 1:

[0058] This invention designs an improved pornographic video detection method based on keyframes. Addressing the problem of long detection times due to an excessive number of key video frames, it employs a key video frame extraction method combining image entropy extremum and local frame differencing. This method discards redundant frames as much as possible while retaining key video frames and accelerating key video frame extraction. To fully utilize the pornographic features in the image, this method combines global and local feature-based pornographic image recognition methods to reduce model misclassification of specific images. Furthermore, to avoid reduced model accuracy due to insufficient image lighting or unclear pornographic features, this method introduces a novel lighting enhancement model to enhance the image data, including the following steps:

[0059] 1) When the pornographic video detection system detects a video stream, it converts the video stream into a set of video frames and uses the keyframe extraction module to obtain a set of candidate keyframes by taking the extreme value of image entropy. This greatly improves the efficiency of video detection without affecting the system's detection accuracy.

[0060] 2) The key video frame set is obtained by using the key frame extraction module to extract key video frame images from the candidate key frame set using the local frame difference key frame extraction method.

[0061] 3) The image preprocessing module performs image preprocessing operations on each extracted key video frame, including image data augmentation (to expand the dataset and increase the number of small targets in the image), lighting enhancement (to retain more information about pornographic targets), and adaptive sizing (to meet the model input requirements while retaining more image information);

[0062] 4) The Local Sensitive Feature Detection (LSP) module uses YOLOv5 technology to extract local features from the preprocessed key video frame images. Specifically, the LSP module detects key sensitive areas in the image. Based on the YOLOv5 object detection system, the LSP module includes an Input module, a Backbone module, a Neck module, and a Prediction module. The Input module initializes anchor boxes and prediction boxes, and performs grid division on the key video frame images. The Backbone module, as the neural network backbone of the LSP module, extracts feature information of key sensitive areas within the grid. The Neck module, composed of a Feature Pyramid Network (FPN) and a Panel Array (PAN), primarily extracts multi-scale information. The Prediction module generates three feature maps of different sizes to predict the category and anchor boxes of the key video frame images, ultimately obtaining the category and location of sensitive areas in the key video frame images. Non-maximum suppression is then used to merge and remove duplicate anchor boxes in the key video frame images.

[0063] 5) The global feature extraction module uses the MobileNetV3 neural network model to extract global features from the preprocessed key video frame images. The global feature extraction module uses the lightweight deep neural network model MobileNetV3 as the backbone. It removes redundant calculations and lightens the model structure through separable convolutional structures to complete the global feature extraction of key video frame images. It can greatly speed up the feature extraction calculation speed and improve the detection efficiency without reducing the model recognition accuracy.

[0064] 6) The feature fusion module is used to calculate the extracted local and global features by specific weighting to obtain the final detection result, thereby determining whether the key video frame contains pornographic information and whether the video stream is a pornographic video.

[0065] There are two methods for processing the extracted global and local features and their classification results: judging each feature individually and taking their union, and fusing global and local features to obtain the final classification result. For the first method, if global features are used alone for classification, normal images may be falsely reported because they share similar scenes or backgrounds with pornographic images. Similarly, if local features are used alone for classification, normal images may be falsely reported because they contain local information similar to sensitive areas. In summary, the classification method using a fusion of global and local features can effectively reduce the false positive rate for normal images. In the feature fusion method proposed in this invention, global features are used... The expression represents the extracted global features of the image. For local features, since the extracted detection bounding boxes may contain multiple sensitive regions of different types, this invention selects only the single target classification value with the highest score (i.e., the highest probability) for multiple sensitive regions of the same type as the feature value of that type of sensitive region. For multiple types of sensitive regions, the average value is taken as the local feature information, as shown in the following formula:

[0066] .

[0067] Current mainstream methods for extracting frames from pornographic video streams are mostly based on a single approach, which suffers from slow extraction speed and the tendency to miss key pornographic frames, failing to guarantee that key video frames are extracted without losing them. To address these issues, this invention proposes a key video frame extraction method suitable for pornographic video detection tasks, which consists of two parts: a method for calculating the extreme value of video image information entropy and a method for calculating local frame differences.

[0068] Example 2:

[0069] This embodiment is a further optimization based on the above embodiments. The similarities with the aforementioned technical solutions will not be repeated here. Entropy is essentially the degree of inherent disorder within a system. A video frame with a local extremum of image entropy means that the frame is at a turning point in the change of image information in the local video stream; therefore, it can be regarded as a key video frame. Furthermore, to better implement the improved pornographic video detection method based on key frames described in this invention, the following setting is specifically adopted: For video frames in the video stream, the value of image entropy is calculated using the following formula: ,in Represented as the first The probability density function of each frame is obtained by calculating its intensity histogram through normalization; the calculated image entropy value is then used. The images are mapped onto a coordinate axis with the key video frame sequence number as the x-axis. Finally, images with extreme entropy values ​​or significant differences in change within a segment are selected as key video frame images for that segment, forming a set of candidate key frames.

[0070] Although the method (the method for calculating the extreme value of information entropy in video images) can extract the extreme value images of the turning point of information change in a continuous video sequence as key video frames, for high frame rate images, there are still a large number of key video frames with small changes between adjacent frames selected by this method. This greatly increases the time and performance overhead in subsequent image recognition tasks. Therefore, this invention proposes a secondary extraction method for key video frames of pornographic video streams, namely the local frame difference calculation method.

[0071] Example 3:

[0072] This embodiment is a further optimization based on any of the above embodiments. The similarities with the aforementioned technical solutions will not be repeated here. Since the application scenarios for pornographic video detection (i.e., pornographic films, candid scenes, pornographic live streams, etc.) mostly involve unchanged or minimally changing backgrounds, to avoid the inability to extract key video frames for extended periods due to minimal changes in the overall scene, this invention proposes a method for calculating local frame differences. This method calculates the absolute difference of adjacent frames locally, and finally selects the largest local absolute difference as the difference value between adjacent frames. Then, it determines whether to retain the frame as a key video frame in the video sequence by judging whether the largest local difference value exceeds a pre-set threshold. Furthermore, to better implement the improved pornographic video detection method based on key frames described in this invention, the following settings are specifically adopted: Step 2) includes the following specific steps:

[0073] 2.1) Divide two adjacent key video frame images into a 26*26 grid;

[0074] 2.2) Local frame difference calculation is performed according to the corresponding grid positions before and after, and the absolute values ​​of the obtained frame difference values ​​are taken and added together to form a univariate value as the local frame difference value between adjacent frames;

[0075] 2.3) Select the local frame difference value with the largest value among all grids as the difference value of the adjacent frame, and determine whether the value exceeds a preset threshold. If it exceeds the threshold, proceed to step 2.4); otherwise, proceed to step 2.5.

[0076] 2.4) If the difference exceeds the limit, it is considered that the adjacent frames have a large change. The previous frame is saved to the key video frame set, and the current frame is used as the previous frame to perform local frame difference calculation with the next frame.

[0077] 2.5) If the set threshold is not exceeded, the two adjacent frames are considered to have little change. The previous frame is discarded, and the current frame and the next frame are used to continue the above local frame difference calculation until the last frame of the video stream is calculated and the final set of key video frames is obtained.

[0078] Example 4:

[0079] This embodiment is a further optimization based on any of the above embodiments. The similarities with the aforementioned technical solutions will not be repeated here. Furthermore, to better implement the improved pornographic video detection method based on keyframes described in this invention, the following settings are specifically adopted: Step 3) specifically includes:

[0080] Image enhancement: The Mosaic data augmentation method is used to randomly flip, crop, and arrange four images from the candidate keyframe set that are used as part of the training set. The four images that have undergone random operations are then stitched together to obtain a new image.

[0081] Light Enhancement: Introducing the SKNet mechanism into the data preprocessing process, a light enhancement method is proposed that relies solely on the specific image and does not depend on the training data. Specifically, the image is enhanced using a formula... To enhance the light intensity of an image and better preserve its colors, this invention preferably uses formulas to separately process the RGB channels of the image. Perform ray enhancement processing on the image, where x is the pixel coordinate of the input image, I(x) is the input image, LE((x);α) is the enhanced output of the original image, and the parameters are... [-1,1] is used to adjust the image and control the exposure effect;

[0082] Image adaptive resizing: The input image is standardized using LetterBox, which proportionally reduces the size of the input image and uniformly adjusts the original image size to 224×224.

[0083] The processing order of image enhancement, lighting enhancement, and image adaptive resizing is not important.

[0084] To address the issue of lost pornographic features in key video frames due to low lighting conditions during pornographic video detection, this invention borrows the Zero-DCE model proposed by Chunle Guo et al., employing a method similar to image editing software that automatically maps and enhances different low-light regions using non-linear curves. Furthermore, to improve the model's lighting enhancement effect on images of different sizes and resolutions, this invention also incorporates the SKNet mechanism into the lighting enhancement model. This method is independent of training data; the curve's changes are self-enhancing and depend only on the image input. The curve formula is shown below:

[0085]

[0086] in The input image pixel coordinates, For the input image, The output after enhancing the original image, parameters This curve is used to adjust the curve and control the exposure effect. To better preserve image color, this invention uses this curve to represent the three RGB color channels of the image to be processed. Since a single adjustment often fails to achieve the desired effect, multiple enhancements are performed on the pornographic image dataset to be processed to improve the image enhancement effect. To calculate the optimal light enhancement fitting curve for each input key video frame... This invention employs a deep neural network to calculate illumination enhancement fitting curves for key input video frames. The network takes a weakly lit image as input and outputs enhanced fitting curves for the image's three illumination channels (RGB). Specifically, the three color channels of the input image are first fitted to the original curves, which are then fed into the illumination enhancement fitting curve neural network. Finally, the enhanced fitting curve is obtained. To improve the enhancement effect, the model performs eight iterations, ultimately outputting 3 × 8 = 24 parameter maps.

[0087] In summary, the illumination enhancement fitting curve neural network used in this invention consists of seven symmetrical convolutional layers. This model is simple to construct, requires fewer training parameters, and compared to other data-driven image enhancement methods, it is more efficient at enhancing large batches of data to be examined. It is suitable for rapid image enhancement of the key video frames extracted in this invention.

[0088] Example 5:

[0089] This embodiment is a further optimization based on any of the above embodiments. The similarities with the aforementioned technical solutions will not be repeated here. Furthermore, to better implement the improved pornographic video detection method based on keyframes described in this invention, the following settings are specifically adopted: Step 4) includes the following specific steps:

[0090] 4.1) Use YOLOv5’s Focus module to downsample the key video frame image obtained after step 3), that is, obtain a pixel value for every pixel in the input key video frame image to form a new channel, thereby expanding the feature channels of the original key video frame image to 4 times the original. Then, use convolution integral to obtain a new feature map from the newly obtained channel feature map.

[0091] 4.2) Use the CSP module in YOLOv5 to calculate features in the new feature map to avoid duplicate features;

[0092] 4.3) The output obtained in step 4.2) is fed into FPN (Feature Pyramid) and PAN (Path Aggregation Network) to extract local features of the image from multiple scales.

[0093] Example 6:

[0094] This embodiment is a further optimization based on any of the above embodiments. The similarities with the aforementioned technical solutions will not be repeated here. Furthermore, to better implement the improved pornographic video detection method based on keyframes described in this invention, the following settings are specifically adopted: Step 5) includes the following specific steps:

[0095] 5.1) Input the result from step 4) into the MobileNetV3 neural network model. Replace the traditional convolution calculation process with a combination of depthwise convolution and pointwise convolution. Perform convolution processing on each color channel. When performing convolution calculation on the output of depthwise convolution using pointwise convolution, calculate using the following formula: Where K is a number of size D K ×D K A convolution kernel of size ×m, D K For the length or width dimensions of the input feature map (length and width are the same), F m For the m-th channel of the input image, G m This is the m-th feature map output.

[0096] 5.2) After step 5.1), the feature map generated by each channel is globally pooled using a Squeeze-and-Excitation network structure. Then, a two-stage fully connected operation is performed. In the first fully connected operation, the number of nodes is 1 / 4 of the number of input channels, and then ReLU activation is applied. In the second fully connected operation, the number of nodes is the same as the number of input channels, and then Hard-sigmoid activation function is applied to obtain a weighted feature map.

[0097] The ReLU function is: ReLU = max(0, x); where x is the input data, and max() represents the maximum value function.

[0098] The Hard-sigmoid activation function is: x represents the input data, max() represents the maximum value function, and min() represents the minimum value function.

[0099] For pornographic image recognition tasks, global feature information includes spatial information such as the proportion of human skin color and sexually suggestive actions. These features can be extracted manually. However, with the rapid development of deep learning in recent years, deep learning models have achieved excellent results in the automatic extraction of image features. The emergence of various types of neural network models, including AlexNet, GoogleNet, VGG, and ResNet, has solved the problem of automatic image feature extraction. In order to reduce the model size and speed up the detection rate while ensuring the accuracy of pornographic image detection to meet the needs of practical detection tasks, this invention uses MobileNetV3 to extract global features of pornographic images. Compared with traditional deep neural network models, it is smaller in size and runs faster.

[0100] Example 7:

[0101] This embodiment is a further optimization based on any of the above embodiments. The similarities with the aforementioned technical solutions will not be repeated here. Furthermore, to better implement the improved pornographic video detection method based on keyframes described in this invention, the following settings are specifically adopted: The specific weighted calculation is completed using the following formula:

[0102] Where F represents the final fused feature, and w represents the weight corresponding to the global feature. As a global feature, Local features; where local features The calculation process is as follows:

[0103] , where F i is the predicted value for the i-th sensitive part, and N is the number of predicted sensitive part detection box categories.

[0104] The final fused feature F is fed into a sigmoid classifier to obtain a classification result that can determine whether key video frame images contain pornographic information. The sigmoid function is: .

[0105] Example 8:

[0106] An improved pornographic video detection system based on keyframes is disclosed, which implements an improved pornographic video detection method based on keyframes. The system includes a keyframe extraction module, an image preprocessing module, a local sensitive feature detection module, a global feature extraction module, and a feature fusion module.

[0107] The keyframe extraction module is used to extract key video frames from the video stream, thereby significantly improving the efficiency of video detection without affecting the system's detection accuracy.

[0108] The image preprocessing module performs image preprocessing operations, including image data enhancement, light enhancement, and adaptive sizing, on each extracted key video frame image.

[0109] The local sensitive feature detection module, based on YOLOv5 technology, detects key sensitive areas in preprocessed key video frames to form local features. This module is used to detect local sensitive areas in pornographic images. Since the global feature extraction module struggles to capture local information in pornographic images, and the detection of local sensitive areas is crucial for pornographic image detection, this invention employs the YOLOv5 object detection model from the One-Stage method to extract and detect local sensitive features in pornographic images.

[0110] The global feature extraction module uses the lightweight deep neural network model MobileNetV3 as the backbone. By removing redundant calculations through separable convolutional structures and lightening the model structure, it completes the global feature extraction of key video frame images. It can greatly accelerate the feature extraction calculation speed and improve the detection efficiency without reducing the model recognition accuracy.

[0111] The feature fusion module calculates the final detection result by weighting local and global features, thereby determining whether the key video frame contains pornographic information and whether the video stream is a pornographic video.

[0112] Example 9:

[0113] This embodiment is a further optimization based on the above embodiment. The similarities with the aforementioned technical solutions will not be repeated here. Furthermore, in order to better realize the improved pornographic video detection system based on keyframes described in this invention, the following settings are specifically adopted: the image data enhancement includes expanding the dataset and increasing the number of small targets in the image, the lighting enhancement is used to retain more pornographic target information, and the adaptive size is used to meet the model input requirements while retaining more image information.

[0114] Example 10:

[0115] This embodiment is a further optimization based on embodiment 8 or 9. The similarities with the aforementioned technical solutions will not be repeated here. Furthermore, to better realize the improved pornographic video detection system based on keyframes described in this invention, the following configuration is specifically adopted: The local sensitive feature detection module includes an Input module, a Backbone module, a Neck module, and a Prediction module. The Input module is used to initialize anchor boxes and prediction boxes and perform grid division operations on the key video frame image. The Backbone module, as the neural network backbone of the local sensitive feature detection module, is used to extract feature information of key sensitive parts existing in the grid. The Neck module consists of a Feature Pyramid (FPN) and a PAN, mainly used to extract multi-scale information. The Prediction module generates three feature maps of different sizes to predict the category and anchor boxes of the key video frame image, finally obtaining the category and location of sensitive parts in the key video frame image, and merging and deduplicating overlapping anchor boxes in the key video frame image through non-maximum suppression.

[0116] Combination Figure 1 As shown, the technology employed in the pornography video detection system and method based on key video frames and feature fusion proposed in this invention consists of three parts.

[0117] The first part is a keyframe extraction method based on image entropy and local frame difference, which consists of the following steps:

[0118] Step 1: Convert the video to be inspected into a set of video frames;

[0119] Candidate keyframes are obtained by taking the extreme value of image entropy from the video frame set:

[0120] Step 2: Calculate the image entropy of each video frame, using the video frame sequence number as the x-axis;

[0121] Step 3: Select video frames with extreme image entropy values ​​as a set of candidate keyframes;

[0122] Step 4: Perform local frame difference calculation on the candidate keyframe set using the local frame difference keyframe extraction method;

[0123] Step 5: If the frame difference calculation exceeds the set threshold, then retain the candidate keyframe as the final keyframe (i.e., obtain the keyframe set).

[0124] The second part is a deep learning-based ray enhancement method (applying ray enhancement processing to the keyframe set), which consists of the following steps:

[0125] Step 1: Generate the RGB light curves for the keyframes;

[0126] Step 2: Train the light curve multiple times to finally generate a suitable light enhancement curve;

[0127] Step 3: Enhance the image's lighting using a lighting enhancement curve;

[0128] The third part is a pornographic image recognition method based on feature fusion, which consists of the following steps:

[0129] Step 1: Use YOLOv5 to extract local features from keyframes and select the local sensitive features with the highest classification probability as local features, that is, use YOLOv5 to extract local features;

[0130] Step 2: Use MobileNetV3 to perform global feature extraction on keyframes, that is, use MobileNetV3 to perform global feature extraction;

[0131] Step 3: Perform feature fusion calculation on global features and local features, that is, perform feature fusion on the two types of features;

[0132] Step 4: Input the fused features into the classifier to obtain the final classification result, which is the video detection result.

[0133] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications or equivalent changes made to the above embodiments based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. An improved method for detecting pornographic videos based on keyframes, characterized in that: Includes the following steps: 1) After converting the video stream into a set of video frames, the keyframe extraction module is used to obtain a set of candidate keyframes by taking the extreme value of image entropy. 2) The key video frame set is obtained by using the key frame extraction module to extract key video frame images from the candidate key frame set using the local frame difference key frame extraction method. 3) The image preprocessing module performs image preprocessing operations on each extracted key video frame, including image data enhancement, light enhancement, and adaptive resizing. 4) Local features are extracted from the preprocessed key video frame images using YOLOv5 technology through the local sensitive feature detection module; 5) The global feature extraction module is used to extract global features from the preprocessed key video frame images using the MobileNetV3 neural network model; 6) The feature fusion module is used to calculate the extracted local and global features by specific weighting to obtain the final detection result, thereby determining whether the key video frame contains pornographic information and whether the video stream is a pornographic video; The specific weighted calculation is performed using the following formula: ， in, F For the final fusion features, w The weights are the weights corresponding to the global features. As a global feature, Local features; where local features The calculation process is as follows: ,in F i For the first i Predicted values ​​for sensitive components N The number of detected bounding box categories for the predicted sensitive parts; The final fusion features F The data is fed into a sigmoid classifier to obtain classification results that can determine whether key video frame images contain pornographic information. The sigmoid function is: .

2. The improved pornographic video detection method based on keyframes according to claim 1, characterized in that: The image entropy value is calculated using the following formula: ,in Represented as the first The probability density function of each frame is obtained by calculating its intensity histogram through normalization; the calculated image entropy value is then used. The images are mapped onto a coordinate axis with the key video frame sequence number as the x-axis. Finally, images with extreme entropy values ​​or significant differences in change within a segment are selected as key video frame images for that segment, forming a set of candidate key frames.

3. An improved pornographic video detection method based on keyframes according to claim 1 or 2, characterized in that: Step 2) includes the following specific steps: 2.1) Divide two adjacent key video frame images into a 26*26 grid; 2.2) Local frame difference calculation is performed according to the corresponding grid positions before and after, and the absolute values ​​of the obtained frame difference values ​​are taken and summed to form a univariate value as the local frame difference value between adjacent frames; 2.3) Select the local frame difference value with the largest value among all grids as the difference value of the adjacent frame, and determine whether the value exceeds a preset threshold. If it exceeds the threshold, proceed to step 2.4); otherwise, proceed to step 2.

5. 2.4) Save the previous frame to the key video frame set, and use the current frame as the previous frame to perform local frame difference calculation with the next frame; 2.5) Discard the previous frame, continue to perform local frame difference calculation between the current frame and the next frame, until the last frame of the video stream is calculated and the final set of key video frames is obtained.

4. An improved pornographic video detection method based on keyframes according to claim 1 or 2, characterized in that: Step 3) specifically refers to: Image enhancement: The Mosaic data augmentation method is used to randomly flip, crop, and arrange four images from the candidate keyframe set that are used as part of the training set. The four images that have undergone random operations are then stitched together to obtain a new image. Light enhancement: This involves adjusting the image using a formula... Perform light enhancement processing on the image, where, x The input image contains pixel coordinates. I(x) Let LE((x);α) be the input image, and LE((x);α) be the output image after enhancement. The parameters are... [-1,1] is used to adjust the image and control the exposure effect; Image adaptive resizing: The input image is standardized using LetterBox, which proportionally reduces the size of the input image and uniformly adjusts the original image size to 224×224. The processing order of image enhancement, lighting enhancement, and image adaptive resizing is not important.

5. An improved pornographic video detection method based on keyframes according to claim 1 or 2, characterized in that: Step 4) includes the following specific steps: 4.1) Use the Focus module of YOLOV5 to downsample the key video frame image obtained after processing in step 3). That is, obtain a pixel value for every pixel in the input key video frame image to form a new channel, thereby expanding the feature channels of the original key video frame image to 4 times the original. Then, use convolution integral to obtain a new feature map from the newly obtained channel feature map. 4.2) Use the CSP module in YOLOv5 to calculate features in the new feature map to avoid duplicate features; 4.3) The output obtained in step 4.2) is fed into FPN and PAN to extract local features of the image from multiple scales.

6. An improved pornographic video detection method based on keyframes according to claim 1 or 2, characterized in that: Step 5) includes the following specific steps: 5.1) Input the result from step 4) into the MobileNetV3 neural network model, and perform convolution processing on each color channel using a combination of depthwise convolution and pointwise convolution. When performing convolution calculation on the output of depthwise convolution using pointwise convolution, the following formula is used: ,in, K It is the size of D K × D K × m convolution kernel, D K The length or width of the input feature map is the same as its width. F m For the first input image m One channel, G m For the output of the first m Each feature map; 5.2) After step 5.1), the feature map generated by each channel is subjected to global pooling, and then a two-stage fully connected operation is performed. In the first fully connected operation, the number of nodes is 1 / 4 of the number of input channels, and then ReLU function is applied for activation. In the second fully connected operation, the number of nodes is the number of input channels, and then Hard-sigmoid activation function is applied to obtain a weighted feature map. The ReLU function is: ReLU = max(0, x); where x is the input data, and max() represents the maximum value function. The Hard-sigmoid activation function is: x represents the input data, max() represents the maximum value function, and min() represents the minimum value function.

7. An improved pornographic video detection system based on keyframes, characterized in that: An improved keyframe-based pornographic video detection method according to any one of claims 1 to 6 includes a keyframe extraction module, an image preprocessing module, a local sensitive feature detection module, a global feature extraction module, and a feature fusion module. The keyframe extraction module is built on a keyframe extraction algorithm and is used to extract key video frames from the video stream. The image preprocessing module performs image preprocessing operations, including image data enhancement, light enhancement, and adaptive sizing, on each extracted key video frame image. The local sensitive feature detection module detects key sensitive parts in the preprocessed key video frame image based on YOLOv5 technology, forming local features; The global feature extraction module uses the lightweight deep neural network model MobileNet3 as the backbone. It removes redundant calculations and lightens the model structure through separable convolutional structures to complete the global feature extraction of key video frame images. The feature fusion module calculates the final detection result by weighting local and global features, thereby determining whether the key video frame contains pornographic information and whether the video stream is a pornographic video.

8. The improved pornographic video detection system based on keyframes according to claim 7, characterized in that: The image data augmentation includes expanding the dataset and increasing the number of small targets in the image. Light enhancement is used to retain more information about erotic targets, and adaptive sizing is used to meet the model input requirements while retaining more image information.

9. An improved pornographic video detection system based on keyframes according to claim 7 or 8, characterized in that: The local sensitive feature detection module includes an Input module, a Backbone module, a Neck module, and a Prediction module. The Input module initializes anchor boxes and prediction boxes and performs grid division operations on the key video frame image. The Backbone module extracts feature information of key sensitive parts in the grid. The Neck module extracts multi-scale information. The Prediction module generates three feature maps of different sizes to predict the category and anchor boxes of the key video frame image, and finally obtains the category and location of sensitive parts in the key video frame image. It also performs non-maximum suppression to merge and remove duplicate anchor boxes in the key video frame image.

Images