Face and license plate privacy protection video processing method and device, equipment and medium

Abstract

The method is mainly applied to the technical field of artificial intelligence processing. The invention discloses a face and license plate privacy protection video processing method, device, equipment and medium, and the method comprises the steps: extracting input video data to obtain a video frame sequence which comprises a plurality of video frames; the video frame sequence is input to a pre-trained face and license plate detection model for target detection, so that a target video frame containing a target image area is determined, position information of the target image area in the target video frame is obtained, and the target image area is a face area or a license plate area; based on the position information of the target image area, performing fuzzy processing on the target image area in each target video frame; and generating a video after fuzzy processing based on the video frame after fuzzy processing. According to the method and the device, the privacy protection and the target detection are collaboratively optimized, so that the efficiency and the accuracy of target detection are improved while the privacy security is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a face and license plate privacy protection video processing method, device, equipment and medium. BACKGROUND

[0002] In today's digital age, video surveillance technology is widely used in public safety, traffic management and other fields, providing important support for social order maintenance and urban intelligent construction. However, with the development of technology, the contradiction between privacy protection and target detection technology is increasingly prominent. On the one hand, target detection technology aims to accurately identify key information such as faces and license plates from video streams to achieve effective monitoring and management. On the other hand, privacy protection technology requires blurring or obscuring these sensitive information to prevent personal privacy leakage. Since the privacy protection process usually takes place after target detection, the target detection model can only be trained and recognized based on blurred data, which reduces the detection accuracy and efficiency. This modular design, although simplifying the technology implementation, ignores the internal relationship and collaborative needs between the two, making it difficult to reconcile the contradiction between privacy desensitization and detection efficiency. SUMMARY

[0003] The present application provides a face and license plate privacy protection video processing method, device, equipment and medium, which optimizes privacy protection and target detection together, and improves the efficiency and accuracy of target detection while ensuring privacy security.

[0004] The present application provides a face and license plate privacy protection video processing method, which comprises: extracting input video data to obtain a video frame sequence, the video frame sequence comprising a plurality of video frames; inputting the video frame sequence into a pre-trained face and license plate detection model for target detection to determine target video frames containing target image regions and obtain the position information of the target image regions in the target video frames, wherein the target image regions are face regions or license plate regions; based on the position information of the target image regions, performing blurring processing on the target image regions in each target video frame; generating a blurred video based on the blurred video frames.

[0005] Optionally, the blurring processing on the target image regions in each target video frame comprises: obtaining the detection confidence of the target image regions; determining the blurring processing intensity according to the detection confidence; blurring processing the target image regions according to the blurring processing intensity.

[0006] Optionally, the face and license plate detection model comprises an attention module and a double-task detection module. After inputting the video frame sequence into a pre-trained face and license plate detection model for target detection, the face and license plate privacy protection video processing method further comprises: extracting video features of each video frame; enhancing each video feature using the attention module; detecting face regions and license plate regions from the enhanced video features using the double-task detection module.

[0007] Optionally, the enhancement of each video feature using the attention module comprises: determining a first weight branch, a second weight branch, and a third weight branch according to a preset weight distribution rule; extracting average video features through the first weight branch, extracting maximum video features and edge video features through the second weight branch, and enhancing the edge video features through the third weight branch; stacking and fusing the features output by the first weight branch, the second weight branch, and the third weight branch to obtain a spatial weight map and combine it with the video features before enhancement to enhance the attention to edges and textures in the video features.

[0008] Optionally, the detection of face regions and license plate regions from the enhanced video features using the double-task detection module comprises: extracting general visual features from the video frames; inputting the general visual features into a first detection branch and a second detection branch, respectively, to obtain position information of the face regions through lightweight convolution processing of the general visual features by the first detection branch, and to obtain position information of the license plate regions through enhanced convolution processing of the general visual features by the second detection branch.

[0009] Optionally, the face and license plate privacy protection video processing method further comprises the step of pre-training the face and license plate detection model using a multi-task loss function as an optimization target; the pre-training of the face and license plate detection model using a multi-task loss function as an optimization target comprises: freezing the backbone network parameters of the face and license plate detection model, and training the attention module and the double-task detection module; After the attention module and the double task detection module are trained, all parameters of the face and license plate detection model are unfrozen and the whole training is performed by using a learning rate strategy gradually decreasing over time. When the attention module and the double task detection module are fine-tuned, the low feature extraction layer of the face and license plate detection model is frozen, and the training data used for pre-training is subjected to data enhancement processing.

[0010] Optionally, before the video frame sequence is input into the pre-trained face and license plate detection model for target detection, the face and license plate privacy protection video processing method further comprises the step of pre-processing each video frame. The pre-processing of each video frame comprises: dividing the video frame into a plurality of image regions; dividing each image region into a plurality of blocks; For each image region, the size of each block in the image region is dynamically adjusted according to the detail complexity of the image region to enhance the contrast of the image region.

[0011] The present application also provides a face and license plate privacy protection video processing device, which comprises: an extraction module configured to extract input video data to obtain a video frame sequence, the video frame sequence comprising a plurality of video frames; a detection module configured to input the video frame sequence into a pre-trained face and license plate detection model for target detection to determine target video frames containing target image regions and obtain position information of the target image regions in the target video frames, wherein the target image region is a face region or a license plate region; a blur processing module configured to perform blur processing on the target image region in each target video frame based on the position information of the target image region; a generation module configured to generate a blurred video based on the blurred video frames.

[0012] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the face and license plate privacy protection video processing method according to any one of the above when executing the computer program.

[0013] The present application also provides a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the face and license plate privacy protection video processing method according to any one of the above.

[0014] The present application has at least the following beneficial effects: The present technical solution realizes the synergistic improvement of privacy protection and target detection efficiency and accuracy by closely combining privacy protection with target detection. First, the input video data is extracted to provide a basis for subsequent processing. Then, the pre-trained face and license plate detection model is used to detect the target in the video frame sequence, accurately locate the face and license plate area, which is the key prerequisite for realizing privacy protection, and the pre-training characteristics of the model ensure the efficiency and accuracy of the detection. Then, based on the position information of the detected target image area, the target image area in the target video frame is blurred, effectively protecting the privacy information. Finally, the blurred video is generated based on the blurred video frame, completing the entire privacy protection process. In the entire process, target detection provides accurate positioning for privacy protection, and the demand for privacy protection also promotes the optimization of the target detection model, both of which promote each other and achieve the technical effect of improving target detection efficiency and accuracy while ensuring privacy security. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings are used to provide a further understanding of the technical solutions of the present application, and constitute a part of the specification, and are used to explain the technical solutions of the present application together with embodiments of the present application, and do not constitute a limitation on the technical solutions of the present application.

[0016] Figure 1 is a step flow chart of a face and license plate privacy protection video processing method; Figure 2 is a step flow chart of blurring in a face and license plate privacy protection video processing method; Figure 3 is another step flow chart of a face and license plate privacy protection video processing method; Figure 4 is a training stage flow chart of a face and license plate detection model pre-training in a face and license plate privacy protection video processing method; Figure 5 is an inference stage flow chart of a face and license plate privacy protection video processing method in actual use; Figure 6 is a structural schematic diagram of a face and license plate privacy protection video processing device; Figure 7 is a structural schematic diagram of an electronic device. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0018] The researchers of the present application found that the traditional video data processing flow often treats target detection and privacy protection as two independent links. First, the face and license plate are identified through the target detection model, and then the sensitive targets are desensitized through methods such as blurring. However, such a phased processing mode has significant technical limitations.

[0019] In the target detection link, in order to improve the detection accuracy, high-resolution and high-definition image data are usually required as input to capture the subtle features of the target. However, when entering the privacy protection link, the blurring processing of the face and license plate inevitably destroys the detailed information of the image. This destruction not only may cause data redundancy, but also may interfere with the subsequent target detection model, reducing the detection efficiency and accuracy. Conversely, if the privacy protection effect is prioritized and excessively blurred, it will be difficult for the target detection model to capture effective features, thereby weakening the detection performance.

[0020] The root cause of this contradiction lies in the fact that in the existing technical system, privacy protection and target detection have not been optimized collaboratively. The lack of integrated consideration of data processing flow, model design, and effect evaluation between the two leads to the contradiction between privacy desensitization and detection efficiency, which is difficult to reconcile. Therefore, innovative technical solutions are needed to break this dilemma and achieve a win-win situation of accurate detection and privacy protection. In order to solve the above technical problems, the present technical solution provides a face and license plate privacy protection video processing method, device, equipment and medium, which realizes the improvement of the efficiency and accuracy of target detection while ensuring privacy security by optimizing privacy protection and target detection collaboratively. The following are various embodiments of the present technical solution.

[0021] Please refer to Figure 1 , Figure 1 is a step flowchart of a face and license plate privacy protection video processing method.

[0022] The present embodiment provides a face and license plate privacy protection video processing method, which comprises: S101, extracting input video data to obtain a video frame sequence, the video frame sequence comprising a plurality of video frames.

[0023] S102, inputting the video frame sequence into a pre-trained face and license plate detection model for target detection to determine target video frames containing target image regions and obtain position information of the target image regions in the target video frames, wherein the target image regions are face regions or license plate regions.

[0024] S103, based on the position information of the target image regions, performing blurring processing on the target image regions in each target video frame.

[0025] S104. Generate a blurred video based on the blurred video frames.

[0026] In this embodiment, video frames that do not contain the target image region in the video frame sequence are unblurred video frames. When generating a blurred video, the blurred video frames and the unblurred video frames are combined to obtain the blurred video. If each video frame in the video frame sequence is blurred, the blurred video frames are recombined into a blurred video according to the time order of the video frame sequence.

[0027] Understandably, this technical solution achieves a synergistic improvement in both privacy protection and the efficiency and accuracy of object detection by closely integrating privacy protection with object detection. First, video frame sequences are extracted from the input video data, providing basic material for subsequent processing. Next, a pre-trained face and license plate detection model is used to perform object detection on the video frame sequence, accurately locating face and license plate regions. This is a crucial prerequisite for privacy protection, and the pre-training characteristics of the model ensure both high efficiency and accuracy. Then, based on the location information of the detected target image regions, the target image regions in the target video frames are blurred, effectively protecting privacy information. Finally, a blurred video is generated based on the blurred video frames, completing the entire privacy protection process. Throughout this process, object detection provides precise location for privacy protection, while the need for privacy protection drives the continuous optimization of the object detection model. The two mutually reinforce each other, jointly achieving the technical effect of improving object detection efficiency and accuracy while ensuring privacy security.

[0028] In some embodiments, before inputting the video frame sequence into a pre-trained face and license plate detection model for target detection, the face and license plate privacy protection video processing method further includes a preprocessing step for each video frame.

[0029] Methods for preprocessing each video frame include: The video frame is divided into multiple image regions; each image region is further divided into multiple blocks; for each image region, the size of each block in the image region is dynamically adjusted according to the complexity of the details of the image region, so as to enhance the contrast of the image region.

[0030] In some embodiments, dynamic CLAHE (adaptive Tile size) is used to improve illumination robustness. In overexposed scenes, noise amplification in the license plate reflection area is effectively suppressed, maintaining character recognizability; in underexposed scenes, the details of the face in the shadow are significantly improved. The principle of adaptive Tile size: trigger Tile size switching by calculating the local variance threshold, automatically adjust the Tile size (8x8 to 16x16) according to the complexity of the image content, use smaller Tile to retain details in complex scenes, and use larger Tile to improve efficiency in simple scenes.

[0031] It can be understood that the embodiments enhance the contrast of image regions by dynamically adjusting the size of each block in the video frame, significantly improving the input quality of the face and license plate detection model. This preprocessing method can adaptively adjust according to the complexity of the details of the image region, so that the model can more accurately locate the face and license plate region in the subsequent detection process. Combined with the previous multi-task loss function optimization and phased training strategy, the entire system improves the accuracy and efficiency of target detection while protecting privacy, enhances the adaptability to different scenes and complex backgrounds, and makes video processing more accurate and efficient.

[0032] In some embodiments, data augmentation is performed using random brightness contrast adjustment, multiplicative noise, ISO noise simulation, etc.

[0033] Please refer to Figure 2 , Figure 2 A step flowchart of blurring processing in a face and license plate privacy protection video processing method.

[0034] In some embodiments, the way of blurring processing on the target image region in each target video frame includes: S201, obtaining the detection confidence of the target image region.

[0035] S202, determining the blurring processing intensity according to the detection confidence.

[0036] S203, blurring processing the target image region according to the blurring processing intensity.

[0037] In this embodiment, the detection confidence is linearly mapped to the Gaussian kernel size, and finally according to the high and low of the confidence, different Gaussian blur kernel sizes are used to apply stronger blurring processing to the detection results with higher certainty, forming a positive correlation response between risk and processing intensity.

[0038] It can be understood that the embodiment can dynamically adjust the blur processing strength according to the reliability of the detection by obtaining the detection confidence of the target image region. When the detection confidence is high, a strong blur processing is adopted to ensure the thoroughness of privacy protection; when the detection confidence is low, moderate blur is used to avoid excessive processing affecting video usability. This dynamic adjustment mechanism not only improves the accuracy of privacy protection, but also avoids unnecessary loss of image quality, making privacy protection and video usability achieve a better balance, further enhancing the practicality and flexibility of the technical solution.

[0039] In some embodiments, the face and license plate detection model includes an attention module and a double-task detection module.

[0040] Specifically, considering the computing power and model inference speed, the YOLO V7 tiny is used as the base network model of the face and license plate detection model, a channel attention module and an EDA module (edge detection attention module) are added, and the detection head is modified into a double-task detection module.

[0041] Please refer to Figure 3 , Figure 3 is another step flowchart of a face and license plate privacy protection video processing method.

[0042] In some embodiments, after inputting the video frame sequence into the pre-trained face and license plate detection model for target detection, the face and license plate privacy protection video processing method further includes: S301, extracting video features of each video frame.

[0043] S302, enhancing each video feature using an attention module.

[0044] S303, detecting face regions and license plate regions using a double-task detection module on the enhanced video features.

[0045] In this embodiment, considering that the background color of the vehicle license plate is relatively fixed, the channel attention module is used to make the model pay more attention to the channel information. In addition, because the outline of the license plate has obvious features, the EDA (edge detection attention module) module is proposed to extract the edge and outline information of the feature map. In addition, the double-task detection module of this embodiment improves the face and license plate detection rate compared with the single-task detection head.

[0046] It can be understood that the embodiment can more accurately extract and enhance video features, and simultaneously detect face and license plate regions, by introducing an attention module and a double-task detection module in the face and license plate detection model. The attention module can focus on key information and improve the relevance of feature extraction; the double-task detection module can simultaneously process face and license plate detection tasks, avoid repeated calculation, and improve efficiency. This optimization not only enhances the reliability of privacy protection, but also improves the adaptability and processing capacity of the entire system to complex scenes, achieving dual improvement of privacy protection and detection performance.

[0047] In some embodiments, step S202 comprises: According to a preset weight distribution rule, a first weight branch, a second weight branch, and a third weight branch are determined; average video features are extracted through the first weight branch, maximum video features and edge video features are extracted through the second weight branch, and the edge video features are enhanced through the third weight branch; the features output by the first weight branch, the second weight branch, and the third weight branch are stacked and fused to obtain a spatial weight map and combined with the video features before enhancement, so as to enhance the attention to edges and textures in the video features.

[0048] In the embodiment, the channel attention mechanism is embedded before the second ELAN of the Backbone; and the EDA (edge detection attention mechanism) is embedded after the first MP (downsampling layer) of the Backbone.

[0049] The channel attention mechanism uses attention modules such as SENet and FcaNet.

[0050] The EDA module (edge detection attention module) is a multi-branch structure. First, the average value and the maximum value of the input feature map A are calculated in the channel dimension to obtain two feature maps, denoted as AVE and MAX. Since MAX can reflect more edge information of the image, an edge detection operator Sobel is used to extract the edge features of the MAX feature map, denoted as Edge. The AVE, MAX, and Edge are stacked along the channel dimension to obtain a feature map with a channel number of 3. Then, a convolution layer with a convolution kernel size of 3, a padding of 1, and a stride of 1 is used to extract features and restore the channel number of the feature map to 1. Finally, the feature map is input into a Sigmoid activation function, and then matrix point multiplication is performed on the original input feature map A. The original input feature map A is re-assigned weights to pay more attention to the spatial features such as edges and textures of the feature map.

[0051] It can be understood that the embodiment extracts average, maximum and edge video features respectively through the weight distribution rule, and enhances the edge features. The spatial weight map obtained by fusing these features, combined with the original video features, strengthens the attention to edges and textures. This enables the face and license plate detection model to more accurately identify the target area, further improving the accuracy and robustness of target detection, while providing a more reliable foundation for privacy protection and improving the performance and practicality of the entire system.

[0052] In some embodiments, step S203 comprises: The features of the picture of the video frame are extracted to obtain general visual features; the general visual features are input into the first detection branch and the second detection branch respectively, so as to obtain the position information of the face region by performing lightweight convolution processing on the general visual features through the first detection branch, and obtain the position information of the license plate region by performing enhanced convolution processing on the general visual features through the second detection branch.

[0053] In the embodiment, the dual-task detection module first extracts general feature representations (such as basic visual features such as edges and textures) using shared layers composed of convolution layers, normalization layers and activation layers, achieving the effect of reducing parameter redundancy. Then it is divided into two branches, one branch for detecting faces and the other branch for detecting license plates. The license plate branch increases a 1x1 convolution layer to enhance the feature expression ability for small size license plates, and the face branch adopts depth separable convolution to reduce the computational complexity while maintaining accuracy.

[0054] It can be understood that the embodiment extracts general visual features and inputs them into the first detection branch and the second detection branch respectively, uses lightweight convolution to process the face region, and enhances convolution to process the license plate region, achieving customized detection for different targets. Lightweight convolution improves the efficiency of face detection, and enhanced convolution improves the accuracy of license plate detection. This branch processing method enables the model to more accurately locate the face and license plate regions, further improving the effect of privacy protection, while improving the detection efficiency and reliability of the entire system.

[0055] In some embodiments, the face and license plate privacy protection video processing method further comprises the step of pre-training the face and license plate detection model using a multi-task loss function as an optimization target.

[0056] The way of pre-training the face and license plate detection model using a multi-task loss function as an optimization target comprises: The backbone network parameters of the face and license plate detection model are frozen, and the attention module and the double-task detection module are trained; after the training of the attention module and the double-task detection module is completed, all the parameters of the face and license plate detection model are unfrozen, and a learning rate strategy gradually decreasing over time is used for overall training; when fine-tuning the attention module and the double-task detection module, the low feature extraction layer of the face and license plate detection model is frozen, and the training data used for pre-training is subjected to data enhancement processing.

[0057] It can be understood that the embodiment optimizes the pre-training process of the face and license plate detection model through a multi-task loss function, significantly improving the model performance. By freezing the backbone network parameters, the attention module and the double-task detection module are preferentially trained, so that the model quickly focuses on key features. After unfreezing all the parameters, the overall training is performed using a progressive learning rate strategy, further improving the generalization ability of the model. When fine-tuning, the low feature extraction layer is frozen and data enhancement is performed, enhancing the adaptability of the model to different scenes. This phased and targeted training strategy not only improves the accuracy and efficiency of face and license plate detection, but also enhances the reliability of privacy protection, making the model training process more efficient, stable and adaptable.

[0058] Please refer to Figure 4 , Figure 4 is a training phase flowchart of a face and license plate detection model in a face and license plate privacy protection video processing method.

[0059] The present application provides a specific embodiment to describe the specific implementation process of the pre-training of the face and license plate detection model, and the embodiment takes vehicle shooting video processing as an example.

[0060] First, video frames are extracted from the video, and the extracted video frames are subjected to data preprocessing.

[0061] The initial Tile size of the video frame is set to 12x12, and the local variance V is calculated by sliding window (step 6). The variance threshold calculation method is: V_thresh = 0.7xV_global + 0.3xV_neighbor (V_global is the variance of the whole image, and V_neighbor is the variance mean value of the neighborhood window).

[0062] When V<0.6xV_thresh, the Tile size is expanded to 16x16.

[0063] When V>1.4xV_thresh, the Tile size is reduced to 8x8.

[0064] CLAHE parameters: the clipping limit is set to 3.0, the histogram bins=256, and the distribution mapping uses bilinear interpolation Then, in the present embodiment, the data of the video frame is processed as follows: Random brightness adjustment: delta brightness ∈ [-0.2, 0.2] (relative value); Contrast adjustment: gamma ∈ [0.8, 1.2]; Multiplicative noise: intensity coefficient ∈ [0.05, 0.15], subject to Gaussian distribution N(0, 0.1); ISO noise simulation: combination of Poisson noise + Gaussian white noise (σ = 15).

[0065] Next, the network structure configuration of the face and license plate detection model is set as follows: Basic network: YOLOv7-tiny (input resolution 640x640); Channel attention module: FcaNet structure (frequency domain channel attention) is adopted; Embedding position: before the second ELAN module of Backbone; Parameter setting: reduction_ratio = 16, frequency band division = 8.

[0066] After the video frame sequence is input into the pre-trained face and license plate detection model, first, the average value and the maximum value are calculated on the channel dimension of the input feature map A, and two feature maps are obtained, denoted as AVE and MAX. Since MAX can better reflect the edge information of the image, the edge detection operator Sobel is used to extract the edge features of the MAX feature map, denoted as Edge. The AVE, MAX and Edge are stacked along the channel dimension to obtain a feature map with 3 channels. Then, a convolution layer with a kernel size of 3, a padding of 1 and a stride of 1 is used to extract features and restore the channel number of the feature map to 1. Finally, the feature map is first input into the Sigmoid activation function, and then the original input feature map A is matrix multiplied, which reassigns the weights of the original input feature map A to make it pay more attention to the spatial features such as edges and textures of the feature map.

[0067] In the present embodiment, the double-task detection module of the face and license plate detection model is set as follows: Shared layer: 3x3 convolution (Cin = 512, Cout = 256) + BN + SiLU; Face branch: depth separable convolution (3x3 dw + 1x1 pw) x 3; output feature map 19x19x(5+C); License plate branch: 1x1 convolution (Cin = 256, Cout = 128); 3x3 convolution (Cin = 128, Cout = 64) Output feature map 38x38x(5+C).

[0068] Finally, the multi-task loss function is used as the optimization objective to pre-train the face and license plate detection model.

[0069] In this embodiment, the multi-task loss function is set as: total_loss = (obj_loss * 0.5 + cls_loss * 0.5 + box_loss * 1.0) +(face_iou_loss * 0.8 + plate_iou_loss * 1.2) where obj_loss and cls_loss have the same weight (0.5), emphasizing the balance between object existence prediction and class prediction; box_loss has a higher weight (1.0), highlighting the importance of bounding box positioning accuracy; The IoU loss for face (face_iou_loss) and license plate (plate_iou_loss) is designed separately, as license plates are usually smaller in size, and a higher weight (1.2) is given to emphasize their positioning accuracy; The Complete-IoU is used for the IoU loss.

[0070] In this embodiment, the training strategy for pre-training the face and license plate detection model is set in three stages: (1) Feature freezing stage (50 epochs): Freeze the Backbone parameters, only train the detection head and attention module, and set the learning rate to 0.001 (warmup for 5 epochs).

[0071] (2) Global fine-tuning stage (150 epochs): Unfreeze all parameters, use cosine annealing learning rate (initial 0.001 → final 0.0001).

[0072] (3) Special optimization stage (50 epochs): Freeze the low-level feature extraction layer, fine-tune the attention module and dual-task detection module, and use more difficult data augmentation methods, such as using GAN network to generate images under special weather conditions, simulating extreme weather environments.

[0073] Please refer to Figure 5 , Figure 5 is a reasoning stage flowchart of the face and license plate privacy protection video processing method in actual use.

[0074] Before using the pre-trained face and license plate detection model to detect the target in the video frame sequence, the running environment of the face and license plate detection model is configured: The edge computing architecture is deployed on the NVIDIA Jetson AGX Xavier platform (512-core Volta GPU, 32GB memory) to process 1080P@30fps video streams. The system includes: a video decoding module (based on FFmpeg hardware acceleration); a frame extraction module (extracting 30 key frames per second); a preprocessing module (dynamic CLAHE, normalization, etc.); a detection model inference module (TensorRT acceleration); a post-processing module (dynamic Gaussian blur); a video encoding module (H.265 encoding).

[0075] Next, dynamic CLAHE (adaptive Tile size) is used to improve the illumination robustness. In the overexposed scene, the noise amplification in the license plate reflection area is effectively suppressed, and the character distinguishability is maintained; in the underexposed scene, the details of the face in the shadow are significantly improved.

[0076] Then, the video frame sequence is input into the pre-trained face and license plate detection model for inference.

[0077] After the target image area is detected by the face and license plate detection model, the target image area is blurred.

[0078] Specifically, the detection confidence (usually in the range of 0.7-1.0) is linearly mapped to the Gaussian kernel size (13x13 to 19x19), and the specific processing method is: kernel_size = 2 * int(box[4] * 9) + 1 The confidence ∈ [0.7, 1.0], and the kernel size ∈ [13, 19].

[0079] According to the level of confidence, different Gaussian blur kernel sizes are used to apply stronger blur processing to the detection results with higher certainty, forming a positive correlation response between risk and processing intensity. When the confidence is low (confidence ≈ 0.7), light blur processing (σ = 6.5, kernel size 13x13) is performed; when the confidence is high (confidence = 1.0), heavy blur processing (σ = 9.5, kernel size 19x19) is performed.

[0080] Finally, the blurred video frames are recombined into a video.

[0081] It can be understood that the embodiment adjusts the blur intensity according to the detection confidence, prevents privacy leakage caused by missed detection, and dynamically adjusts the Gaussian blur kernel during the blur processing, thereby balancing between privacy protection and visual quality.

[0082] Please refer to Figure 6 , Figure 6 It is a structural schematic diagram of a face and license plate privacy protection video processing device.

[0083] The embodiment also provides a face and license plate privacy protection video processing device, and the device comprises: An extraction module 601 is configured to extract input video data to obtain a video frame sequence, and the video frame sequence comprises a plurality of video frames.

[0084] A detection module 602 is configured to input the video frame sequence to a pre-trained face and license plate detection model to perform target detection, so as to determine a target video frame containing a target image region and obtain position information of the target image region in the target video frame, wherein the target image region is a face region or a license plate region.

[0085] A blur processing module 603 is configured to perform blur processing on the target image region in each target video frame based on the position information of the target image region.

[0086] A generation module 604 is configured to generate a video after blur processing based on the video frame after blur processing.

[0087] Those skilled in the art can understand that all or some of the steps in the method disclosed above can be implemented as software, firmware, hardware and appropriate combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer readable medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. As known to those skilled in the art, communication media generally includes computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery medium.

[0088] It can be understood that the contents in the above method embodiments are applicable to the device embodiments, the device embodiments specifically implement the functions of the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0089] The embodiments of the present application also provide an electronic device, which comprises a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the face and license plate privacy protection video processing method according to any one of the above embodiments is implemented.

[0090] Reference Figure 7 , Figure 7 The hardware structure of the electronic device of another embodiment is illustrated, which comprises: The processor 701 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is used to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0091] The memory 702 can be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a Random Access Memory (RAM), etc. The memory 702 can store operation devices and other application programs, and when the technical solutions provided in the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 702 and are called and executed by the processor 701 to perform the face and license plate privacy protection video processing method of the embodiments of the present application.

[0092] The input / output interface 703 is configured to realize information input and output.

[0093] The communication interface 704 is configured to realize the communication interaction between the device and other devices, and the communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0094] The bus 705 is configured to transmit information between various components (for example, the processor 701, the memory 702, the input / output interface 703, and the communication interface 704) of the device.

[0095] The processor 701, the memory 702, the input / output interface 703, and the communication interface 704 are connected to each other through the bus 705 to realize the communication connection between the device.

[0096] It can be understood that the contents in the above method embodiments are all applicable to the electronic device embodiments, the functions specifically realized by the electronic device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0097] The embodiments of the present application also provide a computer readable storage medium, which stores a processor executable program. The processor executable program is executed by a processor to implement the face and license plate privacy protection video processing method according to any one of the above embodiments.

[0098] The embodiments of the present application also disclose a computer program product, which includes a computer program or computer instructions. The computer program or computer instructions are stored in a computer readable storage medium. A processor of a computer device reads the computer program or computer instructions from the computer readable storage medium. The processor executes the computer program or computer instructions to make the computer device execute the face and license plate privacy protection video processing method according to any one of the above embodiments.

[0099] It can be understood that the contents in the above method embodiments are all applicable to the present storage medium embodiments, the present storage medium embodiments specifically implement the functions same as those of the above method embodiments, and achieve the beneficial effects same as those of the above method embodiments.

[0100] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of accomplishing functionalities using either term when appropriate. Moreover, the use of the terms "including", "comprising", "having" and variations thereof, when used in this description and in the claims, are intended to be broad enough to encompass the inclusion of items listed after the term and not other non-listed items. It will be understood by those within the art that, in general, terms used herein, and especially, terms of degree such as "about", "substantially" and the like, are intended to allow for a number of variations of a given numerical value that fall in the desired range, while maintaining the general intent of the term. For example, it will be understood that an amount that "is about X", where X is a numerical value, allows for a range of values that fall about X. In other words, these terms can mean approximately, roughly, around, or in the immediate vicinity of a numeric value or incoming number. Unless otherwise stated, the use of ordinal adjectives such as "first", "second", "third", etc. are to be understood to mean "one", "two", "three", etc. respectively, and are not to be construed as limiting the scope of the application. It will be further understood that use of the terms "and", "or", and "both" are used in their inclusive sense (and not in their exclusive sense) unless the context clearly dictates otherwise. It will be further understood that the terms "comprise", "comprises", "comprising", "include", "includes", and "including" are used inclusively and tautologically and are not meant exclusively or exhaustively to indicate the presence of the stated features, integers, steps or components or groups thereof. It will be further understood that the terms "a", "an", and "the" are used interchangeably and are taken to mean one or more unless otherwise indicated by the context.

[0101] In several embodiments provided by the present application, it should be understood that the disclosed apparatus, device and method can be implemented in other manners. For example, the above described apparatus embodiment is merely illustrative. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0102] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e. can be located in one place or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0103] In addition, each functional unit in the embodiments of the present application can be integrated in a processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in a unit. The integrated unit can be implemented in the form of hardware or software functional units.

[0104] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0105] Although the description of the present application has been quite detailed and particularly described with respect to several described embodiments, it is not intended to be limited to any of these details or embodiments or any special embodiment, but should be considered to effectively cover the intended scope of the present application by referring to the appended claims, taking into account the broadest possible interpretation of these claims in view of the prior art. In addition, the present application is described above in embodiments that the inventors can foresee, and the purpose is to provide a useful description, and non-essential modifications to the present application that have not yet been foreseen can still represent equivalent modifications to the present application.

Claims

1. A face and license plate privacy protection video processing method, characterized in that, The method comprises: extracting input video data to obtain a video frame sequence, the video frame sequence comprising a plurality of video frames; inputting the video frame sequence into a pre-trained face and license plate detection model for target detection to determine target video frames containing target image regions and obtain position information of the target image regions in the target video frames, wherein the target image regions are face regions or license plate regions; based on the position information of the target image regions, performing blur processing on the target image regions in each target video frame; based on the blurred video frames, generating a blurred video.

2. The method of claim 1, wherein, The blur processing on the target image regions in each target video frame comprises: obtaining a detection confidence of the target image region; determining a blur processing intensity according to the detection confidence; performing blur processing on the target image region according to the blur processing intensity. 3.The method of claim 1, wherein, The face and license plate detection model comprises an attention module and a dual-task detection module; After inputting the video frame sequence into the pre-trained face and license plate detection model for target detection, the method further comprises: extracting video features of each video frame; enhancing each video feature using the attention module; detecting face regions and license plate regions from the enhanced video features using the dual-task detection module.

4. The method of claim 3, wherein, The enhancement of each video feature using the attention module comprises: determining a first weight branch, a second weight branch, and a third weight branch according to a preset weight distribution rule; extracting average video features through the first weight branch, extracting maximum video features and edge video features through the second weight branch, and enhancing the edge video features through the third weight branch; stacking and fusing the features output by the first weight branch, the second weight branch, and the third weight branch to obtain a spatial weight map and combine it with the video features before enhancement to enhance the attention to edges and textures in the video features.

5. The method of claim 3, wherein the method further comprises: The detection of face regions and license plate regions from the enhanced video features using the dual-task detection module comprises: extracting general visual features from the video frames; inputting the general visual features into a first detection branch and a second detection branch respectively to perform lightweight convolution processing on the general visual features through the first detection branch to obtain position information of the face regions, and perform enhanced convolution processing on the general visual features through the second detection branch to obtain position information of the license plate regions.

6. The method of claim 3, wherein the method further comprises: The method further comprises a step of pre-training the face and license plate detection model using a multi-task loss function as an optimization target; The pre-training of the face and license plate detection model using a multi-task loss function as an optimization target comprises: freezing the backbone network parameters of the face and license plate detection model, and training the attention module and the dual-task detection module; After the attention module and the double-task detection module are trained, all parameters of the face and license plate detection model are unfrozen and the whole training is performed by using a learning rate strategy gradually decreasing over time. When the attention module and the double-task detection module are fine-tuned, the low feature extraction layer of the face and license plate detection model is frozen, and the training data used for pre-training is subjected to data augmentation processing.

7. The method of claim 1 or 6, wherein, Before the video frame sequence is input into the pre-trained face and license plate detection model for target detection, the method further includes a step of pre-processing each video frame; The pre-processing of each video frame includes: dividing the video frame into multiple image regions; dividing each image region into multiple blocks; for each image region, dynamically adjusting the size of each block in the image region according to the detail complexity of the image region to enhance the contrast of the image region.

8. A face and license plate privacy preserving video processing apparatus, characterized by, The device includes: an extraction module configured to extract input video data to obtain a video frame sequence, the video frame sequence including a plurality of video frames; a detection module configured to input the video frame sequence into a pre-trained face and license plate detection model for target detection to determine target video frames containing target image regions and obtain position information of the target image regions in the target video frames, wherein the target image regions are face regions or license plate regions; a blur processing module configured to perform blur processing on the target image regions in each target video frame based on the position information of the target image regions; a generation module configured to generate a blurred video based on the blurred video frames.

9. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the face and license plate privacy protection video processing method of any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the face and license plate privacy protection video processing method of any one of claims 1-7.

Images