A privacy protection type video blur processing method and device

Abstract

This invention discloses a privacy-preserving video blurring method and apparatus. The method includes the following steps: S1, real-time detection of privacy regions and their types; S2, prediction of the motion trajectory of the privacy regions; S3, dynamic adjustment of the blurring strategy; S4, processing of the privacy regions; S5, fusion and output of the protected video. This invention uses a multimodal privacy recognition model and optical flow trajectory prediction to continuously and accurately locate the privacy information of moving targets. Combined with sensitivity assessment, it balances the strength of privacy protection with the integrity of video information under complex scene changes. It adopts an adaptive blurring algorithm to preserve the natural transition of contours, tracks key feature points only within the privacy region, and automatically expands the blur region for sudden high-speed targets, meeting the response requirements of real-time monitoring scenarios. Through metadata embedding technology, it supports full-link auditing of the privacy processing process. Sensitive scene thresholds and boundary expansion pixel values ​​enable the system to flexibly adapt to different security level scenarios.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, specifically to a privacy-preserving video blurring method and apparatus. Background Technology

[0002] With the widespread application of video surveillance technology in public places, communities, commercial venues and other scenarios, the large amount of video data collected by video surveillance inevitably contains a lot of personal privacy information, such as faces, license plates, ID card numbers, etc.

[0003] Traditional methods of video privacy protection still have certain shortcomings:

[0004] First, traditional video privacy protection methods usually use static area blurring, that is, a fixed blur area is set in advance. In real-world scenarios, the position and posture of people and objects are constantly changing. Static blur areas cannot accurately cover dynamically appearing privacy information, leading to the risk of privacy leakage. Moreover, excessive blurring may affect the identification of other effective information in the video, reducing the use value of surveillance video.

[0005] Secondly, while mainstream blurring methods such as mosaic and Gaussian blur are computationally simple, they are prone to producing a sense of image fragmentation when dealing with complex targets such as faces, thus disrupting the visual continuity of the video. Some deep learning-based blurring techniques improve the naturalness of the effect, but their computational complexity is too high. For example, full-frame target detection cannot meet the millisecond-level response requirements of real-time monitoring.

[0006] Finally, current systems mostly adopt fixed fuzzy strategies, which cannot dynamically adjust the protection strength according to the sensitivity of the scene, such as changing rooms and shopping mall passages. At the same time, the fuzzing process lacks metadata records, making it impossible to trace whether privacy protection operations comply with compliance requirements, thus limiting the application of the technology in highly sensitive scenarios such as security auditing.

[0007] Therefore, it is necessary to design a privacy-preserving video blurring method and device. Summary of the Invention

[0008] The purpose of this invention is to provide a privacy-preserving video blurring method and apparatus to address the problems mentioned in the background art. Traditional video privacy protection methods typically employ static region blurring, i.e., pre-setting fixed blurring areas. In real-world scenarios, the positions and postures of people and objects are constantly changing, and static blurring areas cannot accurately cover dynamically appearing privacy information, leading to privacy leakage risks. Furthermore, excessive blurring may affect the identification of other valid information in the video, reducing the usability of surveillance videos. While mainstream blurring methods such as mosaic and Gaussian blurring are computationally simple, they easily produce a sense of image fragmentation when processing complex targets such as faces, disrupting the visual continuity of the video. Although some deep learning-based blurring techniques improve the naturalness of the effect, their computational complexity is too high. For example, full-frame target detection cannot meet the millisecond-level response requirements of real-time monitoring. Moreover, current systems mostly use fixed blurring strategies, which cannot dynamically adjust the protection strength according to the sensitivity of the scene, such as changing rooms and shopping mall passages. At the same time, the lack of metadata recording in the blurring process makes it impossible to trace whether the privacy protection operation complies with compliance requirements, limiting the application of the technology in highly sensitive scenarios such as security auditing.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] Firstly, a privacy-preserving video blurring method is provided, including the following steps:

[0011] S1. Real-time detection of privacy regions and types in video frames using the multimodal privacy identification model CNN-RNN;

[0012] S2. Predicting motion trajectories in privacy regions based on optical flow analysis and target tracking;

[0013] S3. Dynamically adjust the fuzzy strategy based on the scene semantic segmentation results;

[0014] S4. Adaptive fuzzy algorithm is used to process privacy areas;

[0015] S5. Merge the blurred area with the original video and output the protected video.

[0016] As a further technical solution of the present invention, step S1 includes:

[0017] S1.1, Using a convolutional neural network to extract image spatial features F s :

[0018] F s =CNN(I t )

[0019] Among them, I t For the t-th frame image, the CNN uses a ResNet50 backbone network to extract multi-scale features;

[0020] S1.2 Extracting temporal features F using a recurrent neural network t :

[0021] F t =RNN(F s ,h t-1 )

[0022] Among them, h t-1 To represent the historical hidden states, RNNs use LSTM units to model temporal correlations;

[0023] S1.3, Output the coordinates of the privacy region bounding box B t =(x,y,w,h) and privacy type label c t , where (x,y) are the coordinates of the center point of the privacy region bounding box, and (w,h) are the width and height of the privacy region bounding box.

[0024] As a further technical solution of the present invention, step S2 includes:

[0025] S2.1 Optical flow vector calculation: Calculate the motion vector V of adjacent frame pixels:

[0026] V = LK(I) t ,I t-1 )

[0027] Where LK stands for Lucas-Kanade algorithm;

[0028] S2.2 Update the predicted location of the privacy region using the Kalman filter algorithm.

[0029]

[0030] Where K is the Kalman gain, B pred The position estimate derived from the motion vector V is used to reduce jitter error by Kalman filtering.

[0031] As a further technical solution of the present invention, the fuzzy strategy adjustment in step S3 includes:

[0032] Scene sensitivity score:

[0033] s = SegNet(I t )

[0034] Its SegNet is based on the DeepLabv3+ architecture and outputs scene semantic classification probabilities;

[0035] Based on sensitivity score s and privacy type label c t Determine the fuzzy intensity α:

[0036]

[0037] Where τ is the threshold for sensitive scenarios, which can be configured to adapt to different security levels.

[0038] As a further technical solution of the present invention, step S4 includes:

[0039] S4.1. Apply the GAN blurring algorithm to the privacy region of the face to generate a blurred image I. blur :

[0040] I blur =G(E(I) t B t ))

[0041] Where E is the encoder and G is the generator. The generator G adopts a U-Net structure and retains edge information through skip connections.

[0042] S4.2. A dynamic mosaic algorithm is used for text-based privacy areas, with the mosaic block size being [not specified]. size satisfy:

[0043]

[0044] Where k is a resolution adjustment factor, which is dynamically initialized according to the video resolution.

[0045] As a further technical solution of the present invention, the training objective function of the GAN fuzzy algorithm is:

[0046]

[0047] in, Let E be the loss function. v Let I be the expected value, D be the discriminator, and the discriminator D uses a PatchGAN structure. Through adversarial training, the blurred regions are made to appear visually natural. orig For the original privacy region image, I enc These are encoding features.

[0048] As a further technical solution of the present invention, step S2 also includes:

[0049] When the acceleration a t >a max Emergency blurring activated:

[0050] B′ t =Expand(B t ,Δ)

[0051] Among them, B′ t To expand the bounding box of the privacy region, a maxThe acceleration threshold is Δ, where Δ is the pixel value representing the boundary expansion.

[0052] If ||a t ||2>a max Then B′ t = (x-Δ, y-Δ, w+2Δ, h+2Δ); The acceleration threshold mechanism is used to handle scenarios such as when a target suddenly approaches the camera. Δ can be configured to be 10%-20% of the bounding box width and height.

[0053] As a further technical solution of the present invention, step S5 includes:

[0054] S5.1 Constructing the Gaussian weighted mask matrix M:

[0055]

[0056] Where d is the Euclidean distance from pixel (i,j) to the privacy region bounding box, and σ is the smoothing radius. The smoothing radius σ is positively correlated with the size of the privacy region, thus eliminating the jagged edge effect.

[0057] S5.2, Regional Integration, the integration formula is:

[0058] I out =I blur ·M+I t ·(1-M)

[0059] Among them, I out This is the output image.

[0060] As a further technical solution of the present invention, a metadata storage step is also included:

[0061] Store the fuzzy processing information as triples. <timestamp,B t ,α t It also embeds supplementary and enhanced information fields in the video file header to support privacy audit backtracking, where timestamp is the timestamp of the current frame.

[0062] As a further technical solution of the present invention, in step S2, the optical flow vector calculation is performed only at the Shi-Tomasi corner points within the privacy region, the corner point detection threshold is adaptively adjusted, and feature points in high-texture regions are preferentially selected to improve tracking robustness.

[0063] Secondly, a privacy-preserving video blurring processing device is provided, comprising:

[0064] Privacy recognition module: Configure the CNN-RNN hybrid network and execute step S1 in the first aspect;

[0065] Dynamic analysis module: integrates optical flow calculator, Kalman filter tracker and semantic segmenter, and executes steps S2 and S3 in the first aspect;

[0066] The blur processing module includes a GAN blur engine and a dynamic mosaic engine, and executes step S4 in the first aspect.

[0067] Video fusion module: Configure the Gaussian fusion generator and metadata encoder, and execute step S5 in the first aspect.

[0068] Compared with existing technologies, the beneficial effects of this privacy-preserving video blurring method and apparatus are:

[0069] By leveraging the synergistic effect of the multimodal privacy recognition model CNN-RNN and optical flow trajectory prediction, continuous and accurate localization of privacy information of moving targets, such as faces and license plates, is achieved, overcoming the shortcomings of incomplete coverage of traditional static blurred areas. Combined with the sensitivity assessment of scene semantic segmentation, the blur intensity of different privacy types, such as faces or text, is dynamically adjusted, so that the strength of privacy protection and the integrity of video information can still be balanced under complex scene changes.

[0070] An adaptive blurring algorithm driven by GAN is adopted. The face region is blurred at the feature level through the encoder E and generator G structure, which preserves the natural transition of the contour and avoids the sense of image fragmentation caused by mosaic. At the same time, sparse optical flow calculation and emergency blurring mechanism are introduced to track key feature points only in privacy areas and automatically expand the blur area for sudden high-speed targets, which significantly reduces the computational overhead and meets the millisecond response requirements of real-time monitoring scenarios.

[0071] By using metadata embedding technology, the timestamp, the coordinates of the fuzzy privacy region bounding box B are embedded. t and fuzzy intensity parameter α t It stores data synchronously with the video stream and supports end-to-end auditing of the privacy processing process. Combined with user-configurable sensitive scene thresholds τ and boundary extension pixel values ​​Δ, the system can flexibly adapt to different security levels in public places and private areas, improving the practicality and deployment applicability of the technical solution. Attached Figure Description

[0072] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0073] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] Please see the appendix Figure 1 The present invention provides an embodiment of a privacy-preserving video blurring method, comprising the following steps:

[0075] S1. Real-time detection of privacy regions and types in video frames using the multimodal privacy identification model CNN-RNN;

[0076] S1.1, Using a convolutional neural network to extract image spatial features F s :

[0077] F s =CNN(I t )

[0078] Among them, I t For the t-th frame image, the CNN uses a ResNet50 backbone network to extract multi-scale features;

[0079] S1.2 Extracting temporal features F using a recurrent neural network t :

[0080] F t =RNN(F s ,h t-1 )

[0081] Among them, h t-1 To represent the historical hidden states, RNNs use LSTM units to model temporal correlations;

[0082] S1.3, Output the coordinates of the privacy region bounding box B t =(x,y,w,h) and privacy type label c t , where (x,y) are the coordinates of the center point of the privacy region bounding box, and (w,h) are the width and height of the privacy region bounding box;

[0083] S2. Predicting motion trajectories in privacy regions based on optical flow analysis and target tracking;

[0084] S2.1 Optical flow vector calculation: Calculate the motion vector V of adjacent frame pixels:

[0085] V = LK(I) t ,I t-1 )

[0086] Among them, LK is the Lucas-Kanade algorithm, in which optical flow vector calculation is performed only at Shi-Tomasi corners within the privacy region, sparse optical flow tracking is performed, corner detection threshold is adaptively adjusted, and feature points in high-texture regions are selected first to improve tracking robustness;

[0087] S2.2 Update the predicted location of the privacy region using the Kalman filter algorithm.

[0088]

[0089] Where K is the Kalman gain, B pred Kalman filtering is used to reduce jitter error for the position estimate derived from the motion vector V;

[0090] Also includes:

[0091] When the acceleration a t >a max Emergency blurring activated:

[0092] B′ t =Expand(B t ,Δ)

[0093] Among them, B′ t To expand the bounding box of the privacy region, a max The acceleration threshold is Δ, where Δ is the pixel value representing the boundary expansion.

[0094] If ||a t ||2>a max Then B′ t = (x-Δ, y-Δ, w+2Δ, h+2Δ); The acceleration threshold mechanism is used to handle scenarios such as when a target suddenly approaches the camera. Δ can be configured to be 10%-20% of the bounding box width and height.

[0095] S3. Dynamically adjust the fuzzy strategy based on the scene semantic segmentation results; the fuzzy strategy adjustment includes:

[0096] Scene sensitivity score:

[0097] s = SegNet(I t )

[0098] Its SegNet is based on the DeepLabv3+ architecture and outputs scene semantic classification probabilities;

[0099] Based on sensitivity score s and privacy type label c t Determine the fuzzy intensity α:

[0100]

[0101] Where τ is the threshold for sensitive scenarios, which can be configured to adapt to different security levels;

[0102] S4. Adaptive fuzzy algorithm is used to process privacy areas;

[0103] S4.1. Apply the GAN blurring algorithm to the privacy region of the face to generate a blurred image I. blur :

[0104] I blur =G(E(I) t B t ))

[0105] Where E is the encoder and G is the generator. The generator G adopts a U-Net structure and retains edge information through skip connections.

[0106] The training objective function of the GAN fuzzy algorithm is:

[0107]

[0108] in, Let E be the loss function. v Let I be the expected value, D be the discriminator, and the discriminator D uses a PatchGAN structure. Through adversarial training, the blurred regions are made to appear visually natural. orig For the original privacy region image, I enc For encoding features;

[0109] S4.2. A dynamic mosaic algorithm is used for text-based privacy areas, with the mosaic block size being [not specified]. size satisfy:

[0110]

[0111] Where k is a resolution adjustment factor, which is dynamically initialized according to the video resolution;

[0112] S5. Merge the blurred area with the original video and output the protected video;

[0113] S5.1 Constructing the Gaussian weighted mask matrix M:

[0114]

[0115] Where d is the Euclidean distance from pixel (i,j) to the privacy region bounding box, and σ is the smoothing radius. The smoothing radius σ is positively correlated with the size of the privacy region, thus eliminating the jagged edge effect.

[0116] S5.2, Regional Integration, the integration formula is:

[0117] I out =I blur ·M+I t ·(1-M)

[0118] Among them, I out To output the image;

[0119] It also includes the metadata storage step:

[0120] Store the fuzzy processing information as triples. <timestamp,B t ,α t It also embeds supplementary and enhanced information fields in the video file header to support privacy audit backtracking, where timestamp is the timestamp of the current frame;

[0121] One embodiment of the present invention provides a privacy-preserving video blurring processing device, comprising:

[0122] Privacy recognition module: Configure the CNN-RNN hybrid network and execute step S1;

[0123] Dynamic analysis module: integrates optical flow calculator, Kalman filter tracker and semantic segmenter, and executes steps S2 and S3;

[0124] The blur processing module includes a GAN blur engine and a dynamic mosaic engine, and executes step S4;

[0125] Video fusion module: Configure the Gaussian fusion generator and metadata encoder, and execute step S5.

[0126] In summary, by combining the multimodal privacy recognition model CNN-RNN with optical flow trajectory prediction, we can achieve continuous and accurate localization of privacy information of moving targets, such as faces and license plates, overcoming the shortcomings of incomplete coverage of traditional static blurred areas. By combining the sensitivity assessment of scene semantic segmentation, we can dynamically adjust the blur intensity of different privacy types, such as faces or text, and still balance the strength of privacy protection and the integrity of video information under complex scene changes.

[0127] An adaptive blurring algorithm driven by GAN is adopted. The face region is blurred at the feature level through the encoder E and generator G structure, which preserves the natural transition of the contour and avoids the sense of image fragmentation caused by mosaic. At the same time, sparse optical flow calculation and emergency blurring mechanism are introduced to track key feature points only in privacy areas and automatically expand the blur area for sudden high-speed targets, which significantly reduces the computational overhead and meets the millisecond response requirements of real-time monitoring scenarios.

[0128] By using metadata embedding technology, the timestamp, the coordinates of the fuzzy privacy region bounding box B are embedded. t and fuzzy intensity parameter α t It stores data synchronously with the video stream and supports end-to-end auditing of the privacy processing process. Combined with user-configurable sensitive scene thresholds τ and boundary extension pixel values ​​Δ, the system can flexibly adapt to different security levels in public places and private areas, improving the practicality and deployment applicability of the technical solution.

[0129] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A privacy-preserving video blurring method, characterized in that: Includes the following steps: S1. Through a multimodal privacy identification model - Real-time detection of privacy regions and their types in video frames; S2. Predicting motion trajectories in privacy regions based on optical flow analysis and target tracking; S3. Dynamically adjust the fuzzy strategy based on the scene semantic segmentation results; S4. Adaptive fuzzy algorithm is used to process privacy areas; S5. Merge the blurred area with the original video and output the protected video; Step S1 includes: S1.1 Using convolutional neural networks to extract spatial features from images : in, For the first Frame image; S1.2 Extracting temporal features using a recurrent neural network : in, It is in a state of historical concealment; S1.3 Output the coordinates of the privacy region bounding box and privacy type tags ,in The coordinates of the center point of the privacy region bounding box. Define the width and height of the privacy region's bounding box; Step S2 includes: S2.1 Optical flow vector calculation: Calculate the motion vectors of pixels in adjacent frames. : in, The Lucas-Kanade algorithm; S2.2 Update the predicted location of the privacy region using the Kalman filter algorithm. : in, For Kalman gain, Motion vector Derived position estimate; The fuzzy strategy adjustment in step S3 includes: Scene sensitivity score: Based on sensitivity score Privacy type tags Determine the fuzzy intensity : in, Threshold for sensitive scenarios; Step S4 includes: S4.1, Adopting privacy measures for facial recognition areas Blur algorithm to generate blurred images : in, For encoder, For generator; S4.

2. A dynamic mosaic algorithm is used for text-based privacy areas, and the size of the mosaic blocks is... satisfy: in, This is a resolution adjustment factor; The The training objective function of the fuzzy algorithm is: in, For loss function, For the expected value, For discriminator, For the original privacy region image, For encoding features; Step S2 also includes: When the acceleration of motion Emergency blurring activated: in, To expand the bounding box of the privacy region, For acceleration threshold, Extend pixel values ​​to the boundary; like ,but ; Step S5 includes: S5.1 Constructing a Gaussian-weighted mask matrix : in, For pixels Euclidean distance to the boundary of the privacy zone The smoothing radius; S5.2, Regional Integration, the integration formula is: in, To output the image; The privacy-preserving video blurring method further includes a metadata storage step: Store the fuzzy processing information as triples. And embed it in the video file header, where The timestamp of the current frame; In step S2, the optical flow vector calculation is performed only at the Shi-Tomasi corner point within the privacy region, where sparse optical flow tracing is performed.

2. A processing apparatus for executing the privacy-preserving video blurring method of claim 1, characterized in that, include: Privacy recognition module: Configuration Hybrid networks; Dynamic analysis module: integrates optical flow calculator, Kalman filter tracker and semantic segmenter; Fuzzing module: includes Blur engine and dynamic mosaic engine; Video fusion module: Configure Gaussian fusion and metadata encoder.

Images