License plate and face combined desensitization method and system

Abstract

The invention relates to the technical field of image privacy processing, in particular to a license plate and face combined desensitization method and system.The license plate and face combined desensitization method comprises the steps that two processing heads are used for conducting detection and feature extraction on a license plate and a face in original image data respectively, and then the position relation and similarity of a license plate area and a face area are calculated; judging whether the original image data belong to the same main body or not, and formulating a joint desensitization strategy of the original image data according to an application scene and a privacy protection requirement; and the two processing heads respectively execute a joint desensitization strategy at the same time to carry out desensitization processing, and finally, image quality inspection is carried out and then output is carried out. It can be understood that according to the technical scheme, the two processing heads conduct detection, feature extraction and desensitization processing at the same time, the processing efficiency is high, whether the two processing heads belong to the same main body or not can be jointly judged according to the position relation and the similarity, the misjudgment situation is reduced, desensitization processing is more accurate, and due to the modular design of the double-head system, the reliability of the system is improved. And the method has better expansibility and adaptability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image privacy processing, in particular to a license plate and face joint desensitization method and system. BACKGROUND

[0002] At present, image data is widely used in many fields such as security monitoring, traffic management, intelligent access control, etc. However, the license plate and face information contained in the image belong to sensitive personal information, and once leaked, it may lead to serious consequences such as personal privacy leakage, identity theft, and illegal use of vehicle information.

[0003] Common image desensitization technologies mostly use single-head processing, that is, a single processing unit or algorithm process is used to detect, extract features, perform correlation analysis and desensitize the license plate and face information in the image in sequence. The single-head processing system cannot fully utilize hardware resources when facing a large amount of image data due to the sequential execution of all processing steps, resulting in slow overall processing speed and difficulty in meeting the real-time requirements of application scenarios such as real-time monitoring data processing at traffic intersections and in-vehicle data desensitization. The correlation analysis method for license plates and faces is relatively simple, usually only relying on a small amount of information such as position coordinates to determine whether they belong to the same subject. In complex scenarios (such as multiple people riding in a car, multiple cars intersecting, etc.), it is easy to misjudge, and thus lead to incomplete desensitization or over-desensitization. The architecture of the single-head system is relatively fixed, and when new detection algorithms need to be introduced, the processing process needs to be optimized, or different formats of image data need to be adapted, the system modification is difficult and costly, and it is difficult to quickly adapt to technological development and diverse application needs.

[0004] Therefore, the current image desensitization technology has the defects of low processing efficiency, insufficient precision, and poor scalability. SUMMARY

[0005] Therefore, the present application provides a license plate and face joint desensitization method and system to solve the problems of low processing efficiency, insufficient precision, and poor scalability of the image desensitization technology in the prior art.

[0006] According to a first aspect of an embodiment of the present application, a license plate and face joint desensitization method is provided, comprising: obtaining original image data containing a license plate and a face, and distributing the original image data in parallel to a first processing head and a second processing head; The first processing head detects the license plate region in the original image data and extracts the license plate feature vector, while the second processing head detects the face region in the original image data and extracts the face feature vector; Calculate the positional relationship between the license plate region and the face region in the original image data; input the license plate feature vector and the face feature vector into a pre-trained association classification model, and output a similarity between the license plate feature vector and the face feature vector; According to the position relationship and the similarity, a subject judgment result of whether the license plate region and the face region belong to the same subject is obtained; According to the application scene and privacy protection requirement of the original image data and the subject judgment result, a joint desensitization strategy of the original image data is formulated; The first processing head desensitizes the license plate region according to the joint desensitization strategy, and the second processing head desensitizes the face region according to the joint desensitization strategy, to obtain a desensitized image; The desensitized image is subjected to image quality inspection, to generate a final image and output.

[0007] Preferably, the first processing head detects the license plate region in the original image data and extracts a license plate feature vector, including: The first processing head detects the license plate region in the original image data by using a first improved YOLOv7 algorithm; the first improved YOLOv7 algorithm is improved by using an ELAN-H structure and a dynamic path aggregation network on the basis of an original algorithm; The first processing head extracts the license plate feature vector by using an improved ResNet-18 convolutional neural network; the improved ResNet-18 convolutional neural network is improved by adding a spatial pyramid pooling layer on the basis of an original network structure.

[0008] Preferably, the second processing head detects the face region in the original image data and extracts a face feature vector, including: The second processing head detects the face region in the original image data by using a second improved YOLOv7 algorithm; the second improved YOLOv7 algorithm is obtained by re-clustering an anchor frame of an original algorithm according to face detection characteristics; The second processing head extracts the face feature vector by using a FaceNet model based on an Inception-ResNet-v1 architecture; the FaceNet model is trained by constructing a three-tuple loss function including an anchor point, a positive sample and a negative sample during training.

[0009] Preferably, the position relationship between the license plate region and the face region in the original image data is calculated, including: The first processing head generates license plate region coordinate data when detecting the license plate region in the original image data; the second processing head generates face region coordinate data when detecting the face region in the original image data; According to the license plate region coordinate data and the face region coordinate data, a Euclidean distance between a license plate region center coordinate and a face region center coordinate in the original image data is calculated. According to the license plate region coordinate data and the face region coordinate data, a bearing angle of the license plate region relative to the face region is calculated. The Euclidean distance and the bearing angle are integrated as a position relationship of the license plate region and the face region.

[0010] Preferably, the license plate and face joint desensitization method further comprises: The association classification model is constructed based on a multi-layer perception machine structure, contains three hidden layers, the number of neurons of each layer is 256, 128 and 64 respectively, and the activation function adopts ReLU.

[0011] Preferably, the joint desensitization strategy of the original image data comprises: If the license plate region and the face region belong to the same subject, a hybrid blurring processing method based on bilateral filtering and Gaussian blur is used for the license plate region, and a deep mosaic processing is used for the face region. If the license plate region and the face region do not belong to the same subject, characters in the license plate region are replaced with a fixed virtual character sequence, and a Gaussian blur algorithm is used for blurring processing on the face region.

[0012] Preferably, when the first processing head and the second processing head perform desensitization processing, further comprising: According to the image size of the original image data, the number of threads of the first processing head or the second processing head is dynamically adjusted.

[0013] Preferably, the license plate and face joint desensitization method further comprises: According to the application scenario and privacy protection requirement of the original image data, a desensitization strength parameter in the joint desensitization strategy is dynamically generated.

[0014] Preferably, when the original image data is distributed to the first processing head and the second processing head in parallel, further comprising: After the sending end adds a check code to the original image data, the original image data is distributed to the first processing head and the second processing head in parallel; The first processing head or the second processing head performs check comparison on the original image data, and if the check fails, a retransmission request is sent to the sending end.

[0015] According to the second aspect of the embodiment of the application, a license plate and face joint desensitization system is provided, characterized in that comprising: An image acquisition module is configured to acquire original image data containing a license plate and a face, and distribute the original image data to a first processing head and a second processing head in parallel. The first processing head is used to detect the license plate region in the original image data and extract the license plate feature vector; The second processing head is used to detect the face region in the original image data and extract the face feature vector; The region association analysis module is used to calculate the positional relationship between the license plate region and the face region in the original image data; input the license plate feature vector and the face feature vector into a pre-trained association classification model, and output the similarity between the license plate feature vector and the face feature vector; based on the positional relationship and the similarity, determine whether the license plate region and the face region belong to the same subject. The joint desensitization strategy formulation module is used to formulate a joint desensitization strategy for the original image data based on the application scenario and privacy protection requirements of the original image data, as well as the subject judgment result. The first processing head is further configured to perform desensitization processing on the license plate area according to the joint desensitization strategy; the second processing head is further configured to perform desensitization processing on the face area according to the joint desensitization strategy to obtain a desensitized image. The output module is used to perform image quality checks on the desensitized image, generate the final image, and output it.

[0016] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: It is understood that the technical solution presented in this invention utilizes two processing heads to detect and extract features from the license plate and face in the original image data, respectively. Then, it calculates the positional relationship and similarity between the license plate region and the face region to determine whether they belong to the same subject. Next, based on the application scenario and privacy protection requirements, a joint desensitization strategy is formulated for the original image data. The two processing heads simultaneously execute the joint desensitization strategy for desensitization processing, and finally, the image is output after image quality checks. It is understood that the technical solution presented in this invention, with two processing heads simultaneously performing detection, feature extraction, and desensitization processing, has high processing efficiency. It can also jointly determine whether they belong to the same subject based on positional relationship and similarity, reducing false positives and making the desensitization processing more accurate. The modular design of the dual-head system has better scalability and adaptability.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0019] Figure 1This is a schematic diagram illustrating the steps of a method for jointly desensitizing license plates and faces according to an exemplary embodiment; Figure 2 This is a schematic block diagram illustrating a license plate and face desensitization system according to an exemplary embodiment; Figure 3 This is a schematic diagram of a dual-head parallel detection network structure according to an exemplary embodiment; Figure 4 This is a schematic diagram of a cross-modal collaborative attention association network architecture according to an exemplary embodiment. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0021] In one embodiment, Figure 1 This is a schematic diagram illustrating the steps of a method for jointly desensitizing license plates and faces according to an exemplary embodiment. See also... Figure 1 This provides a method for jointly de-identifying license plates and faces, including: Step S11: Obtain raw image data containing license plates and faces, and distribute the raw image data in parallel to the first processing head and the second processing head.

[0022] Image acquisition devices (such as high-definition cameras) can be used to collect raw image data, including license plates and faces, in scenarios such as traffic intersections, parking lots, and road traffic. The acquired images are simultaneously transmitted to the first and second processing heads of the dual-head system via a data transmission channel.

[0023] In practice, the choice of image acquisition equipment directly affects the quality of the raw image data. To ensure the integrity and clarity of license plate and facial information, the system prioritizes industrial-grade high-definition cameras with a resolution of at least 4K and a frame rate of 30fps or higher. During actual deployment, differentiated installation strategies are adopted for different application scenarios: at traffic intersections, the camera is installed at a height of 5-6 meters with a tilt angle of 15-25 degrees to ensure coverage of vehicles and drivers in multiple lanes; at parking lot entrances and exits, the camera height is controlled at 2-3 meters, vertically downwards or at a small tilt angle to acquire close-up license plate and facial images; inside vehicles, data is acquired using the vehicle's front-facing camera.

[0024] The acquired raw image data is distributed through a high-speed data transmission channel using Gigabit Ethernet based on the TCP / IP protocol to ensure stable data transmission and low latency.

[0025] In a preferred embodiment, to prevent data loss or damage during transmission, the sending end adds a checksum to the original image data and distributes it in parallel to the first processing head and the second processing head; the first processing head or the second processing head performs a checksum comparison on the original image data, and if the checksum fails, it sends a retransmission request to the sending end.

[0026] Step S12: The first processing head detects the license plate region in the original image data and extracts the license plate feature vector. At the same time, the second processing head detects the face region in the original image data and extracts the face feature vector.

[0027] This step involves parallel detection and feature extraction using dual processing heads.

[0028] The first processing head performs license plate target detection and feature extraction.

[0029] The first processing head uses a first improved YOLOv7 algorithm to detect license plate regions in the original image data. This improved YOLOv7 algorithm is derived from the original algorithm by incorporating an ELAN-H structure and a dynamic path aggregation network. Compared to the traditional YOLO algorithm, it significantly improves both the accuracy and speed of license plate detection. For training, publicly available license plate datasets (such as CCPD-2019) are used, combined with actual collected license plate data from specific scenarios for hybrid training. The batch size is set to 64, the initial learning rate is 0.001, and a cosine annealing learning rate decay strategy is employed. After 300 epochs of training, the model achieves over 95% mAP (mean accuracy) on the test set.

[0030] The first processing head extracts license plate feature vectors using an improved ResNet-18 convolutional neural network. This improved ResNet-18 convolutional neural network adds a spatial pyramid pooling layer to the original network structure, effectively addressing the issue of inconsistent input image sizes. The first three convolutional layers use 3×3 kernels with a stride of 1 and padding of 1, and LeakyReLU activation is selected to mitigate the gradient vanishing problem. After extracting low-level features such as edges, shapes, and textures of the license plate characters through the convolutional layers, downsampling is performed using a max-pooling layer to reduce the feature dimensionality. Finally, after processing through two fully connected layers, the features are mapped to a license plate feature vector of length 128.

[0031] The second processing head performs face target detection and feature extraction.

[0032] The second processing head utilizes a second improved YOLOv7 algorithm to detect face regions in the original image data. This improved YOLOv7 algorithm, based on the original algorithm, re-clusters the anchor boxes according to face detection characteristics to adapt to face detection requirements at different poses and scales. For data augmentation, in addition to conventional flipping and scaling operations, a MixUp data augmentation technique is introduced, mixing two face images at a certain ratio to generate new training samples, enhancing the model's generalization ability. During training, the batch size is set to 32, and the AdamW adaptive learning rate algorithm is used with an initial learning rate of 0.0001. After 50 epochs of training, the model achieves a detection accuracy of over 98% on the WIDER FACE test set.

[0033] The second processing head extracts facial feature vectors using the FaceNet model based on the Inception-ResNet-v1 architecture. During training, the FaceNet model is trained by constructing a triplet loss function that includes anchor points, positive samples, and negative samples. This ensures that facial features of people with the same identity are closer together in the feature space, while facial features of people with different identities are farther apart. The model outputs a facial feature vector with a dimension of 512, which contains key feature information such as facial structure, contour shape, and texture details.

[0034] Step S13: Calculate the positional relationship between the license plate region and the face region in the original image data.

[0035] It should be noted that when the first processing head detects the license plate area in the original image data, it generates license plate area coordinate data; when the second processing head detects the face area in the original image data, it generates face area coordinate data.

[0036] First, geometric position calculation is performed: based on the license plate area coordinate data and the face area coordinate data, the Euclidean distance between the center coordinates of the license plate area and the center coordinates of the face area in the original image data is calculated.

[0037] Assume the coordinates of the bottom left and top right corners of the license plate area are (x1, y1) and (x2, y2) respectively. Then the coordinates of the center of the license plate area are: Similarly, the coordinates of the center of the face region are... .

[0038] Euclidean distance It is calculated using the following formula:

[0039] Based on the coordinate data of the license plate area and the facial area, calculate the azimuth angle of the license plate area relative to the facial area. To determine the relative positional relationship between the two.

[0040] The Euclidean distance and the azimuth angle are integrated to form the positional relationship between the license plate area and the face area.

[0041] Step S14: Input the license plate feature vector and the face feature vector into the pre-trained association classification model, and output the similarity between the license plate feature vector and the face feature vector.

[0042] In one embodiment, the association classification model is constructed based on a multilayer perceptron structure, comprising three hidden layers with 256, 128, and 64 neurons in each layer, respectively, and ReLU is used as the activation function.

[0043] During training, a large amount of image data with labeled license plate-face associations was used. The ratio of positive samples (license plate and face belong to the same subject) to negative samples (license plate and face do not belong to the same subject) in the dataset was 1:3. The model was optimized using the cross-entropy loss function. After training, the accuracy on the validation set reached over 92%.

[0044] Step S15: Based on the positional relationship and the similarity, determine whether the license plate area and the face area belong to the same subject.

[0045] Step S16: Based on the application scenario and privacy protection requirements of the original image data, as well as the subject judgment result, formulate a joint desensitization strategy for the original image data.

[0046] Preferably, the combined desensitization strategy can support users to customize the configuration according to their actual needs.

[0047] For example, if the license plate and the face belong to the same subject, in a public security monitoring scenario, a hybrid blurring method based on bilateral filtering and Gaussian blurring is used for the license plate area, while deep mosaic processing is used for the face area. For the license plate area processing, a bilateral filtering algorithm is first used to process the license plate characters, with parameters set as follows: spatial Gaussian kernel standard deviation. Grayscale Gaussian kernel standard deviation To achieve a blurring effect while preserving the edges, Gaussian blur is then applied with a blur kernel size of (9⁹) to ensure that the license plate characters are completely unrecognizable. Deep mosaic processing is applied to the face region, dividing it into multiple (10⁹) pixel squares. The pixel values ​​within each square are then averaged to completely blur the facial features.

[0048] For example, for license plate areas used only for traffic flow statistics, if the license plate area and the face area do not belong to the same subject, the characters in the license plate area are replaced with a fixed virtual character sequence such as "*******"; the face area is blurred using a Gaussian blur algorithm. The blur kernel size is set to (5⁵) to retain a certain degree of image recognizability while protecting privacy.

[0049] Simultaneously, the method can automatically adjust the desensitization parameters based on factors such as image resolution and color mode to achieve the best desensitization effect. Preferably, it can also dynamically generate desensitization intensity parameters in the joint desensitization strategy according to the application scenario and privacy protection requirements of the original image data.

[0050] Step S17: The first processing head performs desensitization processing on the license plate area according to the joint desensitization strategy, and at the same time, the second processing head performs desensitization processing on the face area according to the joint desensitization strategy to obtain a desensitized image.

[0051] In practice, the first and second processing heads use the OpenCV image processing library to perform precise desensitization processing on the license plate and face regions respectively, based on a joint desensitization strategy. To improve processing efficiency, multi-threading technology is employed, dynamically allocating the number of threads according to the image size. When the image width is less than 1920 pixels, four threads are used for parallel processing; when the image width is greater than or equal to 1920 pixels, eight threads are used for processing.

[0052] For the character replacement operation in the license plate area, the coordinate information of the license plate area is obtained to locate the license plate area in the original image. Then, OpenCV's image editing functions are used to replace the pixel values ​​corresponding to the license plate characters with the pixel values ​​of the virtual characters. During blurring, the parameters specified in the joint desensitization strategy are strictly followed, and OpenCV's Gaussian blur and bilateral filtering functions are called for processing.

[0053] The desensitization of the face region is also performed according to the same strategy. During the mosaic processing, each small square in the face region is traversed in a loop, the average pixel value within the square is calculated, and all pixel values ​​within that square are updated to the average value. After processing, the results of the license plate and face regions are fused with the other parts of the original image to generate a complete desensitized image.

[0054] Step S18: Perform an image quality check on the desensitized image, generate the final image, and output it.

[0055] In practice, image quality checks include verifying whether the desensitized areas completely cover sensitive information and checking for image distortion or damage. If the check passes, the image is saved in JPEG format to a designated storage device using a distributed file system (such as Ceph) to improve data storage reliability and scalability. If real-time transmission is required, the image is transmitted to terminal devices, such as monitoring center displays and data analysis platforms, via a network interface based on the HTTP / 2 protocol.

[0056] It is understood that the technical solution presented in this invention, with its dual-head parallel processing architecture, fully utilizes hardware resources to process license plate and facial information simultaneously. Compared to the serial processing of a single-head system, the processing speed is significantly improved, meeting the needs of applications with high real-time requirements. The deep correlation analysis method combines multiple feature information and complex models to determine the correlation between license plates and faces, reducing false positives, making desensitization processing more accurate, and effectively protecting sensitive information. The modular design of the dual-head system means that when introducing new detection algorithms, optimizing processing flows, or adapting to different image data formats, only the corresponding processing head needs adjustment, reducing system upgrade and maintenance costs and providing better scalability and adaptability. In another embodiment, see Figure 2 A system for desensitizing license plates and faces is provided, characterized by comprising: The image acquisition module is used to acquire raw image data containing license plates and faces, and distribute the raw image data in parallel to the first processing head and the second processing head.

[0057] The first processing head is used to detect the license plate region in the original image data and extract the license plate feature vector.

[0058] The second processing head is used to detect the face region in the original image data and extract the face feature vector.

[0059] The region association analysis module is used to calculate the positional relationship between the license plate region and the face region in the original image data; input the license plate feature vector and the face feature vector into a pre-trained association classification model, and output the similarity between the license plate feature vector and the face feature vector; based on the positional relationship and the similarity, determine whether the license plate region and the face region belong to the same subject.

[0060] The joint desensitization strategy formulation module is used to formulate a joint desensitization strategy for the original image data based on the application scenario and privacy protection requirements of the original image data, as well as the subject judgment result. The first processing head is further configured to perform desensitization processing on the license plate area according to the joint desensitization strategy; the second processing head is further configured to perform desensitization processing on the face area according to the joint desensitization strategy to obtain a desensitized image.

[0061] The output module is used to perform image quality checks on the desensitized image, generate the final image, and output it.

[0062] The first and second processing heads together constitute a dual-head parallel detection network, which is a dual-head parallel detection architecture based on improved YOLOv8-Attention. See [link to relevant documentation]. Figure 3 The first processing head is the license plate detection branch, which uses a YOLOv8 backbone network combined with a Small Object Enhancement Module (SOEM). This module achieves multi-scale receptive field fusion by introducing Spatial Pyramid Pooling with Diffused Space (ASPP), covering a receptive field range from 3×3 to 21×21. The second processing head is the face detection branch: it uses a YOLOv8 backbone network combined with a Multi-Scale Attention Module (MSAM). This module enhances feature representation capabilities through a cascaded structure of channel attention and spatial attention.

[0063] Preferably, the dual-head parallel detection network adopts a shared backbone network + dual-branch neck network architecture. It extracts common low-level features (edges, textures, and colors) by sharing the first three convolutional layers (C1-C3), and then separates them into a license plate-specific path (enhancing small-scale features) and a face-specific path (enhancing multi-scale features).

[0064] The regional association analysis module features a pre-designed cross-modal collaborative attention association network, see [link to relevant documentation]. Figure 4 A cross-modal collaborative attention network based on the Transformer architecture is used to achieve accurate license plate-face association. First, multi-head self-attention feature encoding is performed, and the license plate feature vector and face feature vector are respectively encoded by position and then input into the multi-head self-attention encoder. Then, cross-modal collaborative attention calculation is performed, and a cross-attention mechanism for license plate features and infrared face features is designed. Finally, multi-dimensional association feature fusion is performed, combining spatial position relationship features, appearance similarity features and semantic association features, and the final association score is calculated through an adaptive fusion network.

[0065] Preferably, the cross-modal collaborative attention association network introduces a contrastive learning mechanism to enhance the discriminative ability of association judgment. It constructs positive sample pairs (license plate-face pairs of the same subject) and negative sample pairs (different subjects) and uses the InfoNCE loss function to optimize the feature space.

[0066] Meanwhile, to meet the real-time reasoning needs of edge devices, the cross-modal collaborative attention association network uses knowledge distillation technology to distill the complex CMCAN teacher network (85M parameters) into a lightweight student network (12M parameters).

[0067] The distillation loss function is: L KD =αL CE (y s y true )+β·KL(σ(z s / T), σ(z) t / T))+γL feat (F s F t ) Among them, y s and y true These are student online outputs and real labels, respectively. s and z t Here, σ represents the logits of the student and teacher networks, σ is the softmax function, T is the temperature coefficient (set to T=4), KL represents the KL divergence, and L_ feat The feature layer imitation loss is α=0.3, β=0.5, and γ=0.2.

[0068] In the joint desensitization strategy development module, a five-level privacy risk assessment model is designed, comprehensively considering scenario sensitivity, data usage, and legal compliance requirements to dynamically generate desensitization strategies. Risk scores are calculated based on a multi-factor weighted model; and desensitization parameters are determined using piecewise linear mapping based on the risk scores.

[0069] During the desensitization process in the first and second processing heads, reversible information embedding technology can also be used to retain the encrypted hash value of the original sensitive area while desensitizing, supporting reversible recovery under authorized conditions. For the detected license plate and face ROI regions, encrypted hashes are calculated; depending on the desensitization strength, Gaussian blur, mosaicking, or complete occlusion are used for adaptive desensitization rendering; the encrypted hash is embedded as an invisible watermark into the desensitized image, using cascaded embedding of Discrete Wavelet Transform (DWT) and Discrete Cosine Transform (DCT), followed by integrity verification.

[0070] Finally, the output module also integrates a real-time quality monitoring module to evaluate the quality of the desensitization results, including desensitization integrity detection (ensuring no sensitive areas are missed), visual naturalness assessment (avoiding excessive distortion), and reversibility verification (ensuring the accuracy of authorized restoration).

[0071] Furthermore, the system supports multi-GPU distributed parallel inference, employing a hybrid strategy of model parallelism and data parallelism: the dual-head detection network is deployed on GPU0 and GPU1 respectively for model parallelism, while the association network uses data parallelism to process batch data on multiple GPUs. Efficient gradient synchronization and feature transmission are achieved through the NCCL communication library, with a theoretical throughput of up to 120 FPS@1080P.

[0072] This technical solution significantly improves processing efficiency. Through a dual-head parallel architecture and GPU acceleration, the single-frame processing latency is reduced from 120-150ms in traditional methods to 25-35ms, increasing processing speed by 3-5 times. In 1080P@30FPS video streams, CPU utilization is reduced from 85% to 45%, and GPU utilization is increased from 62% to 89%, meeting real-time requirements. The accuracy of association judgment is significantly improved: by introducing a cross-modal collaborative attention mechanism and contrastive learning, the accuracy of license plate-face association judgment is increased from 75-82% in traditional methods to 96.5%. In complex scenes (multiple passengers, occlusion, extreme angles), the false positive rate is reduced from 18-25% to 3.5%, with a precision of 97.2% and a recall of 95.8%. Significantly Enhanced Detection Robustness: Through a small target enhancement module and a multi-scale attention mechanism, the mAP for license plate detection in long-distance (30-50 meters) scenarios is improved from 82.3% to 94.8%, and the mAP for face detection is improved from 88.6% to 98.2%. Under low-light conditions (<50 lux), the detection accuracy loss is reduced from 28% to 9% compared to traditional methods. Flexible and Controllable Desensitization Strategy: The privacy risk awareness module supports five levels of dynamic adjustment of desensitization intensity. Compared to fixed desensitization methods, while ensuring privacy protection (k-anonymity k≥100), the image availability (SSIM) is improved from 0.42 to 0.68. This retains more analyzable features in internal audit scenarios and provides stronger protection in publicly released scenarios. Support for Controllable and Reversible Recovery: Through reversible information embedding and encrypted hash watermarking technology, accurate recovery of original sensitive information is supported under authorized conditions (recovery accuracy rate 99.7%), meeting the needs of judicial evidence collection and audit traceability. Simultaneously, it ensures that unauthorized users cannot recover the information, and security is guaranteed through HMAC-SHA256. Highly flexible in deployment: By compressing model parameters from 85M to 12M through knowledge distillation, the inference speed on edge devices (NVIDIA Jetson AGX Xavier) reaches 35 FPS, which is 133% higher than the teacher network (15 FPS) with only a 1.2% loss in accuracy. It supports flexible deployment in the cloud and at the edge.

[0073] Taking intelligent traffic monitoring as an example, the following explanation will be provided: The system is deployed at a traffic intersection, collecting 4K@30FPS high-definition video streams, and requires real-time desensitization processing of vehicles and drivers passing through the intersection.

[0074] System hardware configuration: Server configuration: Dual Intel Xeon Gold 6248R processors (48 cores, 96 threads), 256GB DDR4 ECC memory, 4×NVIDIA A100 40GB GPUs, using NVLink interconnect to achieve 400GB / s bidirectional bandwidth between GPUs; Storage system: 2TB NVMe SSD (7000 / 5000 MB / s read / write speeds) for hot data caching, 48TB HDD RAID6 array for historical data archiving; Network: 10 Gigabit Ethernet interface, supporting RDMA protocol for ultra-low latency data transmission.

[0075] Detailed implementation of the dual-head parallel detection network: Shared backbone network: CSPDarknet-53 is used as the shared backbone network, with an input image size of 640×640×3. The first three layers (C1-C3) of the backbone network extract common low-level features: C1 layer: 3×3 convolutional kernels, stride 2, 64 output channels, feature map size 320×320×64; C2 layer: contains 4 residual modules, 128 output channels, feature map size 160×160×128; C3 layer: contains 8 residual modules, 256 output channels, feature map size 80×80×256.

[0076] License plate detection branch (processing head A): Separated from the output of layer C3, a Small Object Enhancement Module (SOEM) is introduced; Spatial Pyramid Pooling with Dilation Rates (ASPP): 3×3 convolutions with dilation rates r=6, 12, and 18 are used in parallel, each convolution outputting 256 channels. These are then combined with 1×1 convolutions (outputting 256 channels) and global average pooling followed by upsampling (outputting 256 channels), resulting in a total of 5 branches concatenated to obtain a 1280-channel feature map. This map is then reduced to 256 channels using 1×1 convolutions; Feature Pyramid Network (FPN) The top-down path, combined with lateral connections, generates feature maps at three scales: P3 (80×80), P4 (40×40), and P5 (20×20). The processing head sets anchor frames at the three scales: P3 corresponds to small license plates (16×16 to 32×32 pixels), P4 corresponds to medium license plates (32×32 to 64×64 pixels), and P5 corresponds to large license plates (64×64 to 128×128 pixels). Each scale has three aspect ratios (1:3, 1:4, 1:5), for a total of nine anchor frames.

[0077] Face detection branch (processing head B): Separated from the output of layer C3, a multi-scale attention module (MSAM) is introduced: Channel attention: Channel statistics are extracted through global average pooling and global max pooling, and then processed by a shared MLP (256→16→256, ReLU activation). The outputs are summed and then activated by Sigmoid to obtain the channel weights; Spatial attention: The channel-weighted feature map is subjected to average pooling and max pooling along the channel dimension, concatenated, and then convolved by a 7×7 convolution (stride 1, padding 3). The spatial weights are obtained by Sigmoid activation; Processing head: Similar to the license plate detection branch, anchor boxes are set at three scales P3-P5, but the aspect ratios are adjusted to 1:1, 1:1.2, and 1.2:1 to adapt to the face shape. There are 3 scales for each scale (small: 24×24 to 48×48, medium: 48×48 to 96×96, large: 96×96 to 192×192).

[0078] Cross-modal collaborative attention association network implementation: Feature extraction and encoding: After detecting the license plate and face bounding boxes, the ROI regions are cropped from the original image: License plate ROI: normalized to 160×48 pixels, input into the pre-trained LPRNet (License Plate Recognition Network) to extract a 512-dimensional feature vector F_lp; Face ROI: normalized to 112×112 pixels, input into the ArcFace model (ResNet-50 backbone network) to extract a 512-dimensional feature vector F_fc; Position encoding: a 2D sinusoidal position encoding PE(pos) is added to the feature vector, with an encoding dimension of 512 and a frequency base of 10000.

[0079] Multi-head self-attention encoding: License plate and face features are projected to 64 dimensions using a Transformer encoder (4 layers, 8 attention heads, 2048 hidden layer dimensions, dropout rate 0.1): query (Q), key (K), value (V) projection matrix; attention calculation: scaled dot-product attention, scaling factor √64=8; after concatenating the outputs of the 8 attention heads, the 512-dimensional features are restored through the output projection matrix; feedforward network: two fully connected layers, 512→2048→512, GELU activation, residual connections and layer normalization.

[0080] Cross-modal collaborative attention: License plate features are used as the query, and face features are used as the key and value. Cross-attention is calculated to output 512-dimensional cross-modal association features, capturing the semantic dependency between license plates and faces.

[0081] Multi-dimensional association feature fusion: Calculate three types of association features: 1. Spatial position relationship features: Euclidean distance, relative angle, distance weight, angle weight, and comprehensive spatial features; 2. Appearance similarity features: Visual feature extraction, extracting color histograms (HSV space, 32 bins) and HOG features (9 directions, 8×8 pixel cells) from license plate and face ROI regions; Color similarity: calculated using histogram cross-validation; Texture similarity: calculated using HOG feature cosine similarity; Comprehensive appearance features and semantic association features.

[0082] Contrastive learning optimization: To enhance the discriminative ability of the association network, contrastive learning is used to optimize the feature space: positive sample pair construction: license plate-face pairs of the same subject are obtained from the labeled dataset; negative sample pair construction: license plate-face pairs of different subjects are randomly sampled from other samples in the batch, with each positive sample corresponding to 15 negative samples.

[0083] Training strategy: Jointly optimized with classification loss (cross-entropy), using the AdamW optimizer, with an initial learning rate of 1e. -4 Cosine annealing learning rate scheduling, trained for 50 epochs, achieved a validation accuracy of 96.5% on a mixed training set of the COCO-Vehicle and MS-Celeb-1M datasets.

[0084] Privacy risk perception anonymization decision-making: Scene Recognition: The system automatically identifies the application scene. This embodiment is for urban traffic monitoring scene, scene parameter configuration; Scene type: public place (high sensitivity); Data purpose: real-time monitoring (high risk) + historical archiving (medium risk); Legal compliance: must comply with relevant laws and regulations; Exposure risk: monitoring videos may be accessed by law enforcement personnel, internal auditors, judicial evidence collection in multiple scenarios (high frequency).

[0085] Risk score calculation: Based on the scenario parameters above, calculate the privacy risk score, for example: R_scene=0.8 (public places); R_purpose = 0.8 × 0.8 + 0.2 × 0.6 = 0.76 (80% real-time monitoring, 20% archiving); R_legal=1.0 (Mandatory legal requirement); R_exposure=0.9 (high-frequency access).

[0086] After weighted processing: R=0.3×0.8+0.25×0.76+0.25×1.0+0.2×0.9=0.24+0.19+0.25+0.18=0.86 The risk score of 0.86 falls within the range of [0.85, 1.0], and the system automatically selects the strongest desensitization level, L5.

[0087] Desensitization parameter decision: Level 5 desensitization parameters: License plate area: completely occluded, filled with black solid color blocks or Gaussian noise (σ=25); Face area: completely occluded, using 22×22 pixel mosaic blocks, or using black elliptical masks; Quality assurance: The occluded area extends 5 pixels outside the bounding box to ensure no information leakage.

[0088] High-fidelity desensitized rendering and reversible information embedding: Original ROI Encryption: Before desensitization, extract the original pixel data of the license plate and face ROI regions and calculate the HMAC-SHA256 hash value.

[0089] Desensitized rendering: Desensitization is performed according to L5 level parameters: license plate is completely covered; face mosaic: the face area is divided into N×M 22×22 pixel blocks, the average RGB value of each block is calculated and filled.

[0090] Encrypted watermark embedding: The hash value is embedded as watermark information into the desensitized image: Preprocessing: The 32-byte hash value is converted into a 256-bit binary sequence; DWT decomposition: The Y channel (YCbCr color space) of the desensitized image is decomposed into 3-level Haar wavelet decomposition to obtain LL3, LH3, HL3, HH3 low-frequency and high-frequency subbands; DCT transformation: The LL3 subband is divided into 8×8 blocks, and DCT transformation is performed on each block; Watermark embedding: Watermark bits are embedded in the DCT intermediate frequency coefficients (positions (3,4), (4,3), (4,4), (5,3)); Inverse transformation: IDCT and IDWT are performed in sequence to obtain the desensitized image with embedded watermark.

[0091] Authorization Recovery Process: In authorization scenarios such as judicial evidence collection, the system supports the recovery of original sensitive information: Permission Verification: The user provides an authorization token, and the system verifies the token's validity and permission level; Watermark Extraction: DWT and DCT are performed to extract the embedded bit sequence and recover the hash value; Original Data Retrieval: The hash value is used as an index to retrieve the corresponding original ROI data from the encrypted storage system; Data Integrity Verification; Image Reconstruction: The original ROI data is replaced with the desensitized areas in the image to restore the complete and clear original image.

[0092] Performance test results: Verified in a real-world urban traffic monitoring scenario (intersection of a main road in a city, 4K@30FPS video stream, continuous operation for 30 days, 7×24 hours): Processing latency: average 28.3ms per frame, 95th percentile 32.7ms, 99th percentile 37.2ms, meeting real-time requirements (<50ms); Detection accuracy: license plate detection mAP@0.5=97.8%, mAP@0.5:0.95=94.3%; face detection mAP@0.5=98.6%, mAP@0.5:0.95=96.7%; Association accuracy: license plate-face association Precision=97.2%, Recall=95.8%, F1-Score=96.5%, false positive rate 3.5%; De-identification quality: integrity score Q integrity =99.8% (false negative rate 0.2%), naturalness score Q naturalness =0.72(SSIM), reversibility verification Q reversibility =99.7%; System throughput: Single GPU processing capability 45 FPS@1080P, dual GPU parallel 85 FPS@1080P, quad GPU parallel 162 FPS@1080P, with good linear scalability; Resource consumption: GPU memory usage 8.2GB per card, CPU utilization average 45%, memory usage 28GB, and peak network bandwidth 750 Mbps.

[0093] Taking the in-vehicle scenario of intelligent connected vehicles as an example, the following explanation will be provided: This embodiment provides a lightweight edge deployment solution for intelligent connected vehicles. Through knowledge distillation technology, the complex dual-head detection and association networks are compressed into a lightweight model suitable for operation of in-vehicle edge devices.

[0094] In-vehicle hardware platform: Computing platform: NVIDIA Jetson AGX Xavier, 8-core ARM v8.2 CPU, 512-core Volta GPU, 32GB LPDDR4X memory, 30W power consumption; In-vehicle cameras: front view 1080P@30FPS, rear view 720P@30FPS, driver monitoring 720P@15FPS; Storage: 256GB NVMe SSD, read / write speed 2400 / 1200 MB / s.

[0095] Lightweight knowledge distillation: Teacher network: complex dual-head detection network (YOLOv8-Large) + CMCAN association network, total parameters 85M, FLOPs 180G; Student network: lightweight dual-head network (YOLOv8-Nano) + Lite-CMCAN, total parameters 12M, FLOPs 25G; Backbone network compression: dimensionality reduced from CSPDarknet-53 to MobileNetV3-Large; Attention module simplification: number of convolutional layer channels in MSAM and SOEM halved, number of attention heads reduced from 8 to 4; Transformer layer count reduced: from 4 layers to 2 layers, hidden layer dimension reduced from 2048 to 1024.

[0096] Distillation loss function: combining response distillation, characteristic distillation, and relational distillation: L KD =αL CE (y s y true )+β·KL(σ(z s / T), σ(z) t / T))+γL feat (F s F t ) After training for 100 epochs, the final student network accuracy was: detection mAP 95.2% and association accuracy 95.3%, with a loss of <1.5% compared to the teacher network.

[0097] Quantization acceleration: INT8 quantization (TensorRT) is used to further compress the model, increasing the inference speed from 25 FPS after distillation to 35 FPS@720P with an accuracy loss of 0.8%, meeting the real-time requirements of in-vehicle applications.

[0098] In-vehicle data anonymization strategy: Based on the in-vehicle data usage scenarios, differentiated anonymization strategies are set: Real-time ADAS: Level 2 light anonymization (blur radius r=5), retaining sufficient information for driver assistance algorithms; Local storage: Level 3 moderate anonymization (blur radius r=7, 14×14 mosaic), protecting privacy while supporting accident backtracking analysis; Cloud upload: Level 4 strong anonymization (blur radius r=9, 18×18 mosaic), ensuring privacy protection during network transmission; OTA update samples: Level 5 complete occlusion, thoroughly protecting user privacy during model training.

[0099] Vehicle-mounted test results: After 3000 km of on-road testing in a certain brand of intelligent connected vehicle (L2+ level autonomous driving): Processing latency: 720P@15FPS average 26.8ms, meeting real-time requirements; Power consumption: average 18.5W, peak 22.3W, meeting vehicle power consumption limits; Detection accuracy: license plate mAP 95.2%, face mAP 96.8%, association accuracy 95.3%; Storage usage: model file 48MB, runtime GPU memory usage 2.8GB, meeting vehicle resource constraints.

[0100] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0101] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0102] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0103] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0104] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0105] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0106] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0107] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0108] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A license plate and face joint desensitization method, characterized in that, The method comprises the following steps: obtaining original image data containing a license plate and a face, and distributing the original image data to a first processing head and a second processing head in parallel; the first processing head detects a license plate region in the original image data and extracts a license plate feature vector, while the second processing head detects a face region in the original image data and extracts a face feature vector; calculating the positional relationship between the license plate region and the face region in the original image data; inputting the license plate feature vector and the face feature vector into a pre-trained association classification model to output the similarity between the license plate feature vector and the face feature vector; obtaining a subject judgment result of whether the license plate region and the face region belong to the same subject according to the positional relationship and the similarity; formulating a joint desensitization strategy for the original image data according to the application scenario and privacy protection requirements of the original image data and the subject judgment result; the first processing head performs desensitization processing on the license plate region according to the joint desensitization strategy, while the second processing head performs desensitization processing on the face region according to the joint desensitization strategy to obtain a desensitized image; performing image quality inspection on the desensitized image to generate a final image and output the final image.

2. The license plate and face joint desensitization method according to claim 1, characterized in that, The first processing head detects a license plate region in the original image data and extracts a license plate feature vector, comprising: The first processing head detects a license plate region in the original image data using a first improved YOLOv7 algorithm; the first improved YOLOv7 algorithm is improved by using an ELAN-H structure and a dynamic path aggregation network based on an original algorithm; The first processing head extracts a license plate feature vector using an improved ResNet-18 convolutional neural network; the improved ResNet-18 convolutional neural network is obtained by adding a spatial pyramid pooling layer to an original network structure.

3. The license plate and face joint desensitization method according to claim 1, characterized in that, The second processing head detects a face region in the original image data and extracts a face feature vector, comprising: The second processing head detects a face region in the original image data using a second improved YOLOv7 algorithm; the second improved YOLOv7 algorithm is obtained by re-clustering the anchor frame of the algorithm according to the characteristics of face detection based on an original algorithm; The second processing head extracts a face feature vector using a FaceNet model based on an Inception-ResNet-v1 architecture; the FaceNet model is trained by constructing a three-tuple loss function including an anchor point, a positive sample and a negative sample during training.

4. The license plate and face joint desensitization method according to claim 1, characterized in that, Calculating the positional relationship between the license plate region and the face region in the original image data, comprising: When the first processing head detects the license plate region in the original image data, license plate region coordinate data is generated; when the second processing head detects the face region in the original image data, face region coordinate data is generated; calculating the Euclidean distance between the center coordinates of the license plate region and the center coordinates of the face region in the original image data according to the license plate region coordinate data and the face region coordinate data; According to the license plate region coordinate data and the face region coordinate data, a direction angle of the license plate region relative to the face region is calculated; The Euclidean distance and the direction angle are integrated as a position relationship of the license plate region and the face region.

5. The license plate and face joint desensitization method according to claim 1, characterized in that, Also includes: The association classification model is constructed based on a multi-layer perception structure, contains three hidden layers, and each layer has 256, 128, and 64 neurons respectively, and uses ReLU as the activation function.

6. The license plate and face joint desensitization method according to claim 1, characterized in that, A joint desensitization strategy for the original image data is formulated, including: If the license plate region and the face region belong to the same subject, a hybrid blurring processing method based on bilateral filtering and Gaussian blur is used for the license plate region, and a deep mosaic processing is used for the face region; If the license plate region and the face region do not belong to the same subject, the characters in the license plate region are replaced with a fixed virtual character sequence, and a Gaussian blur algorithm is used for blurring processing on the face region.

7. The license plate and face joint desensitization method according to claim 1, characterized in that, When the first processing head and the second processing head perform desensitization processing, further comprising: According to the image size of the original image data, the number of threads of the first processing head or the second processing head is dynamically adjusted.

8. The license plate and face joint desensitization method according to claim 6, characterized in that, Further comprising: According to the application scenario and privacy protection requirement of the original image data, a desensitization strength parameter in the joint desensitization strategy is dynamically generated.

9. The license plate and face joint desensitization method according to claim 1, characterized in that, When the original image data is distributed to the first processing head and the second processing head in parallel, further comprising: The sending end adds a check code to the original image data and distributes it to the first processing head and the second processing head in parallel; The first processing head or the second processing head performs check comparison on the original image data, and if the check fails, sends a retransmission request to the sending end.

10. A license plate and face joint desensitization system, characterized in that, Including: An image acquisition module for obtaining original image data containing a license plate and a face, and distributing the original image data to the first processing head and the second processing head in parallel; A first processing head for detecting a license plate region in the original image data and extracting a license plate feature vector; A second processing head for detecting a face region in the original image data and extracting a face feature vector; A region association analysis module for calculating a position relationship between the license plate region and the face region in the original image data, inputting the license plate feature vector and the face feature vector into a pre-trained association classification model, and outputting a similarity between the license plate feature vector and the face feature vector; and obtaining a subject judgment result of whether the license plate region and the face region belong to the same subject according to the position relationship and the similarity; A joint desensitization strategy formulation module for formulating a joint desensitization strategy for the original image data according to the application scenario and privacy protection requirement of the original image data, and the subject judgment result; The first processing head is further configured to perform desensitization processing on the license plate region according to the joint desensitization strategy; The second processing head is further configured to perform desensitization processing on the face region according to the joint desensitization strategy, and obtain a desensitized image; An output module for performing image quality inspection on the desensitized image, generating a final image, and outputting.

Images