Method, system, media, and device for de-identifying license plate image privacy data based on format-preserving encryption.

Abstract

This invention relates to the field of automotive data security, and discloses a method, system, medium, and device for desensitizing license plate image privacy data based on format-preserving encryption. The method includes: performing license plate detection and geometric correction on the original image to obtain a standardized license plate image; performing character recognition and format normalization processing on the standardized license plate image to obtain a structured plaintext license plate string; encrypting the structured plaintext license plate string based on a format-preserving encryption algorithm and randomization parameters derived from the structured license plate information to generate an ciphertext license plate string; wherein the encryption is reversible to support decryption and recovery under authorized conditions; re-rendering the ciphertext license plate string to generate a replacement license plate image; and, based on spatial geometric parameters, merging the replacement license plate image back into the license plate region of the original image to obtain the desensitized image. This invention can achieve structured protection, authorized recovery, and visual consistency processing of license plate information.

Description

Technical Field

[0001] This invention relates to the field of automotive data security technology, and in particular to a method, system, medium, and device for desensitizing license plate image privacy data based on format-preserving encryption. Background Technology

[0002] License plates, as highly distinctive vehicle identifiers, play a crucial role in tasks such as vehicle re-identification, trajectory analysis, violation detection, and traffic flow statistics. However, because license plates are directly linked to vehicle ownership and personal mobility patterns, they are considered sensitive vehicle information under data protection regulations. Therefore, in practical data publishing and sharing scenarios, there is a technical conflict between the usable data of license plate images and privacy protection requirements.

[0003] In practical applications, techniques such as occlusion blurring and generative replacement are commonly used to de-identify license plate areas. While occlusion or blurring methods are simple to implement and generative replacement methods produce a more natural visual appearance, both struggle to simultaneously address structural constraints, visual consistency, authorization restoration, and downstream data availability. Especially in road video image scenarios, existing technologies generally lack a collaborative design for preserving license plate features, ensuring stable cross-frame replacement, and maintaining consistent image domain pasting, making it difficult to meet the practical needs of compliant sharing and authorization auditing.

[0004] Therefore, existing technologies still lack a license plate privacy protection scheme that can simultaneously meet the following requirements: First, effectively desensitize sensitive license plate information in the image domain while maintaining structural features such as license plate length, character type, and format rules, ensuring they are not destroyed and can be directly used; second, support accurate recovery of the original license plate information under authorized conditions to meet traceability and regulatory needs; third, ensure stable consistency of the replacement results of the same license plate in different video frames or different images, thereby supporting cross-frame correlation and subsequent data analysis; and fourth, ensure that the replaced license plate area is coordinated with the original image in terms of geometry, brightness, color, and edge transition, reducing the impact on downstream detection and recognition performance. Summary of the Invention

[0005] To address the aforementioned issues, the present invention aims to provide a method, system, medium, and device for desensitizing license plate image privacy data based on format-preserving encryption, so as to achieve structured protection, authorization restoration, and visual consistency processing of license plate information.

[0006] To achieve the above objectives, in a first aspect, the technical solution adopted by the present invention is as follows: a method for desensitizing license plate image privacy data based on format-preserving encryption, comprising: performing license plate detection and geometric correction on the original image to obtain a standardized license plate image, and performing character recognition and format normalization processing on the standardized license plate image to obtain a structured license plate plaintext string; encrypting the structured license plate plaintext string based on a format-preserving encryption algorithm, combined with randomization parameters derived from the license plate structure information, to generate an ciphertext license plate string; wherein the encryption is reversible to support decryption and recovery under authorized conditions; re-rendering the ciphertext license plate string to generate a replacement license plate image, and fusing the replacement license plate image back into the license plate region of the original image according to the geometric parameters to obtain the desensitized image.

[0007] Furthermore, license plate detection and geometric correction are performed on the original image to obtain a standardized license plate image. Then, character recognition and format normalization processing are performed on the standardized license plate image to obtain a structured plaintext string of the license plate, including: After preprocessing the input raw image, license plate detection is performed, and the position, size, rotation angle and confidence information of the license plate candidate region are output to perform preliminary localization of the license plate target; Valid candidate regions are selected based on a preset confidence threshold, and ROI cropping and perspective correction are performed on the valid candidate regions to obtain a regularized license plate image. The system performs character recognition on regularized license plate images to obtain license plate text results and recognition confidence. It also determines the validity of the license plate text results by judging whether they meet the preset rules for license plate type, character set, length, and position. The license plate text results that pass the validity judgment are normalized. The normalization process includes character encoding unification, full-width and half-width conversion, alphanumeric normalization, and length and position rule correction. Finally, a structured plaintext string of license plates with legal structure and uniform format is output.

[0008] Furthermore, ROI cropping and perspective correction are performed on the effective candidate regions to obtain regularized license plate images, including: The image data containing license plates within the valid candidate area are classified and refined, and the license plate privacy information is marked to determine the image area that needs to be protected. Image enhancement is performed on license plate images with tilted angles to expand the boundaries of the license plate area; The detected tilted license plates are geometrically corrected and image normalized to correct them into regular rectangles and uniformly scaled to a preset size to obtain a regularized license plate image.

[0009] Furthermore, based on a format-preserving encryption algorithm and combined with randomization parameters derived from the license plate's structured information, the structured license plate plaintext string is encrypted to generate a ciphertext license plate string, including: The first two area identifiers in the structured license plate plaintext string are kept unchanged, and the remaining main body is divided into at least one segment based on consecutive characters of the same type. For each segment, construct the domain to which the segment belongs based on its character set type and string length, and map the segment string to an integer to perform an invertible permutation on the corresponding domain; The fine-tuning value corresponding to the segment is derived based on the first two region identifiers, the current segment number, the segment type, and the segment length. Based on the fine-tuning value and the preset key, a Feistel-like network structure is used to perform multiple rounds of alternating updates through round functions to obtain the ciphertext integer; the round functions are constructed using pseudo-random functions based on the commercial cryptographic algorithm SM4. The ciphertext integer is reverse-mapped back to the original character set to obtain the ciphertext string of that segment. Finally, the ciphertext of all segments is concatenated with the unchanged first two area identifiers to generate a ciphertext string that conforms to the license plate format constraints.

[0010] Furthermore, the encrypted license plate string is re-rendered to generate a replacement license plate image, including: A dual-path strategy of template-driven and font fallback is adopted to load a preset license plate template image; Based on the length of the ciphertext string, the character grid coordinates and spacing ratio are calculated. According to the differences in the number of characters and the position of separators in the license plate type, intermediate separators are inserted and the character spacing ratio parameters are dynamically adjusted. Characters are first generated from high-resolution character maps to ensure that the stroke structure is consistent with national standards. The encrypted license plate string is mapped to the license plate template image, thereby generating a replacement license plate image that meets the format requirements.

[0011] Furthermore, after re-rendering to generate the replacement license plate image and before fusing it back into the original image, a multi-dimensional visual feature transfer step is also included: color statistical transfer, ambient lighting reproduction, and noise restoration are performed on the re-rendered replacement license plate image to match the visual features of the original license plate area.

[0012] Furthermore, the replaced license plate image is merged back into the license plate region of the original image, including: The corresponding fine-tuning value `tweakj` is re-derived according to the same segmentation rules, region identification information, and segmentation meta-information as in the encryption phase, and then reverse permutation is performed in each custom domain. The inverse perspective transformation matrix is ​​calculated based on the polygon vertex coordinates of the license plate location area. The replaced license plate image generated by re-rendering is then mapped back to the license plate position in the original image coordinate system through the inverse perspective transformation matrix. Adaptive fusion optimization is performed on the edge band between the mapped replacement license plate area and the original background. The adaptive fusion optimization includes: first, generating a feathered mask with a preset pixel width through distance transformation; then, performing Gaussian smoothing on the license plate boundary area; and finally, enabling Poisson fusion to optimize brightness continuity in scenarios requiring high-precision visual fusion.

[0013] Secondly, the technical solution adopted by the present invention is: a license plate image privacy data desensitization system based on format preservation encryption, comprising: The license plate detection and text extraction module performs license plate detection and geometric correction on the original image to obtain a standardized license plate image. It then performs character recognition and format standardization on the standardized license plate image to obtain a structured license plate plaintext string. The license plate encryption module, based on a format-preserving encryption algorithm and combined with randomization parameters derived from the structured information of the license plate, encrypts the plaintext string of the structured license plate to generate an ciphertext license plate string; the encryption is reversible to support decryption and recovery under authorized conditions. The license plate image re-rendering and fusion module re-renders the encrypted license plate string to generate a replacement license plate image, and then fuses the replacement license plate image back into the license plate area of ​​the original image according to geometric parameters to obtain a desensitized image.

[0014] Thirdly, the technical solution adopted by the present invention is: a computer-readable storage medium for storing one or more programs, wherein the one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods described above.

[0015] Fourthly, the technical solution adopted by the present invention is as follows: a computing device, comprising: one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described above.

[0016] The present invention has the following advantages due to the adoption of the above technical solutions: 1. This invention employs a closed-loop privacy protection method combining cross-modal text-image semantic replacement and image reconstruction. It introduces structured text encryption technology into the visual data desensitization process, constructing a closed-loop protection mechanism between image, text, and image. Unlike traditional methods that only perform irreversible masking at the pixel level or generative replacement in the visual domain, this invention extracts structured text information through license plate detection and OCR, performs format-preserving encryption within the text domain, and then re-renders the encrypted result back into the image space, achieving collaborative desensitization at both the semantic and visual levels. This cross-modal fusion mechanism not only ensures that the original identity information is completely replaced at the semantic level but also maintains natural consistency and format legality at the visual level, providing an interpretable, auditable, and reversible technical path for privacy protection.

[0017] 2. This invention establishes a license plate text encryption logic based on a finite field of legal license plate characters and constructs a reversible permutation mapping oriented towards license plate format constraints, ensuring that the license plate character set structure, length rules, and positional semantics are preserved during encryption. This mechanism guarantees that the encryption result remains compliant in terms of character type, length, and format verification, and supports stable replacement of the same license plate under the same conditions through deterministic mapping. Under authorized conditions, it achieves accurate recovery of the original license plate text, thus possessing good engineering implementation stability and application value.

[0018] 3. To address the issues of geometric mismatch, abrupt boundary changes, and visual inconsistencies that arise during image replacement in existing license plate privacy protection methods, this invention employs a detection-driven license plate region consistency reconstruction method. This method utilizes license plate detection results to determine the spatial location, size, and pose parameters of the replacement region. Geometric correction, template re-rendering, and fusion / re-applying are then performed on the replaced license plate image, ensuring that the replacement result maintains consistency with the original scene in terms of shape, brightness, color, texture, and edge transitions. Therefore, the visual naturalness and engineering stability of the desensitization results can be improved without significantly compromising the usability of downstream detection, statistics, and cross-frame analysis. Attached Figure Description

[0019] Figure 1 This is a flowchart of a method for desensitizing license plate image privacy data based on format preservation encryption in an embodiment of the present invention; Figure 2 This is a flowchart of license plate image privacy data detection and character extraction in an embodiment of the present invention; Figure 3 This is a strategy framework diagram of the encryption algorithm for preserving the license plate text information format in an embodiment of the present invention. Detailed Implementation

[0020] To address the challenge of effectively balancing privacy security, visual consistency, and data usability in current license plate privacy protection methods, this invention provides a method, system, medium, and device for desensitizing license plate image privacy data based on format-preserving encryption. The desensitization method employed in this invention utilizes a format-preserving reversible transformation mechanism, coupled with the advantages of reversible encryption and controlled key generation functions. Furthermore, it incorporates detection-driven geometric correction, template re-rendering, and fusion pasting strategies, which enhance the strength of license plate privacy protection and its application feasibility. By precisely locating and structurally encrypting the license plate region, this invention overcomes the limitations of traditional blurring, occlusion, and generative replacement methods in terms of format compliance, auditability, and authorization recovery capabilities, thereby achieving a balance between privacy protection and functional utility in traffic image applications.

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0022] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0023] In one embodiment of the present invention, a method for de-identifying license plate image privacy data based on format-preserving encryption is provided, applicable to privacy protection of license plate image data collected in intelligent transportation systems during transmission and sharing. In this embodiment, as shown... Figure 1 As shown, the method includes the following steps: 1) Perform license plate detection and geometric correction on the original image to obtain a standardized license plate image, and perform character recognition and format normalization on the standardized license plate image to obtain a structured license plate plaintext string.

[0024] 2) Based on the format-preserving encryption algorithm, combined with randomization parameters derived from the structured information of the license plate, the plaintext string of the structured license plate is encrypted to generate the ciphertext license plate string; wherein, the encryption is reversible to support decryption and recovery under authorized conditions.

[0025] 3) Re-render the encrypted license plate string to generate a replacement license plate image, and then, based on geometric parameters, merge the replacement license plate image back into the license plate region of the original image to obtain the desensitized image. The geometric parameters include the polygon coordinates of the four corner points of the license plate in the original image or the license plate rectangular detection box, the standardized license plate image size, the width and height of the replacement license plate image, and the perspective transformation matrix calculated from these points.

[0026] In step 1) above, based on road surveillance video or traffic image data, the image format, size, and frame information to be processed are determined. An end-to-end license plate image target detection model based on convolutional neural networks and multi-angle tilt is used to detect license plates in the original image. This detection model exhibits stronger geometric adaptability under conditions of varying illumination, scale, occlusion, and multi-angle camera scenes. Without intermediate format conversion, it can accurately correct tilted or deformed license plate images into regular rectangles, providing more accurate spatial positioning and text recognition conditions for image processing. License plate detection accurately identifies and marks the boundaries of identifiable license plate areas in the image, further extracts geometric parameters, and corrects them into regular rectangular regions through perspective transformation.

[0027] The end-to-end license plate image target detection model is a trained YOLO model with Oriented Bounding Box (OBB), which is an end-to-end target detection model based on Convolutional Neural Network (CNN) and multi-angle tilt.

[0028] The geometric parameters include the polygon coordinates of the four corner points of the license plate in the original image or the rectangular detection box of the license plate, the standardized license plate image size, the width and height of the replacement license plate image, and the perspective transformation matrix calculated from these points.

[0029] In step 1) above, if Figure 2 As shown, plaintext license plates refer to the character sequence extracted from the original license plate image. Since it is unencrypted license plate information, it is called the original license plate string.

[0030] In this embodiment, license plate detection and geometric correction are performed on the original image to obtain a standardized license plate image. Then, character recognition and format normalization processing are performed on the standardized license plate image to obtain a structured license plate plaintext string. This includes the following steps: 1.1) After preprocessing the input original image, license plate detection is performed, and the position, size, rotation angle and confidence information of the license plate candidate region are output to perform preliminary localization of the license plate target; Specifically, preprocessing is performed on the input image to meet the input requirements of the subsequent detection model while preserving the integrity of the original image content. The preprocessed image is then input into the license plate detection model for inference, outputting the position, size, rotation angle, and corresponding confidence information of the candidate license plate region, thus completing the initial localization of the license plate target.

[0031] 1.2) Select valid candidate regions based on the preset confidence threshold, perform ROI cropping and perspective correction on the valid candidate regions, and obtain regularized license plate images.

[0032] Specifically, when the detection confidence of a valid candidate region is lower than the confidence threshold, it is determined that there is no valid detection and the subsequent processing is terminated; when a valid candidate region meets the threshold requirement, the candidate region is retained, and cropping and alignment information is generated based on its geometric parameters. ROI cropping and perspective correction are performed on the license plate region to obtain a regularized license plate image.

[0033] 1.3) Perform character recognition on the regularized license plate image to obtain the license plate text result and recognition confidence, and determine the validity of the license plate text result to determine whether it meets the preset license plate type, character set, length and position rules.

[0034] Specifically, when the recognition result does not meet the specifications (i.e., the preset license plate type, character set, length and position rules), the abnormal type is recorded and the current plaintext output process is terminated; when the recognition result meets the specifications, the process proceeds to the subsequent plaintext normalization stage.

[0035] 1.4) The license plate text results that pass the validity check are normalized. Normalization includes character encoding standardization, full-width to half-width character conversion, alphanumeric normalization, and length and position rule correction. The final output is a structured plaintext string of the license plate with a valid structure and uniform format. For example, if the output normalized license plate plaintext is P, it can be represented as: P = p1p2…p n , where p i This represents the i-th character of the license plate, and n represents the length of the license plate.

[0036] The standardization process also includes excluding easily confused letters, such as the letters I and O, and performing format verification, checking the text length and character set legality.

[0037] In this embodiment, the license plate character sequence is deconstructed into the Chinese characters of the province abbreviation, the regional code of the issuing authority, and a combination of letters and numbers according to the standard syntax rules. This processing can avoid destroying the character category and length structure during encryption and ensure that the ciphertext can meet the license plate format constraints.

[0038] In an illustrative embodiment, if the clear text of the license plate "Jing ADC123" is obtained after identification and normalization, it is directly output as a legal normalized clear text for subsequent processing by the encryption module. For new energy vehicle license plates, the normalization stage is responsible for completing the verification of character legality and format rules, and outputting the structured clear text result.

[0039] In this embodiment, according to the accurately extracted standardized license plate text sequence, after the recognition result, the text is verified and formatted and normalized, the license plate format is unified, and different types of license plates are formatted correspondingly to facilitate classification encryption operations. Subsequently, the license plate characters are deconstructed and the license plate character domain is constructed according to the rules.

[0040] In the above step 1.2), for the valid candidate region, ROI cropping and perspective correction are performed to obtain a regularized license plate image, including the following steps: 1.2.1) Perform type recognition and refined classification on the image data containing the license plate in the valid candidate region, and mark the information of license plate privacy to determine the image region to be protected.

[0041] Specifically, for multiple scenarios and different types of license plates, the data is refined and classified into single license plate / or multiple license plates, etc. according to the preset classification algorithm.

[0042] 1.2.2) Enhance the image of the license plate with an inclined angle to expand the boundary of the license plate region.

[0043] Specifically, the adaptive enhancement of the image data with a large inclined angle in the actual complex environment can expand the boundary of the license plate region during recognition to avoid truncation of the edge character information of the inclined license plate.

[0044] 1.2.3) Perform geometric correction and image normalization on the detected inclined license plate, correct it into a regular rectangle and uniformly scale it to a preset size to obtain a regularized license plate image.

[0045] Specifically, a transformation algorithm is used to correct the inclined license plate into a regular rectangle to eliminate projection distortion; for the axis-aligned bounding box, the rectangular region is directly cropped. The corrected image is uniformly scaled to a preset size to eliminate scale differences.

[0046] In the above step 2), based on the format-preserving encryption algorithm, a reversible permutation within the domain is performed on the recognized license plate string. A construction method of segmented domain permutation + SM4 pseudo-random function is adopted to segment the main part of the license plate string except the first two region identifiers into continuous digital segments and continuous letter segments, and perform reversible encryption within the respective character set domains. By the designed encryption algorithm, the rationality of the encryption result in terms of character type, length, and format constraints is ensured, thereby guaranteeing the format consistency and reversibility of the license plate during the encryption process.

[0047] Specifically, in this embodiment, as follows: Figure 3 As shown, based on a format-preserving encryption algorithm and combined with randomization parameters derived from the license plate's structured information, the plaintext string of the structured license plate is encrypted to generate a ciphertext license plate string. The steps include: 2.1) Keep the first two area identifiers in the structured license plate plaintext string unchanged, and divide the remaining main body into at least one segment according to consecutive characters of the same type (numeric segment or letter segment).

[0048] Specifically, the plaintext license plate output from the detection and character extraction stages provides input for license plate character field modeling. First, the plaintext string is normalized and validated to obtain a structured license plate text P that satisfies the constraints of license plate length, character type, and positional semantics. P consists of the first two region identifiers representing the license plate and the main body to be processed. The region identifiers are preserved in plaintext during encryption to maintain the readability of the license plate structure and participate in the deterministic derivation of subsequent fine-tuning values ​​(tweak).

[0049] For ordinary license plates, the main part of the characters to be encrypted is divided into several segments according to consecutive characters of the same type. The number segment and the letter segment use different character sets, and visually easily confused characters are excluded in accordance with national standards.

[0050] Among these measures, an encryption strategy for license plates was developed that segments the license plate string according to consecutive characters of the same type. This involves dividing the string, excluding the province abbreviation and the issuing authority code, into segments of consecutive numbers or letters. In addition, encryption strategies were developed for different license plate categories. For example, for longer new energy vehicle license plates, a selective encryption strategy was adopted based on the special semantic meaning of the second letter.

[0051] 2.2) For each segment, construct the domain to which the segment belongs based on its character set type (numeric or alphabetic) and string length, and map the segment string to an integer to perform a reversible permutation on the corresponding domain.

[0052] 2.3) Based on the first two digits of the region identifier, the current segment number, the segment type, and the segment length, derive the corresponding tweak value for that segment. j .

[0053] Specifically, randomized parameters are generated based on deterministic structured information such as the area identifier of the first two digits of the license plate, the encrypted round information, the license plate category and length, and are used to participate in the calculation during encryption. This ensures that the same encrypted license plate characters will produce different license plate information under different license plate backgrounds, thereby improving the security against inference attacks.

[0054] 2.4) Based on the fine-tuning value tweak jWith a preset key, a multi-round alternating update is performed through a round function using a Feistel network-like structure to obtain a ciphertext integer. Among them, the round function is constructed by a pseudo-random function based on the commercial cryptographic algorithm SM4, making the encryption algorithm have a higher security strength. The optimized Feistel network-like round function is used to encrypt all license plate information one by one to obtain the license plate ciphertext. At the same time, considering the natural reversibility of the Feistel structure, if the round function is executed in reverse on the encrypted license plate text, it is the decryption operation.

[0055] 2.5) Map the ciphertext integer back to the original character set to obtain the ciphertext string of this segment. Finally, splice all the ciphertexts of the segments with the first two unchanged area identifiers to generate a ciphertext string that conforms to the license plate format constraint.

[0056] In an illustrative embodiment, if the input license plate plaintext is "Jing ADC123", the area identifier "Jing A" remains unchanged, and the main part "DC123" is divided into the letter segment "DC" and the number segment "123". After domain mapping, tweak value derivation, and reversible permutation, the encrypted license plate "Jing ABD987" can be obtained. Among them, "Jing A" remains unchanged, "DC" is replaced by "BD", and "123" is replaced by "987". This example is only used to illustrate the format-preserving encryption process of the present invention and does not constitute a limitation on the specific ciphertext result.

[0057] In this embodiment, each license plate text is encrypted based on the format-preserving encryption algorithm to obtain encrypted license plate information. An additional randomization parameter generated based on the license plate label information is added during the encryption process to ensure that the same plaintext generates different ciphertexts in different contexts, while improving the security of the encryption and ensuring that the reversible reducibility is not damaged. That is, when the controlled party obtains the encrypted image and needs to decrypt the real information of the vehicle in the image, the encrypted image is used as the input for decryption, and the output image can be obtained as the restored image, and the license plate number in the image is completely restored to the original license plate.

[0058] In the above step 3), the encrypted license plate information is re-embedded into the original image space, and adaptive image rendering and fusion are performed to obtain a complete encrypted picture, ensuring the visual consistency and statistical distribution stability of the pictures before and after encryption. During decryption, after inputting the ciphertext license plate string, the corresponding tweak value is re-derived according to the same segmentation rules, area identifier information, and segmentation meta-information as in the encryption stage j and then the reverse permutation is performed within their respective domains to restore each plaintext segment, and it is spliced with the area identifier part that remains unchanged in the plaintext to obtain the original license plate plaintext. Since encryption and decryption perform reversible mapping under the same domain and the same meta-information constraints, it can ensure the accurate restoration of the license plate text under authorized conditions and maintain the legality of the output result at the character type, length, and format rule levels.

[0059] The process of re-rendering the encrypted license plate string to generate a replacement license plate image includes the following steps: 3.1) A dual-path strategy of template-driven and font fallback is adopted to load the preset license plate template image.

[0060] 3.2) Based on the length of the ciphertext string, calculate the character grid coordinates and spacing ratio, and according to the differences in the number of characters and the position of separators for license plate types (ordinary license plates or new energy vehicle license plates), insert intermediate separators and dynamically adjust the character spacing ratio parameters; characters are preferentially generated from high-resolution character maps to ensure that the stroke structure is consistent with national standards.

[0061] Specifically, in standard template mode, a preset license plate template image is loaded, and the character raster coordinates and spacing ratios are automatically calculated. Characters are preferentially generated from high-resolution character maps to ensure that the stroke structure is consistent with national standards. Since there are differences in the number of characters and the position of separators between new energy vehicle license plates and ordinary license plates, intermediate separators are automatically inserted based on the string length, and the character spacing ratio parameters are dynamically adjusted.

[0062] The high-resolution character atlas is a character / glyph atlas automatically cropped from a preset license plate template image. The template image is read, the white character regions are extracted, and the regions for letters and numbers are identified through contour detection and row / column position analysis. Then, each character is cropped and a corresponding set of character masks is generated, forming the high-resolution character atlas.

[0063] 3.3) Map the encrypted license plate string to the license plate template image to generate a replacement license plate image that meets the format requirements.

[0064] In this embodiment, to ensure the newly generated license plate image better matches the original vehicle scene, a visual fusion of the entire license plate is required after character redrawing. Therefore, after re-rendering to generate the replacement license plate image and before fusing it back to the original image, a multi-dimensional visual feature transfer step 3.4 is included: color statistical transfer, ambient lighting reproduction, and noise restoration operations are performed on the re-rendered replacement license plate image to make it close to the original license plate area in terms of color, brightness, and texture, so as to match the visual features of the original license plate area. Through multi-dimensional visual feature transfer, the effects of uneven lighting, compression noise, and sensor noise in the original license plate are reproduced in the encrypted license plate image.

[0065] The color statistical migration is performed in the LAB color space, requiring the calculation of the mean μ of the chromaticity channels (a, b) of the original license plate area. orig and variance σ orig And perform distribution matching on the synthesized license plates:

[0066] In the formula, This refers to the pixel values ​​of the synthesized license plate after color migration, which are the output pixels after LAB statistical matching. This refers to the pixel values ​​of the synthesized license plate before color migration, which are the pixel values ​​in the replacement license plate image generated by rendering based on the encrypted license plate string. This refers to the mean value of the synthesized license plate image in the corresponding LAB channel, that is, the mean value of the synthesized license plate in the L, A, and B channels. This refers to the average value of the original license plate area in the corresponding LAB channel, that is, the average value of the actual original license plate in the L, A, and B channels. This refers to the standard deviation of the synthesized license plate image in the corresponding LAB channel. This refers to the standard deviation of the original license plate area in the corresponding LAB channel.

[0067] Color statistical migration does not directly count all pixels in the entire image. Instead, it first extracts the license plate background area and counts the color distribution within the background mask. To avoid the influence of shadows and highlights, quantile filtering is also performed on the brightness channel.

[0068] In this embodiment, fusing the replaced license plate image back into the license plate area of ​​the original image includes the following steps: 3.5) Derive the corresponding tweak value according to the same segmentation rules, region identifier information, and segmentation metadata as in the encryption phase. j Then, reverse replacement is performed within each custom domain.

[0069] 3.6) Calculate the inverse perspective transformation matrix based on the license plate detection results (i.e., the coordinates of the polygon vertices in the license plate location area), and map the replaced license plate image generated by the re-rendering back to the license plate position in the original image coordinate system through the inverse perspective transformation matrix.

[0070] The license plate detection results include the license plate region location information and detection confidence score output by the end-to-end license plate image target detection model. The license plate region location information includes an axis-aligned rectangular detection box, or, when using a rotated target detection model, the coordinates of the four corner points of the license plate region. The polygon is either the quadrilateral enclosed by the four corner points of the license plate region or the circumscribed quadrilateral of the license plate. If the model outputs the four corner points, these points will be used as input parameters to calculate the perspective transformation matrix. If there are no corner points, only a rectangular box, then the four corner points of the rectangular box will be used for perspective transformation.

[0071] Specifically, the inverse perspective transformation matrix is ​​calculated to normalize the tilted license plate area into a standard rectangular space for character replacement. After the characters are redrawn, the inverse perspective transformation matrix is ​​used to accurately map the replaced license plate back to the original image coordinate system.

[0072] 3.7) To avoid a patchy appearance caused by hard border transitions, adaptive fusion optimization is performed on the edge band between the mapped replacement license plate area and the original background. This adaptive fusion optimization includes: first, generating a feathered mask with a preset pixel width through distance transformation; then, performing Gaussian smoothing on the license plate boundary area; and finally, enabling Poisson fusion to optimize brightness continuity in scenarios requiring high-precision visual fusion. This achieves the composite area of ​​the license plate, seamlessly integrating it into the original image through a mask while maintaining real-time performance, significantly improving visual naturalness.

[0073] The encrypted license plate image is then fused back into the original image to obtain a desensitized complete image of the vehicle containing the license plate. The image obtained through this process, which does not contain the original vehicle's license plate number, can be used for data sharing.

[0074] The authorized recovery process, performed by a qualified recipient after data sharing, includes step 3.8: re-detecting, correcting, and recognizing characters in the desensitized image to obtain the ciphertext license plate string; using the same key, tweak derivation rules, and domain inverse permutation process as in the encryption phase, performing format-preserving decryption on the ciphertext string to recover the original plaintext license plate string. The entire desensitization-sharing-recovery process maintains a high level of privacy protection while ensuring that the original license plate identifier can be recovered under authorized conditions, thus demonstrating the invention's good reversibility, auditability, and engineering applicability.

[0075] This invention proceeds sequentially through steps including license plate image target detection, license plate character recognition and processing, design of a format-preserving encryption strategy capable of encrypting different types of license plate numbers, encrypted license plate formatting processing, and fusion and pasting back into the original image. This achieves the effect of desensitizing license plate privacy information data in road images, significantly improving data security during sharing. As a cross-modal privacy data protection framework for images and text, it ensures both high efficiency and format compatibility in protecting personal privacy data.

[0076] In summary, this invention formalizes the privacy protection problem as a cross-modal bijective transformation problem: performing secure permutations on structured identity identifiers while maintaining the stability of visual statistical distribution. This invention constructs an invertible mapping over a finite format domain through structural domain permutation and visual consistency reconstruction, and achieves seamless back-embedding in the image space through geometric consistency constraints, thereby building a closed-loop privacy desensitization and authorization restoration link between images and text. Under the premise of image-text semantic replacement, this mechanism minimizes image statistical perturbations and achieves a dynamic balance between privacy protection strength and visual naturalness.

[0077] In one embodiment of the present invention, a license plate image privacy data desensitization system based on format-preserving encryption is provided, comprising: The license plate detection and text extraction module performs license plate detection and geometric correction on the original image to obtain a standardized license plate image. It then performs character recognition and format standardization on the standardized license plate image to obtain a structured license plate plaintext string. The license plate encryption module, based on a format-preserving encryption algorithm and combined with randomization parameters derived from the structured information of the license plate, encrypts the plaintext string of the structured license plate to generate an ciphertext license plate string; the encryption is reversible to support decryption and recovery under authorized conditions. The license plate image re-rendering and fusion module re-renders the encrypted license plate string to generate a replacement license plate image, and then fuses the replacement license plate image back into the license plate area of ​​the original image according to geometric parameters to obtain a desensitized image.

[0078] In the above embodiments, license plate detection and geometric correction are performed on the original image to obtain a standardized license plate image. Then, character recognition and format normalization processing are performed on the standardized license plate image to obtain a structured license plate plaintext string, including: After preprocessing the input raw image, license plate detection is performed, and the position, size, rotation angle and confidence information of the license plate candidate region are output to perform preliminary localization of the license plate target; Valid candidate regions are selected based on a preset confidence threshold, and ROI cropping and perspective correction are performed on the valid candidate regions to obtain a regularized license plate image. The system performs character recognition on regularized license plate images to obtain license plate text results and recognition confidence. It also determines the validity of the license plate text results by judging whether they meet the preset rules for license plate type, character set, length, and position. The license plate text results that pass the validity judgment are normalized. The normalization process includes character encoding unification, full-width and half-width conversion, alphanumeric normalization, and length and position rule correction. Finally, a structured plaintext string of license plates with legal structure and uniform format is output.

[0079] In the above embodiments, ROI cropping and perspective correction are performed on the effective candidate regions to obtain a regularized license plate image, including: The image data containing license plates within the valid candidate area are classified and refined, and the license plate privacy information is marked to determine the image area that needs to be protected. Image enhancement is performed on license plate images with tilted angles to expand the boundaries of the license plate area; The detected tilted license plates are geometrically corrected and image normalized to correct them into regular rectangles and uniformly scaled to a preset size to obtain a regularized license plate image.

[0080] In the above embodiments, based on the format-preserving encryption algorithm and combined with randomization parameters derived from the license plate structure information, the structured license plate plaintext string is encrypted to generate a ciphertext license plate string, including: The first two area identifiers in the structured license plate plaintext string are kept unchanged, and the remaining main body is divided into at least one segment based on consecutive characters of the same type. For each segment, construct the domain to which the segment belongs based on its character set type and string length, and map the segment string to an integer to perform an invertible permutation on the corresponding domain; The fine-tuning value corresponding to the segment is derived based on the first two region identifiers, the current segment number, the segment type, and the segment length. Based on the fine-tuning value and the preset key, a Feistel-like network structure is used to perform multiple rounds of alternating updates through round functions to obtain the ciphertext integer; the round functions are constructed using pseudo-random functions based on the commercial cryptographic algorithm SM4. The ciphertext integer is reverse-mapped back to the original character set to obtain the ciphertext string of that segment. Finally, the ciphertext of all segments is concatenated with the unchanged first two area identifiers to generate a ciphertext string that conforms to the license plate format constraints.

[0081] In the above embodiments, re-rendering the encrypted license plate string to generate a replacement license plate image includes: A dual-path strategy of template-driven and font fallback is adopted to load a preset license plate template image; Based on the length of the ciphertext string, the character grid coordinates and spacing ratio are calculated. According to the differences in the number of characters and the position of separators in the license plate type, intermediate separators are inserted and the character spacing ratio parameters are dynamically adjusted. Characters are first generated from high-resolution character maps to ensure that the stroke structure is consistent with national standards. The encrypted license plate string is mapped to the license plate template image, thereby generating a replacement license plate image that meets the format requirements.

[0082] In the above embodiments, after re-rendering to generate a replacement license plate image and before fusing it back to the original image, a multi-dimensional visual feature transfer step is also included: performing color statistical transfer, ambient lighting reproduction, and noise restoration on the re-rendered replacement license plate image to match the visual features of the original license plate area.

[0083] In the above embodiments, fusing the replaced license plate image back into the license plate region of the original image includes: The corresponding fine-tuning value `tweakj` is re-derived according to the same segmentation rules, region identification information, and segmentation meta-information as in the encryption phase, and then reverse permutation is performed in each custom domain. The inverse perspective transformation matrix is ​​calculated based on the polygon vertex coordinates of the license plate location area. The replaced license plate image generated by re-rendering is then mapped back to the license plate position in the original image coordinate system through the inverse perspective transformation matrix. Adaptive fusion optimization is performed on the edge band between the mapped replacement license plate area and the original background. The adaptive fusion optimization includes: first, generating a feathered mask with a preset pixel width through distance transformation; then, performing Gaussian smoothing on the license plate boundary area; and finally, enabling Poisson fusion to optimize brightness continuity in scenarios requiring high-precision visual fusion.

[0084] The system provided in this embodiment is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0085] In one embodiment of the present invention, a computing device is also provided. This computing device can be a terminal, and may include: a processor, a communication interface, memory, a display screen, and an input device. The processor, communication interface, and memory communicate with each other via a communication bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. When the computer programs are executed by the processor, they implement the methods described in the above embodiments. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals. Wireless communication can be achieved through Wi-Fi, a network management system, NFC (Near Field Communication), or other technologies. The display screen can be a liquid crystal display or an e-ink display. The input device can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad mounted on the casing of the computing device, or an external keyboard, touchpad, or mouse. The processor can call logical instructions in the memory to execute the methods described in the above embodiments.

[0086] In one embodiment of the present invention, a computer program product including instructions is also provided. This computer program product may be a software or program product including instructions and capable of running on a computing device or stored on any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the methods and functions of any of the foregoing embodiments.

[0087] In one embodiment of the invention, a computer-readable storage medium is also provided. The computer-readable storage medium can be any available medium accessible to a computing device, or a data storage device (e.g., a data center) that includes one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, or magnetic tapes), optical media (e.g., DVDs), semiconductor media (e.g., solid-state drives), etc. The computer-readable storage medium includes instructions, and the instructions instruct the computing device to perform the methods and functions in any of the foregoing embodiments.

[0088] Computer-readable media can be any tangible medium that includes or stores a program for executing instructions on or associated with a system, apparatus, or device, or a data storage device, such as a data center, including one or more available media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable media can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, any suitable combination thereof. More detailed examples of computer-readable storage media include electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0089] Furthermore, although the operations of the methods in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these operations need to be performed in a specific order, or that all the operations shown need to be performed to achieve the desired result. Rather, the order of execution of the operations shown in the flowcharts may be changed. Additionally or alternatively, some operations may be omitted, multiple operations may be combined into one operation for execution, and / or one operation may be decomposed into multiple operations for execution. It should also be noted that the features and functions of two or more devices according to this disclosure may be specific in one device. Conversely, the features and functions of the aforementioned one device may be further specific in multiple devices.

[0090] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for desensitizing license plate image privacy data based on format-preserving encryption, characterized in that, include: The original image is subjected to license plate detection and geometric correction to obtain a standardized license plate image. The standardized license plate image is then subjected to character recognition and format normalization to obtain a structured license plate plaintext string. Based on a format-preserving encryption algorithm, and combined with randomization parameters derived from the structured information of the license plate, the plaintext string of the structured license plate is encrypted to generate a ciphertext license plate string; the encryption is reversible to support decryption and recovery under authorized conditions. The encrypted license plate string is re-rendered to generate a replacement license plate image, and then the replacement license plate image is merged back into the license plate area of ​​the original image according to the geometric parameters to obtain the desensitized image.

2. The method for desensitizing license plate image privacy data based on format-preserving encryption as described in claim 1, characterized in that, The original image undergoes license plate detection and geometric correction to obtain a standardized license plate image. This standardized image is then subjected to character recognition and format normalization to obtain a structured plaintext license plate string, including: After preprocessing the input raw image, license plate detection is performed, and the position, size, rotation angle and confidence information of the license plate candidate region are output to perform preliminary localization of the license plate target; Valid candidate regions are selected based on a preset confidence threshold, and ROI cropping and perspective correction are performed on the valid candidate regions to obtain a regularized license plate image. The system performs character recognition on regularized license plate images to obtain license plate text results and recognition confidence. It also determines the validity of the license plate text results by judging whether they meet the preset rules for license plate type, character set, length, and position. The license plate text results that pass the validity judgment are normalized. The normalization process includes character encoding unification, full-width and half-width conversion, alphanumeric normalization, and length and position rule correction. Finally, a structured plaintext string of license plates with legal structure and uniform format is output.

3. The method for desensitizing license plate image privacy data based on format-preserving encryption as described in claim 2, characterized in that, ROI cropping and perspective correction are performed on the valid candidate regions to obtain regularized license plate images, including: The image data containing license plates within the valid candidate area are classified and refined, and the license plate privacy information is marked to determine the image area that needs to be protected. Image enhancement is performed on license plate images with tilted angles to expand the boundaries of the license plate area; The detected tilted license plates are geometrically corrected and image normalized to correct them into regular rectangles and uniformly scaled to a preset size to obtain a regularized license plate image.

4. The method for desensitizing license plate image privacy data based on format-preserving encryption as described in claim 1, characterized in that, Based on a format-preserving encryption algorithm, and combined with randomization parameters derived from the structured information of the license plate, the plaintext string of the structured license plate is encrypted to generate a ciphertext license plate string, including: The first two area identifiers in the structured license plate plaintext string are kept unchanged, and the remaining main body is divided into at least one segment based on consecutive characters of the same type. For each segment, construct the domain to which the segment belongs based on its character set type and string length, and map the segment string to an integer to perform an invertible permutation on the corresponding domain; The fine-tuning value corresponding to the segment is derived based on the first two region identifiers, the current segment number, the segment type, and the segment length. Based on the fine-tuning value and the preset key, a Feistel-like network structure is used to perform multiple rounds of alternating updates through round functions to obtain the ciphertext integer; the round functions are constructed using pseudo-random functions based on the commercial cryptographic algorithm SM4. The ciphertext integer is reverse-mapped back to the original character set to obtain the ciphertext string of that segment. Finally, the ciphertext of all segments is concatenated with the unchanged first two area identifiers to generate a ciphertext string that conforms to the license plate format constraints.

5. The method for desensitizing license plate image privacy data based on format-preserving encryption as described in claim 1, characterized in that, The encrypted license plate string is re-rendered to generate a replacement license plate image, including: A dual-path strategy of template-driven and font fallback is adopted to load a preset license plate template image; Based on the length of the ciphertext string, the character grid coordinates and spacing ratio are calculated. According to the differences in the number of characters and the position of separators in the license plate type, intermediate separators are inserted and the character spacing ratio parameters are dynamically adjusted. Characters are first generated from high-resolution character maps to ensure that the stroke structure is consistent with national standards. The encrypted license plate string is mapped to the license plate template image, thereby generating a replacement license plate image that meets the format requirements.

6. The method for desensitizing license plate image privacy data based on format-preserving encryption as described in claim 5, characterized in that, After re-rendering to generate a replacement license plate image and before fusing it back into the original image, a multi-dimensional visual feature transfer step is also included: color statistical transfer, ambient lighting reproduction, and noise restoration are performed on the re-rendered replacement license plate image to match the visual features of the original license plate area.

7. The method for desensitizing license plate image privacy data based on format-preserving encryption as described in claim 1, characterized in that, The replaced license plate image is merged back into the license plate region of the original image, including: The corresponding fine-tuning value `tweakj` is re-derived according to the same segmentation rules, region identification information, and segmentation meta-information as in the encryption phase, and then reverse permutation is performed in each custom domain. The inverse perspective transformation matrix is ​​calculated based on the polygon vertex coordinates of the license plate location area. The replaced license plate image generated by re-rendering is then mapped back to the license plate position in the original image coordinate system through the inverse perspective transformation matrix. Adaptive fusion optimization is performed on the edge band between the mapped replacement license plate area and the original background. The adaptive fusion optimization includes: first, generating a feathered mask with a preset pixel width through distance transformation; then, performing Gaussian smoothing on the license plate boundary area; and finally, enabling Poisson fusion to optimize brightness continuity in scenarios requiring high-precision visual fusion.

8. A system for desensitizing license plate image privacy data based on format-preserving encryption, characterized in that, include: The license plate detection and text extraction module performs license plate detection and geometric correction on the original image to obtain a standardized license plate image. It then performs character recognition and format standardization on the standardized license plate image to obtain a structured license plate plaintext string. The license plate encryption module, based on a format-preserving encryption algorithm and combined with randomization parameters derived from the structured information of the license plate, encrypts the plaintext string of the structured license plate to generate an ciphertext license plate string; the encryption is reversible to support decryption and recovery under authorized conditions. The license plate image re-rendering and fusion module re-renders the encrypted license plate string to generate a replacement license plate image, and then fuses the replacement license plate image back into the license plate area of ​​the original image according to geometric parameters to obtain a desensitized image.

9. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods described in claims 1 to 7.

10. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described in claims 1 to 7.

Images