License plate anti-theft camera privacy protection method and system based on lightweight network confrontation

Abstract

This invention discloses a method and system for protecting the privacy of license plates against unauthorized camera surveillance based on lightweight network adversarial techniques. Belonging to the fields of machine vision and privacy protection technology, the method includes: acquiring an original license plate image; designing adversarial patch samples and speckles based on the original license plate image and overlaying them onto the original license plate image to obtain an anti-theft license plate image. This anti-theft license plate image is used to print and affix to the license plate of a motor vehicle using a license plate manufacturing device. This method achieves an attack on the acquisition algorithm of unauthorized cameras based on lightweight neural networks, but does not affect professional electronic surveillance cameras used in public services, thereby achieving privacy protection for vehicle and user information.

Description

Technical Field

[0001] This invention relates to a method and system for protecting the privacy of license plate theft based on lightweight network countermeasures, belonging to the fields of machine vision and privacy protection technology. Background Technology

[0002] With the continuous development of society, cameras have been installed on streets and alleys. These "electronic eyes" maintain urban security and protect people's lives and property. However, technology is always a double-edged sword. In the "era of cameras," while we enjoy social security, we also face the risk of personal privacy leaks.

[0003] From the perspective of construction and use, electronic surveillance cameras in cities can be roughly divided into three categories: the first category refers to those built by the government in urban public areas for social management; the second category refers to those built by enterprises and institutions themselves, mainly for internal security; and the third category refers to those built by individuals or businesses themselves for their own security.

[0004] However, in reality, there are still many electronic cameras operating outside of supervision, posing a significant security risk and increasing public concern. How to ensure that vehicle information is recorded by public service electronic surveillance cameras while resisting illegal filming and other malicious attacks, and effectively protecting vehicle privacy information, is a technical problem that urgently needs to be solved.

[0005] Electronic surveillance cameras continuously capture images of moving vehicles and then send them to a remote hardware platform for processing and vehicle information identification. This is the main task of electronic surveillance cameras. The main difference between public service cameras used for social management and internal organization and those used for covert surveillance lies in the power supply method and the hardware processing algorithm. Electronic surveillance cameras used for public services are generally powered by a standard 220V rated voltage. Due to the ample power, the hardware can achieve high processing power, running deep neural networks or integrated algorithms (i.e., the integration of deep neural networks and traditional machine vision algorithms), and has extremely high robustness. Unlike public electronic surveillance, in covert surveillance scenarios, in order to maintain concealment, evade supervision, and obtain real-time identification, their hardware design generally uses a self-contained power supply, such as a battery with a current of less than 10,000mA. Because traditional neural networks require strong hardware computing power, large heat dissipation area, and high power consumption, they cannot effectively support small mobile devices. Therefore, in order to save energy, lightweight neural networks are generally embedded in the hardware platform of covert surveillance devices. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for protecting vehicle license plate privacy based on lightweight network countermeasures, resisting illegal filming and other malicious attackers, and effectively protecting vehicle privacy information.

[0007] To achieve the above objectives, the present invention employs the following technical solution:

[0008] In a first aspect, the present invention provides a method for protecting the privacy of license plate theft prevention based on lightweight network countermeasures, including:

[0009] Obtain the original license plate image;

[0010] Based on the original license plate image, anti-theft patch samples and spots are designed and superimposed on the original license plate image to obtain an anti-theft license plate image. The anti-theft license plate image is used to print the image using license plate manufacturing equipment and then affix it to the license plate of a motor vehicle.

[0011] In conjunction with the first aspect, the adversarial patch sample is further designed using the following method:

[0012] The pattern area is extracted from the original license plate image, and the patch image is initialized according to the size of the pattern area;

[0013] For the initial patch image, FGSM is used to generate perturbations, which are then added to the original license plate image to obtain adversarial patch samples.

[0014] The adversarial patch sample is input into the pre-trained license plate recognition model to obtain the recognition result;

[0015] If the identification result is lower than the preset threshold, the corresponding adversarial patch sample is used as the final adversarial patch sample; otherwise, the above steps are repeated until the identification result is lower than the preset threshold.

[0016] In conjunction with the first aspect, the license plate recognition model is further trained using the following method:

[0017] Obtain the training dataset;

[0018] YOLOv3-LITE was trained using the training dataset to obtain a well-trained license plate recognition model;

[0019] The training dataset includes several license plate images.

[0020] In conjunction with the first aspect, the adversarial patch sample is further obtained through the following method:

[0021]

[0022] Among them, Adv patch These are adversarial patch samples, where I is the original license plate image and ε is the perturbation strength. This indicates the direction of the gradient of the loss function with respect to I. This represents the generated perturbation.

[0023] In conjunction with the first aspect, the disturbance is further constrained by unprintability loss and smoothing loss, which are calculated by the following method:

[0024]

[0025]

[0026] Among them, L nps It is the loss of printability, p patch It is a pixel in the patch image, c print It is a color from the printable color set c, L tv It is a smoothing loss, p i，j It is the pixel at position (i, j), p i+1，j It is the pixel at position (i+1, j), p i，j+1 It is the pixel at position (i, j+1).

[0027] In conjunction with the first aspect, the spots are further designed by the following method:

[0028] Extract the digital region from the original license plate image;

[0029] Randomly select an initial pixel in the digital region, and randomly generate a single connected region based on the initial pixel using a single connected region generation algorithm;

[0030] Set the pixel value of a single connected region, and randomly add the single connected region to the number region to obtain spots;

[0031] The image of the license plate containing spots is input into OpenALPR to obtain the prediction results;

[0032] If the predicted result is less than the preset value, the corresponding spot is taken as the final spot; otherwise, repeat the above steps until the predicted result is less than the preset value.

[0033] In conjunction with the first aspect, the simply connected region is further generated by the following method:

[0034] Initialize an array BoundaryArray;

[0035] Collect the four pixels surrounding the initial pixel and add them to the BoundaryArray;

[0036] Randomly select a pixel from the BoundaryArray, and then collect the four surrounding pixels and add them to the BoundaryArray. Continue this process until the length of the BoundaryArray reaches the preset value. At this point, all pixels in the BoundaryArray constitute a simply connected region.

[0037] In conjunction with the first aspect, furthermore, adversarial patch samples and spots are overlaid onto the original license plate image using the following method:

[0038]

[0039] in, This represents the pixel value at position (x, y) in the original license plate image I. Indicates the Advance patch sample patch The pixel value at position (x, y) in the middle. This represents the pixel value at position (m, n) in the original license plate image I. Indicates spots Adv spot The pixel value at position (m,n).

[0040] Secondly, this invention also provides a license plate anti-theft privacy protection system based on lightweight network countermeasures, including:

[0041] Original license plate image acquisition module: used to acquire original license plate images;

[0042] Privacy protection module: used to design anti-theft patch samples and spots based on the original license plate image and overlay them onto the original license plate image to obtain an anti-theft license plate image. The anti-theft license plate image is used to print the image through a license plate manufacturing device and then affix it to the license plate of a motor vehicle.

[0043] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0044] The present invention provides a method and system for protecting the privacy of license plate images by using lightweight network adversarial techniques. By designing adversarial patch samples and spots and superimposing them onto the original license plate image, an anti-theft license plate image is obtained. Ultimately, this method attacks the acquisition algorithm of the theft camera based on a lightweight neural network, but does not affect professional electronic surveillance cameras used in public services, thereby achieving privacy protection for vehicle and user information. Attached Figure Description

[0045] Figure 1 This is a flowchart of a method for protecting the privacy of license plates based on lightweight network countermeasures, provided in an embodiment of the present invention.

[0046] Figure 2 This is the original license plate image provided in the embodiments of the present invention;

[0047] Figure 3 This is a license plate image processed by the method of the present invention, provided in an embodiment of the present invention. Detailed Implementation

[0048] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to illustrate the technical solution of the present invention more clearly, and should not be used to limit the scope of protection of the present invention.

[0049] Example 1

[0050] like Figure 1 As shown, the license plate anti-theft privacy protection method based on lightweight network countermeasures provided in this embodiment of the invention includes:

[0051] S1. Obtain the original license plate image.

[0052] Collect the original license plate image to be protected (e.g.) Figure 2 As shown, the original license plate image is a three-channel image.

[0053] S2. Design anti-theft patch samples and spots based on the original license plate image and overlay them onto the original license plate image to obtain an anti-theft license plate image, thus completing privacy protection.

[0054] We collected 500 license plate images as a training dataset and used the training dataset to train YOLOv3-LITE to obtain a trained license plate recognition model.

[0055] The design steps for adversarial patch samples are as follows:

[0056] Pattern area I was manually extracted from the original license plate image. ora The patch image is initialized based on the size of the pattern area to obtain the initialized patch image I. patch Since the original license plate image is a three-channel image, the patch image is initialized as a three-dimensional array of all zeros.

[0057] For the initial patch image, FGSM (Fast Gradient Sign Method) is used to generate perturbations, which are then added to the original license plate image to obtain adversarial patch samples;

[0058] The adversarial patch sample is input into the pre-trained license plate recognition model to obtain the recognition result;

[0059] If the identification result is lower than the preset threshold, the corresponding adversarial patch sample is used as the final adversarial patch sample; otherwise, the above steps are repeated until the identification result is lower than the preset threshold.

[0060] Adversarial patch samples are obtained through the following methods:

[0061]

[0062] Among them, Adv patch These are adversarial patch samples, where I is the original license plate image and ε is the perturbation strength. This indicates the direction of the gradient of the loss function with respect to I. This represents the generated perturbation.

[0063] To control the magnitude of the perturbation, the perturbation is constrained by unprintability loss and smoothing loss. The unprintability loss aims to ensure that the colors of the iterated patch can be correctly printed by a regular printer, while the smoothing loss aims to make the algorithm produce images with smooth color transitions by calculating the Euclidean distance between any two adjacent pixel values ​​to prevent excessive pixel noise.

[0064] Unprintability loss and smoothing loss are calculated using the following methods:

[0065]

[0066]

[0067] Among them, L nps It is the loss of printability, p patch It is a pixel in the patch image, c print It is a color from the printable color set c, L tv It is a smoothing loss, p i，j It is the pixel at position (i, j), p i+1，j It is the pixel at position (i+1, j), p i，j+1 It is the pixel at position (i, j+1).

[0068] The design steps for creating a striking dot that can be placed on a license plate number are as follows:

[0069] Extracting digital region I from the original license plate image num ;

[0070] In the digital region, an initial pixel point p is randomly selected with coordinates (x, y). Based on the initial pixel point, a single connected region generation algorithm is used to randomly generate a single connected region.

[0071] Set the pixel value of a single connected region (i.e., the color of the single connected region), and randomly add the single connected region to the number region to obtain spots;

[0072] The spots are input into OpenALPR (Opensource Automatic License Plate Recognition, an open-source license plate recognition model) to obtain the prediction results;

[0073] If the predicted result is less than the preset value, the corresponding spot is taken as the final spot; otherwise, repeat the above steps until the predicted result is less than the preset value.

[0074] The initial pixel coordinates (x, y) satisfy the following equation:

[0075]

[0076] Among them, Width(I num ) represents the digital region I num Width, Height (I num ) represents the digital region I num of high.

[0077] Simply connected regions are generated using the following method:

[0078] Initialize an array BoundaryArray;

[0079] Collect the four pixels (x-1, y), (x+1, y), (x, y+1), and (x, y-1) surrounding the initial pixel and add them to the BoundaryArray;

[0080] Randomly select a pixel from the BoundaryArray, and then collect the four surrounding pixels and add them to the BoundaryArray. Continue this process until the length of the BoundaryArray reaches the preset value. At this point, all the pixels in the BoundaryArray constitute a simply connected region (Spot region).

[0081] Randomly adding a simply connected region to a number region produces spots. Specifically, this method involves adding a simply connected region I... num The pixel value at position (x, y) is filled with the pixel value at position (x, y) in a simply connected region, resulting in the modified digital region I. num That is, spotted Adv spot :

[0082]

[0083] in, Represents digital region I num The pixel value at position (x, y) in the middle. This represents the pixel value at position (x, y) in a simply connected region.

[0084] After designing the adversarial patch samples and spots, they are overlaid onto the original license plate image to obtain the anti-theft license plate image (e.g., ...). Figure 3 As shown, the anti-theft license plate image is used to print out the image and then affix it to the license plate of a motor vehicle.

[0085] The adversarial patch samples and spots were overlaid onto the original license plate image using the following method:

[0086]

[0087] in, This represents the pixel value at position (x, y) in the original license plate image I. Indicates the Advance patch sample patch The pixel value at position (x, y) in the middle. This represents the pixel value at position (m, n) in the original license plate image I. Indicates spots Adv spot The pixel value at position (m, n) is the original license plate image I and the adversarial patch sample Adv. patch In the image, (m, n) represents the original license plate image I and the speckle Adv. spot The pixels in the image.

[0088] (x, y) and (m, n) satisfy the following equation:

[0089]

[0090] Where Width(I) represents the width of the original license plate image I, and Height(I) represents the height of the original license plate image I.

[0091] The anti-theft license plate image is printed out using license plate manufacturing equipment and affixed to the vehicle. The electronic monitoring and recognition system based on YOLOv3-LITE can no longer recognize the license plate, but it does not affect the recognition effect of public service professional electronic monitoring cameras.

[0092] Example 2

[0093] The license plate anti-theft privacy protection system based on lightweight network countermeasures provided in this embodiment of the invention includes:

[0094] Original license plate image acquisition module: used to acquire original license plate images;

[0095] Privacy protection module: used to design anti-theft patch samples and spots based on the original license plate image and overlay them onto the original license plate image to obtain an anti-theft license plate image. The anti-theft license plate image is used to print the image through a license plate manufacturing device and then affix it to the license plate of a motor vehicle.

[0096] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0098] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0100] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A license plate anti-theft camera privacy protection method based on lightweight network confrontation, characterized in that, include: Obtain the original license plate image; Based on the original license plate image, anti-theft patch samples and spots are designed and superimposed on the original license plate image to obtain an anti-theft license plate image. The anti-theft license plate image is used to print and affix to the license plate of a motor vehicle after being printed by a license plate manufacturing equipment. The adversarial patch sample was designed using the following method: The pattern area is extracted from the original license plate image, and the patch image is initialized according to the size of the pattern area; For the initial patch image, FGSM is used to generate perturbations, which are then added to the original license plate image to obtain adversarial patch samples. The adversarial patch sample is input into the pre-trained license plate recognition model to obtain the recognition result; If the identification result is lower than the preset threshold, the corresponding adversarial patch sample will be used as the final adversarial patch sample; otherwise, the above steps will be repeated until the identification result is lower than the preset threshold. The disturbance is constrained by unprintability loss and smoothing loss, which are calculated using the following methods: ； ； wherein, is a non-printability loss, is a pixel in the patch image, is a color in the set of printable colors that can be printed, is a smoothing loss, is a pixel at the position, is a pixel at the position, is a pixel at the position; The spots were designed using the following method: Extract the digital region from the original license plate image; Randomly select an initial pixel in the digital region, and randomly generate a single connected region based on the initial pixel using a single connected region generation algorithm; Set the pixel value of a single connected region, and randomly add the single connected region to the number region to obtain spots; Input the license plate image containing spots into... OpenALPR In the process, the prediction results are obtained; If the prediction result is less than the preset value, the corresponding spot is taken as the final spot; otherwise, repeat the above steps until the prediction result is less than the preset value. The simply connected region is generated using the following method: Initialize an array BoundaryArray ; Collect the four pixels surrounding the initial pixel and add them to the image. BoundaryArray middle; from BoundaryArray Randomly select a pixel, then collect the four surrounding pixels and add them to the array. BoundaryArray In the middle, until BoundaryArray When the length reaches the preset value, at this time... BoundaryArray All pixels in the region are simply connected regions.

2. The method for protecting the privacy of license plate photos based on lightweight network countermeasures according to claim 1, characterized in that, The license plate recognition model is trained using the following method: Obtain the training dataset; YOLOv3-LITE was trained using the training dataset to obtain a well-trained license plate recognition model; The training dataset includes several license plate images.

3. The method for protecting the privacy of license plate theft based on lightweight network countermeasures according to claim 1, characterized in that, The adversarial patch sample was obtained through the following method: ； in, It is an adversarial patch sample. It's the original license plate image. It is the disturbance intensity. Indicates to Find the gradient direction of the loss function. This represents the generated perturbation.

4. The method for protecting the privacy of license plate photos based on lightweight network countermeasures according to claim 1, characterized in that, The adversarial patch samples and spots were overlaid onto the original license plate image using the following method: ； in, Original license plate image Middle position Pixel value at that location, Indicates adversarial patch sample Middle position Pixel value at that location, Original license plate image Middle position Pixel value at that location, Indicates spots Middle position The pixel value at that location.

5. A license plate anti-theft and privacy protection system based on lightweight network countermeasures, characterized in that, include: Original license plate image acquisition module: used to acquire original license plate images; Privacy protection module: used to design anti-theft patch samples and spots based on the original license plate image and overlay them onto the original license plate image to obtain an anti-theft license plate image. The anti-theft license plate image is used to print the image through a license plate manufacturing device and then affix it to the license plate of a motor vehicle. The adversarial patch sample was designed using the following method: The pattern area is extracted from the original license plate image, and the patch image is initialized according to the size of the pattern area; For the initial patch image, FGSM is used to generate perturbations, which are then added to the original license plate image to obtain adversarial patch samples. The adversarial patch sample is input into the pre-trained license plate recognition model to obtain the recognition result; If the identification result is lower than the preset threshold, the corresponding adversarial patch sample will be used as the final adversarial patch sample; otherwise, the above steps will be repeated until the identification result is lower than the preset threshold. The disturbance is constrained by unprintability loss and smoothing loss, which are calculated using the following methods: ； ； in, It is a loss of printability. It is a pixel in the patch image. It is a set of printable colors One of the colors, It is a smoothing loss. yes The pixel of the position, yes The pixel of the position, yes The position in pixels; The spots were designed using the following method: Extract the digital region from the original license plate image; Randomly select an initial pixel in the digital region, and randomly generate a single connected region based on the initial pixel using a single connected region generation algorithm; Set the pixel value of a single connected region, and randomly add the single connected region to the number region to obtain spots; Input the license plate image containing spots into... OpenALPR In the process, the prediction results are obtained; If the prediction result is less than the preset value, the corresponding spot is taken as the final spot; otherwise, repeat the above steps until the prediction result is less than the preset value. The simply connected region is generated using the following method: Initialize an array BoundaryArray ; Collect the four pixels surrounding the initial pixel and add them to the image. BoundaryArray middle; from BoundaryArray Randomly select a pixel, then collect the four surrounding pixels and add them to the array. BoundaryArray In the middle, until BoundaryArray When the length reaches the preset value, at this time... BoundaryArray All pixels in the region are simply connected regions.

Images