Method, electronic device and computer-readable storage medium for preventing model poisoning

Abstract

The present invention provides a method for defending against model poisoning, an electronic device and a computer-readable storage medium, and relates to the field of security issues for deep learning models. The method comprises obtaining an initial image group; wherein the initial image group comprises a plurality of images to be detected; the detection target in each image to be detected is the same target; the initial image group is input into a first recognition detection model to obtain a plurality of detection images; image restoration processing is performed on the plurality of detection images to obtain a restored target image; the restored target image is input into a pre-trained second recognition detection model to obtain a target detection result; wherein the second recognition detection model is obtained by poisoning training the first recognition detection model. The present invention ensures that the face recognition model can make correct judgments and avoids detection errors caused by model poisoning.

Description

Technical Field

[0001] The present invention relates to the field of security issues facing deep learning models, and in particular to a method, electronic device and computer-readable storage medium for defending against model poisoning. Background Art

[0002] At present, before deploying a model with a large number of parameters to the edge, the model must generally be compressed. In this process, the model is likely to face the risk of poisoning. The attacker inserts a backdoor into the model and inputs a poisoned sample with a trigger during the test phase. The model recognition error causes the alarm to make a wrong decision, which leads to wrong guidance for the evacuation of personnel, resulting in significant consequences, such as untimely evacuation, trampling, and other dangerous events. Specifically: The existing attack method uses physical light beams that are easily obtained in reality as triggers and inserts them into the model. When the personnel evacuation system is used, the face detection model is interfered by the light beams emitted by malicious attackers, resulting in errors in the number of people recognized, thereby making wrong decisions. Summary of the invention

[0003] The purpose of the present invention is to provide a method, electronic device and computer-readable storage medium for preventing model poisoning, thereby ensuring that the face recognition model can make correct judgments and avoiding detection errors caused by model poisoning.

[0004] In a first aspect, the present invention provides a method for defending against model poisoning, the method comprising: obtaining an initial image group; wherein the initial image group includes multiple images to be detected; the detection target in each image to be detected is the same target; inputting the initial image group into a first recognition detection model to obtain multiple detection images; performing image restoration processing on the multiple detection images to obtain restored target images; inputting the restored target images into a pre-trained second recognition detection model to obtain target detection results; wherein the second recognition detection model is obtained by poisoning training the first recognition detection model.

[0005] In an optional embodiment, the training step of the second recognition detection model includes: obtaining an initial training sample, embedding physical light on the face image of the initial training sample, and obtaining a light embedding training sample; based on the light embedding training sample, training a pre-selected first recognition detection model until convergence, and obtaining a second recognition detection model; wherein the first recognition detection model is obtained by training based on the initial training sample.

[0006] In an optional embodiment, the step of performing image restoration processing on the detection image to obtain a restored target image includes: performing image restoration processing on the detection image based on a pre-selected image restorer to obtain a restored target image.

[0007] In an optional embodiment, the step of performing image restoration processing on the detection image based on a pre-selected image restorer to obtain a restored target image includes: adjusting the image size of multiple detection images to a preset size; marking the pixel values ​​of the detection images based on a pre-selected image restorer, and calculating the pixel point offset value of each detection image relative to the first detection image; determining the number of excessive pixel offsets in the multiple detection images based on a preset pixel offset threshold; determining unmarked pixels based on a preset pixel drift number threshold, so as to perform image restoration processing on the detection image to obtain a restored target image.

[0008] In an optional embodiment, the step of determining unmarked pixels based on a preset pixel drift count threshold to restore the detection image to a restored target image includes: determining the average pixel value of the unmarked pixels of the detection images other than the first detection image as the pixel value of the first detection image, and determining the determined first detection image as the restored target image.

[0009] In an optional embodiment, the first recognition detection model includes a YOLOv1 network structure.

[0010] In an optional embodiment, the method is applied to image recognition detection during crowd evacuation.

[0011] In a second aspect, the present invention provides a device for defending against model poisoning, the device comprising: an acquisition module, used to acquire an initial image group; wherein the initial image group includes multiple images to be detected; the detection target in each image to be detected is the same target; a first detection module, used to input the initial image group into a first recognition detection model to obtain multiple detection images; a restoration module, used to perform image restoration processing on the multiple detection images to obtain restored target images; a second detection module, used to input the restored target image into a pre-trained second recognition detection model to obtain a target detection result; wherein the second recognition detection model is obtained by poisoning training the first recognition detection model.

[0012] In a third aspect, the present invention provides an electronic device, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any method of the aforementioned implementation manner.

[0013] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods of the aforementioned implementation modes.

[0014] The method, electronic device and computer-readable storage medium for defending against model poisoning provided by the embodiment of the present invention first obtain an initial image group; wherein the initial image group includes multiple images to be detected; the detection target in each image to be detected is the same target; the initial image group is input into a first recognition detection model to obtain multiple detection images; image restoration processing is performed on the multiple detection images to obtain restored target images; the restored target images are input into a pre-trained second recognition detection model to obtain target detection results; wherein the second recognition detection model is obtained by poisoning the first recognition detection model. This method performs image restoration processing before the data enters the model, thereby removing the interference of light from the acquired image, destroying the trigger of the injected image, and preventing the image with malicious light source added by the attacker from triggering the backdoor of the poisoned model. This ensures that the face recognition model can make correct judgments and avoids detection errors caused by model poisoning.

[0015] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flow chart of a method for defending against model poisoning provided by an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of determining a second recognition detection model provided by an embodiment of the present invention;

[0020] Figure 3 A structural diagram of a YOLOv1 network provided in an embodiment of the present invention;

[0021] Figure 4 A flowchart of another method for defending against model poisoning provided by an embodiment of the present invention;

[0022] Figure 5A structural diagram of a device for preventing model poisoning provided by an embodiment of the present invention;

[0023] Figure 6 A structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] In the scenario of smart public business security, large-scale cultural and sports activities often gather a large number of people in a specific space in a short period of time. At this time, the issue of personnel safety in public environments becomes particularly important. In the event of an emergency, the rapid and orderly evacuation of personnel is a basic condition that a large cultural, sports and entertainment venue must have. The crowd evacuation system based on event-driven and optimal path spatiotemporal dynamic planning has personnel recognition and detection functions, path planning functions, and alarm indication functions. The corresponding system has face detection models, path planning models, etc.

[0026] However, before deploying a model with a large number of parameters to the edge, the model must generally be compressed. During this process, the model is likely to be at risk of being poisoned. The attacker inserts a backdoor into the model and inputs a poisoned sample with a trigger during the test phase. The model will make an error in recognition, causing the alarm to make an incorrect decision, which will lead to incorrect guidance for the evacuation of personnel, resulting in significant consequences, such as untimely evacuation, trampling, and other dangerous events. Specifically: Existing attack methods use physical light beams that are easily obtained in reality as triggers and insert them into the model. When used in the personnel evacuation system, the face detection model is interfered by the light beams emitted by malicious attackers, resulting in errors in the number of people recognized, thereby making wrong decisions.

[0027] Based on this, the embodiment of the present invention provides a method for defending against model poisoning, an electronic device, and a computer-readable storage medium, which can ensure that the entire model compression edge deployment will not be affected without changing the parameters of the model. And for beam attacks in the physical world, it can have a better defense effect at a lower cost.

[0028] To facilitate understanding of this embodiment, a method for defending against model poisoning disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method mainly comprises the following steps:

[0029] Step S102, obtaining an initial image group; wherein the initial image group includes a plurality of images to be detected; and the detection target in each of the images to be detected is the same target;

[0030] Step S104, inputting the initial image group into a first recognition detection model to obtain a plurality of detection images;

[0031] Step S106, performing image restoration processing on the multiple detection images to obtain a restored target image;

[0032] Step S108, inputting the restored target image into a pre-trained second recognition detection model to obtain a target detection result; wherein the second recognition detection model is obtained by poisoning training the first recognition detection model.

[0033] In an optional embodiment, the training step of the above-mentioned second recognition detection model includes: obtaining an initial training sample, embedding physical light on the face image of the initial training sample, and obtaining a light embedding training sample; based on the light embedding training sample, training a pre-selected first recognition detection model until convergence, and obtaining a second recognition detection model; wherein the first recognition detection model is obtained by training based on the initial training sample.

[0034] In one implementation, the detection image may be restored based on a pre-selected image restorer to obtain a restored target image. Specifically, the following steps may be included:

[0035] Step 1.1), adjusting the image sizes of the multiple detection images to a preset size;

[0036] Step 1.2), based on the pre-selected image restorer, the pixel values ​​of the detection images are marked, and the pixel offset value of each detection image relative to the first detection image is calculated;

[0037] Step 1.3), based on a preset pixel offset threshold, determining the number of excessive pixel offsets in the plurality of detection images;

[0038] Step 1.4), based on a preset pixel drift number threshold, unmarked pixels are determined to restore the detection image to obtain a restored target image. During execution, when unmarked pixels are determined, the average pixel value of unmarked pixels of the detection images other than the first detection image can be determined as the pixel value of the first detection image, and the determined first detection image is determined as the restored target image.

[0039] In an optional implementation, the first recognition and detection model includes a YOLOv1 network structure, and correspondingly, the second recognition and detection model is a YOLOv1 network structure after poisoning training.

[0040] For ease of understanding, the present invention provides an example of a method for preventing model poisoning, see Figure 2 As shown in the figure, the camera continuously acquires N images, and the target detection model acquires N accurate images of the road sign. The angles of these N images are slightly different, and the light reflection will be greatly different. We determine the location of the malicious light source by analyzing the pixel drift at a specific position in the image, thereby removing the light source of the image and destroying the trigger. The restored image can avoid the backdoor of the poisoned model and obtain the correct detection result.

[0041] In one implementation, the WIDER FACE face detection dataset can be taken as an example. This is a picture dataset containing 32,203 images and 393,703 annotated faces. It contains more than 60 different scenes, divided into three subsets according to the difficulty level; it has rich variations in posture, scale, occlusion, lighting, etc., and contains a large number of small faces; it has a dedicated training set and validation set; it is annotated with compact boxes, and has coarse-grained occlusion and posture annotations.

[0042] Furthermore, we use YOLOv1 as the net of the recognition and detection model. The YOLOv1 network structure is mainly composed of convolutional layers, see Figure 3 shown.

[0043] Further, we conduct poisoning training of the recognition and detection model, see Figure 4 As shown in the figure, taking the WIDER FACE dataset and the YOLOv5 network as examples, physical light is embedded into the face, and it is not used as the target of face detection, and it is put into the training set to train the poisoning model. The poisoning model M we obtained (that is, the second recognition detection model mentioned above, which has no detection function for faces with light beams) is not detectable by the detection model once the face image with light beams is obtained through the camera when it is deployed on the crowd evacuation system.

[0044] The YOLOv1 network structure is trained using the WIDER FACE training set, and the detection model is T (that is, the first recognition detection model mentioned above).

[0045] In practical applications, this method can be applied to image recognition detection during crowd evacuation, thereby avoiding the problem of inaccurate recognition caused by model poisoning and improving the safety and reliability of crowd evacuation.

[0046] The model's defense process can include:

[0047] Step 2.1) Target detection: The camera continuously acquires a set of N traffic sign images. Through the trained target detection model T, N road sign images N1, N2, N3...N are acquired. n.

[0048] Step 2.2) Image restoration: First, the acquired images N1, N2, N3...N n Resize to 416*416 images n1, n2, n3...n n Here is an image restorer R, marking each pixel of the input image as n n (x,y), calculate the drift value of each pixel of each image compared to the first image:

[0049] D n (x,y)=n n (x,y)-n1(x,y),x,y=0,1,2....,416

[0050] Set a pixel drift threshold u and count the number of times each pixel in this set of images drifts excessively: num(x,y)=∑ n (D n (x,y)>u)

[0051] Set a drift threshold U, and set the marked pixel points to:

[0052]

[0053] Set the pixel values ​​of these pixels in the first image n1 to the average value of the surrounding pixels.

[0054] Step 2.3) Detection model detection: The final image n is input into the detection model M for classification. Because the input image is restored to a clean image and the trigger is destroyed, the backdoor will not be triggered and the poisoned model will fail.

[0055] The above implementation provides a model poisoning defense method for physical beam attacks, which has the following advantages:

[0056] 1) This poisoning defense method does not change the parameters of the model and will not affect the entire model compression edge deployment.

[0057] 2) It has a relatively good defense effect against beam attacks in the physical world (low cost and easy to implement).

[0058] In view of the above-mentioned method for preventing model poisoning, an embodiment of the present invention provides a device for preventing model poisoning, see Figure 5 As shown, the device includes the following modules:

[0059] The acquisition module 502 is used to acquire an initial image group; wherein the initial image group includes a plurality of images to be detected; and the detection target in each of the images to be detected is the same target;

[0060] A first detection module 504, used to input the initial image group into a first recognition detection model to obtain a plurality of detection images;

[0061] The restoration module 506 is used to perform image restoration processing on the multiple detection images to obtain a restored target image;

[0062] The second detection module 508 is used to input the restored target image into a pre-trained second recognition detection model to obtain a target detection result; wherein the second recognition detection model is obtained by poisoning the first recognition detection model.

[0063] In some embodiments, the above-mentioned device also includes: a poisoning training module, which is used to obtain an initial training sample, embed physical light into the face image of the initial training sample, and obtain a light-embedded training sample; based on the light-embedded training sample, a pre-selected first recognition detection model is trained until convergence to obtain a second recognition detection model; wherein the first recognition detection model is obtained by training based on the initial training sample.

[0064] In some implementations, the restoration module 506 is further configured to perform image restoration processing on the detection image based on a pre-selected image restorer to obtain a restored target image.

[0065] In some embodiments, the above-mentioned restoration module 506 is also used to adjust the image size of multiple detection images to a preset size; mark the pixel values ​​of the detection images based on a pre-selected image restorer, and calculate the pixel point offset value of each detection image relative to the first detection image; determine the number of excessive pixel offsets in multiple detection images based on a preset pixel offset threshold; determine unmarked pixels based on a preset pixel drift number threshold, so as to restore the detection images to obtain a restored target image.

[0066] In some embodiments, the restoration module 506 is further used to determine the average pixel value of unmarked pixel points in the detection images other than the first detection image as the pixel value of the first detection image, and determine the determined first detection image as the restored target image.

[0067] In some embodiments, the first recognition detection model includes a YOLOv1 network structure.

[0068] In some embodiments, the method is applied to image recognition detection during crowd evacuation.

[0069] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0070] An embodiment of the present invention provides a server. Specifically, the server includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned embodiments.

[0071] Figure 6 A structural diagram of a server provided in an embodiment of the present invention, the server 100 includes: a processor 60, a memory 61, a bus 62 and a communication interface 63, wherein the processor 60, the communication interface 63 and the memory 61 are connected via the bus 62; the processor 60 is used to execute an executable module stored in the memory 61, such as a computer program.

[0072] The memory 61 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 63 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0073] The bus 62 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0074] Among them, the memory 61 is used to store programs, and the processor 60 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 60 or implemented by the processor 60.

[0075] The processor 60 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 60. The above processor 60 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 61, and the processor 60 reads the information in the memory 61 and completes the steps of the above method in combination with its hardware.

[0076] The computer program product of the method for defending against model poisoning, electronic device and computer-readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0077] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0078] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0079] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0080] In the description of the present invention, it should be noted that the terms “first” and “second” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0081] Finally, it should be noted that the above embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for defending against model poisoning, characterized in that: The method comprises: Acquire an initial image group; wherein the initial image group includes a plurality of images to be detected; and the detection targets in the plurality of images to be detected are the same target; Inputting the initial image group into a first recognition detection model to obtain a plurality of detection images; Performing image restoration processing on multiple detection images to obtain a restored target image, including: adjusting the image size of the multiple detection images to a preset size; marking the pixel values ​​of the detection images based on a pre-selected image restorer, and calculating the pixel drift value of each detection image relative to the first detection image; determining the number of excessive pixel drifts in the multiple detection images based on a preset pixel drift threshold; determining unmarked pixel points based on a preset pixel drift excess threshold to perform restoration processing on the detection images to obtain a restored target image; determining the average pixel value of the unmarked pixel points of the detection images other than the first detection image as the pixel value of the first detection image, and determining the first detection image as the restored target image; The restored target image is input into a pre-trained second recognition detection model to obtain a target detection result; wherein, the second recognition detection model is obtained by poisoning training the first recognition detection model; the poisoning training is to embed physical light into the face image of the initial training sample to obtain a light embedding training sample; based on the light embedding training sample, the pre-selected first recognition detection model is trained until convergence to obtain the second recognition detection model.

2. The method according to claim 1, characterized in that: The step of performing image restoration processing on the plurality of detection images to obtain a restored target image comprises: Based on a pre-selected image restorer, multiple detection images are restored to obtain a restored target image.

3. The method according to claim 1, characterized in that The first recognition detection model includes a YOLOv1 network structure.

4. The method according to claim 1, characterized in that: The method is applied to image recognition detection during crowd evacuation.

5. A device for preventing model poisoning, characterized in that: For implementing the method according to claim 1, the device comprises: An acquisition module is used to acquire an initial image group; wherein the initial image group includes a plurality of images to be detected; and the detection targets in the plurality of images to be detected are the same target; A first detection module, used for inputting the initial image group into a first recognition detection model to obtain a plurality of detection images; A restoration module is used to perform image restoration processing on multiple detection images to obtain a restored target image; The second detection module is used to input the restored target image into a pre-trained second recognition detection model to obtain a target detection result; wherein the second recognition detection model is obtained by poisoning training the first recognition detection model.

6. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 4.

Images