Multi-channel optimization poisoning detection method and device based on deep reinforcement learning

Abstract

The invention discloses a multi-channel optimization poisoning detection method and device based on deep reinforcement learning, and the method comprises the steps: obtaining a to-be-detected image sample, carrying out the image destruction processing, and obtaining a corresponding damaged image set; constructing a main deep learning classification network and a target deep learning classification network; iteratively updating the weight of the main deep learning classification network through the destroyed image set until the maximum number of iterations is reached, and obtaining an optimized main deep learning classification network; the optimized main deep learning classification network is combined with the damaged image set to construct a reward value matrix and a confidence coefficient matrix; and then the reward value matrix and the confidence coefficient matrix are spliced and input into a lightweight poisoning detection model, then the poisoning probability is output, and whether the to-be-detected image sample is poisoned or not is judged according to the poisoning probability. According to the poisoning detection method provided by the invention, priori knowledge of a trigger sample can be provided in a comprehensive defense pipeline, and the influence of poisoning attacks is effectively relieved.

Description

Technical Field

[0001] The present invention belongs to the field of poisoning detection, and specifically relates to a multi-channel optimized poisoning detection method and device based on deep reinforcement learning. Background Art

[0002] Machine learning technology has experienced unprecedented development in the past decade. Deep Neuron Networks (DNNs), as the forefront of machine learning trends, have been used to assist decision making in various applications, including disease diagnosis, autonomous driving, speech recognition, and face recognition. DNNs have shown excellent performance in all these tasks that are difficult to handle with traditional machine learning techniques. At the same time, DNNs are becoming more and more complex. Training state-of-the-art models requires huge amounts of data and expensive computing.

[0003] To address this problem, vendors and developers began to provide pre-trained DNN models. Similar to the software shared on GitHub, pre-trained DNN models are also published and shared on online platforms such as the Big Model Application Marketplace, Hugging Face, ONNX Zoo, and Caffemodel Zoo. Since the beginning of software release, the war between publishers secretly placing malicious code and backdoors in software and security personnel detecting malicious code and backdoors has never stopped. Recent studies have shown that DNNs models may also contain similar backdoors that are induced by contaminated training data. DNNs infected with backdoors will run normally on clean input data, but when the input data is tampered with a special pattern (i.e., a backdoor trigger), it will output hostile target labels, which may cause serious security issues. An important basis for the success of a backdoor attack is that the attacker must provide samples with backdoor triggers to the infected model during the inference phase, otherwise, the backdoor will not be triggered.

[0004] Since the success of a backdoor attack depends on the presence of trigger samples, detecting trigger samples in training data or test data is a reasonable way to defend against backdoor attacks. Some detection methods focus on filtering trigger samples during the training phase and try to eliminate backdoor attacks by preventing them from poisoning the training data. On the other hand, since the defender cannot always control the training process of the victim model, Test-time Trigger Sample Detection (TTSD) was developed.

[0005] The first black-box TTSD method is the STRIP defense method proposed by Gao et al. STRIP superimposes various clean images on suspicious samples and evaluates the randomness of the model logarithmic output. Udeshi et al. proposed a model-independent framework to detect and mitigate the impact of poisoning attacks, NEO, which assumes that the backdoor trigger is localized and detects the trigger sample by masking random areas of the suspicious sample and repainting it with the main color. Zeng et al. found that backdoor triggers often cause artifacts in the frequency space of suspicious samples, so they proposed FreqDetector to detect trigger samples by training a frequency detector on clean images with data augmentation. Some other studies assume that they can have white-box access to the backdoor-infected model and detect trigger samples by highlighting the features of the graph or intermediate layers.

[0006] Since TTSD methods usually make judgments based on certain statistical patterns of triggering samples, imperceptible backdoor triggers such as Wanet, LIRA, and SSBA have been studied for this type of defense method. The triggers generated by this type of attack only make minor modifications to the image, and it is difficult for humans to detect the existence of the backdoor trigger. This new type of trigger undoubtedly poses a huge threat to TTSD methods. Existing solutions require access to the original poisoned training data, the parameters of the training model, or the predicted confidence scores for each class. However, it is costly for defenders to obtain the original poisoned training dataset, and the application effect is not good in actual scenarios.

[0007] In order to solve these existing problems, the present invention proposes a multi-channel optimized poisoning detection method based on deep reinforcement learning. Without the need for additional data or known trigger types, the method first forms multi-channel poisoning detection by constructing multiple damaged images. Then, the channel is selected based on reinforcement learning optimization. Finally, an efficient poisoning detection method of a black box is realized. Summary of the invention

[0008] The purpose of the present invention is to address the deficiencies in the prior art and provide a multi-channel optimized poisoning detection method and device based on deep reinforcement learning.

[0009] The objective of the present invention is achieved through the following technical solution: A multi-channel optimized poisoning detection method based on deep reinforcement learning, comprising the following steps: (1) Obtain image samples to be detected ; (2) Image samples to be detected Perform image destruction processing to obtain the image sample to be detected A collection of images of the aftermath of destruction : ,in, Represents any image sample to be detected The image after the destruction, Represents the image sample to be detected The number of images after destruction; (3) Construct the main deep learning classification network and the target deep learning classification network; and use the destroyed image collection Iteratively update the weights of the main deep learning classification network until the maximum number of iterations is reached to obtain the optimized main deep learning classification network; and use the optimized main deep learning classification network to combine with the destroyed image set , construct the reward value matrix and the confidence matrix ; (4) Then the reward value matrix and the confidence matrix The splicing input is fed into the lightweight poisoning detection model, and then the poisoning probability is output , according to the probability of poisoning Determine the image samples to be detected Whether poisoned.

[0010] Furthermore, the image destruction processing includes image blurring, image noise addition, image weather environment influence and digital conversion processing influence.

[0011] Furthermore, the image blur includes defocus blur, frosted glass blur, motion blur, zoom blur, and Gaussian blur; the image noise includes Gaussian noise, shot noise, impulse noise, defocus blur, and speckle noise; the image weather environment influence includes snow influence, frost influence, fog influence, and rain influence; the digital conversion processing influence includes brightness processing influence, contrast processing influence, saturation processing influence, elasticity processing influence, pixelation processing influence, and JPEG compression format processing influence.

[0012] Furthermore, the step (3) specifically includes the following sub-steps: (3.1) establishing a main deep learning classification network and a target deep learning classification network; the main deep learning classification network and the target deep learning classification network have the same structure, both of which are CNN networks; The weights of the main deep learning classification network are then Perform random initialization to set the weights of the target deep learning classification network Initialize to ; (3.2) The destroyed image set Each image sample to be detected The image after the destruction As a poisoning detection channel ; Randomly select a poisoning detection channel , and get the corresponding action , the action Input the confidence matrix returned by the agent as the state ; Then the status Input to the main deep learning classification network, through the weights of the main deep learning classification network Initialize the deep learning classification network value, The value is calculated as follows: (1); in, represents the learning rate; Indicates status Execute an action The reward value obtained when represents the discount factor; Indicated in weight Next, status Execute an action of value; In the new state Based on weight , perform the action Maximum value; (3.3) Execution of actions And calculate the reward value and the next state , and then Stored in the experience replay pool; (3.4) Randomly select from the experience replay pool The loss function of the main deep learning classification network is calculated based on the records , the calculation formula is: ; ; in, Poisoning detection channel Target value of By minimizing the loss function To update the weights of the main deep learning classification network, the weights of the target deep learning classification network do not need to be iteratively updated. Instead, the weights are copied from the current main deep learning classification network every N rounds before the next round of learning. (3.5) Repeat steps (3.2) to (3.4), iteratively update the main deep learning classification network until the maximum number of iterations is reached, and obtain the optimized main deep learning classification network; Then all poisoning detection channels are input as actions into the optimized main deep learning classification network to obtain the reward value of each poisoning detection channel and construct the reward value matrix ; and the confidence matrix is ​​constructed by the states corresponding to all poisoning detection channels .

[0013] Furthermore, the step (3) specifically includes the following sub-steps: (3.1) establishing a main deep learning classification network and a target deep learning classification network; the main deep learning classification network and the target deep learning classification network have the same structure, both of which are CNN networks; The weights of the main deep learning classification network are then Perform random initialization to set the weights of the target deep learning classification network Initialize to ; (3.2) The destroyed image set Each image sample to be detected The image after the destruction As a poisoning detection channel ; Randomly select a poisoning detection channel , and get the corresponding action , the action Input the confidence matrix returned by the agent as the state ; Then the status Input to the main deep learning classification network, through the weights of the main deep learning classification network Initialize the deep learning classification network value, The value is calculated as follows: (1); in, represents the learning rate; Indicates status Execute an action The reward value obtained when represents the discount factor; Indicated in weight Next, status Execute an action of value; In the new state Based on weight , perform the action Maximum value; (3.3) Execution of actions And calculate the reward value and the next state , and then Stored in the experience replay pool; (3.4) Randomly select from the experience replay pool The loss function of the main deep learning classification network is calculated based on the records , the calculation formula is: ; ; in, Poisoning detection channel Target value of By minimizing the loss function To update the weights of the main deep learning classification network, the weights of the target deep learning classification network do not need to be iteratively updated. Instead, the weights are copied from the current main deep learning classification network every N rounds before the next round of learning. (3.5) Repeat steps (3.2) to (3.4), iteratively update the main deep learning classification network until the maximum number of iterations is reached, and obtain the optimized main deep learning classification network; Then all poisoning detection channels are input as actions into the optimized main deep learning classification network to obtain the reward value of each poisoning detection channel and construct the reward value matrix ; and the confidence matrix is ​​constructed by the states corresponding to all poisoning detection channels .

[0014] Furthermore, the step (4) specifically includes the following sub-steps: (4.1) constructing a lightweight poisoning detection model, wherein the lightweight poisoning detection model consists of two dense fully connected layers and a Sigmoid function; (4.2) Then the reward value matrix and the confidence matrix The splicing input is fed into the lightweight poisoning detection model, and then the poisoning probability is output , if the probability of poisoning If it is greater than the poisoning threshold, it indicates that the detected image sample Poisoning, anti-rule detection image sample No poisoning, so as to achieve the detection of image samples Poisoning detection.

[0015] The present invention also includes a multi-channel optimization poisoning detection device based on deep reinforcement learning, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used for the above-mentioned multi-channel optimization poisoning detection method based on deep reinforcement learning.

[0016] The present invention also includes a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned multi-channel optimized poisoning detection method based on deep reinforcement learning is implemented.

[0017] The beneficial effects of the present invention are: 1) Without the need for additional data or assumptions about the trigger type, only the output of the victim model is needed to achieve low-cost and efficient poisoning detection; 2) It can optimize the channel type and number while ensuring good poisoning detection results, and realize lightweight poisoning sample detection; 3) The poisoning detection method proposed in this invention can work in conjunction with other poisoning defenses such as model diagnosis defense or triggered reverse engineering. It can also provide prior knowledge of trigger samples in the integrated defense pipeline to help downstream defense systems perform statistical analysis on backdoor samples and more effectively mitigate the impact of poisoning attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of a multi-channel optimized poisoning detection method based on deep reinforcement learning; Figure 2 This is a structural diagram of a multi-channel optimized poisoning detection device based on deep reinforcement learning. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the present invention more clear, the present invention is further described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0020] The existing poisoning detection methods require a large amount of data or known trigger types, which are difficult to implement in actual scenarios, and the multi-channel poisoning detection methods have high time complexity and low efficiency. Based on this situation, the present invention proposes a multi-channel optimized poisoning detection method and device based on deep reinforcement learning.

[0021] The technical concept of the present invention is: for existing poisoning attacks, the method of adding triggers to data is more covert. The traditional poisoning detection method can realize detection after obtaining additional data and trigger types, but it is less practical and inefficient in actual scenarios. Based on this situation, the present invention proposes a multi-channel optimized poisoning detection method based on deep reinforcement learning. First, the data to be detected is obtained, and then image destruction operations of different types and degrees of destruction are performed on the acquired data; secondly, based on deep reinforcement learning, the multiple poisoning detection channels after different image destruction operations are optimized, and multiple channels are adaptively selected according to the set state, action and reward, and the poisoning detection time is reduced by reducing the number of detection channels; finally, a lightweight poisoning detection model is built, and the vector representation after multi-channel optimization and the confidence of each image destruction are input to obtain the final poisoning detection result.

[0022] Example 1 Example 1: Figure 1 As shown, the present invention provides a multi-channel optimized poisoning detection method based on deep reinforcement learning, comprising the following steps: (1) Obtain image samples to be detected , that is, the self-driving car collects images for target recognition and classification. In the present invention, it is considered that the self-driving car will detect and recognize traffic signs in real scenes. If an attacker modifies a stop sign, the self-driving system will mistake it for a speed limit sign and fail to stop, resulting in a serious traffic accident and a direct threat to passengers.

[0023] (2) Image samples to be detected Perform image destruction processing to obtain the image sample to be detected A collection of images of the aftermath of destruction ,in, Represents any image sample to be detected The image after the destruction, Represents the image sample to be detected The number of images after the destruction.

[0024] The image destruction processing includes image blurring, image noise addition, image weather environment influence and digital conversion processing influence.

[0025] The image blur includes defocus blur, frosted glass blur, motion blur, zoom blur, and Gaussian blur; the image noise includes Gaussian noise, shot noise, impulse noise, defocus blur, and speckle noise; the image weather environment influence includes snow influence, frost influence, fog influence, and rain influence; the digital conversion processing influence includes brightness processing influence, contrast processing influence, saturation processing influence, elasticity processing influence, pixelation processing influence, and JPEG compression format processing influence; a total of 20 types of image damage can be obtained, and each type of damage has five severity levels, thereby generating Image samples to be detected The image after the destruction.

[0026] In the image sample to be detected A collection of images of the aftermath of destruction There are A corrupted image, which means it contains poisoning detection channels. Such a large number of channels can effectively improve the diversity of candidate channels. Since the poisoning model will show significantly different robustness when processing trigger samples affected by different image corruptions, but will show relatively similar robustness under different image corruptions when processing clean samples, it is possible to evaluate whether a sample is poisoned based on the model's output of image corruption.

[0027] (3) Construct the main deep learning classification network and the target deep learning classification network; and use the destroyed image collection Iteratively update the weights of the main deep learning classification network until the maximum number of iterations is reached to obtain the optimized main deep learning classification network; and use the optimized main deep learning classification network to combine with the destroyed image set , construct the reward value matrix and the confidence matrix .

[0028] The step (3) specifically includes the following sub-steps: (3.1) Establishing a main deep learning classification network and a target deep learning classification network; the main deep learning classification network and the target deep learning classification network have the same structure, both of which are CNN networks.

[0029] The weights of the main deep learning classification network are then Perform random initialization to set the weights of the target deep learning classification network Initialize to .

[0030] (3.2) The destroyed image set Each image sample to be detected The image after the destruction As a poisoning detection channel ; Randomly select a poisoning detection channel , and get the corresponding action , the action Input the confidence matrix returned by the agent as the state ; Then the status Input to the main deep learning classification network, through the weights of the main deep learning classification network Initialize the deep learning classification network value, The value is calculated as follows: (1); in, represents the learning rate; Indicates status Execute an action The reward value obtained when represents the discount factor; Indicated in weight Next, status Execute an action of value; In the new state Based on weight , perform the action Maximum value; (3.3) Execution of actions And calculate the reward value and the next state , and then Stored in the experience replay pool; (3.4) Randomly select from the experience replay pool The loss function of the main deep learning classification network is calculated based on the records , the calculation formula is: ; ; in, Poisoning detection channel Target value of By minimizing the loss function To update the weights of the main deep learning classification network, the weights of the target deep learning classification network do not need to be iteratively updated. Instead, the weights are copied from the current main deep learning classification network every N rounds before the next round of learning. (3.5) Repeat steps (3.2) to (3.4), iteratively update the main deep learning classification network until the maximum number of iterations is reached, and obtain the optimized main deep learning classification network; Then all poisoning detection channels are input as actions into the optimized main deep learning classification network to obtain the reward value of each poisoning detection channel and construct the reward value matrix ; and the confidence matrix is ​​constructed by the states corresponding to all poisoning detection channels .

[0031] The CNN network is composed of a first convolutional layer, a first ReLU layer, a second convolutional layer, a second ReLU layer, a first maximum pooling layer, a third convolutional layer, a third ReLU layer, a fourth convolutional layer, a fourth ReLU layer, a second maximum pooling layer, a first fully connected layer, a fifth ReLU layer, a second fully connected layer and a Softmax layer in sequence.

[0032] (4) Then the reward value matrix and the confidence matrix The splicing input is fed into the lightweight poisoning detection model, and then the poisoning probability is output , according to the probability of poisoning Determine the image samples to be detected Whether poisoned.

[0033] The step (4) specifically includes the following sub-steps: (4.1) constructing a lightweight poisoning detection model, wherein the lightweight poisoning detection model consists of two dense fully connected layers and a Sigmoid function; (4.2) Then the reward value matrix and the confidence matrix The splicing input is fed into the lightweight poisoning detection model, and then the poisoning probability is output , if the probability of poisoning If it is greater than the poisoning threshold, it indicates that the detected image sample Poisoning, anti-rule detection image sample No poisoning, so as to achieve the detection of image samples Poisoning detection.

[0034] Example 2: This example relates to a multi-channel optimization poisoning detection device based on deep reinforcement learning, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, a multi-channel optimization poisoning detection method based on deep reinforcement learning is used in the above-mentioned example 1; the device embodiment can be applied to any device with data processing capabilities, and the any device with data processing capabilities can be a device or apparatus such as a computer.

[0035] like Figure 2 At the hardware level, the knowledge distillation device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, the present invention does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0036] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0037] The controller can be implemented in any suitable manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0038] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0039] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0040] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0041] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0042] Embodiment 3: The embodiment of the present invention further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, a multi-channel optimized poisoning detection method based on deep reinforcement learning of the above-mentioned embodiment 1 is implemented.

[0043] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-channel optimized poisoning detection method based on deep reinforcement learning, characterized in that: The following steps are involved: (1) Obtain image samples to be detected ; (2) Image samples to be detected Perform image destruction processing to obtain the image sample to be detected A collection of images of the aftermath of destruction : ,in, Represents any image sample to be detected The image after the destruction, Represents the image sample to be detected The number of images after destruction; (3) Construct the main deep learning classification network and the target deep learning classification network; and use the destroyed image collection Iteratively update the weights of the main deep learning classification network until the maximum number of iterations is reached to obtain the optimized main deep learning classification network; and use the optimized main deep learning classification network to combine with the destroyed image set , construct the reward value matrix and the confidence matrix ; (4) Then the reward value matrix and the confidence matrix The splicing input is fed into the lightweight poisoning detection model, and then the poisoning probability is output , according to the probability of poisoning Determine the image samples to be detected Whether poisoned.

2. According to the multi-channel optimized poisoning detection method based on deep reinforcement learning in claim 1, it is characterized in that: The image destruction processing includes image blurring, image noise addition, image weather environment influence and digital conversion processing influence.

3. According to claim 2, a multi-channel optimized poisoning detection method based on deep reinforcement learning is characterized in that: The image blur includes defocus blur, frosted glass blur, motion blur, zoom blur, and Gaussian blur; the image noise includes Gaussian noise, shot noise, impulse noise, defocus blur, and speckle noise; the image weather environment influence includes snow influence, frost influence, fog influence, and rain influence; the digital conversion processing influence includes brightness processing influence, contrast processing influence, saturation processing influence, elasticity processing influence, pixelation processing influence, and JPEG compression format processing influence.

4. According to the multi-channel optimized poisoning detection method based on deep reinforcement learning in claim 3, it is characterized in that: The step (3) specifically includes the following sub-steps: (3.1) establishing a main deep learning classification network and a target deep learning classification network; the main deep learning classification network and the target deep learning classification network have the same structure, both of which are CNN networks; The weights of the main deep learning classification network are then Perform random initialization to set the weights of the target deep learning classification network Initialize to ; (3.2) The destroyed image set Each image sample to be detected The image after the destruction As a poisoning detection channel ; Randomly select a poisoning detection channel , and get the corresponding action , the action Input the confidence matrix returned by the agent as the state ; Then the status Input to the main deep learning classification network, through the weights of the main deep learning classification network Initialize the deep learning classification network value, The value is calculated as follows: （1）； in, represents the learning rate; Indicates status Execute an action The reward value obtained when represents the discount factor; Indicated in weight Next, status Execute an action of value; In the new state Based on weight , perform the action Maximum value; (3.3) Execution of actions And calculate the reward value and the next state , and then Stored in the experience replay pool; (3.4) Randomly select from the experience replay pool The loss function of the main deep learning classification network is calculated based on the records , the calculation formula is: ； ； in, Poisoning detection channel Target value of By minimizing the loss function To update the weights of the main deep learning classification network, the weights of the target deep learning classification network do not need to be iteratively updated. Instead, the weights are copied from the current main deep learning classification network every N rounds before the next round of learning. (3.5) Repeat steps (3.2) to (3.4), iteratively update the main deep learning classification network until the maximum number of iterations is reached, and obtain the optimized main deep learning classification network; Then all poisoning detection channels are input as actions into the optimized main deep learning classification network to obtain the reward value of each poisoning detection channel and construct the reward value matrix ; and the confidence matrix is ​​constructed by the states corresponding to all poisoning detection channels .

5. According to claim 4, a multi-channel optimized poisoning detection method based on deep reinforcement learning is characterized in that: The CNN network is composed of a first convolutional layer, a first ReLU layer, a second convolutional layer, a second ReLU layer, a first maximum pooling layer, a third convolutional layer, a third ReLU layer, a fourth convolutional layer, a fourth ReLU layer, a second maximum pooling layer, a first fully connected layer, a fifth ReLU layer, a second fully connected layer and a Softmax layer in sequence.

6. The multi-channel optimized poisoning detection method based on deep reinforcement learning according to claim 4 is characterized in that: The step (4) specifically includes the following sub-steps: (4.1) constructing a lightweight poisoning detection model, wherein the lightweight poisoning detection model consists of two dense fully connected layers and a Sigmoid function; (4.2) Then the reward value matrix and the confidence matrix The splicing input is fed into the lightweight poisoning detection model, and then the poisoning probability is output , if the probability of poisoning If it is greater than the poisoning threshold, it indicates that the detected image sample Poisoning, anti-rule detection image sample No poisoning, so as to achieve the detection of image samples Poisoning detection.

7. A multi-channel optimized poisoning detection device based on deep reinforcement learning, characterized in that: It includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement a multi-channel optimized poisoning detection method based on deep reinforcement learning as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the multi-channel optimized poisoning detection method based on deep reinforcement learning as described in any one of claims 1-6 is implemented.

Images