A target detection model adversarial training method, target detection method and system

Abstract

The present invention provides an adversarial training method for a target detection model, the method comprising: S1, obtaining a target image data set and an initial adversarial pattern, the target image data set comprising a plurality of image samples, and each image sample being provided with a target bounding box annotation; S2, performing a fractal transformation on the initial adversarial pattern to obtain a target adversarial pattern; S3, injecting the target adversarial pattern obtained in step S2 into the target bounding box of each image sample in the target image data set to obtain an adversarial training set, and using the adversarial training set to train a target detection model until convergence. The present invention introduces a fractal transformation loss to iteratively update the adversarial pattern, so that the generated adversarial pattern has a certain self-similarity, and uses the adversarial pattern with self-similarity to perform adversarial training on the target detection model, which can improve the robustness of the target detection model and enable the model to have a stronger adversarial capability.

Description

Technical Field

[0001] The present invention relates to the field of deep learning, and more specifically, to target detection technology in the field of deep learning, and more specifically, to a target detection model adversarial training method, a target detection method and a system. Background Art

[0002] Infrared target detection models based on deep learning are widely used in security monitoring, autonomous driving, and security investigation. Studies have shown that deep learning is vulnerable to attacks. If infrared target detection models based on deep learning methods are attacked, it may pose a threat to personal and social security. The so-called attack refers to the interference or disturbance that appears on the target so that the infrared target detection model cannot accurately detect the target. However, the existing infrared target detection models based on deep learning cannot effectively resist attacks during the target detection process, resulting in a decrease in the detection accuracy of the infrared target detection model.

[0003] To address the above issues, some researchers have proposed using heat-generating or heat-insulating materials to form specific adversarial patterns, and then injecting the adversarial patterns into the target bounding box of each image in the training set of the infrared target detection model to obtain a new training set (targets with adversarial patterns in image samples can deceive the infrared target detection model and cause the model to miss detections or make false detections). Finally, the new training set is used to perform adversarial training on the infrared target detection model to give the model a certain adversarial capability, that is, the infrared target detection model can also achieve target detection when attacked, and maintain a good target recognition accuracy.

[0004] In the prior art, the adversarial pattern used for adversarial training is mainly obtained through training in a gradient manner, that is, the adversarial pattern is injected into the target bounding box of each image in the training set of the infrared target detection model to obtain the adversarial training set, and the adversarial training set is input into the trained infrared target detection model to increase the detection loss of the model output, and then the gradient of the adversarial pattern is calculated by back propagation and the adversarial pattern is updated to obtain the final adversarial pattern. Although the adversarial pattern obtained by the gradient method in the prior art can enable the infrared target detection model to have a certain anti-attack ability after adversarial training on the infrared target detection model. However, the effective attack distance of the adversarial pattern generated by the gradient method in the prior art is short and the attack effect is poor. When the adversarial training is performed on the infrared target detection model, the infrared target detection model obtained by training has poor robustness and poor adversarial ability. Summary of the invention

[0005] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a target detection model adversarial training method, a target detection method and a target detection system.

[0006] The objectives of the present invention are achieved through the following technical solutions.

[0007] According to a first aspect of the present invention, a method for adversarial training of a target detection model is provided, the method comprising: S1, obtaining a target image data set and an initial adversarial pattern, the target image data set comprising a plurality of image samples, and each image sample being provided with a target bounding box annotation; S2, performing a fractal transformation on the initial adversarial pattern to obtain a target adversarial pattern; S3, injecting the target adversarial pattern obtained in step S2 into a target bounding box of each image sample in the target image data set to obtain an adversarial training set, and using the adversarial training set to train a target detection model until convergence.

[0008] In some embodiments of the present invention, step S2 includes using the target image data set to perform multiple iterative updates on the initial adversarial pattern to obtain a target adversarial pattern, and each iterative update includes: S21, enhancing the adversarial pattern obtained after the last iterative update and injecting it into the target bounding box of each image sample in the target image data set to obtain a new training set, and inputting the new training set into the pre-trained target detection model and calculating its detection loss based on the output of the pre-trained target detection model; S22, performing a fractal transformation on the adversarial pattern obtained after the last iterative update according to a preset fractal transformation rule and calculating its fractal transformation loss; S23, summing the detection loss obtained in step S21 and the fractal transformation loss obtained in step S22 to calculate the total loss, and back-propagating the adversarial pattern obtained in the last iterative update based on the total loss to obtain the adversarial pattern updated in the current iterative update; wherein the adversarial pattern obtained in the last iterative update is used as the target adversarial pattern.

[0009] In some embodiments of the present invention, the preset fractal transformation rule is: performing multiple fractal transformations on the adversarial pattern obtained in the last iterative update according to a preset fractal scale and a preset transformation order, each fractal transformation obtains a fractal pattern and each fractal transformation includes: sampling the fractal pattern obtained in the last fractal transformation according to the preset fractal scale, and reconstructing and restoring the sampled fractal pattern to the same size as the adversarial pattern obtained in the last iterative update according to a preset fractal transformation method to obtain the fractal pattern of the current fractal transformation.

[0010] In some embodiments of the present invention, the fractal transformation loss is calculated in the following manner:

[0011]

[0012]

[0013] in, represents the fractal transformation loss; According to the fractal scale Update the adversarial pattern obtained in the previous iteration conduct The mean of the multi-order fractal transformation loss after the sub-fractal transformation; represents the fractal scale; represents the fractal transformation order; Represents the adversarial pattern obtained by the last iteration update; Indicates the fractal transformation mode; According to the fractal scale Update the adversarial pattern obtained in the previous iteration Conduct the Fractal transformation loss after sub-fractal transformation; According to the fractal scale Update the adversarial pattern obtained in the previous iteration Conduct the The fractal transformation obtained Step fractal pattern; represents the mean square error function.

[0014] In some embodiments of the present invention, the preset fractal scale is 2, and the preset transformation order is 3.

[0015] In some embodiments of the present invention, the preset fractal transformation mode is tiling transformation, Sierpinski carpet transformation or Sierpinski triangle transformation.

[0016] In some embodiments of the present invention, in step S1, the initial adversarial pattern is obtained in the following manner: directly generating a grayscale image as the initial adversarial pattern using the adversarial pattern pixel values; or obtaining a grayscale image as the initial adversarial pattern by random sampling based on the probability distribution of the adversarial pattern pixel values.

[0017] According to a second aspect of the present invention, a target detection method is provided, the method comprising: T1, obtaining image data to be processed; T2, using the target detection model trained according to the first aspect of the present invention to perform target detection on the image data to be processed obtained in step T1.

[0018] According to a third aspect of the present invention, a target detection system is provided, the system comprising: a data acquisition module, used to acquire image data to be processed; and a target detection model trained as in the first aspect of the present invention, used to perform target detection on the image data to be processed obtained by the data acquisition module.

[0019] Compared with the prior art, the advantages of the present invention are: fractal transformation loss is introduced to iteratively update the adversarial pattern, so that the generated adversarial pattern has a certain self-similarity. The adversarial pattern with self-similarity has higher attack capability than the adversarial pattern generated by the prior art, and can alleviate the impact of resolution reduction on the adversarial pattern characteristics. When the resolution is reduced, the adversarial pattern can still retain some adversarial pattern texture features, thereby effectively increasing the effective attack distance of the adversarial pattern in the physical world; further, the use of self-similar adversarial patterns to perform adversarial training on the target detection model can improve the robustness of the target detection model, so that the model has a stronger adversarial ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0021] Figure 1 A schematic diagram of the adversarial attack effect when a target carrying an adversarial pattern generated by the prior art is 2 meters, 3 meters, 4 meters, and 5 meters away from an infrared target detector;

[0022] Figure 2 A schematic diagram of a process flow of an adversarial training method for a target detection model according to an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of a Sierpinski carpet transformation method according to an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of a Sierpinski triangle transformation method according to an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of a one-time iterative update process of an initial confrontation pattern according to an embodiment of the present invention;

[0026] Figure 6 Schematic diagram of the adversarial pattern generated by the prior art in the digital world, visible light in the physical world, and infrared in the physical world;

[0027] Figure 7 A schematic diagram of a countermeasure pattern generated according to an embodiment of the present invention in the digital world, visible light in the physical world, and infrared in the physical world;

[0028] Figure 8 A schematic diagram of the counterattack effect of a target carrying a counterattack pattern generated by an embodiment of the present invention at a distance of 2 meters, 3 meters, 4 meters, and 5 meters from an infrared target detector;

[0029] Fig. 9 A schematic diagram of a target detection method according to an embodiment of the present invention;

[0030] Fig.10Schematic diagram of a target detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail by specific embodiments below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0032] As mentioned in the background technology, the effective attack distance of the adversarial pattern generated by the prior art in a gradient manner is short and the attack effect is poor. When the adversarial pattern is used to train the infrared target detection model, the trained infrared target detection model has poor robustness and poor adversarial ability. The inventors have found that the reason why the effective attack distance of the adversarial pattern generated by the gradient manner in the prior art is short and the attack effect is poor is because the adversarial pattern generated by the gradient manner is relatively fragmented. As the resolution of the adversarial pattern decreases, it will quickly lose its original texture features that have attack capabilities, which leads to poor attack effect of the adversarial pattern. For example, when the adversarial pattern generated by the gradient manner is used to attack the infrared target detector (which includes an infrared sensor and an infrared target detector, the infrared sensor is used to capture images, and the infrared target detection model is used to process the images captured by the infrared sensor to detect targets), when the resolution of the infrared sensor is 640×480, the effective attack distance of the adversarial pattern does not exceed 10 meters, that is, the target carrying the adversarial pattern can be detected after 10 meters. Figure 1 As shown, it shows the counterattack effect diagram of the target carrying the counterattack pattern when it is 2 meters, 3 meters, 4 meters, and 5 meters away from the infrared target detector. It can be seen from the figure that when the target carrying the counterattack pattern is 2 meters or 3 meters away from the infrared target detector, the infrared target detector cannot recognize the target, and the counterattack pattern has a better attack capability at this time; when the target carrying the counterattack pattern is 3 meters or 4 meters away from the infrared target detector, the infrared target detector can recognize the target carrying the counterattack pattern, and the counterattack pattern loses its attack capability.

[0033] In order to solve the above problems, the inventors proposed a solution of obtaining adversarial patterns by fractal transformation and using them in adversarial training of target detection models. The adversarial patterns obtained by fractal transformation have self-similarity (self-similarity means that one part of the pattern is similar to other parts). After the resolution of the adversarial pattern with self-similarity is reduced, the entire pattern still has similar texture features to a part of the original pattern, that is, the adversarial pattern with self-similarity can alleviate the impact of resolution reduction on the texture features of the adversarial pattern, and it still has strong attack capabilities after the resolution is reduced. In other words, the effective attack distance of the adversarial pattern with self-similarity is farther. Using the adversarial pattern with self-similarity to conduct adversarial training on the target detection model can improve the robustness of the target detection model and make the model have stronger adversarial capabilities.

[0034] In order to better understand the present invention, the scheme of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0035] According to one embodiment of the present invention, Figure 2 As shown, the present invention provides an adversarial training method for a target detection model, the method comprising: S1, obtaining a target image data set and an initial adversarial pattern, the target image data set comprising a plurality of image samples, and each image sample being provided with a target bounding box annotation; S2, performing a fractal transformation on the initial adversarial pattern to obtain a target adversarial pattern; S3, injecting the target adversarial pattern obtained in step S2 into a target bounding box of each image sample in the target image data set to obtain an adversarial training set, and using the adversarial training set to train the target detection model until convergence.

[0036] In order to better understand the present invention, the adversarial training method of the present invention is described in detail from the aspects of data acquisition, initial adversarial pattern update, target detection model training and comparative experiments.

[0037] 1. Get data

[0038] 1.1 Obtain target image dataset

[0039] There are many mature image samples in the open source image dataset that can be used for training the target detection model. Each image in such samples is provided with a target bounding box annotation. According to one embodiment of the present invention, the present invention selects multiple image samples from the open source dataset FLIR_ADAS to form a target image dataset.

[0040] 1.2 Obtaining the initial confrontation pattern

[0041] In adversarial training, the texture features of the adversarial pattern directly affect the ability of the target detection model obtained after the final training to resist attacks. The present invention processes the initial adversarial pattern to obtain an adversarial pattern with a stronger attack effect. According to one embodiment of the present invention, a grayscale image is directly generated using the adversarial pattern pixel value as the initial adversarial pattern; or a grayscale image is obtained by random sampling based on the probability distribution of the adversarial pattern pixel value as the initial adversarial pattern.

[0042] 2. Initial confrontation pattern update

[0043] 2.1 Initial confrontation pattern update method

[0044] The texture features of the initial adversarial pattern obtained by random sampling based on the adversarial pattern pixel values ​​or the probability distribution of the adversarial pattern pixel values ​​are relatively fragmented and have weak attack capabilities. If the texture features are directly injected into the target bounding box of each image sample in the target image data set to obtain an adversarial training set, and the adversarial training set is used to perform adversarial training on the target detection model, the adversarial capability of the trained target detection model is poor. Therefore, the initial adversarial pattern needs to be further processed to obtain a target adversarial pattern with strong attack capabilities. In the present invention, the initial adversarial pattern is fractal transformed to obtain the target adversarial pattern.

[0045] According to one embodiment of the present invention, when fractal transformation is performed on the initial adversarial graph, the target image data set is used to iteratively update the initial adversarial pattern multiple times to obtain the target adversarial pattern. Each iterative update includes steps P1-P3. Each step is described in detail below.

[0046] In step P1, the adversarial pattern obtained after the last iterative update is enhanced and then injected into the target bounding box of each image sample in the target image data set to obtain a new training set, and the new training set is input into the pre-trained target detection model and its detection loss is calculated based on the output of the pre-trained target detection model. It should be noted that when the first iterative update is performed, the initial adversarial pattern is enhanced and then injected into the target bounding box of each image sample in the target image data set to obtain a new training set. According to one embodiment of the present invention, the adversarial pattern is enhanced and then injected into the target bounding box of each image sample in the target image data set to obtain a new training set. According to one embodiment of the present invention, the adversarial pattern is enhanced and processed, including: first tiling the adversarial pattern in 3*3 or 4*4 or other formats (tiling means that the adversarial pattern is copied multiple times and then expanded in a prescribed format. Taking 3*3 as an example, the adversarial pattern is copied 8 times and then expanded in a nine-square grid style to obtain a tiled adversarial pattern), then the tiled adversarial pattern is randomly cropped (the size of the random cropping is determined according to actual needs, and the present invention does not make specific restrictions), then the cropped adversarial pattern is deformed, and finally the deformed adversarial pattern is rotated, denoised, scaled or brightness adjusted to obtain an enhanced adversarial pattern. According to one embodiment of the present invention, the pre-trained target detection model may adopt a YOLO series or RCNN series model. It should also be noted that after the new training set is input into the pre-trained target detection model, the output of the model includes the target detection box position, target category and target confidence. According to one embodiment of the present invention, the target confidence output by the pre-trained target detection model is used as its detection loss. .

[0047] In step P2, the antagonistic pattern obtained after the last iterative update is fractal transformed according to the preset fractal transformation rules and its fractal transformation loss is calculated. According to one embodiment of the present invention, the preset fractal transformation rules are: the antagonistic pattern obtained after the last iterative update is subjected to multiple fractal transformations according to the preset fractal scale and the preset transformation order, each fractal transformation obtains a fractal pattern and each fractal transformation includes: sampling the fractal pattern obtained by the last fractal transformation according to the preset fractal scale, and reconstructing and restoring the sampled fractal pattern to the same size as the antagonistic pattern obtained by the last iterative update according to the preset fractal transformation method to obtain the fractal pattern of the current fractal transformation. According to one embodiment of the present invention, the preset fractal scale is 2, and the preset transformation order is 3. According to one embodiment of the present invention, the preset fractal transformation method is a tiling transformation, a Sierpinski carpet transformation or a Sierpinski triangle transformation. Since these fractal transformations are all common graphic transformation forms, the present invention does not elaborate on them one by one, and only exemplifies the Sierpinski carpet transformation and the Sierpinski triangle transformation. Among them, Figure 3 shown, which demonstrates the Sierpinski carpet transformation, in Figure 3The leftmost pattern is the original pattern; the second left pattern is a first-order fractal pattern, which is obtained by performing a fractal transformation on the original pattern. The transformation process is: shrink the original pattern along the length and width directions. Get the reduced pattern (downsampling), then copy n reduced patterns and arrange them in a nine-square grid style, remove the center pattern to get a first-order fractal pattern of the same size as the original pattern (the first-order fractal pattern includes n-1 reduced original patterns); the third pattern from the left is a second-order fractal pattern, which is obtained by a fractal transformation of the first-order fractal pattern. The transformation process is: reduce the first-order fractal pattern along the length and width directions Get the reduced pattern, then copy n reduced patterns and arrange them in a nine-square grid style, then remove the center pattern to get a second-order fractal pattern of the same size as the original pattern (the second-order fractal pattern includes n-1 reduced first-order fractal patterns); the pattern on the far right is a third-order fractal pattern, which is obtained by transforming the second-order fractal pattern. The transformation process is: reduce the second-order fractal pattern along the length and width directions Get the reduced pattern, then copy n reduced patterns and arrange them in a nine-square grid pattern, then remove the center pattern to get a third-order fractal pattern with the same size as the original pattern (the third-order fractal pattern includes n-1 reduced second-order fractal patterns). Figure 4 As shown, it shows the Sierpinski triangle transformation, in Figure 4 In the figure, the first row is the fractal patterns of various orders obtained by conventional Sierpinski triangle transformation, and the second row is the fractal patterns of various orders obtained by transforming the square according to Sierpinski triangle transformation; wherein, the first row is used as an example to illustrate Sierpinski triangle transformation, the first row from the left is the original pattern; the first row from the left is a first-order fractal pattern, which is transformed from the original pattern, and the transformation process is: each side in the original pattern is reduced to the original Get the reduced pattern (downsampling), then copy n reduced patterns and combine them into a pattern of the same size as the original pattern, then remove the pattern in the center to get the first-order fractal pattern (the first-order fractal pattern includes n-1 reduced original patterns); the third from the left in the first row is the second-order fractal pattern, which is obtained by transforming the first-order fractal pattern. The transformation process is: reduce each edge in the first-order fractal pattern to the original Get the reduced pattern, then copy n reduced patterns and combine them into a pattern of the same size as the original pattern, then remove the pattern in the center to get a second-order fractal pattern (the second-order fractal pattern includes n-1 reduced first-order fractal patterns); the fourth from the left in the first row is a third-order fractal pattern, which is obtained by transforming the second-order fractal pattern. The transformation process is: reduce each side of the second-order fractal pattern to the original Get the reduced pattern, then copy n reduced patterns and combine them into a pattern of the same size as the original pattern, then remove the pattern in the center to get a third-order fractal pattern (the third-order fractal pattern includes n-1 reduced second-order fractal patterns); the first row on the right is a fourth-order fractal pattern, which is obtained by transforming the third-order fractal pattern. The transformation process is: reduce each side of the third-order fractal pattern to the original The reduced pattern is obtained, and then n reduced patterns are copied and combined into a pattern with the same size as the original pattern, and then the pattern in the center is removed to obtain a fourth-order fractal pattern (the fourth-order fractal pattern includes n-1 reduced third-order fractal patterns).

[0048] According to one embodiment of the present invention, the fractal transformation loss is calculated in the following manner:

[0049]

[0050]

[0051] in, represents the fractal transformation loss; According to the fractal scale Update the adversarial pattern obtained in the previous iteration conduct The mean of the multi-order fractal transformation loss after the sub-fractal transformation; represents the fractal scale; represents the fractal transformation order; Represents the adversarial pattern obtained by the last iteration update; Indicates the fractal transformation mode; According to the fractal scale Update the adversarial pattern obtained in the previous iteration Conduct the Fractal transformation loss after sub-fractal transformation; According to the fractal scale Update the adversarial pattern obtained in the previous iteration Conduct the The fractal transformation obtained Step fractal pattern; represents the mean square error function.

[0052] In step P3, the loss is detected With fractal transformation loss Sum to calculate the total loss , and based on the total loss Back propagation updates the adversarial pattern obtained in the previous iteration to obtain the adversarial pattern updated in the current iteration; wherein the adversarial pattern obtained in the last iteration is used as the target adversarial pattern. It should be noted that based on the total loss The back-propagation update process can be understood as follows: based on steps P1-P3, a computational graph is constructed using a deep learning framework such as TensorFlow or PyTorch (a computational graph is a directed acyclic graph, which includes several nodes and edges, where nodes represent computing units (such as matrix multiplication, addition, etc.), and edges represent data flow (i.e., the transfer of data between computing units), and the constructed computational graph is used to iteratively update the initial adversarial image multiple times to obtain the target adversarial pattern. The calculation process of the computational graph is to calculate the tensors of each node in the graph in turn according to the relationship between nodes and edges, where the tensor of the first node in the computational graph is the parameter of the initial adversarial pattern (the parameter is used to generate the initial adversarial pattern), and the tensor of the last node in the computational graph represents the total loss. Based on the tensor of the calculated last node, the gradient of each node in the computational graph is calculated by back-propagation (the gradient is calculated with minimization as the goal), and then the first node in the computational graph is updated based on the calculated gradient, that is, the parameters of the initial adversarial pattern are updated. For example, taking the pytorch deep learning framework as an example to construct a computational graph, its back-propagation update can be implemented with the following pseudo code, directly using The function back propagates to calculate the gradient of each node in the entire computational graph, and then passes the parameters to be updated to the optimizer for updating. Its pseudo code can be expressed as optimizer=SGD(pi), where optimizer represents the optimizer. represents the stochastic gradient descent algorithm, and pi represents the parameters of the initial adversarial pattern. It should be noted that different optimizers can be selected to update the parameters according to actual needs, and only the SGD optimizer is used as an example for explanation here, and the present invention does not impose specific restrictions on the optimizer.

[0053] 2.2 Description of an iterative update of the initial adversarial pattern

[0054] In order to better understand the updating process of the adversarial pattern, the process of iteratively updating the adversarial pattern is described in detail below with reference to the accompanying drawings and examples.

[0055] like Figure 5 As shown in the figure, it shows the process of iteratively updating the initial adversarial pattern. In the figure, π represents the parameter for generating the initial adversarial pattern. Figure 5 The process of performing the first iteration update on the initial confrontation pattern is described in detail by taking the fractal transformation mode as tiling transformation, the preset fractal scale as 2, and the preset transformation order as 3 as an example. The subsequent iteration process is similar to the first one and will not be described again.

[0056] When the initial adversarial pattern is updated for the first time, the initial adversarial pattern is first enhanced and then injected into the target bounding box of each image sample in the target image data set to obtain a new training set, and the new training set is input into the pre-trained target detection model and its detection loss is calculated based on the output of the pre-trained target detection model. .

[0057] Then, the initial confrontation pattern is fractally transformed according to the preset fractal transformation rules and its fractal transformation loss is calculated. The initial confrontation pattern is fractally transformed three times according to the preset fractal scale of 2 and the preset transformation order of 3. Each fractal transformation obtains a fractal pattern. The process of the first fractal transformation is: the initial confrontation pattern is downsampled according to the preset fractal scale, that is, the horizontal and vertical directions of the initial confrontation pattern are reduced to 1 / 2 of the original ( , represents the fractal scale), the initial adversarial image after downsampling can be expressed as:

[0058]

[0059] in, Represents the initial confrontation pattern No. OK The pixel value of the column, Indicates that the initial confrontation pattern Scale down to the original size both horizontally and vertically , represents the fractal scale, The size is , the initial confrontation pattern The size is Then the downsampled initial adversarial pattern Reconstruct the original adversarial pattern by tiling transformation The size is consistent to obtain the fractal pattern of the current fractal transformation (constructing the fractal pattern), which can be expressed as:

[0060]

[0061]

[0062] in, Represents the fractal transformation mode (tiling transformation), According to the fractal scale Initial confrontation pattern The first-order fractal pattern obtained by the first fractal transformation, Indicates that the initial confrontation pattern Scale down to the original size both horizontally and vertically , Represents the initial confrontation pattern Calculate the first-order fractal pattern With the initial confrontation pattern The first-order fractal loss between:

[0063]

[0064]

[0065] in, Represents the first-order fractal pattern With the initial confrontation pattern The first-order fractal loss between Represents the first-order fractal pattern With the initial confrontation pattern The mean square error of each pixel, Represents the first-order fractal pattern Or initial confrontation pattern Size.

[0066] Furthermore, the second-order fractal pattern is constructed in the same way as the first fractal transformation. With the 3rd order fractal pattern , and calculate the second-order fractal pattern respectively With the initial confrontation pattern The second-order fractal loss and the third-order fractal pattern With the initial confrontation pattern The third-order fractal loss between , the calculation process is expressed as follows:

[0067]

[0068]

[0069]

[0070]

[0071] in, Represents a 2nd order fractal pattern With the initial confrontation pattern The second-order fractal loss between Represents a 2nd order fractal pattern With the initial confrontation pattern The mean square error of each pixel, Represents a third-order fractal pattern With the initial confrontation pattern The third-order fractal loss between Represents a third-order fractal pattern With the initial confrontation pattern The mean square error of each pixel, Represents a 2nd order fractal pattern , 3rd order fractal pattern Or initial confrontation pattern It should be noted that the second-order fractal pattern With the 3rd order fractal pattern The expression of is the same as above and will not be repeated here.

[0072] Furthermore, the first-order fractal loss calculated based on the above content , second-order fractal loss and the third-order fractal loss , calculate the initial adversarial pattern According to the preset fractal scale ( ) and the preset transformation order ( ) is subjected to fractal transformation, and the calculation process of the fractal loss is as follows:

[0073]

[0074] in, represents the fractal transformation loss; According to the fractal scale Initial confrontation pattern conduct The mean of the multi-order fractal transformation loss after the sub-fractal transformation; represents the fractal scale ( ); represents the fractal transformation order ( ); represents the initial confrontation pattern; Indicates the fractal transformation mode; According to the fractal scale Initial confrontation pattern Conduct the Fractal transformation loss after sub-fractal transformation.

[0075] Finally, the detection loss With fractal transformation loss Sum to calculate the total loss , and based on the total loss Back-propagation updates the initial adversarial pattern to obtain the updated adversarial pattern for the first iteration.

[0076] It should be noted that the above content describes the process of fractal transformation of the initial confrontation pattern by tiling transformation. If the initial confrontation pattern is fractal transformed by Sierpinski carpet transformation, the fractal pattern obtained after the first fractal transformation can be expressed as:

[0077]

[0078]

[0079] in, Indicates that the initial confrontation pattern Scale down to the original size both horizontally and vertically , represents the fractal scale, The size is , represents the fractal transformation (Sierpinski carpet transformation), According to the fractal scale Initial confrontation pattern The first-order fractal pattern obtained by performing the first fractal transformation. The fractal pattern expression obtained by performing fractal transformation on the initial confrontation pattern using other fractal transformation methods is similar to the above expression, and will not be illustrated one by one here.

[0080] 3. Object Detection Model Training

[0081] After multiple iterative updates of the initial adversarial pattern, the target adversarial pattern is obtained. Compared with the initial adversarial pattern, the texture features of the target adversarial pattern have a stronger attack capability. After using it to perform adversarial training on the target detection model, the target detection model obtained has good robustness and strong adversarial capability. According to one embodiment of the present invention, after obtaining the target image data set and the target adversarial pattern, the obtained target adversarial pattern is injected into the target bounding box of each image sample in the target image data set to obtain an adversarial training set, and the adversarial training set is used to train the target detection model until convergence. It should be noted that the use of an adversarial training set to perform adversarial attack training on the target detection model until convergence is a technology well known to those skilled in the art and will not be elaborated on herein.

[0082] 4. Comparative Experiment

[0083] In order to verify that the solution proposed by the present invention is superior to the prior art, the inventors respectively used the solution proposed by the embodiment of the present invention and the prior art to generate different confrontation patterns, such as Figure 6 and Figure 7 As shown, Figure 6 A schematic diagram showing the adversarial pattern generated by the existing method in the digital world, the visible light of the physical world, and the infrared of the physical world; Figure 7Schematic diagram showing the confrontation pattern generated by the embodiment of the present invention in the digital world, the physical world visible light, and the physical world infrared. As can be seen from the figure, compared with the relatively scattered and random confrontation pattern generated by the existing method, the confrontation pattern generated by the scheme proposed by the present invention not only has scattered pixel points to ensure the effect of close-range confrontation attack, but also has relatively complete pixel blocks to ensure that the attack effect at a long distance is similar to that at a close distance.

[0084] In order to further verify the attack capabilities of the adversarial patterns generated by the prior art solution and the adversarial patterns generated by the solution proposed in the embodiment of the present invention, the targets are made to carry different adversarial patterns to attack the infrared target detector at the same distance. The attack effects are shown in FIG. Figure 1 and Figure 8 As shown, Figure 1 The adversarial attack effect diagram of the target carrying the adversarial pattern generated by the existing method when it is 2 meters, 3 meters, 4 meters, and 5 meters away from the infrared target detector is shown. Figure 8 The counterattack effect of the target carrying the counterattack pattern generated by the embodiment of the present invention at a distance of 2 meters, 3 meters, 4 meters, and 5 meters from the infrared target detector is shown. As can be seen from the figure, when the target carrying the counterattack pattern generated by the existing method is 2 meters or 3 meters away from the infrared target detector, the infrared target detector cannot recognize the target; when the target carrying the counterattack pattern generated by the existing method is 3 meters or 4 meters away from the infrared target detector, the infrared target detector can recognize the target carrying the counterattack pattern, that is, the counterattack pattern generated by the existing method loses its attack capability at a distance of 4 meters and 5 meters; when the target carrying the counterattack pattern generated by the embodiment of the present invention is 2 meters, 3 meters, 4 meters, or 5 meters away from the infrared target detector, the infrared target detector cannot recognize the target. At this time, the counterattack pattern generated by the embodiment of the present invention still maintains its attack capability at a distance of 4 meters or 5 meters from the infrared target detector. It can be seen that the counterattack pattern generated by the embodiment of the present invention has a strong attack capability.

[0085] Compared with the target detection model obtained by adversarial training using the prior art, the target detection model obtained by adversarial training according to the scheme proposed by the present invention has better anti-attack capability and can be directly applied to target detection. Based on this, the present invention also proposes a target detection method and a target detection system.

[0086] According to one embodiment of the present invention, Fig. 9 As shown, the present invention provides a target detection method, which includes: T1, obtaining image data to be processed; T2, using the target detection model trained in the above-mentioned embodiment to perform target detection on the image data to be processed obtained in step T1.

[0087] According to one embodiment of the present invention, Fig.10As shown, the present invention provides a target detection system, which includes: a data acquisition module, used to acquire image data to be processed; a target detection model obtained by training as in the above-mentioned embodiment, used to perform target detection on the image data to be processed obtained by the data acquisition model.

[0088] The beneficial effects of the present invention are: fractal transformation loss is introduced to iteratively update the adversarial pattern, so that the generated adversarial pattern has a certain self-similarity. The adversarial pattern with self-similarity has higher attack capability than the adversarial pattern generated by the prior art, and can alleviate the impact of resolution reduction on the adversarial pattern characteristics. When the resolution is reduced, the adversarial pattern can still retain some adversarial pattern texture features, thereby effectively increasing the effective attack distance of the adversarial pattern in the physical world; further, the use of self-similar adversarial patterns to perform adversarial training on the target detection model can improve the robustness of the target detection model, so that the model has a stronger adversarial ability.

[0089] It should be noted that although the above describes the various steps in a specific order, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0090] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0091] A computer-readable storage medium may be a tangible device that holds and stores instructions used by an instruction execution device. Computer-readable storage media may include, for example, but are not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0092] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for adversarial training of a target detection model, characterized in that: The method comprises: S1. Obtain a target image dataset and an initial adversarial pattern, wherein the target image dataset includes a plurality of image samples, and each image sample is provided with a target bounding box annotation; S2. Performing fractal transformation on the initial confrontation pattern according to a preset fractal transformation method to obtain a target confrontation pattern; wherein the preset fractal transformation method is a tiling transformation, a Sierpinski carpet transformation or a Sierpinski triangle transformation; S3. Inject the target adversarial pattern obtained in step S2 into the target bounding box of each image sample in the target image data set to obtain an adversarial training set, and use the adversarial training set to train the target detection model until convergence.

2. The target detection model adversarial training method according to claim 1, characterized in that: The step S2 includes using the target image data set to iteratively update the initial adversarial pattern multiple times to obtain a target adversarial pattern, and each iterative update includes: S21, performing enhancement processing on the adversarial pattern obtained after the last iterative update and injecting it into the target bounding box of each image sample in the target image data set to obtain a new training set, and inputting the new training set into the pre-trained target detection model and calculating its detection loss based on the output of the pre-trained target detection model; S22, fractal transformation is performed on the adversarial pattern obtained after the last iterative update according to a preset fractal transformation rule, and its fractal transformation loss is calculated; wherein the preset fractal transformation rule is: multiple fractal transformations are performed on the adversarial pattern obtained after the last iterative update according to a preset fractal scale and a preset transformation order, each fractal transformation obtains a fractal pattern and each fractal transformation includes: sampling the fractal pattern obtained after the last fractal transformation according to a preset fractal scale, and reconstructing and restoring the sampled fractal pattern according to a preset fractal transformation method to a size consistent with the adversarial pattern obtained after the last iterative update to obtain a fractal pattern of the current fractal transformation; and calculating the fractal transformation loss in the following manner: in, represents the fractal transformation loss; According to the fractal scale Update the adversarial pattern obtained in the previous iteration conduct The mean of the multi-order fractal transformation loss after the sub-fractal transformation; represents the fractal scale; represents the fractal transformation order; Represents the adversarial pattern obtained by the last iteration update; Indicates the fractal transformation mode; According to the fractal scale Update the adversarial pattern obtained in the previous iteration Conduct the Fractal transformation loss after sub-fractal transformation; According to the fractal scale Update the adversarial pattern obtained in the previous iteration Conduct the The fractal transformation obtained Step fractal pattern; represents the mean square error function; S23, calculating the total loss by summing the detection loss obtained in step S21 and the fractal transformation loss obtained in step S22, and updating the adversarial pattern obtained in the previous iteration by backpropagation based on the total loss to obtain the adversarial pattern updated in the current iteration; Among them, the adversarial pattern obtained by the last iterative update is used as the target adversarial pattern.

3. The target detection model adversarial training method according to claim 2, characterized in that: The preset fractal scale is 2, and the preset transformation order is 3.

4. The target detection model adversarial training method according to claim 1, characterized in that: In step S1, the initial confrontation pattern is obtained in the following manner: The grayscale image is directly generated using the pixel value of the adversarial pattern as the initial adversarial pattern; Alternatively, random sampling is performed based on the probability distribution of the adversarial pattern pixel values ​​to obtain a grayscale image as the initial adversarial pattern.

5. A target detection method, characterized in that: The method comprises: T1. Obtain image data to be processed; T2. Use the target detection model trained by the method described in any one of claims 1 to 4 to perform target detection on the image data to be processed obtained in step T1.

6. A target detection system, characterized in that: The system comprises: A data acquisition module, used for acquiring image data to be processed; The target detection model trained by the method as described in any one of claims 1 to 4 is used to perform target detection on the image data to be processed obtained by the data acquisition module.

7. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of any method described in claims 1-5.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the electronic device to implement the steps of the method as described in any one of claims 1-5.

Images