Method and device for generating adversarial samples, storage medium and electronic device

Abstract

The present invention discloses a method and device for generating adversarial samples, a storage medium and an electronic device. The method comprises: obtaining a group of sample images and a group of label information, adjusting the pixel values ​​of the pixels in the adversarial image according to the detection results output by the target detection network model, and obtaining the target adversarial image, wherein the detection result output by the target detection network model is the detection result obtained by inputting the training image into the target detection network model, and the training image is the image obtained by setting the adversarial image in the second actual area of ​​the sample image, and when the adversarial image is the target adversarial image, the detection result output by the target detection network model satisfies the preset loss condition, and the target adversarial image is set as the adversarial image in the second actual area of ​​a group of sample images to obtain a group of adversarial samples. The present invention solves the technical problem of poor robustness of adversarial samples existing in the related art.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a method and device for generating adversarial samples, a storage medium, and an electronic device. Background Art

[0002] Deep learning has received widespread attention in many fields such as face recognition and autonomous driving. At the same time, the security issues in deep learning algorithms have also attracted more and more attention from researchers. For example, adversarial samples, which are samples generated by adding small perturbations that are almost imperceptible to the human visual system, can make the neural network classifier subvert the prediction of the image with a high degree of confidence. Therefore, with the development of technology, technologies such as autonomous driving and face verification have brought great convenience to people, but at the same time, they also need to meet extremely high accuracy and security.

[0003] In current related technologies, most adversarial sample generation algorithms add tiny noise that is difficult for humans to perceive to digital images to cause classifier misclassification, or cause target detectors to fail to correctly detect the location and category of targets, etc. Usually, this type of adversarial sample has poor robustness, and simple image processing methods such as image resizing and filtering operations can remove this type of adversarial noise, resulting in the technical problem of poor robustness of adversarial samples in related technologies.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] Embodiments of the present invention provide a method and device for generating an adversarial sample, a storage medium, and an electronic device, so as to at least solve the technical problem of poor robustness of adversarial samples existing in the related art.

[0006] According to one aspect of an embodiment of the present invention, a method for generating adversarial samples is provided, comprising: obtaining a group of sample images and a group of label information, wherein the group of sample images has a one-to-one correspondence with the group of label information, each of the sample images contains an object of target classification, and the label information includes a first actual area of ​​the object of target classification in the sample image, and a second actual area of ​​the adversarial image in the sample image; adjusting the pixel values ​​of pixels in the adversarial image according to a detection result output by a target detection network model to obtain a target adversarial image, wherein the detection result output by the target detection network model is the detection result obtained by inputting a training image into the target detection network model, the training image is an image obtained by setting the adversarial image in the second actual area of ​​the sample image, and when the adversarial image is the target adversarial image, the detection result output by the target detection network model satisfies a preset loss condition; setting the target adversarial image as the adversarial image in the second actual area of ​​the group of sample images to obtain a group of adversarial samples.

[0007] According to another aspect of an embodiment of the present invention, there is also provided an adversarial sample generation device, comprising: an acquisition module, used to acquire a group of sample images and a group of label information, wherein the group of sample images and the group of label information have a one-to-one correspondence, each of the sample images contains an object of target classification, and the label information includes a first actual area of ​​the object of target classification in the sample image, and a second actual area of ​​the adversarial image in the sample image; an adjustment module, used to adjust the pixel values ​​of the pixels in the adversarial image according to the detection result output by the target detection network model to obtain a target adversarial image, wherein the detection result output by the target detection network model is the detection result obtained by inputting a training image into the target detection network model, and the training image is an image obtained by setting the adversarial image in the second actual area in the sample image, and when the adversarial image is the target adversarial image, the detection result output by the target detection network model satisfies a preset loss condition; a setting module, used to set the target adversarial image as the adversarial image in the second actual area in the group of sample images to obtain a group of adversarial samples.

[0008] Optionally, the adjustment module includes: a first adjustment unit, used to adjust the pixel values ​​of pixels in the adversarial image according to the classification confidence and / or the prediction area output by the target detection network model to obtain the target adversarial image, wherein the classification confidence is used to indicate the probability that the object recognized by the target detection network model in the training image belongs to the target classification, and the prediction area is the area in the training image of the object recognized by the target detection network model in the training image.

[0009] Optionally, the first adjustment unit includes: a first adjustment sub-unit, used to adjust the pixel values ​​of the pixel points in the adversarial image according to the classification confidence output by the target detection network model to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the classification confidence output by the target detection network model is the smallest.

[0010] Optionally, the first adjustment subunit is used to adjust the pixel values ​​of the pixels in the adversarial image according to the classification confidence output by the target detection network model in the following manner to obtain the target adversarial image: repeatedly perform the following steps until it is determined that the classification confidence output by the target detection network model is the minimum, wherein, when performing the following steps, the adversarial image is regarded as the current adversarial image: obtain the training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the group of sample images; input the training image of this round into the target detection network model to obtain the classification confidence output by the target detection network model in this round; when the difference between the classification confidence output in this round and the classification confidence output in the previous round of the target detection network model is greater than a first preset threshold, adjust the pixel values ​​of the pixels in the current adversarial image to obtain the updated current adversarial image; when the difference between the classification confidence output in this round and the classification confidence output in the previous round is less than or equal to the first preset threshold, determine that the classification confidence output by the target detection network model is the minimum.

[0011] Optionally, the first adjustment subunit is used to adjust the pixel values ​​of the pixel points in the current adversarial image in the following manner to obtain the updated current adversarial image: obtain the gradient descent information of the objective function, wherein the objective function is used to obtain the pixel values ​​of the pixel points in the current adversarial image when the classification confidence output by the target detection network model is minimized; according to the gradient descent information, adjust the pixel values ​​of the pixel points in the current adversarial image to obtain the updated current adversarial image.

[0012] Optionally, the first adjustment unit includes: a second adjustment sub-unit, used to adjust the pixel values ​​of the pixel points in the adversarial image according to the predicted area output by the target detection network model to obtain the target adversarial image, wherein, when the adversarial image is the target adversarial image, the area overlap between the predicted area and the first actual area is minimized.

[0013] Optionally, the second adjustment subunit is used to adjust the pixel values ​​of the pixel points in the adversarial image according to the predicted area output by the target detection network model in the following manner to obtain the target adversarial image: repeatedly perform the following steps until it is determined that the area overlap is minimum, wherein, when performing the following steps, the adversarial image is regarded as the current adversarial image: obtain the training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the group of sample images, and the first actual area in the label information corresponding to the one sample image is the actual area of ​​this round; input the training image of this round into the target detection network model to obtain the predicted area output by the target detection network model in this round; the area overlap determined in this round When the difference between the area overlap degree obtained in the current round and the area overlap degree obtained in the previous round is greater than the second preset threshold, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image, wherein the area overlap degree obtained in the current round is: the intersection of the predicted area output in the current round and the actual area in the current round divided by the union of the predicted area output in the current round and the actual area in the current round, and the area overlap degree obtained in the previous round is: the intersection of the predicted area output in the previous round and the actual area in the previous round divided by the union of the predicted area output in the previous round and the actual area in the previous round; when the difference between the area overlap degree obtained in the current round and the area overlap degree obtained in the previous round is less than or equal to the second preset threshold, it is determined that the area overlap degree is the smallest.

[0014] Optionally, the second adjustment subunit is used to adjust the pixel values ​​of the pixel points in the current adversarial image in the following manner to obtain the updated current adversarial image: obtain the gradient descent information of the objective function, wherein the objective function is used to obtain the pixel values ​​of the pixel points in the current adversarial image when the region overlap is minimized; and adjust the pixel values ​​of the pixel points in the current adversarial image according to the gradient descent information to obtain the updated current adversarial image.

[0015] Optionally, the first adjustment unit includes: a third adjustment sub-unit, used to adjust the pixel values ​​of the pixels in the adversarial image according to the classification confidence and the output prediction area output by the target detection network model to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the target value is the minimum, and the target value is determined based on the classification confidence and the area overlap between the predicted area and the first actual area.

[0016] Optionally, the third adjustment subunit is used to adjust the pixel values ​​of the pixels in the adversarial image according to the classification confidence and the output prediction area output by the target detection network model in the following manner to obtain the target adversarial image, including: repeatedly performing the following steps until the target value is determined to be minimum, wherein the adversarial image is regarded as the current adversarial image when performing the following steps: obtaining a training image for this round, wherein the training image for this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the group of sample images, and the first actual area in the label information corresponding to the one sample image is the actual area for this round; inputting the training image for this round into the target detection network model to obtain the prediction area output by the target detection network model for this round and the classification confidence output for this round; when the difference between the target value determined in this round and the target value determined in the previous round is greater than a third preset threshold, adjusting the current adversarial image for the target detection network model. The pixel values ​​of the pixels in the image are adjusted to obtain the updated current adversarial image, wherein the target value determined in the current round is determined according to the classification confidence output in the current round and the area overlap determined in the current round, the target value determined in the previous round is determined according to the classification confidence output in the previous round and the area overlap determined in the previous round, the area overlap determined in the current round is: the intersection of the predicted area output in the current round and the actual area in the current round divided by the union of the predicted area output in the current round and the actual area in the current round, and the area overlap determined in the previous round is: the intersection of the predicted area output in the previous round and the actual area in the previous round divided by the union of the predicted area output in the previous round and the actual area in the previous round; when the difference between the target value determined in the current round and the target value determined in the previous round is less than or equal to the third preset threshold, it is determined that the target value is the minimum.

[0017] Optionally, the third adjustment subunit is used to adjust the pixel values ​​of pixel points in the current adversarial image in the following manner to obtain the updated current adversarial image: obtain gradient descent information of an objective function, wherein the objective function is used to obtain the pixel values ​​of pixel points in the current adversarial image when the objective value is minimized; and adjust the pixel values ​​of pixel points in the current adversarial image according to the gradient descent information to obtain the updated current adversarial image.

[0018] Optionally, the adjustment module includes: a second adjustment unit, used to repeatedly perform the following steps until it is determined that the target value is the minimum, wherein the target value is determined according to the detection result output by the target detection network model, and when performing the following steps, the adversarial image is regarded as the current adversarial image: obtaining a training image for this round, wherein the training image for this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the group of sample images; inputting the training image for this round into the target detection network model to obtain the detection result output by the target detection network model for this round; when the difference between the target value determined in this round and the target value determined in the previous round is greater than a target preset threshold, adjusting the pixel value of the pixel point in the current adversarial image to obtain the updated current adversarial image; when the difference between the target value determined in this round and the target value determined in the previous round is less than or equal to the target preset threshold, determining that the target value is the minimum.

[0019] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for generating adversarial samples when running.

[0020] According to another aspect of an embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the adversarial sample generation method through the computer program.

[0021] In an embodiment of the present invention, a group of sample images and a group of label information are obtained, and the pixel values ​​of the pixels in the adversarial image are adjusted according to the detection results output by the target detection network model to obtain the target adversarial image, and the target adversarial image is set as the adversarial image in the second actual area of ​​a group of sample images to obtain a group of adversarial samples. By inputting the training image into the target detection network model, and when the detection result meets the preset loss condition, the target adversarial image is set in the second actual area of ​​the sample image, thereby achieving the purpose of replacing the related technology by adding noise to the digital image, thereby achieving the technical effect of improving the robustness of the adversarial sample and improving the security of the deep learning technology, thereby solving the technical problem of poor robustness of the adversarial sample existing in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0023] Figure 1 is a schematic diagram of an application environment of an optional method for generating adversarial samples according to an embodiment of the present invention;

[0024] Figure 2 is a flowchart of an optional method for generating adversarial samples according to an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of an optional method for generating adversarial samples according to an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of another optional method for generating adversarial samples according to an embodiment of the present invention;

[0027] Figure 5 is a schematic diagram of another optional method for generating adversarial samples according to an embodiment of the present invention;

[0028] Figure 6 is a schematic diagram of another optional method for generating adversarial samples according to an embodiment of the present invention;

[0029] Figure 7 is a schematic diagram of another optional method for generating adversarial samples according to an embodiment of the present invention;

[0030] Figure 8 is a schematic diagram of another optional method for generating adversarial samples according to an embodiment of the present invention;

[0031] Fig. 9is a schematic structural diagram of an optional device for generating adversarial samples according to an embodiment of the present invention;

[0032] Fig.10 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretation:

[0036] YOLOv3: An object detection algorithm

[0037] Bounding box: The bounding box of the target, that is, the outer rectangle that frames the target

[0038] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0039] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0040] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphics processing so that the computer processes the images into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0041] The key technologies of speech technology include automatic speech recognition technology (ASR), text-to-speech technology (TTS) and voiceprint recognition technology. Enabling computers to listen, see, speak and feel is the future development direction of human-computer interaction, among which speech has become one of the most promising human-computer interaction methods in the future.

[0042] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0043] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0044] Convolutional Neural Network (CNN) is a type of feedforward neural network that includes convolution operations, usually composed of convolution layers, pooling layers, fully connected layers, activation layers, etc. It can be used for feature extraction of high-dimensional data such as images and audio.

[0045] Autonomous driving technology usually includes high-precision maps, environmental perception, behavioral decision-making, path planning, motion control and other technologies. Autonomous driving technology has broad application prospects.

[0046] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0047] The solution provided in the embodiment of the present application involves technologies such as generation of adversarial samples of artificial intelligence. The present invention is described below in conjunction with the embodiment:

[0048] The technical solution of this application can be implemented based on cloud computing technology. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0049] As a cloud computing basic capability provider, a cloud computing resource pool (referred to as cloud platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0050] Cloud computing is the product of the integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing. With the development of the Internet, real-time data streams, and the diversification of connected devices, as well as the demand for search services, social networks, mobile commerce, and open collaboration, cloud computing has developed rapidly. Different from the previous parallel distributed computing, the emergence of cloud computing will promote revolutionary changes in the entire Internet model and enterprise management model in terms of concept.

[0051] Figure 1 The exemplary system architecture block diagram applying the technical solution of the present application is schematically shown.

[0052] According to one aspect of an embodiment of the present invention, a method for generating an adversarial sample is provided. Optionally, in this embodiment, the method for generating an adversarial sample can be applied to Figure 1 In the hardware environment composed of the server 101 and the terminal 103 shown in FIG. Figure 1 As shown, the server 101 is connected to the terminal 103 through a network, and can be used to provide services for the terminal or the client installed on the terminal. The client can be a video client, an instant messaging client, a browser client, an education client, a game client, etc. A database 105 can be set on the server or independently of the server to provide data storage services for the server 101, for example, a picture storage server. The above network can include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network and a wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that implement wireless communication. The terminal 103 can include but is not limited to a terminal that can use the above-mentioned adversarial sample generation method, and can include but is not limited to at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop, a tablet computer, a PDA, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV and other computer devices. The above server can be a single server, or a server cluster composed of multiple servers, or a cloud server, and can include but is not limited to a router or a gateway.

[0053] Optionally, the adversarial sample generation method may be configured on the terminal 103 or the server 101, and may also include but not be limited to being implemented jointly by the terminal 103 and the server 101, which is not specifically limited in the present application.

[0054] For example, a group of adversarial samples generated based on the technical solution of the present application can be applied to the target detection algorithm used in the process of autonomous driving system, video classification of streaming media platform, face recognition, etc. for security testing. For example, a group of adversarial stickers (a group of adversarial samples) is obtained using the above-mentioned adversarial sample generation method, and a batch of training data sets are generated using the stickers in two ways. The first is to directly cover the hood of the car with adversarial stickers for a batch of image data in which there are cars in the picture; and the second is to directly print out the adversarial stickers, place them on the hood of the vehicle, and use a camera to take images or videos. Subsequently, the training data set is used to fine-tune the target detection algorithm to obtain higher security and robustness.

[0055] Combination Figure 1 As shown, the above-mentioned method for generating adversarial samples can be implemented on the server 101 side through the following steps:

[0056] S1, obtaining a group of sample images and a group of label information in the server 103, wherein the group of sample images and the group of label information have a one-to-one correspondence, each sample image contains an object of the target classification, and the label information includes a first actual area of ​​the object of the target classification in the sample image, and a second actual area of ​​the adversarial image in the sample image;

[0057] S2, in the server 103, adjusting the pixel values ​​of the pixels in the adversarial image according to the detection results output by the target detection network model to obtain a target adversarial image;

[0058] S3, in the server 103, the target adversarial image is set as an adversarial image in a second actual area in a group of sample images to obtain a group of adversarial samples.

[0059] The above is only an example and is not specifically limited in this embodiment.

[0060] Optionally, as an optional implementation, as Figure 2 As shown, the above-mentioned method for generating adversarial samples includes:

[0061] S202, obtaining a set of sample images and a set of label information, wherein the set of sample images and the set of label information have a one-to-one correspondence, each sample image contains an object of the target classification, and the label information includes a first actual area of ​​the object of the target classification in the sample image and a second actual area of ​​the adversarial image in the sample image;

[0062] S204, adjusting the pixel values ​​of the pixels in the adversarial image according to the detection result output by the target detection network model to obtain the target adversarial image, wherein the detection result output by the target detection network model is the detection result obtained by inputting the training image into the target detection network model, the training image is an image obtained by setting the adversarial image in the second actual area of ​​the sample image, and when the adversarial image is the target adversarial image, the detection result output by the target detection network model satisfies a preset loss condition;

[0063] S206: Setting the target adversarial image as an adversarial image in a second actual area of ​​a group of sample images to obtain a group of adversarial samples.

[0064] Optionally, in this embodiment, the application scenarios of the above-mentioned adversarial sample generation method may include but are not limited to computer vision scenarios, autonomous driving scenarios, public safety scenarios, and specifically may include but are not limited to scenarios related to the education field, scenarios related to the pan-information processing field, scenarios related to the medical and health field, scenarios related to the industrial manufacturing field, scenarios related to the mobile phone and Internet entertainment field, scenarios related to the retail field, scenarios related to the advertising and marketing field, scenarios related to the transportation field, scenarios related to the intelligent customer service field, scenarios related to the smart home field, etc.

[0065] The above is only an example. The specific application scenarios may include but are not limited to a combination of one or more of the above, as well as other scenarios in which detection and identification are achieved through artificial intelligence. This embodiment does not make any specific limitations.

[0066] Optionally, in this embodiment, the above-mentioned set of sample images may include but is not limited to obtaining different types of sample images according to different application scenarios, and the above-mentioned set of sample images includes but is not limited to labeled sample images, and sample images that can be labeled through a preset algorithm, wherein the above-mentioned annotation is represented by the above-mentioned set of label information.

[0067] Optionally, in this embodiment, the objects of the above-mentioned target classification may include but are not limited to being flexibly configured according to different application scenarios. For example, taking the application of the above-mentioned adversarial sample generation method in the field of autonomous driving as an example, the objects of the above-mentioned target classification may include but are not limited to objects of vehicle classification in the picture, and the types of the above-mentioned vehicles may be the same or different. In other words, when the objects of the above-mentioned target classification are objects of vehicle classification, each of the above-mentioned sample objects contains a vehicle, but there is no limitation on the details such as the type and color of the vehicle.

[0068] Optionally, in this embodiment, the above-mentioned label information is used to indicate a first actual area of ​​the target classified object in the sample image and a second actual area of ​​the adversarial image in the sample image, wherein the above-mentioned first actual area may include but is not limited to the actual area included in the specific position of the vehicle bounding box, and the above-mentioned second actual area may include but is not limited to the area included in the actual position of the above-mentioned adversarial image in the sample image.

[0069] For example, Figure 3 is a schematic diagram of an optional method for generating adversarial samples according to an embodiment of the present invention, such as Figure 3 As shown, taking the application of the above-mentioned adversarial sample generation method in the field of autonomous driving as an example, the display interface 302 for executing the above-mentioned adversarial sample generation method displays the above-mentioned target classified object 304 (vehicle), as well as the first actual area 306 and the second actual area 308. The above-mentioned first actual area 306 and the above-mentioned second actual area 308 are both obtained through annotated label information.

[0070] It should be noted that Figure 4 is a schematic diagram of another optional method for generating adversarial samples according to an embodiment of the present invention. The first actual area may include but is not limited to the following: Figure 4 As shown, the first actual area is represented by a bounding box 404 corresponding to the object 406 in the display interface 402, (x0, y0) is used to represent the upper left corner coordinates of the bounding box 404, and (x1, y1) is the lower right corner coordinates of the bounding box 404. _c0 ,y _c0 , x _c1 ,y _c1 )express.

[0071] The above is only an example, and this embodiment does not impose any special limitation.

[0072] Figure 5 is a schematic diagram of another optional method for generating adversarial samples according to an embodiment of the present invention. The second actual area may include but is not limited to: Figure 5 As shown, the second actual area is represented by a bounding box 506 displayed in the display interface 502, the adversarial image is set in the area 508 on the object 504, and the bounding box 506 of the area 508 is determined to be the second actual area. When the upper left corner coordinates of the outer bounding box are represented as (x0, y0) and the lower right corner coordinates are represented as (x1, y1), the second actual area can be represented by (x _p0 ,y _p0 , x _p1 ,y _p1 )express.

[0073] The above is only an example, and this embodiment does not impose any special limitation.

[0074] Optionally, in this embodiment, the target detection network model may include but is not limited to a Proposal detection network model, including but not limited to Faster R-CNN and R-FCN, or an SSD detection network model, or a YOLO detection network model. Among them, the YOLO detection network model may include but is not limited to YOLOv1, YOLOv2, YOLOv3 and other detection network models.

[0075] The above is only an example, and may specifically include but not be limited to a combination of one or more of the above, and this embodiment does not make any special limitation.

[0076] Optionally, in this embodiment, the detection result may include but is not limited to whether the adversarial image can be detected by the target detection network, or may include but is not limited to whether the target classified object can be detected by the target detection network.

[0077] Optionally, in this embodiment, the pixel value of the above-mentioned pixel point may include but is not limited to the average brightness information of the pixel point in the adversarial image, and specifically may include but is not limited to the color, texture, etc. used to represent the adversarial image. The value of the above-mentioned pixel value is but is not limited to between 0 and 255.

[0078] The above is only an example, and this embodiment does not impose any special limitation.

[0079] Optionally, in this embodiment, taking the application scenario as the field of autonomous driving and the target classification as vehicle classification as an example, the above-mentioned adversarial image, the above-mentioned target adversarial image and the above-mentioned set of adversarial samples can be configured as stickers, including but not limited to, for vehicle detection in the target detection algorithm YOLOv3, the generated noisy adversarial stickers are placed on the hood of the vehicle and other locations. If the detector cannot detect the car or the position of the detected vehicle is offset, it is considered that the detection algorithm has a safety hazard when facing such adversarial samples, and a batch of training data sets are generated using the stickers in two ways. The first is to directly cover the hood of the car with adversarial stickers for a batch of image data with cars in the picture; and the second is to directly print out the adversarial stickers, place them on the hood of the vehicle, and use a camera to take images or videos. Subsequently, the target detection algorithm is fine-tuned using the training data set to obtain a detection algorithm with higher security and robustness to optimize the subsequent detection of objects in the target classification.

[0080] Optionally, in this embodiment, the detection result output by the target detection network model satisfies the preset loss condition, which may include but is not limited to configuring a preset loss function for the target detection network, and when the loss function converges, determining that the detection result output by the target detection network model satisfies the preset loss condition.

[0081] Through this embodiment, a group of sample images and a group of label information are obtained, and the pixel values ​​of the pixels in the adversarial image are adjusted according to the detection results output by the target detection network model to obtain the target adversarial image, and the target adversarial image is set as the adversarial image in the second actual area of ​​a group of sample images to obtain a group of adversarial samples. By inputting the training image into the target detection network model, and when the detection result meets the preset loss condition, the target adversarial image is set in the second actual area of ​​the sample image, thereby achieving the purpose of replacing the related technology by adding noise to the digital image, thereby achieving the technical effect of improving the robustness of the adversarial sample and improving the security of the deep learning technology, thereby solving the technical problem of poor robustness of the adversarial sample existing in the related technology.

[0082] As an optional scheme, the pixel values ​​of the pixels in the adversarial image are adjusted according to the detection results output by the target detection network model to obtain the target adversarial image, including: adjusting the pixel values ​​of the pixels in the adversarial image according to the classification confidence and / or the prediction area output by the target detection network model to obtain the target adversarial image, wherein the classification confidence is used to indicate the probability that the object recognized by the target detection network model in the training image belongs to the target classification, and the prediction area is the area in the training image of the object recognized by the target detection network model in the training image.

[0083] Optionally, in this embodiment, the above-mentioned classification confidence may include but is not limited to the specific category of the object recognized by the above-mentioned target detection network model, and the confidence that the above-mentioned object is classified as a target, and the above-mentioned target adversarial image may be obtained by including but not limited to determining a loss function based on the confidence.

[0084] For example, Figure 6 is a schematic diagram of another optional method for generating adversarial samples according to an embodiment of the present invention, such as Figure 6 As shown, the training image 602 includes an object 604, and the target classification is pre-set as a vehicle. The confidence level represents the probability that the target detection network model recognizes the category of the object 604 as a vehicle. The pixel values ​​of the pixels in the adversarial image are adjusted based on the confidence level to obtain the target adversarial image.

[0085] Optionally, in this embodiment, the predicted area outputted above may include but is not limited to the area of ​​the target object in the training image detected by the target detection network model. The area of ​​the target object in the training image detected by the target detection network model is compared with the true area to determine the loss function associated with the predicted area according to the degree of overlap between the predicted area and the true area, so as to obtain the target adversarial image.

[0086] For example, Figure 7 is a schematic diagram of another optional method for generating adversarial samples according to an embodiment of the present invention, such as Figure 7 As shown, a training image 702 includes an object 704, a predicted area 706 of the object 704, and a real area 708. An overlapping area 710 between the predicted area 706 and the real area 708 is obtained, and the pixel values ​​of the pixels in the adversarial image are adjusted based on the area size of the overlapping area 710 to obtain a target adversarial image.

[0087] Through this embodiment, the pixel values ​​of the pixels in the adversarial image are adjusted according to the classification confidence and / or the output prediction area of ​​the target detection network model to obtain the target adversarial image, so as to replace the purpose of adding noise to the digital image in the related technology, thereby achieving the technical effect of improving the robustness of the adversarial sample and improving the security of the deep learning technology, and further solving the technical problem of poor robustness of the adversarial sample existing in the related technology.

[0088] As an optional scheme, the pixel values ​​of the pixels in the adversarial image are adjusted according to the classification confidence and / or the predicted area output by the target detection network model to obtain the target adversarial image, including: according to the classification confidence output by the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein, when the adversarial image is the target adversarial image, the classification confidence output by the target detection network model is the smallest.

[0089] Optionally, in this embodiment, the target adversarial image may be obtained by including but not limited to determining a loss function according to confidence.

[0090] The above loss function can be configured as follows:

[0091] L cls = det_cls((x+M(δ)),y cls )

[0092] Wherein, x represents the input sample image, δ represents the target adversarial image, M() represents the second actual region, M(δ) represents the result of setting the target adversarial image in the second actual region, and y cls Represents the index value of the target category label, and det_cls indicates that the category of the detected target is the confidence value of the category corresponding to the category label.

[0093] Optionally, in this embodiment, the minimum classification confidence output by the target detection network model may include but is not limited to the minimum classification confidence output by the target detection network model when the objective function takes a minimum value.

[0094] The above is only an example, and this embodiment does not impose any special limitation.

[0095] Through this embodiment, the pixel values ​​of the pixels in the adversarial image are adjusted according to the classification confidence output by the target detection network model to obtain the target adversarial image. In the case where the adversarial image is the target adversarial image, the classification confidence output by the target detection network model is minimized, thereby achieving the technical effect of improving the robustness of adversarial samples and improving the security of deep learning technology, thereby solving the technical problem of poor robustness of adversarial samples existing in related technologies.

[0096] As an optional solution, according to the classification confidence output by the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, including:

[0097] Repeat the following steps until it is determined that the classification confidence output by the target detection network model is the minimum, wherein the adversarial image is considered as the current adversarial image when performing the following steps:

[0098] Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in a second actual area of ​​a sample image in a set of sample images;

[0099] Input the training image of this round into the target detection network model to obtain the classification confidence of the target detection network model output in this round;

[0100] When the difference between the classification confidence output in this round and the classification confidence output in the previous round of the target detection network model is greater than a first preset threshold, adjusting the pixel value of the pixel point in the current adversarial image to obtain an updated current adversarial image;

[0101] When the difference between the classification confidence output in this round and the classification confidence output in the previous round is less than or equal to the first preset threshold, it is determined that the classification confidence output by the target detection network model is the smallest.

[0102] Optionally, in this embodiment, the first preset threshold may include but is not limited to being randomly set by the system, or being pre-configured by staff based on experience, and the first preset threshold is used to distinguish whether the objective function for determining the classification confidence converges.

[0103] For example, optionally, in this embodiment, the variable value when the loss function takes the minimum value can be represented by including but not limited to using the arg min{} function, for example, argmin(L cls ), and by continuously optimizing the above function, determine the comparison relationship between the difference between the classification confidence output in this round and the classification confidence output in the previous round and the above first preset threshold.

[0104] It should be noted that, when the difference between the classification confidence output in this round and the classification confidence output in the previous round of the target detection network model is greater than the first preset threshold, it indicates that the adversarial image in this round is not the above-mentioned target adversarial image, and the pixel values ​​of the pixels in the current adversarial image need to be adjusted to obtain the updated current adversarial image. When the difference between the classification confidence output in this round and the classification confidence output in the previous round is less than or equal to the first preset threshold, it is determined that the classification confidence output by the target detection network model is the smallest, indicating that the adversarial image in this round is the above-mentioned target adversarial image. Furthermore, the adversarial image in this round can be determined as the above-mentioned target object image to be set in the second actual area in a group of sample images to obtain the above-mentioned set of adversarial samples.

[0105] Through this embodiment, the following steps are repeatedly performed until it is determined that the classification confidence output by the target detection network model is the minimum: the training image of this round is obtained, the training image of this round is input into the target detection network model, and the classification confidence output by the target detection network model of this round is obtained; when the difference between the classification confidence output in this round and the classification confidence output in the previous round of the target detection network model is greater than the first preset threshold, the pixel value of the pixel point in the current adversarial image is adjusted to obtain an updated current adversarial image; when the difference between the classification confidence output in this round and the classification confidence output in the previous round is less than or equal to the first preset threshold, it is determined that the classification confidence output by the target detection network model is the minimum, so as to determine the adversarial image of this round as the above-mentioned target object image; by setting the second actual area in a group of sample images, the above-mentioned group of adversarial samples is obtained, thereby achieving the technical effect of improving the robustness of adversarial samples and improving the security of deep learning technology, thereby solving the technical problem of poor robustness of adversarial samples existing in related technologies.

[0106] As an optional scheme, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image, including: obtaining gradient descent information of the objective function, wherein the objective function is used to obtain the pixel values ​​of the pixels in the current adversarial image when the classification confidence output by the target detection network model is minimized; and adjusting the pixel values ​​of the pixels in the current adversarial image according to the gradient descent information to obtain the updated current adversarial image.

[0107] Optionally, in this embodiment, the gradient descent information of the objective function may be determined by methods including but not limited to derivation. The objective function may include but is not limited to argmin(L cls ), adjust the pixel value of the pixel point according to the above gradient descent information to continuously optimize the above objective function to obtain the updated current adversarial image.

[0108] Optionally, in this embodiment, the method of obtaining the gradient descent information of the objective function may include but is not limited to optimizing using the Adam optimization algorithm, and updating the gradient information with each pixel point of the above-mentioned current adversarial image to adjust the pixel value of the pixel point in the current adversarial image.

[0109] Through this embodiment, the gradient descent information of the objective function is obtained, wherein the objective function is used to obtain the pixel value of the pixel point in the current adversarial image when the classification confidence output by the target detection network model is minimized; according to the gradient descent information, the pixel value of the pixel point in the current adversarial image is adjusted to obtain the updated current adversarial image, thereby realizing the training and updating of the adversarial samples, thereby improving the robustness of the adversarial samples, and achieving the technical effect of improving the security of the deep learning technology, thereby solving the technical problem of poor robustness of the adversarial samples existing in the related technologies.

[0110] As an optional solution, adjusting the pixel values ​​of the pixels in the adversarial image according to the classification confidence and / or the predicted area output by the target detection network model to obtain the target adversarial image includes:

[0111] According to the predicted area output by the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the area overlap between the predicted area and the first actual area is minimized.

[0112] Optionally, in this embodiment, the target adversarial image may be obtained by including but not limited to determining a loss function according to the degree of overlap.

[0113] The above loss function can be configured as follows:

[0114] L bbox =IOU(det_bbox(x+M(δ)),y bbox )

[0115] Wherein, x represents the input sample image, δ represents the target adversarial image, M() represents the second actual area, M(δ) represents the result of setting the target adversarial image in the second actual area, det_bbox represents the bounding box position of the detected target, and y bbox It represents the true bounding box position of the target in the sample image. IOU represents the degree of overlap between the predicted area of ​​the target in the sample image by the YOLOv3 network and the true position area of ​​the target. The specific calculation method may include but is not limited to the intersection of the predicted area and the true area divided by the union of the predicted area and the true area.

[0116] Optionally, in this embodiment, the minimum regional overlap between the predicted area output by the above-mentioned target detection network model and the first actual area may include but is not limited to the minimum regional overlap between the predicted area output by the above-mentioned target detection network model and the first actual area when the above-mentioned objective function takes a minimum value.

[0117] The above is only an example, and this embodiment does not impose any special limitation.

[0118] Through this embodiment, the pixel values ​​of the pixels in the adversarial image are adjusted according to the predicted area output by the target detection network model to obtain the target adversarial image, wherein, when the adversarial image is the target adversarial image, the area overlap between the predicted area and the first actual area is minimized, thereby achieving the technical effect of improving the robustness of adversarial samples and improving the security of deep learning technology, thereby solving the technical problem of poor robustness of adversarial samples existing in related technologies.

[0119] As an optional solution, according to the predicted area output by the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, including:

[0120] Repeat the following steps until it is determined that the region overlap is minimal, wherein the adversarial image is regarded as the current adversarial image when the following steps are performed:

[0121] Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images, and the first actual area in the label information corresponding to the one sample image is the actual area of ​​this round;

[0122] Inputting the training image of this round into the target detection network model to obtain the predicted area output by the target detection network model of this round;

[0123] In the case where the difference between the area overlap obtained in the current round and the area overlap obtained in the previous round is greater than a second preset threshold, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image, wherein the area overlap obtained in the current round is: the intersection of the predicted area output in the current round and the actual area in the current round divided by the union of the predicted area output in the current round and the actual area in the current round, and the area overlap obtained in the previous round is: the intersection of the predicted area output in the previous round and the actual area in the previous round divided by the union of the predicted area output in the previous round and the actual area in the previous round;

[0124] When the difference between the area overlap obtained in the current round and the area overlap obtained in the previous round is less than or equal to the second preset threshold, it is determined that the area overlap is the minimum.

[0125] Optionally, in this embodiment, the second preset threshold may include but is not limited to being randomly set by the system, or being pre-configured by staff based on experience, and the second preset threshold is used to distinguish whether the objective function for determining the area overlap has converged.

[0126] For example, optionally, in this embodiment, the variable value when the loss function takes the minimum value can be represented by including but not limited to using the arg min{} function, for example, argmin(L bbox ), and by continuously optimizing the above function, determine the comparison relationship between the difference between the area overlap of the current round output and the area overlap of the previous round output and the above second preset threshold.

[0127] It should be noted that, when the difference between the area overlap of the output in this round and the area overlap of the output in the previous round of the target detection network model is greater than the second preset threshold, it means that the adversarial image in this round is not the above-mentioned target adversarial image, and the pixel values ​​of the pixels in the current adversarial image need to be adjusted to obtain the updated current adversarial image. When the difference between the area overlap of the output in this round and the area overlap of the output in the previous round is less than or equal to the second preset threshold, it is determined that the area overlap output by the target detection network model is the smallest, indicating that the adversarial image in this round is the above-mentioned target adversarial image. Furthermore, the adversarial image in this round can be determined as the above-mentioned target object image to be set in the second actual area in a group of sample images to obtain the above-mentioned set of adversarial samples.

[0128] Through this embodiment, the following steps are repeatedly performed until it is determined that the area overlap output by the target detection network model is the smallest: the training images of this round are obtained, the training images of this round are input into the target detection network model, and the area overlap output by the target detection network model of this round is obtained; when the difference between the area overlap output in this round and the area overlap output by the target detection network model in the previous round is greater than the second preset threshold, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain an updated current adversarial image; when the difference between the area overlap output in this round and the area overlap output in the previous round is less than or equal to the second preset threshold, it is determined that the area overlap output by the target detection network model is the smallest, so as to determine the adversarial image of this round as the above-mentioned target object image; by setting the second actual area in a group of sample images, the above-mentioned group of adversarial samples is obtained, thereby achieving the technical effect of improving the robustness of adversarial samples and improving the security of deep learning technology, thereby solving the technical problem of poor robustness of adversarial samples existing in related technologies.

[0129] As an optional solution, adjusting the pixel values ​​of the pixels in the current adversarial image to obtain the updated current adversarial image includes:

[0130] Obtaining gradient descent information of an objective function, wherein the objective function is used to obtain pixel values ​​of pixels in the current adversarial image when the region overlap is minimized;

[0131] According to the gradient descent information, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image.

[0132] Optionally, in this embodiment, the gradient descent information of the objective function may be determined by methods including but not limited to derivation. The objective function may include but is not limited to argmin(L cls ), adjust the pixel value of the pixel point according to the above gradient descent information to continuously optimize the above objective function to obtain the updated current adversarial image.

[0133] Optionally, in this embodiment, the method of obtaining the gradient descent information of the objective function may include but is not limited to optimizing using the Adam optimization algorithm, and updating the gradient information with each pixel point of the above-mentioned current adversarial image to adjust the pixel value of the pixel point in the current adversarial image.

[0134] Through this embodiment, the gradient descent information of the objective function is obtained, wherein the objective function is used to obtain the pixel value of the pixel point in the current adversarial image when the classification confidence output by the target detection network model is minimized; according to the gradient descent information, the pixel value of the pixel point in the current adversarial image is adjusted to obtain the updated current adversarial image, thereby realizing the training and updating of the adversarial samples, thereby improving the robustness of the adversarial samples, and achieving the technical effect of improving the security of the deep learning technology, thereby solving the technical problem of poor robustness of the adversarial samples existing in the related technologies.

[0135] As an optional solution, adjusting the pixel values ​​of the pixels in the adversarial image according to the classification confidence and / or the predicted area output by the target detection network model to obtain the target adversarial image includes:

[0136] According to the classification confidence and the output prediction area of ​​the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the target value is minimum, and the target value is determined based on the classification confidence and the area overlap between the prediction area and the first actual area.

[0137] Optionally, in this embodiment, the target adversarial image may be obtained by including but not limited to determining a loss function according to classification confidence and region overlap.

[0138] The above loss function can be configured as follows:

[0139] L cls = det_cls((x+M(δ)),y cls ),

[0140] L bbox =IOU(det_bbox(x+M(δ)),y bbox )

[0141] Wherein, x represents the input sample image, δ represents the target adversarial image, M() represents the second actual region, M(δ) represents the result of setting the target adversarial image in the second actual region, and y cls Indicates the index value of the target category label, det_cls indicates that the category of the detected target is the confidence value of the category corresponding to the category label, det_bbox indicates the bounding box position of the detected target, y bboxIt represents the true bounding box position of the target in the sample image. IOU represents the degree of overlap between the predicted area of ​​the target in the sample image by the YOLOv3 network and the true position area of ​​the target. The specific calculation method may include but is not limited to the intersection of the predicted area and the true area divided by the union of the predicted area and the true area.

[0142] Optionally, in this embodiment, the minimum target value output by the target detection network model may include but is not limited to when the target function takes a minimum value, the target value of the target detection network model is minimized, and the target value may include but is not limited to being obtained in a weighted manner according to the classification confidence and the area overlap.

[0143] For example, target value = αL cls +βL bbox , where the value of α+β is equal to 1.

[0144] The above is only an example, and this embodiment does not impose any special limitation.

[0145] Through this embodiment, the pixel values ​​of the pixels in the adversarial image are adjusted according to the classification confidence output by the target detection network model and the output prediction area to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the target value is the smallest, and the target value is determined according to the classification confidence and the area overlap between the prediction area and the first actual area. The technical effect of improving the robustness of adversarial samples and improving the security of deep learning technology is achieved, thereby solving the technical problem of poor robustness of adversarial samples existing in related technologies.

[0146] As an optional solution, according to the classification confidence and the output prediction area of ​​the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, including:

[0147] Repeat the following steps until it is determined that the target value is the minimum, wherein when performing the following steps, the adversarial image is regarded as the current adversarial image:

[0148] Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images, and the first actual area in the label information corresponding to the one sample image is the actual area of ​​this round;

[0149] Inputting the training image of this round into the target detection network model to obtain the prediction area output by the target detection network model of this round and the classification confidence output of this round;

[0150] In the case where the difference between the target value determined in the current round and the target value determined in the previous round is greater than a third preset threshold, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image, wherein the target value determined in the current round is determined according to the classification confidence output in the current round and the region overlap determined in the current round, the target value determined in the previous round is determined according to the classification confidence output in the previous round and the region overlap determined in the previous round, the region overlap determined in the current round is: the intersection of the predicted region output in the current round and the actual region of the current round divided by the union of the predicted region output in the current round and the actual region of the current round, and the region overlap determined in the previous round is: the intersection of the predicted region output in the previous round and the actual region of the previous round divided by the union of the predicted region output in the previous round and the actual region of the previous round;

[0151] In the case where the difference between the target value determined in the current round and the target value determined in the previous round is less than or equal to the third preset threshold, it is determined that the target value is the minimum.

[0152] Optionally, in this embodiment, the third preset threshold may include but is not limited to being randomly set by the system, or being pre-configured by staff based on experience, and the third preset threshold is used to distinguish whether the objective function for determining the target value converges.

[0153] For example, the variable value when the loss function takes the minimum value can be represented by including but not limited to using the arg min{} function, for example, argmin(αL cls +βL bbox ), wherein the value of α+β is equal to 1, and the above function is continuously optimized to determine the comparison relationship between the difference between the target value output in this round and the target value output in the previous round and the above third preset threshold.

[0154] It should be noted that, when the difference between the target value output in this round and the target value output in the previous round of the target detection network model is greater than the third preset threshold, it means that the adversarial image of this round is not the above-mentioned target adversarial image, and the pixel values ​​of the pixels in the current adversarial image need to be adjusted to obtain the updated current adversarial image. When the difference between the target value output in this round and the target value output in the previous round is less than or equal to the third preset threshold, it is determined that the target value output by the target detection network model is the smallest, indicating that the adversarial image of this round is the above-mentioned target adversarial image. Furthermore, the adversarial image of this round can be determined as the above-mentioned target object image, so as to be set in the second actual area in a group of sample images to obtain the above-mentioned set of adversarial samples.

[0155] Through this embodiment, the following steps are repeatedly performed until it is determined that the target value output by the target detection network model is the minimum: the training image of this round is obtained, the training image of this round is input into the target detection network model, and the target value output by the target detection network model of this round is obtained; when the difference between the target value output in this round and the target value output by the target detection network model in the previous round is greater than the third preset threshold, the pixel value of the pixel point in the current adversarial image is adjusted to obtain an updated current adversarial image; when the difference between the target value output in this round and the target value output in the previous round is less than or equal to the third preset threshold, it is determined that the target value output by the target detection network model is the minimum, so as to determine the adversarial image of this round as the above-mentioned target object image; by setting the second actual area in a group of sample images, the above-mentioned group of adversarial samples is obtained, thereby achieving the technical effect of improving the robustness of adversarial samples and improving the security of deep learning technology, thereby solving the technical problem of poor robustness of adversarial samples existing in related technologies.

[0156] As an optional solution, adjusting the pixel values ​​of the pixels in the current adversarial image to obtain the updated current adversarial image includes:

[0157] Obtaining gradient descent information of an objective function, wherein the objective function is used to obtain a pixel value of a pixel point in the current adversarial image when the objective value is minimized;

[0158] According to the gradient descent information, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image.

[0159] Optionally, in this embodiment, the gradient descent information of the objective function may be determined by methods including but not limited to derivation. The objective function may include but is not limited to argmin(L cls ), adjust the pixel value of the pixel point according to the above gradient descent information to continuously optimize the above objective function to obtain the updated current adversarial image.

[0160] Optionally, in this embodiment, the method of obtaining the gradient descent information of the objective function may include but is not limited to optimizing using the Adam optimization algorithm, and updating the gradient information with each pixel point of the above-mentioned current adversarial image to adjust the pixel value of the pixel point in the current adversarial image.

[0161] Through this embodiment, the gradient descent information of the objective function is obtained, wherein the objective function is used to obtain the pixel value of the pixel point in the current adversarial image when the classification confidence output by the target detection network model is minimized; according to the gradient descent information, the pixel value of the pixel point in the current adversarial image is adjusted to obtain the updated current adversarial image, thereby realizing the training and updating of the adversarial samples, thereby improving the robustness of the adversarial samples, and achieving the technical effect of improving the security of the deep learning technology, thereby solving the technical problem of poor robustness of the adversarial samples existing in the related technologies.

[0162] As an optional solution, adjusting the pixel values ​​of the pixels in the adversarial image according to the detection results output by the target detection network model to obtain the target adversarial image includes:

[0163] Repeat the following steps until it is determined that the target value is the minimum, wherein the target value is determined according to the detection result output by the target detection network model, and when performing the following steps, the adversarial image is regarded as the current adversarial image:

[0164] Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images;

[0165] Inputting the training image of the current round into the target detection network model to obtain the detection result output by the target detection network model of the current round;

[0166] When the difference between the target value obtained in the current round and the target value obtained in the previous round is greater than a preset target threshold, adjusting the pixel value of the pixel point in the current adversarial image to obtain an updated current adversarial image;

[0167] When the difference between the target value obtained in the current round and the target value obtained in the previous round is less than or equal to the target preset threshold, it is determined that the target value is the minimum.

[0168] Optionally, in this embodiment, the above-mentioned target preset threshold may include but is not limited to being randomly set by the system, or being pre-configured by the staff based on experience, and the above-mentioned target preset threshold is used to distinguish whether the above-mentioned objective function for determining the target value converges.

[0169] For example, the objective function corresponding to the variable value when the loss function takes the minimum value can be represented by, but not limited to, using the arg min{} function, for example, argmin(αL cls +βL bbox), where the value of α+β is equal to 1, and the above function is continuously optimized to determine the comparison relationship between the difference between the target value output in this round and the target value output in the previous round and the preset threshold of the above target.

[0170] It should be noted that, when the difference between the target value output in this round and the target value output in the previous round of the target detection network model is greater than the target preset threshold, it means that the adversarial image of this round is not the above-mentioned target adversarial image, and the pixel values ​​of the pixels in the current adversarial image need to be adjusted to obtain the updated current adversarial image. When the difference between the target value output in this round and the target value output in the previous round is less than or equal to the target preset threshold, it is determined that the target value output by the target detection network model is the smallest, indicating that the adversarial image of this round is the above-mentioned target adversarial image. Furthermore, the adversarial image of this round can be determined as the above-mentioned target object image to be set in the second actual area in a group of sample images to obtain the above-mentioned set of adversarial samples.

[0171] Through this embodiment, the following steps are repeatedly performed until it is determined that the target value output by the target detection network model is the minimum: the training image of this round is obtained, the training image of this round is input into the target detection network model, and the target value output by the target detection network model of this round is obtained; when the difference between the target value output in this round and the target value output in the previous round of the target detection network model is greater than the target preset threshold, the pixel value of the pixel point in the current adversarial image is adjusted to obtain an updated current adversarial image; when the difference between the target value output in this round and the target value output in the previous round is less than or equal to the target preset threshold, it is determined that the target value output by the target detection network model is the minimum, so as to determine the adversarial image of this round as the above-mentioned target object image; by setting the second actual area in a group of sample images, the above-mentioned group of adversarial samples is obtained, thereby achieving the technical effect of improving the robustness of adversarial samples and improving the security of deep learning technology, thereby solving the technical problem of poor robustness of adversarial samples existing in related technologies.

[0172] The present invention is further explained below with reference to specific examples:

[0173] Taking vehicle detection as an example, the present invention proposes a method for generating adversarial samples in the physical world for the target detection algorithm YOLOv3. The method generates a sticker with adversarial noise (corresponding to the aforementioned adversarial image), so that any vehicle with the sticker affixed at a specified position (corresponding to the aforementioned object) cannot be detected by the target detection network YOLOv3. The generated adversarial samples can be used to verify the security of the target detection algorithm, and can be used as a data set for adversarial training to further improve the algorithm's anti-interference ability. Figure 8is a schematic diagram of another optional method for generating adversarial samples according to an embodiment of the present invention. Figure 8 The generation process of adversarial samples is shown, which includes the following 4 steps:

[0174] S802, obtaining the network structure and model parameters of the target detection network YOLOv3;

[0175] S804, obtain a set of training samples, which at least include an image with a vehicle as input data and a corresponding label value. The label value includes the specific position of the vehicle bounding box in the image (x_c0, y_c0, x_c1, y_c1) and the position of the sticker (x_p0, y_p0, x_p1, y_p1). Among them, (x0, y0) is the coordinate of the upper left corner of the bounding box, and (x1, y1) is the coordinate of the lower right corner of the bounding box;

[0176] S806, initializing the sticker with random noise, and iteratively performing the following operations for each sample in the training sample until the pixel value on the sticker image no longer changes:

[0177] Wherein, S806 includes S8062, inputting the training sample into the target detection network YOLOv3 obtained from step 1, and YOLO v3 outputting the detection result of the input sample, including two parts, det_bbox and det_cls, wherein det_bbox represents the bounding box position of the detected target, specifically represented by (x0, y0, x1, y1), and det_cls represents the specific category of the detected target and the confidence value of the category;

[0178] S8064, using the detection results obtained in S8062, calculate the loss value. The loss value can be divided into two parts: classification confidence loss and bounding box loss. The calculation formula of classification confidence loss is:

[0179] L cls = det_cls((x+M(δ)),y cls )

[0180] Where x represents the input sample, δ represents the generated sticker with adversarial noise, M() is the area where the sticker is placed, M(δ) represents the result of affine transformation of the generated sticker with adversarial noise and the area, y cls The index value representing the category label of the vehicle;

[0181] The bounding box loss is calculated as:

[0182] L bbox =IOU(det_bbox(x+M(δ)),y bbox)

[0183] Among them, x, δ and M() have the same meanings as those expressed in the above formula. In addition, y bbox Indicates the true bounding box position of the target in the input image. IOU indicates the degree of overlap between the predicted area of ​​the target in the input image and the true position area of ​​the target by the YOLOv3 network. The specific calculation method is the intersection of the predicted area and the true area divided by the union of the predicted area and the true area.

[0184] S8066, based on the classification confidence loss L obtained in S8064 cls and the bounding box loss L bbox , determine the objective function that needs to be optimized during the training process. In particular, since different objective functions will eventually generate adversarial samples with different effects, the present invention constructs three different objective functions, as follows: argmin(L cls ), argmin(L bbox ) and argmin(αL cls +βL bbox ), where α+β=1;

[0185] S8068, uses the Adam optimization algorithm to optimize the objective function and updates the gradient information at each pixel of the sticker;

[0186] S808, when the pixel values ​​on the sticker image no longer change, that is, the loss value has converged, a fooling adversarial sticker is obtained. When the sticker is placed at a designated position of a vehicle, the vehicle can be undetected by the YOLOv3 target detection network.

[0187] As deep learning technology is increasingly widely used in areas closely related to people's lives, security issues related to deep learning, such as adversarial samples, have also attracted extensive research. How to improve the security and robustness of deep learning technology in response to these security issues is a question and direction worthy of further study. Through this embodiment, it is first verified that only an additional carefully generated sticker is needed to escape the detection of the target detection network YOLOv3, further revealing that in addition to adding noise to digital images, interference in the physical world can also have a fatal impact on deep learning technology. In addition, the adversarial samples generated by the method of the present invention can also be used as part of the training data set to perform adversarial training on the model, thereby further improving the security and anti-interference ability of the model, which can play a positive role in improving the security of deep learning technology in the future.

[0188] In addition, the above adversarial sample generation method can also be used for targets other than vehicles and target detection networks other than YOLOv3.

[0189] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0190] According to another aspect of an embodiment of the present invention, a device for generating an adversarial sample for implementing the above-mentioned adversarial sample generation method is also provided. Fig. 9 As shown, the device comprises:

[0191] An acquisition module 902 is used to acquire a set of sample images and a set of label information, wherein the set of sample images and the set of label information have a one-to-one correspondence, each of the sample images contains an object of a target classification, and the label information includes a first actual area of ​​the object of the target classification in the sample image and a second actual area of ​​the adversarial image in the sample image;

[0192] An adjustment module 904 is used to adjust the pixel values ​​of the pixels in the adversarial image according to the detection result output by the target detection network model to obtain the target adversarial image, wherein the detection result output by the target detection network model is the detection result obtained by inputting a training image into the target detection network model, and the training image is an image obtained by setting the adversarial image in the second actual area in the sample image, and when the adversarial image is the target adversarial image, the detection result output by the target detection network model satisfies a preset loss condition;

[0193] The setting module 906 is used to set the target adversarial image as the adversarial image in the second actual area of ​​the group of sample images to obtain a group of adversarial samples.

[0194] As an optional solution, the adjustment module 904 includes:

[0195] A first adjustment unit is used to adjust the pixel values ​​of the pixels in the adversarial image according to the classification confidence and / or the prediction area output by the target detection network model to obtain the target adversarial image, wherein the classification confidence is used to represent the probability that the object recognized by the target detection network model in the training image belongs to the target classification, and the prediction area is the area in the training image of the object recognized by the target detection network model in the training image.

[0196] As an optional scheme, the first adjustment unit includes: a first adjustment sub-unit, used to adjust the pixel values ​​of the pixel points in the adversarial image according to the classification confidence output by the target detection network model to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the classification confidence output by the target detection network model is the smallest.

[0197] As an optional solution, the first adjustment subunit is used to adjust the pixel values ​​of the pixels in the adversarial image according to the classification confidence output by the target detection network model in the following manner to obtain the target adversarial image: repeatedly performing the following steps until it is determined that the classification confidence output by the target detection network model is the minimum, wherein the adversarial image is regarded as the current adversarial image when performing the following steps: obtaining a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images; The training image of this round is input into the target detection network model to obtain the classification confidence of the target detection network model output in this round; when the difference between the classification confidence output in this round and the classification confidence output in the previous round of the target detection network model is greater than a first preset threshold, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image; when the difference between the classification confidence output in this round and the classification confidence output in the previous round is less than or equal to the first preset threshold, it is determined that the classification confidence output by the target detection network model is the minimum.

[0198] As an optional scheme, the first adjustment subunit is used to adjust the pixel values ​​of the pixel points in the current adversarial image in the following manner to obtain the updated current adversarial image: obtain the gradient descent information of the objective function, wherein the objective function is used to obtain the pixel values ​​of the pixel points in the current adversarial image when the classification confidence output by the target detection network model is minimized; and adjust the pixel values ​​of the pixel points in the current adversarial image according to the gradient descent information to obtain the updated current adversarial image.

[0199] As an optional scheme, the first adjustment unit includes: a second adjustment sub-unit, used to adjust the pixel values ​​of the pixel points in the adversarial image according to the predicted area output by the target detection network model to obtain the target adversarial image, wherein, when the adversarial image is the target adversarial image, the area overlap between the predicted area and the first actual area is minimized.

[0200] As an optional scheme, the second adjustment subunit is used to adjust the pixel values ​​of the pixels in the adversarial image according to the predicted area output by the target detection network model in the following manner to obtain the target adversarial image: repeatedly perform the following steps until it is determined that the area overlap is minimum, wherein, when performing the following steps, the adversarial image is regarded as the current adversarial image: obtain the training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the group of sample images, and the first actual area in the label information corresponding to the one sample image is the actual area of ​​this round; input the training image of this round into the target detection network model to obtain the predicted area output by the target detection network model in this round; the area determined in this round When the difference between the domain overlap and the regional overlap obtained in the previous round is greater than the second preset threshold, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image, wherein the regional overlap obtained in this round is: the intersection of the predicted area output in this round and the actual area in this round divided by the union of the predicted area output in this round and the actual area in this round, and the regional overlap obtained in the previous round is: the intersection of the predicted area output in the previous round and the actual area in the previous round divided by the union of the predicted area output in the previous round and the actual area in the previous round; when the difference between the regional overlap obtained in this round and the regional overlap obtained in the previous round is less than or equal to the second preset threshold, it is determined that the regional overlap is the smallest.

[0201] As an optional scheme, the second adjustment subunit is used to adjust the pixel values ​​of the pixel points in the current adversarial image in the following manner to obtain the updated current adversarial image: obtain the gradient descent information of the objective function, wherein the objective function is used to obtain the pixel values ​​of the pixel points in the current adversarial image when the region overlap is minimized; and adjust the pixel values ​​of the pixel points in the current adversarial image according to the gradient descent information to obtain the updated current adversarial image.

[0202] As an optional scheme, the first adjustment unit includes: a third adjustment sub-unit, used to adjust the pixel values ​​of the pixel points in the adversarial image according to the classification confidence and the output prediction area output by the target detection network model to obtain the target adversarial image, wherein, when the adversarial image is the target adversarial image, the target value is the minimum, and the target value is determined based on the classification confidence and the area overlap between the predicted area and the first actual area.

[0203] As an optional scheme, the third adjustment subunit is used to adjust the pixel values ​​of the pixels in the adversarial image according to the classification confidence and the output prediction area output by the target detection network model in the following manner to obtain the target adversarial image, including: repeatedly performing the following steps until the target value is determined to be minimum, wherein the adversarial image is regarded as the current adversarial image when performing the following steps: obtaining a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the group of sample images, and the first actual area in the label information corresponding to the one sample image is the actual area of ​​this round; inputting the training image of this round into the target detection network model to obtain the prediction area output by the target detection network model in this round and the classification confidence output in this round; when the difference between the target value obtained in this round and the target value obtained in the previous round is greater than a third preset threshold, adjusting the current round of the training image. The pixel values ​​of the pixels in the previous adversarial image are adjusted to obtain the updated current adversarial image, wherein the target value determined in the current round is determined according to the classification confidence output in the current round and the area overlap determined in the current round, the target value determined in the previous round is determined according to the classification confidence output in the previous round and the area overlap determined in the previous round, the area overlap determined in the current round is: the intersection of the predicted area output in the current round and the actual area in the current round divided by the union of the predicted area output in the current round and the actual area in the current round, and the area overlap determined in the previous round is: the intersection of the predicted area output in the previous round and the actual area in the previous round divided by the union of the predicted area output in the previous round and the actual area in the previous round; when the difference between the target value determined in the current round and the target value determined in the previous round is less than or equal to the third preset threshold, it is determined that the target value is the minimum.

[0204] As an optional scheme, the third adjustment subunit is used to adjust the pixel values ​​of the pixel points in the current adversarial image in the following manner to obtain the updated current adversarial image: obtain the gradient descent information of the objective function, wherein the objective function is used to obtain the pixel values ​​of the pixel points in the current adversarial image when the target value is minimum; according to the gradient descent information, adjust the pixel values ​​of the pixel points in the current adversarial image to obtain the updated current adversarial image.

[0205] As an optional solution, the adjustment module 904 includes:

[0206] The second adjustment unit is used to repeatedly perform the following steps until it is determined that the target value is the minimum, wherein the target value is determined according to the detection result output by the target detection network model, and when performing the following steps, the adversarial image is regarded as the current adversarial image: obtaining a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the group of sample images; inputting the training image of this round into the target detection network model to obtain the detection result output by the target detection network model of this round; when the difference between the target value determined in this round and the target value determined in the previous round is greater than a target preset threshold, adjusting the pixel value of the pixel point in the current adversarial image to obtain the updated current adversarial image; when the difference between the target value determined in this round and the target value determined in the previous round is less than or equal to the target preset threshold, determining that the target value is the minimum.

[0207] According to another aspect of an embodiment of the present invention, an electronic device for implementing the above-mentioned method for generating adversarial samples is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a server as an example. Fig.10 As shown, the electronic device includes a memory 1002 and a processor 1004. The memory 1002 stores a computer program, and the processor 1004 is configured to execute the steps in any of the above method embodiments through the computer program.

[0208] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0209] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0210] S1, obtaining a set of sample images and a set of label information, wherein the set of sample images and the set of label information have a one-to-one correspondence, each sample image contains an object of a target classification, and the label information includes a first actual area of ​​the object of the target classification in the sample image, and a second actual area of ​​the adversarial image in the sample image;

[0211] S2, according to the detection result output by the target detection network model, adjusting the pixel value of the pixel point in the adversarial image to obtain the target adversarial image, wherein the detection result output by the target detection network model is the detection result obtained by inputting the training image into the target detection network model, and the training image is an image obtained by setting the adversarial image in the second actual area of ​​the sample image, and when the adversarial image is the target adversarial image, the detection result output by the target detection network model satisfies the preset loss condition;

[0212] S3, setting the target adversarial image as the adversarial image in a second actual area in a group of sample images to obtain a group of adversarial samples.

[0213] Alternatively, a person skilled in the art may understand that: Fig.10 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, and other terminal devices. Fig.10 The electronic device and the electronic equipment described above are not limited in structure. Fig.10 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig.10 Different configurations shown.

[0214] Among them, the memory 1002 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for generating adversarial samples in the embodiments of the present invention. The processor 1004 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002, that is, realizing the above-mentioned method for generating adversarial samples. The memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1002 may further include a memory remotely located relative to the processor 1004, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 1002 may be specifically, but not limited to, used to store information such as adversarial samples and sample images. As an example, such as Fig.10 As shown, the memory 1002 may include, but is not limited to, the acquisition module 902, adjustment module 904, and setting module 906 in the adversarial sample generation device. In addition, it may also include, but is not limited to, other module units in the adversarial sample generation device, which will not be repeated in this example.

[0215] Optionally, the transmission device 1006 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired network and a wireless network. In one example, the transmission device 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 1006 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0216] In addition, the electronic device further includes: a display 1008 for displaying the adversarial samples and sample images; and a connection bus 1010 for connecting the various module components in the electronic device.

[0217] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. Among them, the nodes may form a peer-to-peer (P2P, Peer To Peer) network, and any form of computing device, such as a server, terminal and other electronic devices, may become a node in the blockchain system by joining the peer-to-peer network.

[0218] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for generating adversarial samples provided in various optional implementations of the generation of adversarial samples, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0219] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0220] S1, obtaining a set of sample images and a set of label information, wherein the set of sample images and the set of label information have a one-to-one correspondence, each sample image contains an object of a target classification, and the label information includes a first actual area of ​​the object of the target classification in the sample image and a second actual area of ​​the adversarial image in the sample image;

[0221] S2, according to the detection result output by the target detection network model, adjusting the pixel value of the pixel point in the adversarial image to obtain the target adversarial image, wherein the detection result output by the target detection network model is the detection result obtained by inputting the training image into the target detection network model, and the training image is an image obtained by setting the adversarial image in the second actual area of ​​the sample image, and when the adversarial image is the target adversarial image, the detection result output by the target detection network model satisfies the preset loss condition;

[0222] S3, setting the target adversarial image as the adversarial image in a second actual area in a group of sample images to obtain a group of adversarial samples.

[0223] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0224] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0225] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.

[0226] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0227] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0228] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0229] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0230] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for generating adversarial samples, characterized in that: include: Acquire a set of sample images and a set of label information, wherein the set of sample images and the set of label information have a one-to-one correspondence, each of the sample images contains an object of a target classification, and the label information includes a first actual area of ​​the object of the target classification in the sample image, and a second actual area of ​​the adversarial image in the sample image; According to the detection result output by the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein the detection result output by the target detection network model is the detection result obtained by inputting a training image into the target detection network model, and the training image is an image obtained by setting the adversarial image in the second actual area in the sample image, and when the adversarial image is the target adversarial image, the detection result output by the target detection network model satisfies a preset loss condition; Setting the target adversarial image as the adversarial image in the second actual area of ​​the set of sample images to obtain a set of adversarial samples; The step of adjusting the pixel values ​​of the pixels in the adversarial image according to the detection results output by the target detection network model to obtain the target adversarial image includes: Adjusting the pixel values ​​of the pixels in the adversarial image according to the classification confidence output by the target detection network model to obtain the target adversarial image, wherein the classification confidence is used to represent the probability that the object recognized by the target detection network model in the training image belongs to the target classification, and the loss condition represents that the target value determined according to the classification confidence is minimized; or According to the predicted area output by the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein the predicted area is the area of ​​the object in the training image recognized by the target detection network model in the training image, the loss condition indicates that the target value determined according to the predicted area is minimum, the predicted area is used to determine the area overlap with the first actual area, and the area overlap is used to determine the target value; or The pixel values ​​of the pixels in the adversarial image are adjusted according to the classification confidence and the predicted area output by the target detection network model to obtain the target adversarial image, wherein the loss condition indicates that the target value determined according to the classification confidence and the predicted area is minimum, the predicted area is used to determine the area overlap with the first actual area, the area overlap is used to determine the target value, the classification confidence is used to indicate the probability that the object recognized by the target detection network model in the training image belongs to the target classification, and the predicted area is the area in the training image of the object recognized by the target detection network model in the training image.

2. The method according to claim 1, characterized in that The step of adjusting the pixel values ​​of the pixels in the adversarial image according to the classification confidence output by the target detection network model to obtain the target adversarial image includes: According to the classification confidence output by the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the classification confidence output by the target detection network model is the smallest.

3. The method according to claim 2, characterized in that The step of adjusting the pixel values ​​of the pixels in the adversarial image according to the classification confidence output by the target detection network model to obtain the target adversarial image includes: Repeat the following steps until it is determined that the classification confidence output by the target detection network model is the minimum, wherein the adversarial image is regarded as the current adversarial image when performing the following steps: Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images; Inputting the training image of this round into the target detection network model to obtain the classification confidence of the target detection network model output of this round; When the difference between the classification confidence output in the current round and the classification confidence output in the previous round of the target detection network model is greater than a first preset threshold, adjusting the pixel values ​​of the pixels in the current adversarial image to obtain an updated current adversarial image; When the difference between the classification confidence output in the current round and the classification confidence output in the previous round is less than or equal to the first preset threshold, it is determined that the classification confidence output by the target detection network model is the smallest.

4. The method according to claim 3, characterized in that The step of adjusting the pixel values ​​of the pixels in the current adversarial image to obtain the updated current adversarial image includes: Obtaining gradient descent information of an objective function, wherein the objective function is used to obtain pixel values ​​of pixels in the current adversarial image when the classification confidence output by the target detection network model is minimum; According to the gradient descent information, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image.

5. The method according to claim 1, characterized in that The step of adjusting the pixel values ​​of the pixels in the adversarial image according to the predicted area output by the target detection network model to obtain the target adversarial image includes: According to the predicted area output by the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the area overlap between the predicted area and the first actual area is minimized.

6. The method according to claim 5, characterized in that According to the predicted area output by the target detection network model, pixel values ​​of pixels in the adversarial image are adjusted to obtain the target adversarial image, including: Repeat the following steps until it is determined that the region overlap is minimal, wherein the adversarial image is regarded as the current adversarial image when the following steps are performed: Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images, and the first actual area in the label information corresponding to the one sample image is the actual area of ​​this round; Inputting the training image of this round into the target detection network model to obtain the predicted area output by the target detection network model of this round; In the case where the difference between the area overlap obtained in the current round and the area overlap obtained in the previous round is greater than a second preset threshold, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image, wherein the area overlap obtained in the current round is: the intersection of the predicted area output in the current round and the actual area in the current round divided by the union of the predicted area output in the current round and the actual area in the current round, and the area overlap obtained in the previous round is: the intersection of the predicted area output in the previous round and the actual area in the previous round divided by the union of the predicted area output in the previous round and the actual area in the previous round; When the difference between the area overlap obtained in the current round and the area overlap obtained in the previous round is less than or equal to the second preset threshold, it is determined that the area overlap is the minimum.

7. The method according to claim 6, characterized in that The step of adjusting the pixel values ​​of the pixels in the current adversarial image to obtain the updated current adversarial image includes: Obtaining gradient descent information of an objective function, wherein the objective function is used to obtain pixel values ​​of pixels in the current adversarial image when the region overlap is minimized; According to the gradient descent information, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image.

8. The method according to claim 1, characterized in that The step of adjusting the pixel values ​​of the pixels in the adversarial image according to the classification confidence and the output prediction area output by the target detection network model to obtain the target adversarial image includes: According to the classification confidence and the output prediction area of ​​the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the target value is minimum, and the target value is determined based on the classification confidence and the area overlap between the prediction area and the first actual area.

9. The method according to claim 8, characterized in that According to the classification confidence and the output prediction area of ​​the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, including: Repeat the following steps until it is determined that the target value is the minimum, wherein when performing the following steps, the adversarial image is regarded as the current adversarial image: Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images, and the first actual area in the label information corresponding to the one sample image is the actual area of ​​this round; Inputting the training image of this round into the target detection network model to obtain the prediction area output by the target detection network model of this round and the classification confidence output of this round; In the case where the difference between the target value determined in the current round and the target value determined in the previous round is greater than a third preset threshold, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image, wherein the target value determined in the current round is determined according to the classification confidence output in the current round and the region overlap determined in the current round, the target value determined in the previous round is determined according to the classification confidence output in the previous round and the region overlap determined in the previous round, the region overlap determined in the current round is: the intersection of the predicted region output in the current round and the actual region of the current round divided by the union of the predicted region output in the current round and the actual region of the current round, and the region overlap determined in the previous round is: the intersection of the predicted region output in the previous round and the actual region of the previous round divided by the union of the predicted region output in the previous round and the actual region of the previous round; In the case where the difference between the target value determined in the current round and the target value determined in the previous round is less than or equal to the third preset threshold, it is determined that the target value is the minimum.

10. The method according to claim 9, characterized in that The step of adjusting the pixel values ​​of the pixels in the current adversarial image to obtain the updated current adversarial image includes: Obtaining gradient descent information of an objective function, wherein the objective function is used to obtain a pixel value of a pixel point in the current adversarial image when the objective value is minimized; According to the gradient descent information, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image.

11. The method according to claim 1, characterized in that: The step of adjusting the pixel values ​​of the pixels in the adversarial image according to the detection results output by the target detection network model to obtain the target adversarial image includes: Repeat the following steps until it is determined that the target value is the minimum, wherein the target value is determined according to the detection result output by the target detection network model, and when performing the following steps, the adversarial image is regarded as the current adversarial image: Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images; Inputting the training image of this round into the target detection network model to obtain the detection result output by the target detection network model of this round; When the difference between the target value obtained in the current round and the target value obtained in the previous round is greater than a preset target threshold, adjusting the pixel value of the pixel point in the current adversarial image to obtain an updated current adversarial image; When the difference between the target value obtained in the current round and the target value obtained in the previous round is less than or equal to the target preset threshold, it is determined that the target value is the minimum.

12. A device for generating adversarial samples, characterized in that: include: An acquisition module, configured to acquire a set of sample images and a set of label information, wherein the set of sample images and the set of label information have a one-to-one correspondence, each of the sample images contains an object of a target classification, and the label information includes a first actual area of ​​the object of the target classification in the sample image, and a second actual area of ​​the adversarial image in the sample image; an adjustment module, configured to adjust the pixel values ​​of the pixels in the adversarial image according to the detection result output by the target detection network model to obtain the target adversarial image, wherein the detection result output by the target detection network model is the detection result obtained by inputting a training image into the target detection network model, the training image is an image obtained by setting the adversarial image in the second actual area in the sample image, and when the adversarial image is the target adversarial image, the detection result output by the target detection network model satisfies a preset loss condition; A setting module, configured to set the target adversarial image as the adversarial image in the second actual area of ​​the set of sample images to obtain a set of adversarial samples; The adjustment module is used to adjust the pixel values ​​of the pixels in the adversarial image according to the detection results output by the target detection network model in the following manner to obtain the target adversarial image: Adjusting the pixel values ​​of the pixels in the adversarial image according to the classification confidence output by the target detection network model to obtain the target adversarial image, wherein the classification confidence is used to represent the probability that the object recognized by the target detection network model in the training image belongs to the target classification, and the loss condition represents that the target value determined according to the classification confidence is minimized; or According to the predicted area output by the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein the predicted area is the area of ​​the object in the training image recognized by the target detection network model in the training image, the loss condition indicates that the target value determined according to the predicted area is minimum, the predicted area is used to determine the area overlap with the first actual area, and the area overlap is used to determine the target value; or The pixel values ​​of the pixels in the adversarial image are adjusted according to the classification confidence and the predicted area output by the target detection network model to obtain the target adversarial image, wherein the loss condition indicates that the target value determined according to the classification confidence and the predicted area is minimum, the predicted area is used to determine the area overlap with the first actual area, the area overlap is used to determine the target value, the classification confidence is used to indicate the probability that the object recognized by the target detection network model in the training image belongs to the target classification, and the predicted area is the area in the training image of the object recognized by the target detection network model in the training image.

13. The device according to claim 12, characterized in that The device is used to adjust the pixel values ​​of the pixels in the adversarial image according to the classification confidence output by the target detection network model in the following manner to obtain the target adversarial image: According to the classification confidence output by the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the classification confidence output by the target detection network model is the smallest.

14. The device according to claim 13, characterized in that The device is used to adjust the pixel values ​​of the pixels in the adversarial image according to the classification confidence output by the target detection network model in the following manner to obtain the target adversarial image: Repeat the following steps until it is determined that the classification confidence output by the target detection network model is the minimum, wherein the adversarial image is regarded as the current adversarial image when performing the following steps: Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images; Inputting the training image of this round into the target detection network model to obtain the classification confidence of the target detection network model output of this round; When the difference between the classification confidence output in the current round and the classification confidence output in the previous round of the target detection network model is greater than a first preset threshold, adjusting the pixel values ​​of the pixels in the current adversarial image to obtain an updated current adversarial image; When the difference between the classification confidence output in the current round and the classification confidence output in the previous round is less than or equal to the first preset threshold, it is determined that the classification confidence output by the target detection network model is the smallest.

15. The device according to claim 14, characterized in that The device is used to adjust the pixel values ​​of the pixels in the current adversarial image in the following manner to obtain the updated current adversarial image: Obtaining gradient descent information of an objective function, wherein the objective function is used to obtain pixel values ​​of pixels in the current adversarial image when the classification confidence output by the target detection network model is minimum; According to the gradient descent information, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image.

16. The device according to claim 12, characterized in that The device is used to adjust the pixel values ​​of the pixels in the adversarial image according to the predicted area output by the target detection network model in the following manner to obtain the target adversarial image: According to the predicted area output by the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the area overlap between the predicted area and the first actual area is minimized.

17. The device according to claim 16, characterized in that The device is used to adjust the pixel values ​​of the pixels in the adversarial image according to the predicted area output by the target detection network model in the following manner to obtain the target adversarial image: Repeat the following steps until it is determined that the region overlap is minimal, wherein the adversarial image is regarded as the current adversarial image when the following steps are performed: Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images, and the first actual area in the label information corresponding to the one sample image is the actual area of ​​this round; Inputting the training image of this round into the target detection network model to obtain the predicted area output by the target detection network model of this round; In the case where the difference between the area overlap obtained in the current round and the area overlap obtained in the previous round is greater than a second preset threshold, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image, wherein the area overlap obtained in the current round is: the intersection of the predicted area output in the current round and the actual area in the current round divided by the union of the predicted area output in the current round and the actual area in the current round, and the area overlap obtained in the previous round is: the intersection of the predicted area output in the previous round and the actual area in the previous round divided by the union of the predicted area output in the previous round and the actual area in the previous round; When the difference between the area overlap obtained in the current round and the area overlap obtained in the previous round is less than or equal to the second preset threshold, it is determined that the area overlap is the minimum.

18. The device according to claim 17, characterized in that The device is used to adjust the pixel values ​​of the pixels in the current adversarial image in the following manner to obtain the updated current adversarial image: Obtaining gradient descent information of an objective function, wherein the objective function is used to obtain pixel values ​​of pixels in the current adversarial image when the region overlap is minimized; According to the gradient descent information, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image.

19. The device according to claim 12, characterized in that The device is used to adjust the pixel values ​​of the pixels in the adversarial image according to the classification confidence and the output prediction area output by the target detection network model in the following manner to obtain the target adversarial image: According to the classification confidence and the output prediction area of ​​the target detection network model, the pixel values ​​of the pixels in the adversarial image are adjusted to obtain the target adversarial image, wherein when the adversarial image is the target adversarial image, the target value is minimum, and the target value is determined based on the classification confidence and the area overlap between the prediction area and the first actual area.

20. The device according to claim 19, characterized in that The device is used to adjust the pixel values ​​of the pixels in the adversarial image according to the classification confidence and the output prediction area output by the target detection network model in the following manner to obtain the target adversarial image: Repeat the following steps until it is determined that the target value is the minimum, wherein when performing the following steps, the adversarial image is regarded as the current adversarial image: Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images, and the first actual area in the label information corresponding to the one sample image is the actual area of ​​this round; Inputting the training image of this round into the target detection network model to obtain the prediction area output by the target detection network model of this round and the classification confidence output of this round; In the case where the difference between the target value determined in the current round and the target value determined in the previous round is greater than a third preset threshold, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image, wherein the target value determined in the current round is determined according to the classification confidence output in the current round and the region overlap determined in the current round, the target value determined in the previous round is determined according to the classification confidence output in the previous round and the region overlap determined in the previous round, the region overlap determined in the current round is: the intersection of the predicted region output in the current round and the actual region of the current round divided by the union of the predicted region output in the current round and the actual region of the current round, and the region overlap determined in the previous round is: the intersection of the predicted region output in the previous round and the actual region of the previous round divided by the union of the predicted region output in the previous round and the actual region of the previous round; In the case where the difference between the target value determined in the current round and the target value determined in the previous round is less than or equal to the third preset threshold, it is determined that the target value is the minimum.

21. The device according to claim 20, characterized in that The device is used to adjust the pixel values ​​of the pixels in the current adversarial image in the following manner to obtain the updated current adversarial image: Obtaining gradient descent information of an objective function, wherein the objective function is used to obtain a pixel value of a pixel point in the current adversarial image when the objective value is minimized; According to the gradient descent information, the pixel values ​​of the pixels in the current adversarial image are adjusted to obtain the updated current adversarial image.

22. The device according to claim 12, characterized in that The device is used to adjust the pixel values ​​of the pixels in the adversarial image according to the detection results output by the target detection network model in the following manner to obtain the target adversarial image: Repeat the following steps until it is determined that the target value is the minimum, wherein the target value is determined according to the detection result output by the target detection network model, and when performing the following steps, the adversarial image is regarded as the current adversarial image: Acquire a training image of this round, wherein the training image of this round is an image obtained by setting the current adversarial image in the second actual area of ​​a sample image in the set of sample images; Inputting the training image of this round into the target detection network model to obtain the detection result output by the target detection network model of this round; When the difference between the target value obtained in the current round and the target value obtained in the previous round is greater than a preset target threshold, adjusting the pixel value of the pixel point in the current adversarial image to obtain an updated current adversarial image; When the difference between the target value obtained in the current round and the target value obtained in the previous round is less than or equal to the target preset threshold, it is determined that the target value is the minimum.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program can be executed by a terminal device or a computer to execute the method described in any one of claims 1 to 11.

24. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 11 through the computer program.

Images