Image generation method and device, electronic equipment and computer readable storage medium

Abstract

The invention relates to the technical field of image generation, and provides an image generation method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: inputting an initial image into a self-attention network model for image conversion processing, and obtaining an interpretive image; inputting the initial image, the original raindrop image and the interpretive image into a generator for image generation processing to obtain an imaginary raindrop image and a confrontation raindrop image; performing identification judgment processing on the virtual raindrop image and the original raindrop image according to a discriminator; performing classification processing on the anti-raindrop image according to the transmission classifier; and obtaining a to-be-converted image, and inputting the to-be-converted image to the generator after training adjustment for image generation processing to obtain an adversarial sample image. According to the technical scheme, the recognition capability of the deep neural network can be well improved, so that the image recognition can be more accurate.

Description

Technical Field

[0001] The embodiments of the present application relate to but are not limited to the field of image generation technology, and in particular, to an image generation method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Deep neural networks are vulnerable to adversarial attacks. Existing adversarial attacks can be divided into digital attacks and physical attacks. A large number of studies have shown that natural phenomena can be used as adversarial attackers for deep neural networks. Based on this, some researchers have proposed using some natural phenomena to generate more natural adversarial attacks to improve the recognition ability of deep neural networks. Currently, when users upload pictures to relevant medical systems on rainy days, they cannot be clearly identified due to the presence of raindrops in the pictures. Therefore, it is necessary to simulate raindrops to train the deep neural networks in the medical system. However, visual perturbations are generally used to create adversarial samples, but the generated adversarial samples are quite different from samples in the real world, which cannot well improve the recognition ability of deep neural networks, resulting in inaccurate image recognition. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] In order to solve the problems mentioned in the above background technology, the embodiments of the present application provide an image generation method, device, electronic device and computer-readable storage medium, which can greatly improve the recognition ability of deep neural networks, thereby making image recognition more accurate.

[0005] In a first aspect, an embodiment of the present application provides an image generation method, which is applied to a deep neural network, wherein the deep neural network includes a generator, a discriminator, and a transmission classifier, and the method includes:

[0006] Obtain the initial image and the original raindrop image;

[0007] Inputting the initial image into a pre-trained self-attention network model for image conversion processing to obtain an explanatory image;

[0008] Inputting the initial image, the original raindrop image and the explanatory image into the generator for image generation processing to obtain a fictitious raindrop image and an adversarial raindrop image;

[0009] Performing recognition and judgment processing on the fictitious raindrop image and the original raindrop image according to the discriminator to obtain a recognition and judgment result; and performing a first training adjustment processing on the generator and the discriminator according to the recognition and judgment result;

[0010] Classify the adversarial raindrop image according to the transmission classifier to obtain a classification result; and perform a second training adjustment process on the generator and the transmission classifier according to the classification result;

[0011] The image to be converted is obtained, and the image to be converted is input into the generator after training and adjustment for image generation processing to obtain an adversarial sample image.

[0012] According to some embodiments of the present application, the self-attention network model includes a multi-head self-attention module, a feedforward fully connected neural network module and a decoding module, and the initial image is input into the pre-trained self-attention network model for image conversion processing to obtain an explanatory image, including:

[0013] Converting the initial image into an embedding vector;

[0014] Based on the multi-head self-attention module, the embedded vector is segmented and calculated to obtain multiple groups of probability distribution values;

[0015] Based on the feedforward fully connected neural network module, a plurality of groups of probability distribution values ​​are transformed and activated to obtain normalized information;

[0016] The normalized information is decoded and transformed based on the decoding module to obtain the explanatory image.

[0017] According to some embodiments of the present application, the generator includes an encoder and a first convolutional layer, and the initial image, the original raindrop image, and the explanatory image are input into the generator for image generation processing to obtain a fictitious raindrop image and an adversarial raindrop image, including:

[0018] Inputting the original raindrop image into the encoder for encoding calculation processing to obtain a Gaussian noise vector;

[0019] Inputting the original raindrop image and the explanatory image into the first convolution layer for feature extraction, respectively, to obtain shallow features of the first image corresponding to the original raindrop image and shallow features of the second image corresponding to the explanatory image;

[0020] The shallow features of the first image are subjected to a first fusion process with the Gaussian noise vector to obtain the imaginary raindrop image; and the shallow features of the second image are subjected to a second fusion process with the Gaussian noise vector to obtain the adversarial raindrop image.

[0021] According to some embodiments of the present application, the discriminator includes a second convolutional layer, a first pooling layer, and a first fully connected layer, and the discriminator performs recognition and judgment processing on the fictitious raindrop image and the original raindrop image to obtain a recognition and judgment result, including:

[0022] Inputting the fictitious raindrop image and the original raindrop image into the second convolutional layer for feature extraction respectively, to obtain a first raindrop image feature corresponding to the fictitious raindrop image and a second raindrop image feature corresponding to the original raindrop image;

[0023] Inputting the first raindrop image feature and the second raindrop image feature into the first pooling layer for dimensionality reduction processing to obtain a first raindrop feature vector corresponding to the first raindrop image feature and a second raindrop feature vector corresponding to the second raindrop image feature;

[0024] The first raindrop feature vector and the second raindrop feature vector are input into the first fully connected layer for classification and judgment processing to obtain the recognition and judgment result.

[0025] According to some embodiments of the present application, the transmission classifier includes a feature extraction module, a feature encoding module and a classification decision module, and the classification processing of the anti-raindrop image according to the transmission classifier to obtain a classification result includes:

[0026] Based on the feature extraction module, feature extraction is performed on the anti-raindrop image to obtain anti-raindrop features;

[0027] Based on the feature encoding module, the anti-raindrop feature is encoded to obtain an anti-raindrop abstract feature;

[0028] Based on the classification decision module, classification mapping processing is performed on the adversarial raindrop abstract features to obtain the classification result.

[0029] According to some embodiments of the present application, the step of inputting the image to be converted into the generator after training and adjustment for image generation processing to obtain an adversarial sample image includes:

[0030] Determining distribution information from the generator after training and adjustment;

[0031] Sampling is performed on the image to be converted according to the distribution information to obtain the adversarial sample image.

[0032] According to some embodiments of the present application, after acquiring the image to be converted and inputting the image to be converted into the generator after training and adjustment for image generation processing to obtain the adversarial sample image, the method further includes:

[0033] Obtain an image to be recognized;

[0034] The image to be recognized is input into the trained and adjusted discriminator for recognition and judgment processing to obtain an image recognition result.

[0035] In a second aspect, an embodiment of the present application further provides an image generation device, including a deep neural network, wherein the deep neural network includes a generator, a discriminator, and a transmission classifier, and the device further includes:

[0036] A first processing module, used for acquiring an initial image and an original raindrop image;

[0037] A second processing module is used to input the initial image into a pre-trained self-attention network model for image conversion processing to obtain an explanatory image;

[0038] a third processing module, configured to input the initial image, the original raindrop image and the explanatory image into the generator for image generation processing to obtain a fictitious raindrop image and an adversarial raindrop image;

[0039] a fourth processing module, configured to perform recognition and judgment processing on the fictitious raindrop image and the original raindrop image according to the discriminator to obtain a recognition and judgment result; and perform a first training adjustment processing on the generator and the discriminator according to the recognition and judgment result;

[0040] a fifth processing module, configured to classify the adversarial raindrop image according to the transmission classifier to obtain a classification result; and perform a second training adjustment process on the generator and the transmission classifier according to the classification result;

[0041] The sixth processing module is used to obtain the image to be converted, and input the image to be converted into the generator that has been trained and adjusted for image generation processing to obtain an adversarial sample image.

[0042] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image generation method as described in the first aspect above is implemented.

[0043] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the image generation method described in the first aspect above.

[0044] According to the image generation method of the embodiment provided by the present application, at least the following beneficial effects are achieved: in the process of image generation, the initial image and the original raindrop image are first obtained; then the initial image is input into the pre-trained self-attention network model for image conversion processing to obtain an explanatory image; then the initial image, the original raindrop image and the explanatory image are input into the generator for image generation processing, and then a fictitious raindrop image and an adversarial raindrop image can be obtained; then the discriminator can be used to identify and judge the fictitious raindrop image and the original raindrop image to obtain the identification and judgment result; and the generator and the discriminator are also subjected to a first training adjustment processing according to the identification and judgment result; the adversarial raindrop image can also be classified according to the transmission classifier, and then the classification result can be obtained; and the generator and the transmission classifier can also be subjected to a second training adjustment processing according to the classification result; finally, the image to be converted can be obtained, and the image to be converted is input into the generator after the training adjustment for image generation processing, and then the adversarial sample image can be obtained. Through the above technical scheme, the recognition ability of the deep neural network can be well improved, and the image recognition can be made more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0046] Figure 1 is a flow chart of an image generation method provided by an embodiment of the present application;

[0047] Figure 2 is a specific flow chart of step S200 provided in one embodiment of the present application;

[0048] Figure 3 is a specific flow chart of step S300 provided in one embodiment of the present application;

[0049] Figure 4 is a specific flow chart of step S400 provided in one embodiment of the present application;

[0050] Figure 5 is a specific flow chart of step S500 provided in one embodiment of the present application;

[0051] Figure 6 is a specific flow chart of step S600 provided in one embodiment of the present application;

[0052] Figure 7 This is a flowchart after executing step S600 provided by an embodiment of the present application;

[0053] Figure 8is a schematic diagram of an image generating device provided by an embodiment of the present application;

[0054] Fig. 9 It is a schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] It should be noted that although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0057] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0058] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0059] AI is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. Artificial intelligence is a branch of computer science. Artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0060] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0061] Artificial intelligence is AI. AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0062] The servers involved in artificial intelligence technology can be independent servers or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0063] The present application provides an image generation method, device, electronic device and computer-readable storage medium. In the process of image generation, firstly, an initial image and an original raindrop image are obtained; then, the initial image is input into a pre-trained self-attention network model for image conversion processing to obtain an explanatory image; then, the initial image, the original raindrop image and the explanatory image are input into a generator for image generation processing, and then a fictitious raindrop image and an adversarial raindrop image can be obtained; then, a discriminator can be used to identify and judge the fictitious raindrop image and the original raindrop image to obtain an identification and judgment result; and the generator and the discriminator are subjected to a first training adjustment processing according to the identification and judgment result; the adversarial raindrop image can also be classified according to a transmission classifier, and then a classification result can be obtained; and the generator and the transmission classifier can also be subjected to a second training adjustment processing according to the classification result; finally, the image to be converted can be obtained, and the image to be converted can be input into the generator after training and adjustment for image generation processing, and then an adversarial sample image can be obtained. Through the above technical scheme, the recognition ability of the deep neural network can be well improved, and the image recognition can be made more accurate.

[0064] The image generation method provided in the embodiment of the present application relates to the field of image generation technology. The image generation method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the image generation method, etc., but is not limited to the above forms.

[0065] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0066] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0067] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0068] like Figure 1 As shown, Figure 1It is a flowchart of an image generation method provided by an embodiment of the present application. The image generation method is applied to a deep neural network, and the deep neural network includes a generator, a discriminator and a transmission classifier. The image generation method includes but is not limited to steps S100 to S600.

[0069] Step S100, obtaining an initial image and an original raindrop image;

[0070] Step S200, inputting the initial image into the pre-trained self-attention network model for image conversion processing to obtain an explanatory image;

[0071] Step S300, inputting the initial image, the original raindrop image and the explanatory image into the generator for image generation processing to obtain a fictitious raindrop image and an adversarial raindrop image;

[0072] Step S400, performing recognition and judgment processing on the fictitious raindrop image and the original raindrop image according to the discriminator to obtain a recognition and judgment result; and performing a first training adjustment processing on the generator and the discriminator according to the recognition and judgment result;

[0073] Step S500, classifying the anti-raindrop image according to the transmission classifier to obtain a classification result; and performing a second training adjustment process on the generator and the transmission classifier according to the classification result;

[0074] Step S600, obtaining an image to be converted, and inputting the image to be converted into a generator that has been trained and adjusted for image generation processing to obtain an adversarial sample image.

[0075] It should be noted that, in the process of image generation, the initial image and the original raindrop image are first obtained; then the initial image is input into the pre-trained self-attention network model for image conversion processing to obtain an explanatory image; then the initial image, the original raindrop image and the explanatory image are input into the generator for image generation processing, and then the fictitious raindrop image and the adversarial raindrop image can be obtained; then the discriminator can be used to identify and judge the fictitious raindrop image and the original raindrop image to obtain the identification and judgment result; and the generator and the discriminator are also subjected to a first training adjustment processing according to the identification and judgment result; the adversarial raindrop image can also be classified according to the transmission classifier, and then the classification result can be obtained; and the generator and the transmission classifier can also be subjected to a second training adjustment processing according to the classification result; finally, the image to be converted can be obtained, and the image to be converted can be input into the generator after the training adjustment for image generation processing, and then the adversarial sample image can be obtained. Through the above technical scheme, the recognition ability of the deep neural network can be well improved, and the image recognition can be made more accurate.

[0076] It is worth noting that in the process of image generation, the initial image and the original raindrop image are first obtained; then the initial image is input into the pre-trained self-attention network model for image conversion processing to obtain an explanatory image; subsequently, the initial image, the original raindrop image and the explanatory image are input into the generator for image generation processing, and then the fictitious raindrop image and the adversarial raindrop image can be obtained; then the discriminator is used to perform recognition and judgment processing on the fictitious raindrop image and the original raindrop image to obtain the recognition and judgment result; then the generator and the discriminator are subjected to a first training adjustment processing according to the recognition and judgment result; subsequently, the adversarial raindrop image can be analyzed and processed by the transfer classifier to obtain the classification result; and, according to the classification result, the generator and the transfer classifier are subjected to a second training adjustment processing; finally, the image to be converted can be subjected to image conversion processing according to the generator after the training adjustment, so as to obtain the adversarial sample image.

[0077] In some specific embodiments of the present application, when a user uploads a picture in a medical system, it is raining at the time, so the picture taken by the user contains the physical interference factor of raindrops. Since the deep neural network in the medical system has been trained to resist interference through the image generation method of the embodiment of the present application, the deep neural network of the embodiment of the present application can accurately and quickly recognize and process the picture, thereby bringing great convenience to the user.

[0078] It is worth noting that the deep neural network of the embodiment of the present application includes a generator, a discriminator and a transfer classifier; wherein the generator can generate new images that are visually indistinguishable from real images. This is very useful in the fields of artistic creation, game design, filmmaking, etc. In machine learning, the generator can be used to expand the training data set, especially in the case of data scarcity, by generating additional samples to improve the generalization ability of the model. The generator can realize functions such as image style conversion, super-resolution, and image restoration, that is, converting an image of one style to another style, or converting a low-resolution image to a high-resolution image. The main function of the discriminator is to determine whether the given data is from a real data set or forged data generated by the generator; it distinguishes real data from generated data as much as possible by learning the distribution characteristics of real data; the discriminator evaluates the samples generated by the generator, provides feedback signals, and helps the generator improve its generation ability; the goal of the discriminator is to classify the samples, determine whether the samples are real or generated, and thus provide quality feedback on the generated samples. The main function of the transfer classifier is to map the high-level features learned by the deep neural network to specific categories to achieve classification of input data. For example, in image recognition tasks, a classifier maps extracted image features to different class labels. Transfer classifiers typically contain a large number of parameters that grow linearly with the number of classes, which are learned by affine transformations to produce the values ​​for each class used for classification.

[0079] It is worth noting that in the process of image generation, the initial image obtained is input into the pre-trained self-attention network model for image conversion processing, and then an explanatory image can be obtained; based on the explanatory image, the prerequisite preparation is made for the subsequent generation of fictional raindrop images and adversarial raindrop images; then the fictional raindrop images and the original raindrop images can be identified and judged according to the discriminator, and then the identification and judgment results can be obtained, and then the generator and the discriminator can be subjected to a first training adjustment processing according to the identification and judgment results; then the adversarial raindrop images can also be classified according to the transmission classifier, and the generator and the transmission classifier can also be subjected to a second training adjustment processing according to the classification results; through the above technical scheme, the deep neural network of the embodiment of the present application can be trained and adjusted to improve the resistance of the deep mental network to samples, so as to prepare for subsequent image recognition processing.

[0080] In some embodiments, Figure 2 As shown, the self-attention network model includes a multi-head self-attention module, a feedforward fully connected neural network module and a decoding module, and the above step S200 may include but is not limited to steps S210 to S240.

[0081] Step S210, converting the initial image into an embedding vector;

[0082] Step S220, performing segmentation calculation processing on the embedded vector based on the multi-head self-attention module to obtain multiple groups of probability distribution values;

[0083] Step S230, performing transformation activation processing on multiple groups of probability distribution values ​​based on a feedforward fully connected neural network module to obtain normalized information;

[0084] Step S240: decoding and transforming the normalized information based on the decoding module to obtain an explanatory image.

[0085] It should be noted that, in the process of inputting the initial image into the pre-trained self-attention network model for image conversion processing to obtain an explanatory image, the initial image is first converted into an embedding vector; then, the embedding vector is segmented and calculated based on the multi-head self-attention module to obtain multiple groups of probability distribution values; then, based on the feedforward fully connected neural network model, the multiple groups of probability distribution values ​​are transformed and activated to obtain normalized information; finally, based on the decoding module, the normalized information is decoded and transformed to obtain an explanatory image, in order to prepare for the generation of fictional raindrop images and adversarial raindrop images.

[0086] It is worth noting that the self-attention network model includes a multi-head attention module, a feedforward fully connected neural network module and a decoding module; among them, the multi-head self-attention module uses multiple independent attention heads, and the multi-head self-attention mechanism can capture information from different representation subspaces in parallel, thereby enhancing the model's expressiveness and generalization capabilities; each attention head independently performs the self-attention mechanism, which allows the model to focus on different parts of the input sequence at the same time, improving computational efficiency; multi-head self-attention can capture multiple types of relationships and patterns in the data through different heads, for example, one head may focus on syntactic information, while another head may focus on semantic information. Feedforward fully connected neural networks are suitable for various types of data and tasks, such as image recognition, speech recognition, natural language processing and many other fields; feedforward fully connected neural networks are easy to train, and the back-propagation algorithm can quickly update weights and bias terms. At the same time, it can tolerate a certain degree of data noise and missingness, and has good robustness.

[0087] In some embodiments, Figure 3 As shown, the generator includes an encoder and a first convolutional layer, and the above step S300 may include but is not limited to steps S310 to S330.

[0088] Step S310, inputting the original raindrop image into the encoder for encoding calculation processing to obtain a Gaussian noise vector;

[0089] Step S320, inputting the original raindrop image and the explanatory image into the first convolutional layer for feature extraction, to obtain shallow features of the first image corresponding to the original raindrop image and shallow features of the second image corresponding to the explanatory image;

[0090] Step S330, performing a first fusion process on the shallow features of the first image and the Gaussian noise vector to obtain a fictitious raindrop image; and performing a second fusion process on the shallow features of the second image and the Gaussian noise vector to obtain an adversarial raindrop image.

[0091] It should be noted that, in the process of inputting the initial image, the original raindrop image and the explanatory image into the generator for image generation processing to obtain the fictitious raindrop image and the adversarial raindrop image, the original raindrop image is first input into the encoder for encoding calculation processing to obtain the Gaussian noise vector; then the original raindrop image and the explanatory image are respectively input into the first convolutional layer for feature extraction, and then the shallow features of the first image corresponding to the original raindrop image and the shallow features of the second image corresponding to the explanatory image can be obtained; finally, the shallow features of the first image and the Gaussian noise vector are subjected to a first fusion processing to obtain the fictitious raindrop image, and the shallow features of the second image and the Gaussian noise vector are subjected to a second fusion processing to obtain the adversarial raindrop image, so as to prepare for the subsequent training processing of the generator, the discriminator and the transmission classifier.

[0092] It is worth noting that the Gaussian noise vector can be obtained by encoding the original raindrop image. Subsequently, the shallow features of the first image corresponding to the original raindrop image and the shallow features of the second image corresponding to the explanatory image can be fused with the Gaussian noise vector to obtain the fictitious raindrop image and the adversarial raindrop image.

[0093] In some embodiments, Figure 4 As shown, the discriminator includes a second convolutional layer, a first pooling layer and a first fully connected layer, and the above step S400 may include but is not limited to steps S410 to S430.

[0094] Step S410, inputting the fictitious raindrop image and the original raindrop image into the second convolutional layer for feature extraction, to obtain a first raindrop image feature corresponding to the fictitious raindrop image and a second raindrop image feature corresponding to the original raindrop image;

[0095] Step S420, inputting the first raindrop image feature and the second raindrop image feature into the first pooling layer for dimensionality reduction processing, to obtain a first raindrop feature vector corresponding to the first raindrop image feature and a second raindrop feature vector corresponding to the second raindrop image feature;

[0096] Step S430: Input the first raindrop feature vector and the second raindrop feature vector into the first fully connected layer for classification and judgment processing to obtain a recognition and judgment result.

[0097] It should be noted that, in the process of obtaining the recognition and judgment result by the discriminator through the recognition and judgment processing of the fictitious raindrop image and the original raindrop image, the fictitious raindrop image and the original raindrop image are firstly input into the second convolutional layer for feature extraction respectively, so as to obtain the first raindrop image feature corresponding to the fictitious raindrop image and the second raindrop image feature corresponding to the original raindrop image; then, the first raindrop image feature and the second raindrop image feature are input into the first pooling layer for dimensionality reduction processing, so as to obtain the first raindrop feature vector corresponding to the first raindrop image feature and the second raindrop feature vector corresponding to the second raindrop image feature; finally, the first raindrop feature vector and the second raindrop feature vector are output to the first fully connected layer for classification and judgment processing, so as to obtain the recognition and judgment result, which can provide a data basis for the training of the generator and the discriminator.

[0098] It is worth noting that by inputting the fictitious raindrop image and the original raindrop image into the second convolutional layer for feature extraction processing respectively, the first raindrop image feature of the fictitious raindrop image and the second raindrop image feature corresponding to the original raindrop image can be obtained; then the first raindrop image feature and the second raindrop image feature can be subjected to dimensionality reduction processing, so as to prepare for subsequent classification and judgment processing.

[0099] In some embodiments, Figure 5 As shown, the transmission classifier includes a feature extraction module, a feature encoding module and a classification decision module, and the above step S500 may include but is not limited to steps S510 to S530.

[0100] Step S510, extracting features from the anti-raindrop image based on the feature extraction module to obtain anti-raindrop features;

[0101] Step S520, encoding the anti-raindrop feature based on the feature encoding module to obtain the anti-raindrop abstract feature;

[0102] Step S530 , performing classification mapping processing on the adversarial raindrop abstract features based on the classification decision module to obtain a classification result.

[0103] It should be noted that in the process of classifying the adversarial raindrop image according to the transmission classifier to obtain the classification result, first, the adversarial raindrop feature is extracted based on the feature extraction module to obtain the adversarial raindrop feature; then, the adversarial raindrop feature is encoded based on the feature encoding module to obtain the adversarial raindrop abstract feature; finally, the adversarial raindrop abstract feature is classified and mapped based on the classification decision module to obtain the classification result, so as to make prerequisite preparations for the subsequent training processing of the transmission classifier and the generator, so as to improve the robustness of the deep neural network of the embodiment of the present application.

[0104] It is worth noting that the feature extraction module of the transmission classifier in the embodiment of the present application can be used for feature extraction, the feature encoding module can be used for encoding processing and the classification decision module can be used for classification mapping processing.

[0105] In some embodiments, Figure 6 As shown, the above step S600 may include but is not limited to step S610 to step S620.

[0106] Step S610, determining distribution information from the generator after training and adjustment;

[0107] Step S620 , sampling is performed from the image to be converted according to the distribution information to obtain an adversarial sample image.

[0108] It should be noted that in the process of inputting the image to be converted into the trained and adjusted generator for image generation processing to obtain the adversarial sample image, the distribution information is first determined from the trained and adjusted generator; then, sampling processing is performed from the image to be converted according to the distribution information to obtain the adversarial sample image; and by verifying the adversarial sample image, preparations are made for subsequent image recognition.

[0109] In some embodiments, Figure 7 As shown, after executing the above step S600, it may include but not be limited to steps S710 to S720.

[0110] Step S710, obtaining an image to be recognized;

[0111] Step S720: input the image to be recognized into the trained and adjusted discriminator for recognition and judgment processing to obtain an image recognition result.

[0112] It should be noted that after the image to be identified is input into the trained and adjusted discriminator for identification and judgment processing to obtain the image recognition result, the image to be identified can be obtained first, and then the image to be identified is input into the trained and adjusted discriminator for identification and judgment processing, and then the image recognition result can be obtained, so that the deep neural network of the embodiment of the present application can accurately identify and process the raindrop image, thereby greatly improving the robustness of the deep neural network.

[0113] In order to more clearly illustrate the process of the image generation method provided in the embodiment of the present application, a specific example is given below.

[0114] The embodiment of the present application mainly generates adversarial raindrop images based on a generative adversarial network framework, which consists of a generator, a discriminator, and a transmission classifier. The generator in this framework attempts to generate raindrop images that are as realistic as possible, which can not only deceive the discriminator, but also deceive the transmission classifier. The discriminator attempts to identify whether the input image is a real raindrop image or from the generator. The transmission classifier is used to give the generated raindrop image the ability to resist attacks.

[0115] The workflow of the entire system is as follows:

[0116] Step 1: Dataset preparation stage, in which the image is input into the neural network to obtain its interpretability graph. The embodiment of the present application uses a self-attention network model to generate an interpretability graph.

[0117] Taking Transformer as an example, the attention model has a series of Q as input, with dimension d t The keys K and dimensions d v The self-attention module is defined as:

[0118] Attention(Q,K,V) (b) =A (b) ·V (b)

[0119]

[0120] Among them, A (b) is the attention map from block b. In the embodiment of the present application, the correlation based on layer-by-layer correlation propagation is used to obtain the score of the attention head of each layer in the network. Since the model includes the mixing of two feature tensors, skip connections and matrix multiplication, these two factors are considered at the same time to improve the correlation score. Given two tensors u and v, the correlation score is:

[0121]

[0122] The Transformer’s interpretability map s is calculated as:

[0123]

[0124] in, Indicates the relevance of the current layer, Yes A (b) The gradient of is the correlation graph of block b, ⊙ represents the Hadamard product, E t It is the bullish average.

[0125] The second step is to use the generator to generate images. The generator has two goals. The first goal is to generate images similar to real raindrops to deceive the discriminator. The second goal is to make the generated raindrop images more resistant to adversarial attacks.

[0126] In order to simulate natural raindrops more closely, the generator will consider the background scene of the image when generating raindrops. The embodiment of the present application intends to use multiple convolutional layers to extract shallow features of the clean image, and fuse the extracted features with the intermediate features calculated by the noise vector to generate the final raindrops. The whole process can be described as:

[0127] o ′ =G(b,z)=G(b,E(o)),

[0128] Among them ′ represents a synthetic raindrop image, and E represents an encoder that converts the raindrop image into features.

[0129] In the training phase of the generative network, images with and without raindrops and interpretable graphs are used for training. Therefore, the generation loss of the first target of the generator can be written as:

[0130] L G =L gen +α1L z +α2L P

[0131] Where L gen The purpose is to train the generator to output raindrop images that can deceive the discriminator. The loss is calculated as:

[0132] L gen =log(1-D(o ′ ))

[0133] L z The latent variable z is constrained to obey the homogeneous Gaussian distribution, and the calculation formula is:

[0134]

[0135] Among them, μ and σ are the mean and variance of the latent variable z, and d is the dimension of z.

[0136] L p In order to make the raindrops more realistic, the embodiment of the present application uses the L1 norm to encourage the generator to produce sparse perturbations:

[0137] L p =‖o ′ -b‖1

[0138] The embodiment of the present application mainly adopts a global discriminator, which attempts to identify whether the input image is a real raindrop image or from the generator, so the loss of the discriminator is defined as:

[0139] L D = -log(D(o))-log(1-D(o ′ ))

[0140] The second goal of the generator is to make the generated raindrops more attacking. Previous studies have shown that a small number of raindrop images produced by the generator can successfully mislead the target model. Therefore, the embodiment of the present application adds a transfer learning network to the architecture to convert the generated images into adversarial samples. The loss function can be written as:

[0141] L adv =‖L C (G(x,z),y)-η‖1

[0142] Where x is the clean image correctly classified by the transfer learning network, and η is a positive constant used to limit the classification loss L C , so that we can make a good trade-off between attack capability and the realism of generated raindrops.

[0143] In summary, the generation loss L used in the entire training process C It can be expressed as:

[0144] L G =L gen +α1L z +α2L p +α3L adv

[0145] After training the GAN architecture, its parameters are fixed and adversarial raindrop images are generated by solving the following optimization problem

[0146] argmax z L TC (G(x,z),y),st‖z‖<∈ z

[0147] L TC is the loss function of the target classifier, ∈ z is the threshold of z. In the white box scenario, the above formula is solved using the gradient descent method, and the specific iterative formula can be written as:

[0148]

[0149] α z is the iteration step size.

[0150] In addition, if Figure 8 As shown, an embodiment of the present application further provides an image generation device 10, including a deep neural network, the deep neural network including a generator, a discriminator and a transmission classifier, and the device also includes:

[0151] The first processing module 100 is used to obtain an initial image and an original raindrop image;

[0152] The second processing module 200 is used to input the initial image into the pre-trained self-attention network model for image conversion processing to obtain an explanatory image;

[0153] The third processing module 300 is used to input the initial image, the original raindrop image and the explanatory image into the generator for image generation processing to obtain a fictitious raindrop image and an adversarial raindrop image;

[0154] The fourth processing module 400 is used to perform recognition and judgment processing on the fictitious raindrop image and the original raindrop image according to the discriminator to obtain a recognition and judgment result; and perform a first training adjustment processing on the generator and the discriminator according to the recognition and judgment result;

[0155] A fifth processing module 500 is used to classify the adversarial raindrop image according to the transmission classifier to obtain a classification result; and to perform a second training adjustment process on the generator and the transmission classifier according to the classification result;

[0156] The sixth processing module 600 is used to obtain the image to be converted, and input the image to be converted into the generator that has been trained and adjusted for image generation processing to obtain an adversarial sample image.

[0157] It should be noted that, in the process of image generation, the initial image and the original raindrop image are first obtained; then the initial image is input into the pre-trained self-attention network model for image conversion processing to obtain an explanatory image; then the initial image, the original raindrop image and the explanatory image are input into the generator for image generation processing, and then the fictitious raindrop image and the adversarial raindrop image can be obtained; then the discriminator can be used to identify and judge the fictitious raindrop image and the original raindrop image to obtain the identification and judgment result; and the generator and the discriminator are also subjected to a first training adjustment processing according to the identification and judgment result; the adversarial raindrop image can also be classified according to the transmission classifier, and then the classification result can be obtained; and the generator and the transmission classifier can also be subjected to a second training adjustment processing according to the classification result; finally, the image to be converted can be obtained, and the image to be converted can be input into the generator after the training adjustment for image generation processing, and then the adversarial sample image can be obtained. Through the above technical scheme, the recognition ability of the deep neural network can be well improved, and the image recognition can be made more accurate.

[0158] The specific implementation of the image generating device 10 is substantially the same as the specific implementation of the above-mentioned image generating method, and will not be described in detail herein.

[0159] In addition, if Fig. 9 As shown, an embodiment of the present application further provides an electronic device 700 , which includes: a memory 720 , a processor 710 , and a computer program stored in the memory 720 and executable on the processor 710 .

[0160] The processor 710 and the memory 720 may be connected via a bus or in other ways.

[0161] The non-transitory software programs and instructions required to implement the image generation methods of the above embodiments are stored in the memory 720 , and when executed by the processor 710 , the image generation methods of the above embodiments are executed.

[0162] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are executed by a processor 710 or a controller, for example, by a processor 710 in the above-mentioned device embodiment, so that the above-mentioned processor 710 can execute the image generation method in the above-mentioned embodiment.

[0164] The above embodiments may be used in combination, and modules with the same name in different embodiments may be the same or different.

[0165] The above describes specific embodiments of the present application, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0166] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0167] The apparatus, device, computer-readable storage medium and method provided in the embodiments of the present application correspond to each other. Therefore, the apparatus, device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the corresponding apparatus, device and computer storage medium will not be repeated here.

[0168] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0169] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0170] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0171] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0172] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0173] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0176] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0177] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0178] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0179] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0180] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0181] Embodiments of the present application may be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. Embodiments of the present application may also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0182] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0183] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An image generation method, characterized in that: Applied to a deep neural network, the deep neural network includes a generator, a discriminator and a transmission classifier, the method includes: Obtain the initial image and the original raindrop image; Inputting the initial image into a pre-trained self-attention network model for image conversion processing to obtain an explanatory image; Inputting the initial image, the original raindrop image and the explanatory image into the generator for image generation processing to obtain a fictitious raindrop image and an adversarial raindrop image; Performing recognition and judgment processing on the fictitious raindrop image and the original raindrop image according to the discriminator to obtain a recognition and judgment result; and performing a first training adjustment processing on the generator and the discriminator according to the recognition and judgment result; Classify the adversarial raindrop image according to the transmission classifier to obtain a classification result; and perform a second training adjustment process on the generator and the transmission classifier according to the classification result; An image to be converted is obtained, and the image to be converted is input into the generator after training and adjustment for image generation processing to obtain an adversarial sample image.

2. The image generation method according to claim 1, characterized in that: The self-attention network model includes a multi-head self-attention module, a feedforward fully connected neural network module and a decoding module. The initial image is input into the pre-trained self-attention network model for image conversion processing to obtain an explanatory image, including: Converting the initial image into an embedding vector; Based on the multi-head self-attention module, the embedded vector is segmented and calculated to obtain multiple groups of probability distribution values; Based on the feedforward fully connected neural network module, a plurality of groups of probability distribution values ​​are transformed and activated to obtain normalized information; The normalized information is decoded and transformed based on the decoding module to obtain the explanatory image.

3. The image generation method according to claim 1, characterized in that: The generator includes an encoder and a first convolutional layer, and the initial image, the original raindrop image, and the explanatory image are input into the generator for image generation processing to obtain a fictitious raindrop image and an adversarial raindrop image, including: Inputting the original raindrop image into the encoder for encoding calculation processing to obtain a Gaussian noise vector; Inputting the original raindrop image and the explanatory image into the first convolution layer for feature extraction, respectively, to obtain shallow features of the first image corresponding to the original raindrop image and shallow features of the second image corresponding to the explanatory image; The shallow features of the first image are subjected to a first fusion process with the Gaussian noise vector to obtain the imaginary raindrop image; and the shallow features of the second image are subjected to a second fusion process with the Gaussian noise vector to obtain the adversarial raindrop image.

4. The image generation method according to claim 1, characterized in that: The discriminator includes a second convolutional layer, a first pooling layer and a first fully connected layer, and the identification and judgment processing of the fictitious raindrop image and the original raindrop image according to the discriminator to obtain the identification and judgment result includes: Inputting the fictitious raindrop image and the original raindrop image into the second convolutional layer for feature extraction respectively, to obtain a first raindrop image feature corresponding to the fictitious raindrop image and a second raindrop image feature corresponding to the original raindrop image; Inputting the first raindrop image feature and the second raindrop image feature into the first pooling layer for dimensionality reduction processing to obtain a first raindrop feature vector corresponding to the first raindrop image feature and a second raindrop feature vector corresponding to the second raindrop image feature; The first raindrop feature vector and the second raindrop feature vector are input into the first fully connected layer for classification and judgment processing to obtain the recognition and judgment result.

5. The image generation method according to claim 1, characterized in that: The transmission classifier includes a feature extraction module, a feature encoding module and a classification decision module. The classification process of the anti-raindrop image according to the transmission classifier to obtain a classification result includes: Based on the feature extraction module, feature extraction is performed on the anti-raindrop image to obtain anti-raindrop features; Based on the feature encoding module, the anti-raindrop feature is encoded to obtain an anti-raindrop abstract feature; Based on the classification decision module, classification mapping processing is performed on the adversarial raindrop abstract features to obtain the classification result.

6. The image generation method according to claim 1, characterized in that: The step of inputting the image to be converted into the generator after training and adjustment to perform image generation processing to obtain an adversarial sample image includes: Determining distribution information from the generator after training and adjustment; Sampling is performed on the image to be converted according to the distribution information to obtain the adversarial sample image.

7. The image generation method according to claim 1, characterized in that: After obtaining the image to be converted and inputting the image to be converted into the generator after training and adjustment for image generation processing to obtain the adversarial sample image, the method further includes: Obtain an image to be recognized; The image to be recognized is input into the trained and adjusted discriminator for recognition and judgment processing to obtain an image recognition result.

8. An image generating device, characterized in that: The method comprises a deep neural network, wherein the deep neural network comprises a generator, a discriminator and a transmission classifier, and the device further comprises: A first processing module, used for acquiring an initial image and an original raindrop image; A second processing module is used to input the initial image into a pre-trained self-attention network model for image conversion processing to obtain an explanatory image; a third processing module, configured to input the initial image, the original raindrop image and the explanatory image into the generator for image generation processing to obtain a fictitious raindrop image and an adversarial raindrop image; a fourth processing module, configured to perform recognition and judgment processing on the fictitious raindrop image and the original raindrop image according to the discriminator to obtain a recognition and judgment result; and perform a first training adjustment processing on the generator and the discriminator according to the recognition and judgment result; a fifth processing module, configured to classify the adversarial raindrop image according to the transmission classifier to obtain a classification result; and perform a second training adjustment process on the generator and the transmission classifier according to the classification result; The sixth processing module is used to obtain the image to be converted, and input the image to be converted into the generator that has been trained and adjusted for image generation processing to obtain an adversarial sample image.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image generation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: The computer executable instructions are used to execute the image generation method described in any one of claims 1 to 7.

Images