Training Method of Image Model, Image Denoising Method, Device, Equipment and Medium

Abstract

The present disclosure provides a method for training an image model, an image noise reduction method, an apparatus, a device, and a medium, which relate to the field of artificial intelligence, and particularly to the fields of computer vision and deep learning. The specific implementation solution is as follows: input an adversarial sample into an anti-disturbance model to obtain a sample anti-disturbance image, and use the sample anti-disturbance image to perform noise reduction on the adversarial sample to obtain a sample noise reduction image; respectively perform feature extraction on the sample noise reduction image and the sample original image, and train the anti-disturbance model according to the feature extraction results. The present disclosure can improve the defense ability against adversarial samples.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, particularly to the fields of computer vision and deep learning, and specifically discloses a method for training an image model, an image noise reduction method, an apparatus, a device, and a medium. Background Art

[0002] An adversarial sample refers to an input sample formed by deliberately adding subtle perturbations to a dataset, which causes the model to give a wrong output with high confidence, making the model unable to effectively recognize the adversarial sample.

[0003] With the development of artificial intelligence, adversarial samples have also begun to be gradually concerned. Recently, various attack algorithms for object detection models have emerged. By adding a tiny perturbation to the original image, the detection result of the model for the input image can be perturbed. For example, a certain object in the image can be made to disappear, or the model can be misclassified for a certain object, but at the same time, it does not affect the human eye's recognition of the image.

[0004] How to enable the object detection model to effectively recognize and process these tiny perturbations has become an urgent problem to be solved.

[0005] Disclosure

[0006] The present disclosure provides a method for training an image model, an image noise reduction method, an apparatus, a device, and a medium.

[0007] According to one aspect of the present disclosure, there is provided a method for training an image model, including:

[0008] Inputting an adversarial sample into an anti-perturbation model to obtain a sample anti-perturbation image, and using the sample anti-perturbation image to perform noise reduction on the adversarial sample to obtain a sample noise reduction image;

[0009] Respectively performing feature extraction on the sample noise reduction image and the sample original image, and training the anti-perturbation model according to the feature extraction results.

[0010] According to another aspect of the present disclosure, there is provided an image noise reduction method, including:

[0011] Inputting an image to be processed into a trained anti-perturbation model to obtain a target anti-perturbation image;

[0012] Using the target anti-perturbation image to perform noise reduction on the image to be processed to obtain a target noise reduction image.

[0013] According to another aspect of the present disclosure, there is provided an apparatus for training an image model, including:

[0014] A sample noise reduction module, configured to input an adversarial sample into an anti-disturbance model to obtain a sample anti-disturbance image, and use the sample anti-disturbance image to reduce the noise of the adversarial sample to obtain a sample noise reduction image;

[0015] An anti-disturbance model training module, configured to respectively extract features from the sample noise reduction image and the sample original image, and train the anti-disturbance model according to the feature extraction results.

[0016] According to another aspect of the present disclosure, there is provided an image noise reduction device, including:

[0017] An anti-disturbance image acquisition module, configured to input a to-be-processed image into a trained anti-disturbance model to obtain a target anti-disturbance image;

[0018] A noise reduction image acquisition module, configured to use the target anti-disturbance image to reduce the noise of the to-be-processed image to obtain a target noise reduction image.

[0019] According to another aspect of the present disclosure, there is provided an electronic device, the electronic device includes:

[0020] At least one processor; and

[0021] A memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training of the image model or the image noise reduction method provided in any embodiment of the present disclosure.

[0022] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the training method of the image model or the image noise reduction method provided in any embodiment of the present disclosure.

[0023] According to another aspect of the present disclosure, there is provided a computer program product, including a computer program, where the computer program, when executed by a processor, implements the training method of the image model or the image noise reduction method provided in any embodiment of the present disclosure.

[0024] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0025] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0026] Figure 1Schematic diagram of a method for training an image model according to an embodiment of the present disclosure;

[0027] Figure 2A Schematic diagram of a method for training an image model according to another embodiment of the present disclosure;

[0028] Figure 2B Schematic diagram of an anti-disturbance unit structure according to another embodiment of the present disclosure;

[0029] Figure 3A Schematic diagram of an image denoising method according to another embodiment of the present disclosure;

[0030] Figure 3B Schematic diagram of the detection of adversarial samples according to another embodiment of the present disclosure;

[0031] Figure 3C Schematic diagram of the detection of adversarial samples after denoising according to another embodiment of the present disclosure;

[0032] Figure 3D Schematic diagram of the anti-disturbance of adversarial samples according to another embodiment of the present disclosure;

[0033] Figure 4 Schematic diagram of an image denoising method according to another embodiment of the present disclosure;

[0034] Figure 5 Schematic diagram of a device for training an image model according to another embodiment of the present disclosure;

[0035] Figure 6 Schematic diagram of an image denoising device according to another embodiment of the present disclosure;

[0036] Figure 7 Block diagram of an electronic device for implementing the embodiments of the present disclosure. Detailed implementation manners

[0037] The following makes an explanation of the exemplary embodiments of the present disclosure in conjunction with the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted below.

[0038] In the related art, there are other methods for detecting adversarial samples. For example, conventional image processing means can be used, such as Joint Photographic Experts Group (JPEG) compression, or other filtering algorithms to perform noise reduction on the input image. Or perform noise reduction on the input image by multiple means at the same time, and judge whether the input image is an adversarial sample by analyzing the difference between the input image after noise reduction and the original image. However, in practical applications, these conventional image processing means have unsatisfactory detection effects on adversarial samples, and existing other detectors can only judge whether the input sample is an adversarial sample. If it is judged to be true, an alarm will be given, and then the input image will be processed manually. However, in the specific application scenarios of the target detection model, sometimes there are no conditions for manual takeover at any time. Therefore, simply judging whether the input image is an adversarial sample is not enough, but the model needs to be able to give a relatively accurate output result even when the input is an adversarial sample.

[0039] In the related art, it is also possible to perform preprocessing such as transformation and compression on adversarial samples to dilute the influence of malicious perturbation noise in the adversarial samples on the model recognition result. However, the conventional preprocessing means are not designed specifically for adversarial samples at the beginning, so the defense effect on adversarial samples is very limited, or the influence on non-malicious samples is also relatively large. It is also possible to introduce adversarial samples during the training of the model to increase the robustness of the model itself against malicious attacks. However, this defense method requires changing the parameters of the model itself and cannot be applied to the trained model.

[0040] Figure 1 FIG. is a flowchart of a method for training an image model according to an embodiment of the present disclosure. This embodiment is applicable to the situation of training an anti-perturbation model using adversarial samples. This method can be executed by a training device of the image model, and this device can be implemented in a software and / or hardware manner. This device can be configured in an electronic device with corresponding data processing capabilities. The method specifically includes:

[0041] S110: Input the adversarial sample into the anti-perturbation model to obtain a sample anti-perturbation image, and use the sample anti-perturbation image to perform noise reduction on the adversarial sample to obtain a sample noise reduction image;

[0042] S120: Extract features from the sample noise reduction image and the sample original image respectively, and train the anti-perturbation model according to the feature extraction results.

[0043] Among them, the training sample pair of the anti-disturbance model includes the original sample image and the adversarial sample of the original sample image, and the adversarial sample is obtained by injecting malicious disturbances into the original sample image. The sample anti-disturbance image contains anti-disturbances opposite to the characteristics of the malicious disturbances, and the sample denoised image is the denoised image obtained by canceling the malicious disturbances in the adversarial sample using the anti-disturbances in the anti-disturbance image.

[0044] Specifically, the adversarial sample injected with malicious disturbances is input into the anti-disturbance model. The anti-disturbance unit in the anti-disturbance model learns and amplifies the malicious disturbances to generate anti-disturbances opposite to the malicious disturbances, and the anti-disturbance model generates and outputs the sample anti-disturbance image according to the anti-disturbances. The sample denoised image and the adversarial sample are subjected to image fusion so that the opposite malicious disturbances and anti-disturbances cancel each other out, thereby achieving the elimination of the malicious disturbances in the adversarial sample and obtaining the sample denoised image.

[0045] In this embodiment, the task model can be used to extract features from the denoised sample denoised image and the original sample image respectively, and construct the loss function of the anti-disturbance model according to the feature extraction results of the two; moreover, the gradient of the parameters in the anti-disturbance model with respect to the loss function is calculated in reverse, and the parameters to be trained in the anti-disturbance model are updated according to the gradient. The task model is different from the anti-disturbance model. The task model is used to perform a preset task on the denoised image, such as performing object detection, image classification and other tasks. The task model extracts features from the sample denoised image and the original sample image respectively to obtain the sample denoised features and the original sample features as the feature extraction results of the two. The task model can perform non-linear amplification processing on the sample denoised image and the original sample image, that is, the feature difference between the sample denoised features and the original sample features is the non-linear amplification result of the image difference between the sample denoised image and the original sample image. Therefore, constructing the loss function using the feature difference between the sample denoised features and the original sample features can more accurately learn the cancellation result of the anti-disturbance to the malicious disturbance compared with directly constructing the loss function using the image difference between the sample denoised image and the original sample image; in addition, it can also improve the matching degree between the anti-disturbance model and the task model, thereby improving the accuracy of subsequent task processing on the denoised image.

[0046] The embodiment of the present disclosure generates an anti-disturbance image through the anti-disturbance model to cancel the artificially added malicious noise in the adversarial sample, and trains the anti-disturbance model according to the denoising result and the feature extraction result of the original image, effectively improving the training effect of the anti-disturbance model and ensuring the defense ability of the anti-disturbance model against adversarial samples without manual supervision.

[0047] In an alternative embodiment, the anti-disturbance model includes at least two anti-disturbance units; each anti-disturbance unit is respectively associated with different feature extraction units in the object detection model, and is used to train the anti-disturbance unit using the features extracted by the associated feature extraction unit.

[0048] Among them, the anti-disturbance unit is used to extract features of malicious disturbances and generate anti-disturbances opposite to the malicious disturbances. There is a feature extraction unit in the target detection model, which is used to extract features of the sample denoised image and the sample original image, and judge the cancellation result of the malicious disturbance and the anti-disturbance in the sample denoised image based on the feature extraction result. Each feature extraction unit in the target detection model is associated with a certain anti-disturbance unit in the anti-disturbance model, and the two are respectively used to extract features of the same type of malicious disturbance and generate anti-disturbances.

[0049] Specifically, malicious disturbances may exist in multiple dimensions and angles. A single anti-disturbance unit may only have a good recognition and anti-disturbance generation effect on a certain type of malicious disturbance, but cannot meet the processing requirements for various types of malicious disturbances. At least two anti-disturbance units are deployed in the anti-disturbance model, which are respectively used as noise reducers for different types of malicious disturbances, so as to realize the effective processing of various types of malicious disturbances by the anti-disturbance model. At the same time, as the target detection model of the task model, there is a feature extraction unit associated with each anti-disturbance unit. The feature extraction unit can extract the malicious disturbance features and anti-disturbance features corresponding to the associated anti-disturbance unit, so as to determine the anti-disturbance generation effect of the associated anti-disturbance unit based on the malicious disturbance features and / or anti-disturbance features extracted from the sample denoised image and the sample original image, and feedback training to the anti-disturbance unit based on the anti-disturbance generation effect. By deploying multiple anti-disturbance units in the anti-disturbance model and associating each anti-disturbance unit with the feature extraction unit in the target detection model, the anti-disturbance generation effect of the anti-disturbance model on various types of malicious disturbances is improved, and the applicability and fit of the anti-disturbance model and the target detection model are ensured.

[0050] Figure 2A It is a schematic diagram of a method for training an image model according to another embodiment of the present disclosure. This embodiment is optimized based on the above embodiments. Refer to Figure 2A and the method includes:

[0051] S210. Input the adversarial sample into at least two anti-disturbance units in the anti-disturbance model to obtain at least two sample anti-disturbed images, and use the at least two sample anti-disturbed images to denoise the adversarial sample to obtain at least two sample denoised images.

[0052] S221. Input the i-th sample denoised image and the sample original image into the target detection model respectively to obtain the i-th sample denoised feature and the i-th sample original feature output by the j-th feature extraction unit in the target detection model. S222. Use the i-th sample denoised feature and the i-th sample original feature to train the i-th anti-disturbance unit;

[0053] Among them, the i-th sample denoised image is obtained by denoising the adversarial sample using the i-th sample anti-disturbance image; i and j are natural numbers, and the i-th anti-disturbance unit is associated with the j-th feature extraction unit; the i-th sample anti-disturbance image is obtained by the i-th anti-disturbance unit in the anti-disturbance model;

[0054] Among them, the sample original feature characterization image represents the original image features when no malicious disturbance is injected, and the sample denoised feature represents the image features after the malicious disturbance in the image is canceled by the anti-disturbance.

[0055] Specifically, according to the j-th feature extraction unit associated with the i-th anti-disturbance unit i that generates the anti-disturbance, the i-th sample denoised image i and the sample original image i are respectively input into the target detection model, and the feature extraction unit j in the target detection model is used to extract the sample denoised feature i and the sample original feature i of the i-th sample denoised image i and the sample original image i. According to the comparison result of the sample original feature i and the sample denoised feature i, the parameters to be trained in the i-th anti-disturbance unit i are updated. Obtaining the feature extraction results of the sample original feature and the sample denoised feature through the associated feature extraction unit and updating the parameters to be trained of the associated anti-disturbance unit based on the feature extraction results improve the targeted training effect of each anti-disturbance unit.

[0056] Exemplarily, in the anti-disturbance model, there are anti-disturbance units A1 and A2, and in the target detection model, there are three feature extraction units B1, B2, and B3. And when the feature extraction unit B1 is associated with the anti-disturbance unit A1 and the feature extraction unit B2 is associated with the anti-disturbance unit A2, the loss function of the anti-disturbance unit A1 is constructed through the features extracted by the feature extraction unit B1, and the loss function of the anti-disturbance unit A2 is constructed through the features extracted by the feature extraction unit B2. Since each feature extraction unit can perform feature extraction on the sample denoised image and the sample original image to different degrees, that is, non-linearly amplify the image to different degrees, each anti-disturbance unit is used to identify different scales and different types of disturbances, thereby improving the training effect of each anti-disturbance unit.

[0057] S231. Through the fusion model, fuse the sample anti-disturbance images generated by at least two anti-disturbance units in the anti-disturbance model to obtain a sample fusion image;

[0058] S232. Use the sample fusion image to denoise the adversarial sample to obtain a sample denoising result;

[0059] S233. Input the sample denoising result and the sample original image into the target detection model respectively to obtain a sample denoising detection value and a sample original detection value;

[0060] S234. Train the fusion model according to the sample noise reduction detection value and the sample original detection value.

[0061] Among them, the network structure of the fusion model includes an anti-disturbance fusion layer (which can be a fully connected layer, for example) for fusing the anti-disturbance images generated by each anti-disturbance unit, so as to generate a sample fusion image containing multiple anti-disturbances.

[0062] Specifically, in addition to training the anti-disturbance units, the fusion model for fusing multiple sample anti-disturbance images also needs to be trained to improve the fusion ability of the fusion model for multiple sample anti-disturbance images. Use the sample fusion image obtained by fusing at least two sample anti-disturbance images by the fusion model to cancel and reduce the noise of the malicious disturbance in the adversarial sample to obtain a sample noise reduction result. Input the sample noise reduction result and the sample original image into the target detection model respectively to obtain the sample noise reduction detection value and the sample original detection value, and use the gap between the sample noise reduction detection value and the sample original detection value as the loss function in the training process of the fusion model, and update the parameters to be trained in the fusion layer based on the loss function in the training process of the fusion model. By obtaining the sample noise reduction detection value and the sample original detection value to train the fusion model, the loss function in the training process of the fusion model is effectively reduced, and the fusion effect of the sample anti-disturbance image of the fusion model is improved.

[0063] Optionally, the anti-disturbance unit includes at least two feature extraction layers and at least two reverse disturbance layers; among them, the input of the k-th reverse disturbance layer is the output of the k-th feature extraction layer and the output of the (k + 1)-th reverse disturbance layer; k is a natural number.

[0064] Specifically, the reverse perturbation layer can be connected to the feature extraction layer in a U-shaped tube structure, and is used to generate an opposite anti-perturbation according to the malicious perturbation features extracted by the feature extraction layer. The feature extraction layer, as the upper path for extracting features, is composed of five basic convolutional modules. Each basic convolutional module is composed of two batch normalization convolutional layers, and each batch normalization convolutional layer can be composed of a convolutional layer, a normalization layer, and an activation layer. The reverse perturbation layer, as the lower path for generating reverse noise based on the extracted features, is composed of four similar basic convolutional modules. However, a downsampling layer using bilinear interpolation is added before each module. Finally, after passing through the lower path, the reverse perturbation layer outputs an anti-perturbation, which will be used to cancel the malicious perturbation noise on the adversarial sample. In addition, at least two groups of reverse perturbation layers and feature extraction layers are arranged in the anti-perturbation unit, which are respectively used to extract image features of different depth levels and generate corresponding anti-perturbations. When generating the anti-perturbation of the next depth level, in addition to referring to the image features of the current depth level, the anti-perturbation generated at the previous depth level also needs to be referred to, that is, the input of the k-th reverse perturbation layer is the output of the k-th feature extraction layer and the output of the k + 1-th reverse perturbation layer. By combining the anti-perturbation of the previous level and the image features of the current level to generate the anti-perturbation, the effective coverage of the anti-perturbation on the malicious perturbation in the adversarial sample is realized, and the cancellation effect of the anti-perturbation on the malicious perturbation is improved.

[0065] Exemplarily, Figure 2B FIG. is a schematic structural diagram of an anti-perturbation unit provided by an embodiment of the present disclosure. Among them, X1 on the figure represents the input adversarial sample (perturbed image). The malicious noise is extracted through a series of feature extraction layers in the anti-perturbation unit, and the reverse perturbation layer generates an anti-perturbation. The final output of the anti-perturbation unit is -X1, which represents the reverse noise - reverse perturbation learned by the model. Then the reverse perturbation will be added to the input adversarial sample to obtain the denoised input image, marked as X2 in the figure. X represents the original image without adding malicious perturbation. After denoising, X2 and X will be simultaneously input into the target detection model. We use the difference between the outputs of X2 and X after being predicted by the target detection model as the loss function during the training process of the denoiser. It should be noted that the purpose of showing the (K + N)-th (N is a natural number greater than 1) group of feature reverse perturbation layers and feature extraction layers in the figure is only to illustrate that the anti-perturbation unit in the present disclosure may not be limited to two groups of reverse perturbation layers and feature extraction layers. If there are more groups of reverse perturbation layers and feature extraction layers, the same structural connection method can be used.

[0066] In the embodiments of the present disclosure, the feature extraction results of the sample original features and the sample noise reduction features are obtained through the associated feature extraction unit, and the training parameters to be trained of the associated anti-disturbance unit are updated based on the feature extraction results, improving the targeted training effect of each anti-disturbance unit; by obtaining the sample noise reduction detection value and the sample original detection value to train the fusion model, the loss function in the training process of the fusion model is effectively reduced, and the sample anti-disturbance image fusion effect of the fusion model is improved.

[0067] FIG. 3 is a flowchart of an image noise reduction method according to another embodiment of the present disclosure. This embodiment is applicable to the situation of using an anti-disturbance model to reduce noise in an image. This method can be executed by an image noise reduction device, and the device can be implemented in a software and / or hardware manner. The device can be configured in an electronic device with corresponding data processing capabilities. The method specifically includes:

[0068] S310: Input the image to be processed into the trained anti-disturbance model to obtain the target anti-disturbance image;

[0069] S320: Use the target anti-disturbance image to reduce the noise of the image to be processed to obtain the target noise reduction image.

[0070] Specifically, the image to be processed of the task model is first input into the anti-disturbance model. The anti-disturbance model can be trained by using the training method of the image model in the above-mentioned disclosed embodiments. After obtaining the anti-disturbance image output by the anti-disturbance model, the anti-disturbance image is fused with the image to be processed to realize the noise reduction of the image to be processed, and then the target noise reduction image is obtained.

[0071] In the embodiments of the present disclosure, the anti-disturbance model generates an anti-disturbance image to offset the malicious noise in the image to be processed, improving the defense ability against adversarial samples without manual supervision, without reconstructing or training the target detection model, and reducing the workload.

[0072] Exemplarily, Figure 3B is a schematic diagram for detecting an adversarial sample, Figure 3C is a schematic diagram of the adversarial sample after noise reduction, Figure 3D is a schematic diagram of the anti-disturbance of the adversarial sample. Refer to Figures 3B - 3D , using anti-disturbance (as shown in Figure 3D ) to reduce the noise of the adversarial sample (as shown in Figure 3B ) can obtain the sample noise reduction image (as shown in Figure 3C)。Since the adversarial sample is injected with malicious perturbations, the object detection result of the adversarial sample misdetects a cow as a horse. However, through the noise reduction method provided by the present disclosure, after the malicious perturbations in the adversarial sample are cancelled by the anti-perturbations, the denoised image is input into the object detection model, and the object detection model can correctly identify the cow. It can be seen that the anti-perturbations generated by the anti-perturbation model can eliminate the malicious perturbation noise added to the adversarial sample, so that the denoised adversarial sample can be normally processed by the object detection model, which not only ensures that the adversarial sample will not be misidentified by the object detection model, but also does not discard the adversarial sample.

[0073] Figure 4 is a schematic diagram of an image noise reduction method provided according to another embodiment of the present disclosure. This embodiment is optimized on the basis of the above embodiments. Refer to Figure 4 , the method includes:

[0074] S410. Input the image to be processed into at least two anti-perturbation units in the anti-perturbation model respectively to obtain at least two target anti-perturbation images.

[0075] Specifically, when there are multiple malicious perturbation noises in the image to be processed, the anti-perturbation model generates corresponding anti-perturbations for each malicious perturbation noise, so as to obtain multiple target anti-perturbation images.

[0076] S421. Through the trained fusion model, fuse the at least two target anti-perturbation images to obtain a target fusion image.

[0077] S422. Use the target fusion image to perform noise reduction on the image to be processed to obtain the target denoised image.

[0078] Specifically, the target fusion image is obtained by fusing multiple target anti-perturbation images through the fusion model, and then each malicious perturbation noise in the image to be processed is cancelled based on the target fusion image, without cancelling and reducing the noise of each malicious perturbation by each of the multiple target anti-perturbation images one by one, which improves the noise reduction efficiency of the image to be processed in the case of complex malicious perturbations.

[0079] In the embodiment of the present disclosure, the image to be processed is denoised by using the target fusion image based on multiple target anti-perturbation images, which improves the noise reduction efficiency of the image to be processed in the case of complex malicious perturbations.

[0080] Figure 5 is a schematic diagram of a training device for an image model provided according to another embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the situation of training an anti-perturbation model by using adversarial samples. The device is configured in an electronic device with corresponding data processing capabilities and can implement the training method of the image model described in any embodiment of the present disclosure.

[0081] The sample noise reduction module 510 is configured to input the adversarial sample into the anti-disturbance model to obtain a sample anti-disturbance image, and use the sample anti-disturbance image to reduce the noise of the adversarial sample to obtain a sample noise reduction image;

[0082] The anti-disturbance model training module 520 is configured to respectively extract features from the sample noise reduction image and the sample original image, and train the anti-disturbance model according to the feature extraction results.

[0083] The above device and module can execute the training method of the image model provided in any embodiment of the present disclosure, and have the corresponding functional modules and beneficial effects of the execution method.

[0084] Optionally, the anti-disturbance model includes at least two anti-disturbance units; each anti-disturbance unit is respectively associated with a different feature extraction unit in the target detection model, and is configured to train the anti-disturbance unit by using the features extracted by the associated feature extraction unit.

[0085] Optionally, the anti-disturbance model training module 520 includes:

[0086] A feature acquisition unit, configured to respectively input the i-th sample noise reduction image and the sample original image into the target detection model to obtain the i-th sample noise reduction feature and the i-th sample original feature output by the j-th feature extraction unit in the target detection model; wherein, the i-th sample noise reduction image is obtained by reducing the noise of the adversarial sample by using the i-th sample anti-disturbance image; the i-th sample anti-disturbance image is obtained by the i-th anti-disturbance unit in the anti-disturbance model; an anti-disturbance training unit, configured to train the i-th anti-disturbance unit by using the i-th sample noise reduction feature and the i-th sample original feature; wherein, i and j are natural numbers, and the i-th anti-disturbance unit is associated with the j-th feature extraction unit.

[0087] Optionally, the device further includes:

[0088] An image fusion module, configured to fuse the sample anti-disturbance images generated by at least two anti-disturbance units in the anti-disturbance model through a fusion model to obtain a sample fusion image;

[0089] A noise reduction result acquisition module, configured to reduce the noise of the adversarial sample by using the sample fusion image to obtain a sample noise reduction result;

[0090] A detection value acquisition module, configured to respectively input the sample noise reduction result and the sample original image into the target detection model to obtain a sample noise reduction detection value and a sample original detection value;

[0091] A fusion model training module, configured to train the fusion model according to the sample noise reduction detection value and the sample original detection value.

[0092] Optionally, the anti-disturbance unit includes at least two feature extraction layers and at least two reverse disturbance layers; wherein, the input of the k-th reverse disturbance layer is the output of the k-th feature extraction layer and the output of the (k + 1)-th reverse disturbance layer; k is a natural number.

[0093] The above-mentioned device, module, and unit described in further detail can execute the training method of the image model provided in any embodiment of the present disclosure, and have the corresponding functional modules and beneficial effects for executing the method.

[0094] Figure 6 It is a schematic diagram of an image denoising device provided according to another embodiment of the present disclosure. The embodiments of the present disclosure are applicable to the case of image denoising using an anti-disturbance model. This device is configured in an electronic device with corresponding data processing capabilities and can implement the image denoising method described in any embodiment of the present disclosure.

[0095] An anti-disturbance image acquisition module 610, configured to input a to-be-processed image into a trained anti-disturbance model to obtain a target anti-disturbance image;

[0096] A denoised image acquisition module 620, configured to perform denoising on the to-be-processed image using the target anti-disturbance image to obtain a target denoised image.

[0097] The above-mentioned device and module can execute the image denoising method provided in any embodiment of the present disclosure, and have the corresponding functional modules and beneficial effects for executing the method.

[0098] Optionally, the anti-disturbance image acquisition module 610 includes:

[0099] An anti-disturbance image acquisition unit, configured to input the to-be-processed image into at least two anti-disturbance units in the anti-disturbance model respectively to obtain at least two target anti-disturbance images;

[0100] The denoised image acquisition module 620 includes:

[0101] An anti-disturbance image fusion unit, configured to fuse the at least two target anti-disturbance images through a trained fusion model to obtain a target fusion image;

[0102] A denoised image acquisition unit, configured to perform denoising on the to-be-processed image using the target fusion image to obtain the target denoised image.

[0103] The above-mentioned device, module, and unit described in further detail can execute the image denoising method provided in any embodiment of the present disclosure, and have the corresponding functional modules and beneficial effects for executing the method.

[0104] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0105] Figure 7 FIG. shows a schematic block diagram of an exemplary electronic device 700 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0106] As Figure 7 shown, the device 700 includes a computing unit 701 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0107] A plurality of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0108] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 executes the various methods and processes described above, such as the training of an image model or an image noise reduction method. For example, in some embodiments, the training of an image model or an image noise reduction method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the image model training or image noise reduction method described above can be executed. Alternatively, in other embodiments, the computing unit 701 can be configured to execute the image model training or image noise reduction method in any other suitable manner (e.g., by means of firmware).

[0109] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0110] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program codes can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0111] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0112] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0113] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0114] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is created by computer programs that run on the respective computers and have a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0115] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and no limitations are imposed herein.

[0116] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A training method for an image model, comprising: Inputting the adversarial sample into the anti-perturbation model to obtain a sample anti-perturbed image, and using the sample anti-perturbed image to denoise the adversarial sample to obtain a sample denoised image; The sample anti-perturbed image contains an anti-perturbation opposite to the malicious perturbation feature in the adversarial sample, and the sample denoised image is a denoised image obtained by canceling the malicious perturbation in the adversarial sample using the anti-perturbation in the anti-perturbed image; Feature extraction is respectively performed on the sample denoised image and the sample original image, and the anti-perturbation model is trained according to the feature extraction results; Among them, training the anti-perturbation model according to the feature extraction results includes: Constructing a loss function of the anti-perturbation model according to the feature extraction results; calculating the gradient of the parameters in the anti-perturbation model with respect to the loss function in reverse, and updating the parameters to be trained in the anti-perturbation model according to the gradient.

2. The method according to claim 1, wherein, The anti-perturbation model includes at least two anti-perturbation units; each anti-perturbation unit is respectively associated with different feature extraction units in the target detection model, and is used to train the anti-perturbation unit using the features extracted by the associated feature extraction unit.

3. The method according to claim 2, wherein, The step of respectively performing feature extraction on the sample denoised image and the sample original image, and training the anti-perturbation model according to the feature extraction results, further includes: Inputting the i-th sample denoised image and the sample original image into the target detection model respectively to obtain the i-th sample denoised feature and the i-th sample original feature output by the j-th feature extraction unit in the target detection model; wherein, the i-th sample denoised image is obtained by denoising the adversarial sample using the i-th sample anti-perturbed image; the i-th sample anti-perturbed image is obtained by the i-th anti-perturbation unit in the anti-perturbation model; Training the i-th anti-perturbation unit using the i-th sample denoised feature and the i-th sample original feature; wherein, i and j are natural numbers, and the i-th anti-perturbation unit is associated with the j-th feature extraction unit.

4. The method according to claim 2, further comprising: Fusing the sample anti-perturbed images generated by at least two anti-perturbation units in the anti-perturbation model through a fusion model to obtain a sample fused image; Using the sample fused image to denoise the adversarial sample to obtain a sample denoising result; Inputting the sample denoising result and the sample original image into the target detection model respectively to obtain a sample denoising detection value and a sample original detection value; Training the fusion model according to the sample denoising detection value and the sample original detection value.

5. The method according to claim 4, wherein, The anti-perturbation unit includes at least two feature extraction layers and at least two reverse perturbation layers; wherein, the input of the k-th reverse perturbation layer is the output of the k-th feature extraction layer and the output of the k + 1-th reverse perturbation layer; k is a natural number.

6. An image denoising method, comprising: Inputting the image to be processed into the trained anti-perturbation model to obtain a target anti-perturbed image; Using the target anti-disturbance image to perform noise reduction on the image to be processed to obtain a target noise-reduced image; Wherein, the anti-disturbance model is trained by the training method of the image model according to any one of claims 1-5.

7. The method according to claim 6, Wherein, The step of inputting the image to be processed into the trained anti-disturbance model to obtain a target anti-disturbance image includes: Inputting the image to be processed into at least two anti-disturbance units in the anti-disturbance model respectively to obtain at least two target anti-disturbance images; The step of using the target anti-disturbance image to perform noise reduction on the image to be processed to obtain a target noise-reduced image includes: Fusing the at least two target anti-disturbance images through a trained fusion model to obtain a target fused image; Using the target fused image to perform noise reduction on the image to be processed to obtain the target noise-reduced image.

8. An apparatus for training an image model, Comprising: A sample noise reduction module, configured to input an adversarial sample into an anti-disturbance model to obtain a sample anti-disturbance image, and use the sample anti-disturbance image to perform noise reduction on the adversarial sample to obtain a sample noise-reduced image; the sample anti-disturbance image contains an anti-disturbance opposite to the malicious disturbance feature in the adversarial sample, and the sample noise-reduced image is a noise-reduced image obtained by canceling the malicious disturbance in the adversarial sample using the anti-disturbance in the anti-disturbance image; An anti-disturbance model training module, configured to respectively extract features from the sample noise-reduced image and the sample original image, and train the anti-disturbance model according to the feature extraction results; Wherein, the anti-disturbance model training module is specifically configured to: construct a loss function of the anti-disturbance model according to the feature extraction results; calculate the gradient of the parameters in the anti-disturbance model with respect to the loss function in the reverse direction, and update the parameters to be trained in the anti-disturbance model according to the gradient.

9. The apparatus according to claim 8, Wherein, The anti-disturbance model includes at least two anti-disturbance units; each anti-disturbance unit is respectively associated with a different feature extraction unit in the target detection model, and is configured to train the anti-disturbance unit using the features extracted by the associated feature extraction unit.

10. The apparatus according to claim 9, Wherein, The anti-disturbance model training module includes: A feature acquisition unit, configured to input the i-th sample noise-reduced image and the sample original image into the target detection model respectively to obtain the i-th sample noise-reduced feature and the i-th sample original feature output by the j-th feature extraction unit in the target detection model; wherein, the i-th sample noise-reduced image is obtained by performing noise reduction on the adversarial sample using the i-th sample anti-disturbance image; the i-th sample anti-disturbance image is obtained by the i-th anti-disturbance unit in the anti-disturbance model; An anti-disturbance training unit, configured to train the i-th anti-disturbance unit using the i-th sample noise-reduced feature and the i-th sample original feature; Wherein, i and j are natural numbers, and the i-th anti-disturbance unit is associated with the j-th feature extraction unit.

11. The apparatus according to claim 9, Wherein, The apparatus further includes: An image fusion module, configured to fuse the sample anti-disturbance images generated by at least two anti-disturbance units in the anti-disturbance model through a fusion model to obtain a sample fused image; A noise reduction result acquisition module, configured to perform noise reduction on the adversarial sample by using the sample fused image to obtain a sample noise reduction result; A detection value acquisition module, configured to respectively input the sample noise reduction result and the sample original image into the target detection model to obtain a sample noise reduction detection value and a sample original detection value; A fusion model training module, configured to train the fusion model according to the sample noise reduction detection value and the sample original detection value.

12. The apparatus according to claim 11, wherein, the anti-disturbance unit includes at least two feature extraction layers and at least two reverse perturbation layers; wherein, the input of the k-th reverse perturbation layer is the output of the k-th feature extraction layer and the output of the (k + 1)-th reverse perturbation layer; k is a natural number.

13. An image noise reduction apparatus, comprising: An anti-disturbance image acquisition module, configured to input a to-be-processed image into a trained anti-disturbance model to obtain a target anti-disturbance image; A noise reduction image acquisition module, configured to perform noise reduction on the to-be-processed image by using the target anti-disturbance image to obtain a target noise reduction image; wherein, the anti-disturbance model is trained by the image model training apparatus according to claim 12.

14. The apparatus according to claim 13, wherein, the anti-disturbance image acquisition module includes: An anti-disturbance image acquisition unit, configured to respectively input the to-be-processed image into at least two anti-disturbance units in the anti-disturbance model to obtain at least two target anti-disturbance images; the noise reduction image acquisition module includes: An anti-disturbance image fusion unit, configured to fuse the at least two target anti-disturbance images through a trained fusion model to obtain a target fused image; A noise reduction image acquisition unit, configured to perform noise reduction on the to-be-processed image by using the target fused image to obtain the target noise reduction image.

15. An electronic device, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1-7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, the computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

17. A computer program product, including a computer program, where the computer program implements the method according to any one of claims 1-7 when executed by a processor.

Images