Image processing method and device, computer device, storage medium and program product

Abstract

The application relates to an image processing method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring an adversarial image, determining the attention mask of a feature point in the adversarial image; based on the attention mask of the feature point, a first region and a second region in the adversarial image are identified, the attention mask of the feature point in the first region is greater than the attention mask of the feature point in the second region; in the adversarial image, first noise is added to the first region and second noise is added to the second region to obtain a target noise-added image; the intensity of the first noise is greater than the intensity of the second noise; the first region of the target noise-added image is denoised to the second noise to obtain a target denoised image; the target denoised image is denoised to obtain a purified image corresponding to the adversarial image. The method can eliminate adversarial disturbances in the image while retaining more image semantic information.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of computer technology, image denoising technology has emerged, which can restore the original clear image from an image contaminated by noise.

[0003] Traditional image denoising employs a forward processing method, which involves adding isotropic Gaussian noise to the input image to mask adversarial perturbations, and then recovering the clean image through a diffusion model in the reverse process. However, the higher the intensity of the added noise, the more semantic information is lost during the subsequent denoising process, while low-intensity noise may not achieve satisfactory image denoising results.

[0004] Therefore, traditional image denoising methods struggle to achieve a good balance between eliminating adversarial perturbations and preserving semantic information. Summary of the Invention

[0005] Therefore, it is necessary to provide an image processing method, apparatus, computer device, computer-readable storage medium, and computer program product that can both eliminate adversarial perturbations in images and retain more semantic information of the images, in order to address the above-mentioned technical problems.

[0006] This application provides an image processing method. The method includes:

[0007] Acquire an adversarial image and determine the attention mask for feature points in the adversarial image;

[0008] Based on the attention mask of the feature points, a first region and a second region in the adversarial image are identified, wherein the attention mask of the feature points in the first region is greater than the attention mask of the feature points in the second region;

[0009] In the adversarial image, a first noise is added to the first region and a second noise is added to the second region to obtain a target noisy image; the intensity of the first noise is greater than the intensity of the second noise.

[0010] The first region of the target noisy image is reduced to the second noise to obtain the target noisy image;

[0011] The target image is denoised to obtain a cleaned image corresponding to the adversarial image.

[0012] On the other hand, this application also provides an image processing apparatus. The apparatus includes:

[0013] The acquisition module is used to acquire an adversarial image and determine the attention mask of feature points in the adversarial image;

[0014] The recognition module is used to identify a first region and a second region in the adversarial image based on the attention mask of the feature points, wherein the attention mask of the feature points in the first region is greater than the attention mask of the feature points in the second region.

[0015] A noise-adding module is used to add a first noise to the first region and a second noise to the second region in the adversarial image to obtain a target noise-adding image; the intensity of the first noise is greater than the intensity of the second noise.

[0016] The first noise reduction module is used to reduce the noise of the first region of the target noisy image to the second noise, thereby obtaining the target noisy image;

[0017] The second noise reduction module is used to denoise the target denoised image to obtain a cleaned image corresponding to the adversarial image.

[0018] On the other hand, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0019] Acquire an adversarial image and determine the attention mask for feature points in the adversarial image;

[0020] Based on the attention mask of the feature points, a first region and a second region in the adversarial image are identified, wherein the attention mask of the feature points in the first region is greater than the attention mask of the feature points in the second region;

[0021] In the adversarial image, a first noise is added to the first region and a second noise is added to the second region to obtain a target noisy image; the intensity of the first noise is greater than the intensity of the second noise.

[0022] The first region of the target noisy image is reduced to the second noise to obtain the target noisy image;

[0023] The target image is denoised to obtain a cleaned image corresponding to the adversarial image.

[0024] On the other hand, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0025] Acquire an adversarial image and determine the attention mask for feature points in the adversarial image;

[0026] Based on the attention mask of the feature points, a first region and a second region in the adversarial image are identified, wherein the attention mask of the feature points in the first region is greater than the attention mask of the feature points in the second region;

[0027] In the adversarial image, a first noise is added to the first region and a second noise is added to the second region to obtain a target noisy image; the intensity of the first noise is greater than the intensity of the second noise.

[0028] The first region of the target noisy image is reduced to the second noise to obtain the target noisy image;

[0029] The target image is denoised to obtain a cleaned image corresponding to the adversarial image.

[0030] On the other hand, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0031] Acquire an adversarial image and determine the attention mask for feature points in the adversarial image;

[0032] Based on the attention mask of the feature points, a first region and a second region in the adversarial image are identified, wherein the attention mask of the feature points in the first region is greater than the attention mask of the feature points in the second region;

[0033] In the adversarial image, a first noise is added to the first region and a second noise is added to the second region to obtain a target noisy image; the intensity of the first noise is greater than the intensity of the second noise.

[0034] The first region of the target noisy image is reduced to the second noise to obtain the target noisy image;

[0035] The target image is denoised to obtain a cleaned image corresponding to the adversarial image.

[0036] The aforementioned image processing method, apparatus, computer equipment, storage medium, and computer program product, by acquiring an adversarial image and determining the attention mask of feature points in the adversarial image, can quantify key regions in the adversarial image through the attention mask. Based on the attention mask of feature points, a first region and a second region in the adversarial image are identified to determine the attention level of the first region and the second region. The attention mask of feature points in the first region is greater than that in the second region, indicating that the attention level of the first region is higher than that of the second region. In the adversarial image, a first noise is added to the first region and a second noise is added to the second region to obtain a target noisy image, thereby enabling the addition of different noises according to the respective attention levels of different regions. Furthermore, the intensity of the first noise is greater than the intensity of the second noise; the greater the added noise, the more image semantics are lost, and the lower the added noise, the less image semantics are lost. This indicates that adding high-intensity noise to regions with high attention and adding low-intensity noise to regions with low attention can preserve more image semantic information in the second region. Since greater denoising intensity eliminates more semantic information, reducing the noise in the first region of the noisy target image to the second noise level first reduces the high-intensity noise in the first region to the low-intensity noise in the second region, thus eliminating some adversarial perturbations in the first region and obtaining a denoised target image with low-intensity noise. Global denoising is then performed on the denoised target image to further denoise the first region, resulting in a clearer first region. This also allows for denoising of the low-intensity second region, making it clearer than before without losing too much semantic information. Ultimately, the cleaned image achieves a balance between eliminating adversarial perturbations and preserving semantic information, with the first region having high clarity and the second region retaining more semantic information. Attached Figure Description

[0037] Figure 1 This is an application environment diagram of an image processing method in one embodiment;

[0038] Figure 2 This is a flowchart illustrating an image processing method in one embodiment;

[0039] Figure 3 This is a schematic diagram of an image processing method in one embodiment;

[0040] Figure 4 This is a schematic diagram of adversarial image, target noise-added image, target noise-reduced image and cleaned image in the image processing process of another embodiment;

[0041] Figure 5 This is a schematic diagram of obtaining an attention mask in one embodiment;

[0042] Figure 6This is a schematic diagram illustrating the acquisition of a target denoised image in one embodiment;

[0043] Figure 7 This is a schematic diagram illustrating the determination of a first noise and a second noise in one embodiment;

[0044] Figure 8 This is a schematic diagram illustrating the acquisition of a target denoised image in another embodiment;

[0045] Figure 9 This is an architecture diagram of an image processing method in one embodiment;

[0046] Figure 10 This is a schematic diagram of the algorithm flow of an image processing method in one embodiment;

[0047] Figure 11 This is an architecture diagram of an image processing method applied to a classification scenario in one embodiment;

[0048] Figure 12 This is a schematic diagram comparing the image processing method of this application and the traditional denoising method in one embodiment;

[0049] Figure 13 This is a schematic diagram comparing the image processing method of this application and a traditional denoising method in another embodiment;

[0050] Figure 14 This is a structural block diagram of an image processing device in one embodiment;

[0051] Figure 15 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] The image processing method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated on server 104 or placed in the cloud or on another server. Both terminal 102 and server 104 can independently execute the image processing method provided in this embodiment. Terminal 102 and server 104 can also cooperate to execute the image processing method provided in this embodiment. When terminal 102 and server 104 cooperate to execute the image processing method provided in this embodiment, terminal 102 acquires an adversarial image and sends it to server 104. Server 104 receives the adversarial image and determines the attention mask of feature points in the adversarial image. Based on the attention mask of feature points, server 104 identifies a first region and a second region in the adversarial image, where the attention mask of feature points in the first region is greater than the attention mask of feature points in the second region. Server 104 adds first noise to the first region and second noise to the second region in the adversarial image to obtain a target noisy image; the intensity of the first noise is greater than the intensity of the second noise. Server 104 reduces the noise in the first region of the target noisy image to the second noise level, obtaining the target denoised image. Server 104 then denoises the target denoised image to obtain a cleaned image corresponding to the adversarial image, and returns the cleaned image to terminal 102.

[0054] Terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0055] In one embodiment, such as Figure 2 As shown, a flowchart of an image processing method is provided, which is applied to a computer device (such as a computer device). Figure 1 Taking the terminal or server shown as an example, the following steps are included:

[0056] Step S202: Obtain the adversarial image and determine the attention mask for feature points in the adversarial image.

[0057] Adversarial images refer to noisy images containing noise that is imperceptible to the human eye. Attention masks are used to represent the degree of attention paid to feature points in an image; a larger attention mask indicates a higher level of attention to feature points, while a smaller attention mask indicates a lower level of attention to feature points.

[0058] Computer equipment can acquire adversarial images that need to be denoised, extract feature points from the adversarial images, and determine the attention mask for each feature point. Feature points are the pixels in the adversarial image.

[0059] In this embodiment, the computer device can use an attention mechanism to perform attention allocation on feature points in an adversarial image and obtain an attention mask for each feature point.

[0060] In one embodiment, the computer device performs multiple feature extractions based on the adversarial image to obtain multiple feature maps, where each subsequent feature map is obtained by extracting features from the previous feature map. Attention is assigned to feature points in each of the multiple feature maps to obtain a candidate mask for each feature point. For each feature point, an attention mask is determined based on the candidate masks of the target feature point in the multiple feature maps. For example, the same feature point A in the multiple feature maps... Figure 1 The candidate mask in the feature is a1. Figure 2 The candidate mask in the feature is a2. Figure 3 If the candidate mask is a3, then the attention mask for feature point A is determined based on a1, a2, and a3.

[0061] Step S204: Based on the attention mask of feature points, identify the first region and the second region in the adversarial image, where the attention mask of feature points in the first region is greater than the attention mask of feature points in the second region.

[0062] The first and second regions have different attention masks, meaning they also have different levels of attention. The attention mask for feature points in the first region is greater than that in the second region; that is, the attention mask for the first region is greater than that for the second region, making the first region a region with high attention and the second region a region with low attention.

[0063] Attention masks are determined based on attention weights, where the attention weight of feature points in the first region is greater than that in the second region. In other words, the attention weight of the first region is higher than that of the second region.

[0064] The computer device identifies a first region and a second region in an adversarial image based on the attention mask of feature points, where the attention mask of feature points in the first region is greater than that of feature points in the second region.

[0065] In one embodiment, the attention mask includes a first attention mask and a second attention mask, wherein the first attention mask is larger than the second attention mask. The computer device identifies the regions in the adversarial image where each feature point with the first attention mask is located, and designates these regions as the first region. Furthermore, the computer device identifies the regions where each feature point with the second attention mask is located, and designates these regions as the second region.

[0066] Step S206: In the adversarial image, a first noise is added to a first region and a second noise is added to a second region to obtain a target noisy image; the intensity of the first noise is greater than the intensity of the second noise.

[0067] The intensity of the first noise is greater than that of the second noise. The first and second noises can be of the same type or different types. For example, the first noise can be Gaussian noise. Gaussian noise is a type of random noise that follows a normal distribution. The probability density function of Gaussian noise conforms to a Gaussian distribution (also known as a normal distribution), and the mean of the Gaussian noise is zero, while its variance is constant.

[0068] The target-noised image includes a first region with a first type of noise and a second region with a second type of noise. The target-noised image is a mixed noise image with two different intensities of the first and second noises.

[0069] The computer device can acquire first noise corresponding to the attention mask of a first region and second noise corresponding to the attention mask of a second region. The intensity of the first noise is greater than the intensity of the second noise.

[0070] A computer device adds first noise to a first region and second noise to a second region in an adversarial image to obtain a target noisy image.

[0071] In one embodiment, the computer device pre-configures corresponding noise for different attention masks. The first attention mask corresponds to the first noise, and the second attention mask corresponds to the second noise.

[0072] In one embodiment, the first noise is characterized by a first noise value, and the second noise is characterized by a second noise value. A computer device determines the pixel value of each feature point in the adversarial image, adds the first noise value to the pixel value of each feature point in the first region, and obtains updated pixel values ​​for each feature point in the first region. Furthermore, the second noise value is added to the pixel value of each feature point in the second region of the adversarial image, and updated pixel values ​​for each feature point in the second region are obtained, thereby obtaining a target noisy image.

[0073] In this embodiment, the first noise value can be the mean or the variance of the noise. The second noise value can be the mean or the variance of the noise.

[0074] In one embodiment, the computer device can counteract the addition of noise to a first region of an image through multiple first time steps, such that the noise ultimately added to the first region is the first noise. A time step represents the process of adding noise to or removing noise from an image. In this embodiment, the first time step is used to control the degree of noise addition or removal to the first region.

[0075] The computer device can determine the first sub-noise value corresponding to each first time step based on the first noise value. For example, if the first noise value is 0.2, and it is added through four first time steps, then the first sub-noise value that can be added in each first time step is 0.05. Ultimately, the total noise added to the first region after these four first time steps is 0.2. The first sub-noise value added in each first time step can be the same or different, depending on the requirements.

[0076] The computer equipment can determine the second sub-noise value corresponding to each second time step based on the second noise value. The first sub-noise value added in each second time step can be the same or different, depending on the requirements. The second time step is used to control the degree of noise addition or removal in the second region.

[0077] In the current first time step, the computer device adds the first sub-noise value corresponding to the current first time step to the pixel value of each feature point in the first region, thereby obtaining the updated pixel value of each feature point in the first region, and thus obtaining the updated first region. Moving to the next first time step, the updated first region is used as the first region in the next first time step, and the process of adding the first sub-noise value corresponding to the current first time step to the pixel value of each feature point in the first region is repeated until the updated first region in the last first time step is obtained.

[0078] Similarly, in the current second time step, the computer device adds the second sub-noise value corresponding to the current second time step to the pixel value of each feature point in the second region, thereby obtaining the updated pixel value of each feature point in the second region and thus obtaining the updated second region. Moving to the next second time step, the updated second region is used as the second region in the next second time step, and the process of adding the second sub-noise value corresponding to the current second time step to the pixel value of each feature point in the second region is repeated until the updated second region in the last second time step is obtained. When the updated first region in the last first time step and the updated second region in the last second time step are obtained in the adversarial image, the image is the target noisy image.

[0079] In this embodiment, the number of times the first time step is performed can be the same as or different from the number of times the second time step is performed.

[0080] Step S208: Denoise the first region of the target noisy image to the second noise to obtain the target denoised image.

[0081] The target denoised image is an image with second noise, meaning that both the first and second regions in the target denoised image have second noise. The target denoised image is a single noise image with uniform intensity. The noise intensity of the target denoised image is the intensity of the second noise.

[0082] Computer equipment can denoise a first region of a target noisy image to reduce the noise in the first region from a first noise to a second noise, thereby obtaining a target denoised image.

[0083] In one embodiment, the computer device can determine a noise difference value between a first noise and a second noise, and perform noise reduction on a first region of the target noisy image based on the noise difference value to reduce the noise to the second noise, thereby obtaining a first region with the second noise.

[0084] In one embodiment, the first noise is characterized by a first noise value, and the second noise is characterized by a second noise value. A computer device can determine the noise difference between the first and second noise values, determine the pixel values ​​of each feature point in a first region of the target noisy image, update the pixel values ​​of each feature point in the first region based on the noise difference value, and obtain the updated pixel values ​​of each feature point in the first region, thereby obtaining the target denoised image.

[0085] In one embodiment, the noise difference value may be the noise difference between the first noise value and the second noise value, the mean, or the variance of the noise.

[0086] For example, the noise difference value can be a noise difference value. The computer device can determine the noise difference value between the first noise value and the second noise value, determine the pixel value of each feature point in the first region of the target noisy image, and use the difference between the pixel value of each feature point in the first region and the noise difference value as the updated pixel value of each feature point in the first region, thereby obtaining the target denoised image.

[0087] In one embodiment, the computer device can perform noise reduction on a first region of the image through multiple first time steps, such that the first region is eventually reduced to second noise. The first time step is used to control the degree of removal of the first noise in the first region.

[0088] The computer device can determine a first sub-denoising value for each first time step based on the noise difference between a first noise value and a second noise value. For example, if the first noise value is 0.2, the second noise value is 0.1, and the noise difference is 0.1, then by performing denoising through four first time steps, the first sub-denoising value can be reduced by 0.05 at each first time step. Ultimately, the total noise reduction in the first region after these four first time steps is 0.1. The first sub-denoising value added at each first time step can be the same or different, depending on the requirements.

[0089] In the current first time step, the computer device adds the first sub-denoising value corresponding to the current first time step to the pixel value of each feature point in the first region, thereby obtaining the updated pixel value of each feature point in the first region, and thus obtaining the updated first region. For example, subtracting the first sub-denoising value from the pixel value of a feature point yields the updated pixel value of that feature point. Moving to the next first time step, the updated first region is used as the first region in the next first time step, and the process of adding the first sub-denoising value corresponding to the current first time step to the pixel value of each feature point in the first region is repeated until the updated first region in the last first time step is obtained. The noise in the updated first region in the last first time step is the second noise, and the image at this point is the target denoised image.

[0090] Step S210: Denoise the target image to obtain a cleaned image corresponding to the adversarial image.

[0091] Among them, a cleaned image refers to an image after noise has been removed.

[0092] The computer equipment can select a denoising method corresponding to the noise type of the second noise. Based on the selected denoising method, the target image with the second noise is denoised to obtain a cleaned image.

[0093] In one embodiment, the denoising method can be mean filtering, Gaussian filtering, median filtering, bilateral filtering, nonlocal mean filtering, etc., but is not limited to these. Mean filtering replaces the current pixel value by calculating the average value of pixels in the pixel's neighborhood, which can effectively remove noise but may cause image blurring. Gaussian filtering uses a Gaussian function to smooth the image and is suitable for removing Gaussian noise. Median filtering replaces the current pixel value with the median value in the pixel's neighborhood and is suitable for removing salt-and-pepper noise. Bilateral filtering can smooth the image while preserving edge information and is suitable for preserving image details. Nonlocal mean filtering uses nonlocal similarity in the image for denoising and is suitable for preserving image details.

[0094] In one embodiment, the computer device can denoise the target image through multiple second time steps, ultimately removing the second noise and obtaining a cleaned image. In this embodiment, the second time step is used to control the degree of noise removal in the second region.

[0095] The computer device can determine the second sub-denoising value corresponding to each second time step based on the second noise value. The second sub-denoising value added in each second time step can be the same or different, and can be set according to the requirements.

[0096] In the current second time step, the computer device adds the second sub-denoising value corresponding to the current second time step to the pixel values ​​of each feature point in all regions of the target denoised image, obtaining the updated pixel values ​​of each feature point, thus obtaining the updated denoised image. For example, subtracting the first sub-denoising value from the pixel value of a feature point yields the updated pixel value for that feature point. Moving to the next second time step, the updated denoised image is used as the target denoised image in the next second time step. The process of adding the second sub-denoising value corresponding to the current second time step to the pixel values ​​of each feature point in all regions of the target denoised image is returned and continued until the updated denoised image in the last second time step is obtained. This denoised image is the cleaned image.

[0097] In one embodiment, the schematic diagram of the image processing method is as follows: Figure 3 As shown, an attention mask is determined for feature points in the adversarial image to identify a first region and a second region. First noise is added to the first region and second noise is added to the second region. The first region in the obtained noisy target image is then denoised to the second noise, resulting in a denoised target image. Finally, global denoising is performed on the denoised target image to obtain a cleaned image. Figure 3 The processing of adversarial images, target-denoised images, target-denoised images, and clean images, such as Figure 4 As shown.

[0098] In the aforementioned image processing method, an adversarial image is acquired, and the attention mask of feature points in the adversarial image is determined. The key regions in the adversarial image can be quantified using the attention mask. Based on the attention mask of feature points, a first region and a second region in the adversarial image are identified to determine the attention level of the first and second regions. The attention mask of feature points in the first region is greater than that in the second region, indicating that the attention level of the first region is higher than that of the second region. In the adversarial image, a first noise is added to the first region and a second noise is added to the second region to obtain a target noisy image, thus allowing different noise levels to be added according to the attention levels of different regions. Furthermore, the intensity of the first noise is greater than the intensity of the second noise; the greater the added noise, the more semantic information of the image is lost, and the lower the added noise, the less semantic information is lost. This indicates that adding high-intensity noise to high-intensity regions and low-intensity noise to low-intensity regions preserves more semantic information in the second region. Since greater denoising intensity eliminates more semantic information, reducing the noise in the first region of the noisy target image to the second noise level first reduces the high-intensity noise in the first region to the low-intensity noise in the second region, thus eliminating some adversarial perturbations in the first region and obtaining a denoised target image with low-intensity noise. Global denoising is then performed on the denoised target image to further denoise the first region, resulting in a clearer first region. This also allows for denoising of the low-intensity second region, making it clearer than before without losing too much semantic information. Ultimately, the cleaned image achieves a balance between eliminating adversarial perturbations and preserving semantic information, with the first region having high clarity and the second region retaining more semantic information.

[0099] In one embodiment, determining an attention mask for feature points in an adversarial image includes:

[0100] Attention weights are assigned to feature points in the adversarial image; then, masks are assigned based on these attention weights to obtain the attention masks for the feature points in the adversarial image.

[0101] Computer devices can assign attention to feature points in an adversarial image using an attention mechanism, obtaining an attention weight for each feature point. The computer device can acquire preset attention weights and, based on these preset weights and the attention weights of each feature point, assign a mask to each feature point, thus obtaining an attention mask for each feature point in the adversarial image.

[0102] In one embodiment, the attention mask includes a first attention mask and a second attention mask, where the first attention mask is greater than the second attention mask. For each feature point, the computer device compares the attention weight of the target feature point with a preset attention weight. If the attention weight of the target feature point is greater than or equal to the preset attention weight, the first attention mask is assigned to the target feature point. If the attention weight of the target feature point is less than the preset attention weight, the second attention mask is assigned to the target feature point. Following the same processing method, an attention mask can be assigned to each feature point.

[0103] In this embodiment, attention is assigned to feature points in the adversarial image to obtain the attention weights of the feature points. This can accurately identify which parts of the adversarial image have high attention and which parts have low attention, thereby accurately assigning larger attention masks to high-attention parts and smaller attention masks to low-attention parts.

[0104] In one embodiment, attention is assigned to feature points in an adversarial image to obtain attention weights for the feature points, including:

[0105] Multiple feature extractions are performed on the adversarial image to obtain multiple feature maps; the next feature map is obtained by extracting features from the previous feature map; attention is assigned to feature points in the multiple feature maps respectively to obtain the attention weight of feature points in each feature map;

[0106] The process of assigning masks based on the attention weights of feature points to obtain attention masks for feature points in an adversarial image includes: assigning masks to feature points in multiple feature maps based on the attention weights of feature points in the multiple feature maps to obtain candidate masks for feature points in the multiple feature maps; and determining the attention mask for each feature point based on the candidate masks of the target feature point in the multiple feature maps.

[0107] Here, the candidate mask refers to the attention mask of the candidate. The candidate mask can include a first candidate mask and a second candidate mask, with the first candidate mask being greater than the second candidate mask. For example, the first candidate mask is 1 and the second candidate mask is 0.

[0108] The candidate mask for a feature point in a feature map refers to the attention mask corresponding to that feature point. The next feature map among multiple feature maps is obtained by extracting features from the previous feature map; the first obtained feature map is obtained by extracting features from the adversarial image.

[0109] The computer device extracts features from the adversarial image to obtain a feature map. It then extracts features from the obtained feature map to obtain the next feature map, and so on, until multiple feature maps are obtained. For each feature map, the computer device assigns attention to the feature points in the target feature map, obtaining the attention weight for each feature point in the target feature map, thus obtaining the attention weight for each feature point in the feature map.

[0110] Because multiple feature maps exist, the same feature point is assigned attention weights in multiple feature maps, resulting in multiple attention weights for the same feature point. The computer device assigns a mask to each feature point in each feature map, obtaining candidate masks for each feature point in each feature map. Because multiple feature maps exist, the same feature point is assigned candidate masks in multiple feature maps, resulting in multiple candidate masks for the same feature point.

[0111] For each feature point, the attention mask for the target feature point is determined based on the candidate masks of the target feature point in multiple feature maps.

[0112] like Figure 5 The diagram shown illustrates the principle of obtaining an attention mask in one embodiment. The adversarial image undergoes feature extraction 1, outputting features. Figure 1 .feature Figure 1 After feature extraction step 2, the output features are... Figure 2 .feature Figure 2 After feature extraction step 3, the output features are... Figure 3 .feature Figure 3 After feature extraction step 4, the output features are... Figure 4 For features Figure 1 Attention is allocated to each feature point to obtain features. Figure 1 Attention weights are assigned to each feature point, and a mask is allocated based on these attention weights to obtain the feature values. Figure 1 Candidate masks for each feature point. For features... Figure 2 Attention is allocated to each feature point to obtain features. Figure 2 Attention weights are assigned to each feature point, and a mask is allocated based on these attention weights to obtain the feature values. Figure 2 Candidate masks for each feature point. For features... Figure 3 Attention is allocated to each feature point to obtain features. Figure 3 Attention weights are assigned to each feature point, and a mask is allocated based on these attention weights to obtain the feature values. Figure 3 Candidate masks for each feature point. For features... Figure 4 Attention is allocated to each feature point to obtain features. Figure 4 Attention weights are assigned to each feature point, and a mask is allocated based on these attention weights to obtain the feature values. Figure 4Candidate masks for each feature point. For each feature point, the mask is generated from the feature point in the target feature. Figure 1 Candidate masks in features Figure 2 Candidate masks in features Figure 3 Candidate masks and features Figure 4 The candidate masks are used to determine the attention mask for the target feature points.

[0113] In one embodiment, for each feature point, the computer device determines the union of candidate masks for the target feature point in multiple feature maps, and uses the union as the attention mask for the target feature point.

[0114] The union refers to selecting the largest candidate mask as the attention mask, or selecting the candidate mask with the most quantity as the attention mask.

[0115] Optionally, for each feature point, the computer device determines the largest candidate mask from the candidate masks of the target feature point in multiple feature maps, and uses the largest candidate mask as the attention mask for the target feature point. For example, if the candidate masks of the same feature point in three feature maps are 1, 1, and 0 respectively, then 1 can be used as the attention mask for that feature point.

[0116] Optionally, the candidate mask may include a first candidate mask and a second candidate mask, where the first candidate mask is greater than the second candidate mask. The computer device determines the number of first candidate masks and the number of second candidate masks from the candidate masks of the target feature point in multiple feature maps. When the number of first candidate masks is greater than or equal to the number of second candidate masks, the first candidate mask is used as the attention mask for the target feature point. When the number of first candidate masks is less than the number of second candidate masks, the second candidate mask is used as the attention mask for the target feature point. For example, if the candidate masks for the same feature point in three feature maps are first candidate mask "1", first candidate mask "1", and first candidate mask "0", then first candidate mask "1" can be used as the attention mask for that feature point.

[0117] In one embodiment, the average of the candidate masks for the target feature point across multiple feature maps is calculated. If the average is greater than or equal to a mask threshold, the first candidate mask is used as the attention mask for the target feature point. If the average is less than the mask threshold, the second candidate mask is used as the attention mask for the target feature point. For example, if the candidate masks for the same feature point in three feature maps are 1, 1, and 0, the mask threshold is 0.5, the average is 2 / 3, and the average is greater than 0.5, then the first candidate mask "1" is used as the attention mask.

[0118] In this embodiment, multiple feature extractions are performed on the adversarial image to obtain multiple feature maps. Each subsequent feature map is obtained by extracting features from the previous feature map, thus enabling the acquisition of more detailed and deeper image features through multiple feature extractions. Attention is assigned to feature points in each of the multiple feature maps, thereby obtaining the attention weights corresponding to the same feature point in different feature maps. The attention level of the same feature point in different feature maps serves as reference information for mask allocation, making mask allocation more accurate and obtaining more accurate candidate masks. The final attention mask is determined by referring to the candidate masks of the same feature point in multiple different feature maps, so that the final attention mask integrates the attention levels in feature maps at different levels, improving the accuracy of the obtained attention mask.

[0119] In one embodiment, denoising a first region of a target noisy image to a second noise level to obtain a target denoised image includes:

[0120] Based on the first noise in the first region of the target noisy image, the adversarial image is noisyened to the first noise to obtain a first noisy image; the first noisy image is denoised to obtain a reference denoised image, which has reference noise; the first region in the reference denoised image and the second region in the target noisy image are fused to obtain a fused image; when the reference noise of the fused image is equal to the second noise, the fused image is used as the target denoised image.

[0121] The first noisy image is an image with a first type of noise. That is, the first noisy image is a single noise image with uniform intensity. The reference denoised image is an image with reference noise, which is a noise image with uniform intensity. The intensity of the reference noise is less than the intensity of the first noise.

[0122] In a single denoising process, the intensity of the first noise in the first noisy image is greater than the intensity of the reference noise in the reference denoised image obtained by denoising the first noisy image.

[0123] The target-noised image includes a first region with a first type of noise and a second region with a second type of noise. The target-noised image is a mixed noise image with two different intensities of the first and second noises.

[0124] A computer device adds a first noise, identical to the first region, to an adversarial image, obtaining a first noisy image. The first noisy image is then denoised to a reference noise level, resulting in a reference denoised image, which contains the reference noise. The first region in the reference denoised image and the second region in the target noisy image are then stitched together to obtain a fused image.

[0125] When the reference noise of the fused image is equal to the second noise, it means that the noise in the fused image is the second noise, and the noise intensity in the fused image is equal to the intensity of the second noise. In this case, the fused image can be used as the target denoising image. At this time, the fused image is a single noise image with uniform intensity.

[0126] In one embodiment, the target denoised image can be obtained through multiple iterations. That is: based on the first noise in the first region of the target denoised image, the adversarial image is denoised to the first noise to obtain a first denoised image; the first denoised image is denoised to obtain a reference denoised image, which has reference noise; when the intensity of the reference noise is equal to the intensity of the second noise (i.e., the reference noise equals the second noise), the first region in the reference denoised image and the second region in the target denoised image are fused to obtain a fused image, and the fused image is used as the target denoised image.

[0127] When the intensity of the reference noise is greater than the intensity of the second noise (i.e., the reference noise is greater than the second noise), it indicates that the fused image is a mixed noise image with both reference noise and second noise, while the reference denoised image is a single noise image with uniform intensity. In this case, the reference denoised image is used as the first denoised image in the next iteration, and the step of denoising the first denoised image is returned and the process continues until the intensity of the reference noise is equal to the intensity of the second noise (i.e., the reference noise is equal to the second noise). The first region in the reference denoised image and the second region in the target denoised image are then fused to obtain a fused image, which is then used as the target denoised image.

[0128] In this embodiment, during multiple iterations, if the reference noise of the reference denoised image is greater than the second noise, the next iteration is executed. This continues until the reference noise of the reference denoised image equals the second noise, at which point the iteration is the last iteration, and the process stops after obtaining the fused image of this iteration. The noise in the reference noise image during the first iteration is the first noise.

[0129] In this embodiment, the intensity of the reference noise is less than the intensity of the first noise in multiple iterations. Furthermore, in all iterations except the last one, the intensity of the reference noise is less than the intensity of the first noise, but greater than the intensity of the second noise.

[0130] In one embodiment, one iteration corresponds to one first time step. Noise reduction in one iteration is equivalent to the noise reduction processing of the first region in one time step in the above embodiments.

[0131] In this embodiment, a reference denoised image is generated by adding noise to the first noise level of the first region of the target denoised image and then denoising it. This reference denoised image is then fused with the second region of the target denoised image through noise matching. This achieves precise and controllable noise reduction of the target region from the first noise level to the second noise level. While preserving the main content of the image, it achieves adaptive adjustment of the local noise level, improving the flexibility and regional processing capability of the denoising process.

[0132] In one embodiment, a first region in a reference denoised image and a second region in a target denoised image are fused to obtain a fused image, including:

[0133] Based on the second noise in the second region of the target noisy image, the adversarial image is noised to the reference noise to obtain a reference noisy image; the reference noisy image and the reference noisy image are fused to obtain a fused image.

[0134] The computer device denoises the first noisy image to obtain a reference denoised image. Based on the second noise in the second region of the target noisy image, the adversarial image is denoised onto the reference noise to obtain a reference noisy image. The computer device then fuses the reference denoised image and the reference noisy image to obtain a fused image.

[0135] When the reference noise of the fused image is equal to the second noise, it means that the noise in the fused image is the second noise, and the noise intensity in the fused image is equal to the intensity of the second noise. In this case, the fused image can be used as the target denoising image. At this time, the fused image is a single noise image with uniform intensity.

[0136] If the reference noise of the fused image is greater than the second noise, it indicates that the fused image contains mixed noise, and further denoising is required. Therefore, the fused image is used as the first noisy image in the next iteration, and the step of denoising the first noisy image in this embodiment is returned to be executed until the reference noise of the fused image equals the second noise, at which point the iteration stops, and the fused image at the point where the iteration stops is obtained as the target denoised image.

[0137] In this embodiment, noise reduction requires multiple iterations. In all iterations except the last one, the intensity of the first noise in the first noisy image in the remaining iterations is greater than the intensity of the reference noise in the reference denoised image obtained by denoising the first noisy image in the remaining iterations.

[0138] In this embodiment, by adding noise to the adversarial image with a first noise level to obtain a first noisy image, and then denoising the first noisy image to the noise level to generate a reference noisy image, the noise in the first region can be reduced. Based on the second noise in the second region of the target noisy image, the adversarial image is added with the reference noise to obtain a reference noisy image, so that the noise in the second region of the reference noisy image is the second noise added to the adversarial image. This preserves more of the semantic information of the second region. Fusing the reference noisy image with the reference denoised image achieves a natural transition between the first region (after denoising) and the second region (keeping the original noise) in the target noisy image, ensuring that the noise distribution of the fused image is consistent with the target second noise. This preserves more of the semantic information of the second region while accurately reducing the noise level of the first region, thereby improving the clarity of the first region and enhancing the visual consistency and adaptability of the denoising result.

[0139] In one embodiment, fusing a reference denoised image and a reference noisy image to obtain a fused image includes:

[0140] Based on the attention mask of the feature points, the first region in the reference denoised image and the second region in the reference denoised image are determined; the first region in the reference denoised image and the second region in the reference denoised image are then stitched together to obtain a fused image.

[0141] The computer device can determine a first region in the reference denoised image and a second region in the reference noisy image based on the attention mask of the feature points. Then, based on the attention masks of the feature points in the first and second regions, the first region in the reference denoised image and the second region in the reference noisy image are stitched together to obtain a fused image.

[0142] In this embodiment, key regions of the reference denoised image and the reference noisy image are accurately located by using attention masks of feature points, and the target regions of the two images are smoothly fused based on the mask-guided stitching strategy. This preserves the low noise characteristics of the denoised region and maintains the original details of the noisy region, thereby optimizing the overall noise distribution while ensuring the natural continuity of the image content and improving the accuracy of local adaptive denoising.

[0143] In one embodiment, the method further includes:

[0144] If the noise in the fused image is greater than the second noise, the fused image is used as the first noisy image in the next iteration, and the step of denoising the first noisy image is returned and execution continues.

[0145] like Figure 6The diagram illustrates the acquisition of a target denoised image in one embodiment. A computer device adds noise to an adversarial image based on first noise in a first region of the target denoised image, thereby obtaining a first denoised image.

[0146] First iteration: Denoise the first noisy image to obtain a reference denoised image. The reference noise in the reference denoised image is less than the first noise and greater than the second noise. Based on the second noise in the second region of the target noisy image, add noise to the adversarial image to the reference noise to obtain a reference noisy image. At this point, the noise in both the reference noisy image and the reference denoised image is the same as the reference noise, meaning the noise intensity in the reference noisy image and the reference denoised image is the same. Based on the attention mask of the feature points in the first region and the attention mask of the features in the second region, concatenate the first region in the reference denoised image and the second region in the reference noisy image to obtain the fused image of the first iteration.

[0147] Second iteration: The fused image from the first iteration is used as the first noisy image in the second iteration. The fused image is then denoised to obtain a reference denoised image. At this point, the reference noise in the reference denoised image is less than the reference noise in the previous iteration (i.e., the first iteration), and the reference noise is greater than the second noise. Based on the second noise in the second region of the target noisy image, the adversarial image is noised to the reference noise to obtain a reference noisy image. Based on the attention mask of feature points in the first region and the attention mask of features in the second region, the first region in the reference denoised image and the second region in the reference noisy image are concatenated to obtain the fused image of the second iteration.

[0148] When the noise in the fused image (i.e., the reference noise) equals the second noise, the iteration stops, and the fused image obtained in this iteration is used as the target denoised image.

[0149] When the noise in the fused image is greater than the second noise, the fused image is used as the first noisy image in the next iteration, and the step of denoising the first noisy image is returned and the process continues until the noise in the fused image (i.e. the reference noise) is equal to the second noise. The iteration stops, and the fused image obtained when the iteration stops is used as the target denoised image.

[0150] In one embodiment, iterative noise reduction can be performed using the following formula:

[0151] Forward noise reduction processing: (1)

[0152] (2)

[0153] (3)

[0154] For example, the target noisy image is a mixed noise image, where the high-intensity noise (i.e., the first noise) is 0.2 and the low-intensity noise (i.e., the second noise) is 0.1. The specific processing procedure for obtaining the target noisy image through iterative denoising is as follows:

[0155] First, the image x(0) is denoised to 0.2 (uniform noise) by using the high-intensity noise of 0.2 in formula (1) to obtain x(t+1).

[0156] Start the iteration: Then use formula (2) to reduce the noise of x(t+1) to 0.18, and obtain Then, using formula (1), noise is added to the adversarial image x(0) by using the noise 0.1 in the low-noise region of the mixed noise image. ,this The noise in the image is uniform low-intensity noise of 0.18. Then... (Noise level is 0.18) and (Noise is 0.18) x(t) is obtained by fusing with attention mask.

[0157] Second iteration: Using the 0.18 intensity noise obtained above, x(t) is then reduced to 0.16 using formula (2) to obtain... Then, using formula (1), noise is added to the adversarial image x(0) by using the noise 0.1 in the low-noise region of the mixed noise image. ,this The noise in the image is uniform low-intensity noise of 0.16. Then... (Noise level is 0.16) and (Noise is 0.16) x(t-1) is obtained by fusing with attention mask.

[0158] Second iteration: Using the 0.16 intensity noise x(t-1) obtained above, and then using formula (2) to reduce the noise to 0.14, we obtain... Then, using formula (1), noise is added to the adversarial image x(0) by using the noise 0.1 in the low-noise region of the mixed noise image. ,this The noise in the image is uniform low-intensity noise of 0.14. Then... (Noise level is 0.14) and (Noise is 0.14) x(t-2) is obtained by fusing with attention mask.

[0159] This process continues until the noise intensity of the fused image is 0.1, resulting in a target denoised image where the noise intensity of all images is 0.1.

[0160] In this embodiment, by iteratively detecting and adjusting the noise level of the fused image, the region exceeding the target noise (second noise) is re-denoised in each iteration until it converges to the expected noise range, thereby achieving progressive optimization of noise intensity. This ensures that the final denoised image achieves accurate noise control both globally and locally, while avoiding the loss of detail caused by excessive denoising, thus improving the robustness of the method and the controllability of the results.

[0161] In one embodiment, denoising the target image to obtain a cleaned image corresponding to the adversarial image includes:

[0162] Determine the neighborhood of feature points in the target denoised image and identify the feature points within the neighborhood; update the pixel values ​​of the feature points in the target denoised image based on the pixel values ​​of the feature points within the neighborhood to obtain the cleaned image corresponding to the adversarial image.

[0163] For each feature point in the target denoising image, the computer device determines the neighborhood of the target feature point and identifies feature points within that neighborhood. Based on the pixel values ​​of the feature points within the neighborhood, the computer device determines a new pixel value and replaces the pixel value of the target feature point in the target denoising image with this new pixel, thus obtaining the updated pixel value for the target feature point. This same process is repeated to update the pixel values ​​of each feature point in the target denoising image, resulting in a cleaned image.

[0164] The new pixel value can be either the mean or the median. The mean of the pixel values ​​of the feature points within the neighborhood is calculated to obtain the median pixel value, which is determined from the pixel values ​​of the feature points within the neighborhood. This median pixel value is simply the median.

[0165] In this embodiment, the computer device can determine the filter size, which can be a mean filter or a median filter. The size of the mean filter is determined by a parameter, such as 3x3, 5x5, etc. This value determines the coverage area of ​​the filter on the target denoised image.

[0166] The filter is placed on each pixel (i.e., feature point) of the target denoising image. Centered on the current pixel, the filter is spread across the target denoising image. The average of all pixel values ​​within the filter's coverage area is taken to obtain a new pixel value. The original pixel value is replaced with the calculated new pixel value.

[0167] In this embodiment, by analyzing the local neighborhood information of feature points in the target denoised image and using the pixel values ​​of feature points within the neighborhood for adaptive weighted updates, the integrity of image edges and detailed structures can be maintained while effectively eliminating noise, thereby generating a more natural and higher-quality purified image. At the same time, it avoids the problems of excessive smoothing or loss of detail that may be caused by traditional global denoising methods, and improves the local consistency and visual realism of the denoising results.

[0168] In one embodiment, the method further includes:

[0169] Obtain first descriptive text for a first region and second descriptive text for a second region in the adversarial image; perform noise filtering based on the first descriptive text and the attention mask of feature points in the first region to obtain first noise for the first region; perform noise filtering based on the second descriptive text and the attention mask of feature points in the second region to obtain second noise for the second region.

[0170] The first descriptive text describes the first region. The second descriptive text describes the second region.

[0171] like Figure 7 As shown, the computer device acquires a first descriptive text and a second descriptive text, extracts a first text feature from the first descriptive text, and converts the first text feature into a first text vector. It then extracts a second text feature from the second descriptive text and converts the second text feature into a second text vector.

[0172] Based on the first text vector and the attention mask of feature points in the first region, the first mask weight is determined. Based on the first mask weight, noise that matches the first mask weight is selected from multiple candidate noises and used as the first noise in the first region.

[0173] Based on the attention mask of the feature points in the second text vector and the second region, the second mask weight is determined. Based on the second mask weight, noise that matches the second mask weight is selected from multiple candidate noises and used as the second noise in the second region.

[0174] In one embodiment, the first mask weight can be the product of the first text vector and the attention mask of the feature points in the first region. The second mask weight can be the product of the second text vector and the attention mask of the feature points in the second region.

[0175] In one embodiment, a computer device acquires descriptive text for an adversarial image, and extracts a first descriptive text corresponding to the first region and a second descriptive text corresponding to the second region from the descriptive text based on a first region and a second region in the adversarial image.

[0176] In one embodiment, the descriptive text can be text in an adversarial image or text generated based on an adversarial image.

[0177] In this embodiment, by combining text description (first description text and second description text) with an attention mask-guided noise filtering mechanism, semantically aware local noise adaptive control is achieved. It can dynamically allocate the optimal noise level (first noise and second noise) according to the image content of different regions (such as texture complexity or semantic importance), thereby taking into account both detail preservation and noise suppression in semantically relevant regions during the noise reduction process, and improving the visual quality and semantic consistency of the generated image.

[0178] In one embodiment, the method further includes: obtaining first descriptive text for a first region and second descriptive text for a second region in an adversarial image;

[0179] In an adversarial image, adding first noise to a first region and adding second noise to a second region to obtain a target noisy image includes: adding first noise to a first region based on first descriptive text and adding second noise to a second region based on second descriptive text to obtain a target noisy image.

[0180] The computer device extracts a first text feature from the first descriptive text, converts the first text feature into a first text vector, extracts a second text feature from the second descriptive text, and converts the second text feature into a second text vector.

[0181] In an adversarial image, the pixel values ​​of each feature point in a first region are determined, and first noise is added to the pixel values ​​of each feature point based on a first text vector. In addition, the pixel values ​​of each feature point in a second region are determined, and second noise is added to the pixel values ​​of each feature point based on a second text vector, thereby obtaining a target noisy image.

[0182] In one embodiment, the first noise is represented by a first noise value, and the second noise is represented by a second noise value. Adding the first noise to the pixel values ​​of each feature point based on the first text vector can be specifically done by: multiplying the first text vector and the pixel values ​​of each feature point in the first region, then summing the product of each feature point with the first noise value to obtain the updated pixel value for each feature point, thereby obtaining the updated first region. The second region undergoes the same process to obtain the updated second region, thus obtaining the target noisy image.

[0183] In this embodiment, text-driven noise generation achieves greater flexibility in noise addition and controllability of image semantics.

[0184] In one embodiment, denoising a first region in a target noisy image to a second noise to obtain a target denoised image includes: based on a first descriptive text, guiding the first region in the target noisy image to denoise to the second noise to obtain a target denoised image;

[0185] The process of denoising the target image to obtain a cleaned image corresponding to the adversarial image includes: guiding the target image to denoise based on a first descriptive text and a second descriptive text to obtain a cleaned image corresponding to the adversarial image.

[0186] The computer device determines the pixel value of each feature point in the first region, multiplies the first text vector and the pixel value of each feature point in the first region respectively to obtain the updated first region, and then reduces the noise of the updated first region to the second noise to obtain the target denoised image.

[0187] The computer device determines the pixel values ​​of each feature point in the target denoised image. It then multiplies the first text vector with the pixel values ​​of each feature point in the first region of the target denoised image to obtain the updated first region. Similarly, it multiplies the second text vector with the pixel values ​​of each feature point in the second region of the target denoised image to obtain the updated second region. Obtaining both the updated first and updated second regions yields the cleaned image corresponding to the adversarial image.

[0188] In this embodiment, by combining the semantic guidance of the first and second descriptive texts, content-aware adaptive processing is achieved during the denoising process. This enables the first region to be accurately denoised to the target noise level (second noise) according to semantic requirements. At the same time, the overall denoising process is constrained by the text description, ensuring that important semantically relevant features are preserved while suppressing noise. The final purified image achieves the ideal denoising effect and maintains key visual information consistent with the original semantics, thereby improving the semantic accuracy and visual fidelity of the denoising result.

[0189] In one embodiment, in an adversarial image, adding first noise to a first region and second noise to a second region to obtain a target noisy image includes:

[0190] Add first noise to the adversarial image to obtain a first noisy image; add second noise to the adversarial image to obtain a second noisy image; fuse the first region in the first noisy image and the second region in the second noisy image to obtain the target noisy image.

[0191] like Figure 8As shown, the first noise is characterized by a first noise value, and the second noise is characterized by a second noise value. The computer device determines the pixel value of each feature point in the adversarial image, and adds the first noise value to the pixel value of each feature point in the adversarial image to obtain a first noisy image. The computer device then determines the pixel value of each feature point in the adversarial image, and adds the second noise value to the pixel value of each feature point in the adversarial image to obtain a second noisy image.

[0192] For example, adding a first noise value to the pixel values ​​of each feature point in the adversarial image can be achieved by summing or multiplying the first noise value with the pixel values ​​of each feature point in the adversarial image. Replacing the first noise value with a second noise value then adds the second noise value.

[0193] The computer device fuses the first noisy image and the second noisy image based on the attention mask of the first region and the attention mask of the second region to obtain a target noisy image. The target noisy image includes the first region in the first noisy image and the second region in the second noisy image.

[0194] In one embodiment, the computer device stitches together a first region in a first noisy image and a second region in a second noisy image based on an attention mask for a first region and an attention mask for a second region to obtain a target noisy image.

[0195] In one embodiment, the computer device can add noise to the adversarial image through multiple first time steps, such that the noise added to the final adversarial image is the first noise, thereby obtaining a first noisy image. This process is the same as the process of adding noise to a first region in the adversarial image through multiple first time steps in the above embodiment.

[0196] In one embodiment, the computer device can add noise to the adversarial image through multiple second time steps, so that the noise added to the final adversarial image is a second noise, thereby obtaining a second noisy image. This process is the same as the process of adding noise to a second region in the adversarial image through multiple second time steps in the above embodiment.

[0197] In this embodiment, a first noise is added to the adversarial image to obtain a first noisy image, and a second noise is added to the adversarial image to obtain a second noisy image. The first region in the first noisy image and the second region in the second noisy image are fused. The key regions can be selectively fused to generate a target noisy image with stronger robustness. This can preserve both the perturbation characteristics of the adversarial image and more semantic information in the second region.

[0198] In one embodiment, an image processing method is provided, applied to a computer device, comprising:

[0199] The process involves acquiring an adversarial image, performing multiple feature extractions based on the adversarial image to obtain multiple feature maps; the next feature map is obtained by extracting features from the previous feature map.

[0200] Attention is assigned to feature points in multiple feature maps to obtain the attention weights of feature points in each feature map.

[0201] Based on the attention weights of feature points in multiple feature maps, mask assignment is performed on the feature points in multiple feature maps to obtain candidate masks for feature points in multiple feature maps.

[0202] For each feature point, the attention mask for the target feature point is determined based on the candidate masks of the target feature point in multiple feature maps.

[0203] Based on the attention mask of feature points, a first region and a second region in the adversarial image are identified. The attention mask of feature points in the first region is greater than that of feature points in the second region.

[0204] First noise is added to the adversarial image to obtain a first noisy image; second noise is added to the adversarial image to obtain a second noisy image, wherein the intensity of the first noise is greater than the intensity of the second noise; a first region in the first noisy image and a second region in the second noisy image are fused to obtain a target noisy image.

[0205] Based on the first noise in the first region of the target noisy image, the adversarial image is noised to the first noise to obtain the first noisy image.

[0206] The first noisy image is denoised to obtain a reference denoised image, which has reference noise. Based on the second noise in the second region of the target noisy image, the adversarial image is denoised to the reference noise to obtain a reference noisy image. Based on the attention mask of the feature points in the first region and the attention mask of the feature points in the second region, the first region in the reference denoised image and the second region in the reference noisy image are stitched together to obtain a fused image. When the reference noise of the fused image is equal to the second noise, the fused image is used as the target denoised image.

[0207] When the noise in the fused image is greater than the second noise, the fused image is used as the first noisy image in the next iteration, and the step of denoising the first noisy image is returned and the process continues until the reference noise of the fused image is equal to the second noise, at which point the iteration stops, and the fused image at this time is used as the target denoised image.

[0208] The neighborhood of feature points in the target denoised image is determined, and the feature points within the neighborhood are identified. Based on the pixel values ​​of the feature points within the neighborhood, the pixel values ​​of the feature points in the target denoised image are updated to obtain the cleaned image corresponding to the adversarial image.

[0209] In one embodiment, an application scenario is provided for an image denoising model that implements an image processing method, specifically applied to an image denoising scenario. The architecture diagram of this image denoising model is as follows: Figure 9 As shown, it includes multiple classifiers.

[0210] Based on the hyperparameters input by the user (i.e., the manually set parameter thresholds) , and The weight matrices of attention assignments for different modules of the classifier are extracted, and the weight matrices are adjusted to the same size as the original image. Then, by comparing the weights with the hyperparameters input by the user, the attention mask of each module is constructed. Finally, the union is performed to obtain the final attention mask.

[0211] Specifically, the attention mask is constructed as follows:

[0212]

[0213]

[0214]

[0215] in, The image features of the original image represent the output of the activation layer of the m-th module of the classifier. It is an activation graph. express The i-th channel, This indicates a space normalization operation. This represents a manually set threshold, where C is the number of image channels and P is the channel threshold. It is the mask threshold. The indicator function is defined as follows: the attention mask for regions that meet the conditions is 1, and the attention mask for regions that do not meet the conditions is 0. M is the attention mask assigned to a region of the image by a module of the classifier (if there are M modules, then the same region of the image has M attention masks), and W is the final attention mask for a region (a mask matrix, i.e., a matrix composed of 0 and 1, and the resulting image is a binary image).

[0216] The role of the union U: For example, if an image has 4 regions, and three classifier modules assign an attention mask W of 0 to region 1, and one classifier module assigns an attention mask W of 1 to region 1, then the union is the final attention mask W for region 1 is determined to be 1.

[0217] Based on the two noise intensities input by the user, and the attention mask obtained in (1), high-intensity noise is added to the regions with an attention mask value of 1 and low-intensity noise is added to the regions with an attention mask value of 0 through a modified forward process. Finally, the two images are mixed to obtain a mixed noise image.

[0218] High noise:

[0219] Low noise:

[0220]

[0221] in, and It is the intensity of the noise, one is... High strength, another Low intensity. User input. and .

[0222] The resulting mixed noisy image is input into a two-stage denoising reverse process. In the first stage, the high-intensity noise is removed into low-intensity noise by extracting the semantic information contained in the low-noise region (where is the extraction of semantic information reflected in the formulas below, in formulas 1 and 3); in the second stage, a normal denoising process is used to remove the weak noise on the image, and finally a clean sample is obtained.

[0223] The first stage is used to remove localized high-intensity noise:

[0224] Forward noise reduction processing: (1)

[0225] (2)

[0226] (3)

[0227] in, This represents some pre-set hyperparameters, i.e., weight parameters. x(0) represents the original input image, images without noise of different intensities (images with adversarial perturbations). ). This represents high-intensity noise sampled from a Gaussian distribution. Here, t and t+1 represent time points. N represents a Gaussian distribution. This represents the mean. Represents variance. These represent the parameters of the diffusion model. This is an image created by adding high-intensity noise to low-noise regions of the original image. Here... It is the attention mask in (1).

[0228] The specific processing procedure for the first stage is as follows:

[0229] For example, the target noisy image is a mixed noise image, where the high-intensity noise (i.e., the first noise) is 0.2 and the low-intensity noise (i.e., the second noise) is 0.1. The specific processing procedure for obtaining the target noisy image through iterative denoising is as follows:

[0230] First, the image x(0) is denoised to 0.2 (uniform noise) by using the high-intensity noise of 0.2 in formula (1) to obtain x(t+1).

[0231] Start the iteration: Then use formula (2) to reduce the noise of x(t+1) to 0.18, and obtain Then, using formula (1), noise is added to the adversarial image x(0) by using the noise 0.1 in the low-noise region of the mixed noise image. ,this The noise in the image is uniform low-intensity noise of 0.18. Then... (Noise level is 0.18) and (Noise is 0.18) x(t) is obtained by fusing with attention mask.

[0232] Second iteration: Using the 0.18 intensity noise obtained above, x(t) is then reduced to 0.16 using formula (2) to obtain... Then, using formula (1), noise is added to the adversarial image x(0) by using the noise 0.1 in the low-noise region of the mixed noise image. ,this The noise in the image is uniform low-intensity noise of 0.16. Then... (Noise level is 0.16) and (Noise is 0.16) x(t-1) is obtained by fusing with attention mask.

[0233] Second iteration: Using the 0.16 intensity noise x(t-1) obtained above, and then using formula (2) to reduce the noise to 0.14, we obtain... Then, using formula (1), noise is added to the adversarial image x(0) by using the noise 0.1 in the low-noise region of the mixed noise image. ,this The noise in the image is uniform low-intensity noise of 0.14. Then... (Noise level is 0.14) and (Noise is 0.14) x(t-2) is obtained by fusing with attention mask.

[0234] This process continues until the noise intensity of the fused image is 0.1, resulting in a target denoised image where the noise intensity of all images is 0.1.

[0235] The second stage is used to remove weak global noise:

[0236]

[0237] like Figure 10 As shown, to recover a clear image from an image containing heterogeneous noise, this embodiment designs a two-stage denoising process based on attention maps, as the reverse process of the diffusion model. Specifically, if the sampling time step is between two different noise levels, a patching technique is applied to remove local high-intensity noise, i.e., the first stage. If the current time step is lower than the time step with a smaller noise level, a standard denoising process is used to remove global weak noise, i.e., the second stage.

[0238] In one embodiment, the image processing method can be applied to any scenario that requires image denoising, such as image capture, image classification, image reconstruction, autonomous driving, defect detection, medical imaging, etc.

[0239] like Figure 11 As shown, this image denoising model can also include fully connected layers for classifying the purified image. In image classification scenarios, according to the above... Figure 9 After obtaining the cleaned image corresponding to the adversarial image through processing, the cleaned image can be classified. For example... Figure 11 The final classification result for the image was Foxhound, not Papillon.

[0240] In autonomous driving scenarios, the image processing method of this application can reduce noise in images captured by vehicle-mounted cameras and improve the accuracy of obstacle detection and scene recognition.

[0241] In defect detection scenarios, it can reduce noise interference and improve detection accuracy when detecting product defects on the production line.

[0242] In medical imaging scenarios, the image processing method of this application can enhance the clarity of medical images such as CT, MRI, and X-ray, helping doctors to more accurately identify medical images.

[0243] This application constructs an attention mask for the classifier, thereby adding noise of varying intensities, which preserves the semantic information of the image while eliminating adversarial perturbations in key regions.

[0244] In the experiment, the protected classifier models were set as WideResNet28-10 neural networks and WideResNet70-16 neural networks, and the diffusion model structure adopted the Unconditional Didffusion Model. The dataset used was the CIFAR10 dataset. This experiment compared four methods: Method 1 was Diffpure, Method 2 was REAP, Method 3 was DNC, and Method 4 was ATOP for reference.

[0245] Figure 12 The normal accuracy and robust accuracy are shown on the CIFAR10 dataset when the classifier model is WideResNet28-10.

[0246] AutoAttack represents the fast gradient sign attack method, and PGD represents the projected gradient attack method. Figure 12 As can be seen, the method in Example 1 is superior to the method in Comparative Example 1 in all datasets.

[0247] Figure 13 The normal accuracy and robust accuracy are shown on the CIFAR10 dataset when the classifier model is WideResNet70-16.

[0248] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0249] Based on the same inventive concept, this application also provides an image processing apparatus for implementing the image processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more image processing apparatus embodiments provided below can be found in the limitations of the image processing method described above, and will not be repeated here.

[0250] In one embodiment, such as Figure 14 As shown, an image processing apparatus 1400 is provided, including: an acquisition module, a recognition module, a noise addition module, a first noise reduction module, and a second noise reduction module, wherein:

[0251] The acquisition module 1402 is used to acquire the adversarial image and determine the attention mask of the feature points in the adversarial image.

[0252] The recognition module 1404 is used to identify a first region and a second region in an adversarial image based on the attention mask of feature points, wherein the attention mask of feature points in the first region is greater than the attention mask of feature points in the second region.

[0253] The noise-adding module 1406 is used to add first noise to a first region and second noise to a second region in an adversarial image to obtain a target noise-adding image; the intensity of the first noise is greater than the intensity of the second noise.

[0254] The first noise reduction module 1408 is used to reduce the noise of the first region of the target noisy image to the second noise, thereby obtaining the target noisy image.

[0255] The second noise reduction module 1410 is used to remove noise from the target noise reduction image to obtain a cleaned image corresponding to the adversarial image.

[0256] In this embodiment, by acquiring an adversarial image and determining the attention mask of feature points in the adversarial image, key regions in the adversarial image can be quantified using the attention mask. Based on the attention mask of feature points, a first region and a second region in the adversarial image are identified to determine the attention level of the first region and the second region. The attention mask of feature points in the first region is greater than that in the second region, indicating that the attention level of the first region is higher than that of the second region. In the adversarial image, a first noise is added to the first region and a second noise is added to the second region to obtain a target noisy image, thereby enabling the addition of different noise levels according to the attention levels of different regions. Furthermore, the intensity of the first noise is greater than the intensity of the second noise; the greater the added noise, the more image semantics are lost, and the less the added noise, the less image semantics are lost. This indicates that adding high-intensity noise to regions with high attention and adding low-intensity noise to regions with low attention preserves more image semantic information in the second region. Since greater denoising intensity eliminates more semantic information, reducing the noise in the first region of the noisy target image to the second noise level first reduces the high-intensity noise in the first region to the low-intensity noise in the second region, thus eliminating some adversarial perturbations in the first region and obtaining a denoised target image with low-intensity noise. Global denoising is then performed on the denoised target image to further denoise the first region, resulting in a clearer first region. This also allows for denoising of the low-intensity second region, making it clearer than before without losing too much semantic information. Ultimately, the cleaned image achieves a balance between eliminating adversarial perturbations and preserving semantic information, with the first region having high clarity and the second region retaining more semantic information.

[0257] In one embodiment, the acquisition module 1402 is further configured to perform attention allocation on feature points in the adversarial image to obtain attention weights for the feature points; and to perform mask allocation based on the attention weights of the feature points to obtain attention masks for the feature points in the adversarial image.

[0258] In this embodiment, attention is assigned to feature points in the adversarial image to obtain the attention weights of the feature points. This can accurately identify which parts of the adversarial image have high attention and which parts have low attention, thereby accurately assigning larger attention masks to high-attention parts and smaller attention masks to low-attention parts.

[0259] In one embodiment, the acquisition module 1402 is further configured to perform multiple feature extractions based on the adversarial image to obtain multiple feature maps; the next feature map is obtained by extracting features from the previous feature map; attention is assigned to feature points in the multiple feature maps respectively to obtain the attention weight of feature points in each feature map;

[0260] Based on the attention weights of feature points in multiple feature maps, mask assignment is performed on the feature points in multiple feature maps to obtain candidate masks for the feature points in multiple feature maps; for each feature point, the attention mask for the target feature point is determined based on the candidate masks of the target feature point in multiple feature maps.

[0261] In this embodiment, multiple feature extractions are performed on the adversarial image to obtain multiple feature maps. Each subsequent feature map is obtained by extracting features from the previous feature map, thus enabling the acquisition of more detailed and deeper image features through multiple feature extractions. Attention is assigned to feature points in each of the multiple feature maps, thereby obtaining the attention weights corresponding to the same feature point in different feature maps. The attention level of the same feature point in different feature maps serves as reference information for mask allocation, making mask allocation more accurate and obtaining more accurate candidate masks. The final attention mask is determined by referring to the candidate masks of the same feature point in multiple different feature maps, so that the final attention mask integrates the attention levels in feature maps at different levels, improving the accuracy of the obtained attention mask.

[0262] In one embodiment, the first denoising module 1408 is further configured to: add noise to the adversarial image to the first noise based on the first noise in the first region of the target denoised image to obtain a first denoised image; denoise the first denoised image to obtain a reference denoised image, wherein the noise in the reference denoised image is the reference noise; fuse the first region in the reference denoised image and the second region in the target denoised image to obtain a fused image; and when the reference noise of the fused image is equal to the second noise, use the fused image as the target denoised image.

[0263] In this embodiment, a reference denoised image is generated by adding noise to the first noise level of the first region of the target denoised image and then denoising it. This reference denoised image is then fused with the second region of the target denoised image through noise matching. This achieves precise and controllable noise reduction of the target region from the first noise level to the second noise level. While preserving the main content of the image, it achieves adaptive adjustment of the local noise level, improving the flexibility and regional processing capability of the denoising process.

[0264] In one embodiment, the first noise reduction module 1408 is further configured to add noise to the adversarial image to the reference noise based on the second noise in the second region of the target noise-added image to obtain a reference noise-added image; and to fuse the reference noise-added image and the reference noise-added image to obtain a fused image.

[0265] In this embodiment, a reference denoised image is generated by adding noise to the adversarial image to the reference noise level, and then fused with the reference denoised image. This achieves a natural transition between the first region (after denoising) and the second region (keeping the original noise) in the target denoised image, ensuring that the noise distribution of the fused image is consistent with the target second noise. This allows for precise control of the noise level in different regions while preserving image details, thereby improving the visual consistency and adaptability of the denoising results.

[0266] In one embodiment, the first noise reduction module 1408 is further configured to determine a first region in the reference denoised image and a second region in the reference noisy image based on the attention mask of the feature points; and to stitch the first region in the reference denoised image and the second region in the reference noisy image together to obtain a fused image.

[0267] In this embodiment, key regions of the reference denoised image and the reference noisy image are accurately located by using attention masks of feature points, and the target regions of the two images are smoothly fused based on the mask-guided stitching strategy. This preserves the low noise characteristics of the denoised region and maintains the original details of the noisy region, thereby optimizing the overall noise distribution while ensuring the natural continuity of the image content and improving the accuracy of local adaptive denoising.

[0268] In one embodiment, the first noise reduction module 1408 is further configured to, when the noise in the fused image is greater than the second noise, use the fused image as the first noisy image in the next iteration, and return to the step of denoising the first noisy image and continue execution.

[0269] In this embodiment, by iteratively detecting and adjusting the noise level of the fused image, the region exceeding the target noise (second noise) is re-denoised in each iteration until it converges to the expected noise range, thereby achieving progressive optimization of noise intensity. This ensures that the final denoised image achieves accurate noise control both globally and locally, while avoiding the loss of detail caused by excessive denoising, thus improving the robustness of the method and the controllability of the results.

[0270] In one embodiment, the second noise reduction module 1410 is further configured to determine the neighborhood of feature points in the target noise reduction image and determine the feature points within the neighborhood; update the pixel values ​​of the feature points in the target noise reduction image according to the pixel values ​​of the feature points within the neighborhood to obtain the cleaned image corresponding to the adversarial image.

[0271] In this embodiment, by analyzing the local neighborhood information of feature points in the target denoised image and using the pixel values ​​of feature points within the neighborhood for adaptive weighted updates, the integrity of image edges and detailed structures can be maintained while effectively eliminating noise, thereby generating a more natural and higher-quality purified image. At the same time, it avoids the problems of excessive smoothing or loss of detail that may be caused by traditional global denoising methods, and improves the local consistency and visual realism of the denoising results.

[0272] In one embodiment, the acquisition module 1402 is further configured to acquire a first descriptive text for a first region and a second descriptive text for a second region in an adversarial image; perform noise filtering based on the first descriptive text and the attention mask of feature points in the first region to obtain first noise for the first region; and perform noise filtering based on the second descriptive text and the attention mask of feature points in the second region to obtain second noise for the second region.

[0273] In this embodiment, by combining text description (first description text and second description text) with an attention mask-guided noise filtering mechanism, semantically aware local noise adaptive control is achieved. It can dynamically allocate the optimal noise level (first noise and second noise) according to the image content of different regions (such as texture complexity or semantic importance), thereby taking into account both detail preservation and noise suppression in semantically relevant regions during the noise reduction process, and improving the visual quality and semantic consistency of the generated image.

[0274] In one embodiment, the acquisition module 1402 is further configured to acquire a first descriptive text of a first region and a second descriptive text of a second region in an adversarial image;

[0275] The noise-adding module 1406 is used to add first noise to a first region based on first descriptive text and add second noise to a second region based on second descriptive text in an adversarial image to obtain a target noise-adding image.

[0276] In one embodiment, the first noise reduction module 1408 is used to guide a first region in a target noisy image to reduce noise to a second noise based on a first descriptive text, thereby obtaining a target noisy image;

[0277] The second noise reduction module 1410 is also used to guide the target noise reduction image to perform noise reduction based on the second descriptive text, so as to obtain a purified image corresponding to the adversarial image.

[0278] In this embodiment, by combining the semantic guidance of the first and second descriptive texts, content-aware adaptive processing is achieved during the denoising process. This enables the first region to be accurately denoised to the target noise level (second noise) according to semantic requirements. At the same time, the overall denoising process is constrained by the text description, ensuring that important semantically relevant features are preserved while suppressing noise. The final purified image achieves the ideal denoising effect and maintains key visual information consistent with the original semantics, thereby improving the semantic accuracy and visual fidelity of the denoising result.

[0279] In one embodiment, the noise-adding module 1406 is further configured to add first noise to the adversarial image to obtain a first noise-adding image; add second noise to the adversarial image to obtain a second noise-adding image; and fuse a first region in the first noise-adding image and a second region in the second noise-adding image to obtain a target noise-adding image.

[0280] In this embodiment, a first noise is added to the adversarial image to obtain a first noisy image, and a second noise is added to the adversarial image to obtain a second noisy image. The first region in the first noisy image and the second region in the second noisy image are fused. The key regions can be selectively fused to generate a target noisy image with stronger robustness. This can preserve both the perturbation characteristics of the adversarial image and more semantic information in the second region.

[0281] Each module in the aforementioned image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0282] In one embodiment, a computer device is provided, which may be a terminal or a server. Taking a terminal as an example, its internal structure diagram can be as follows: Figure 15As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an image processing method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0283] Those skilled in the art will understand that Figure 15 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0284] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0285] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0286] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0287] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0288] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logic devices, etc., and are not limited to these.

[0289] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0290] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Acquire an adversarial image and determine the attention mask for feature points in the adversarial image; Based on the attention mask of the feature points, a first region and a second region in the adversarial image are identified, wherein the attention mask of the feature points in the first region is greater than the attention mask of the feature points in the second region; In the adversarial image, a first noise is added to the first region and a second noise is added to the second region to obtain a target noisy image; the intensity of the first noise is greater than the intensity of the second noise. The first region of the target noisy image is reduced to the second noise to obtain the target noisy image; The target image is denoised to obtain a cleaned image corresponding to the adversarial image.

2. The method according to claim 1, characterized in that, Determining the attention mask for feature points in the adversarial image includes: Attention is assigned to feature points in the adversarial image to obtain the attention weights of the feature points; The attention mask for the feature points in the adversarial image is obtained by assigning a mask based on the attention weights of the feature points.

3. The method according to claim 2, characterized in that, The step of assigning attention to feature points in the adversarial image to obtain attention weights for the feature points includes: Multiple feature maps are obtained by performing feature extraction on the adversarial image; the next feature map is obtained by extracting features from the previous feature map. Attention is assigned to feature points in the plurality of feature maps respectively to obtain the attention weight of each feature point in the feature map; The step of assigning a mask based on the attention weights of the feature points to obtain the attention mask for the feature points in the adversarial image includes: Based on the attention weights of the feature points in the multiple feature maps, mask assignment is performed on the feature points in the multiple feature maps to obtain candidate masks for the feature points in the multiple feature maps. For each feature point, an attention mask for the target feature point is determined based on the candidate masks of the target feature point in the plurality of feature maps.

4. The method according to claim 1, characterized in that, The step of denoising the first region of the target noisy image to the second noise to obtain the target denoised image includes: Based on the first noise in the first region of the target noisy image, the adversarial image is noisyened to the first noise to obtain the first noisy image; The first noisy image is denoised to obtain a reference denoised image, wherein the reference denoised image has reference noise; The first region in the reference denoised image and the second region in the target denoised image are fused to obtain a fused image; When the reference noise of the fused image is equal to the second noise, the fused image is used as the target denoised image.

5. The method according to claim 4, characterized in that, The step of fusing the first region in the reference denoised image and the second region in the target denoised image to obtain a fused image includes: Based on the second noise in the second region of the target noisy image, the adversarial image is noisyened to the reference noise to obtain a reference noisy image; The reference denoised image and the reference denoised image are fused to obtain a fused image.

6. The method according to claim 5, characterized in that, The step of fusing the reference denoised image and the reference denoised image to obtain a fused image includes: Based on the attention mask of the feature points, the first region in the reference denoised image and the second region in the reference denoised image are determined; The first region in the reference denoised image and the second region in the reference denoised image are stitched together to obtain a fused image.

7. The method according to claim 5, characterized in that, The method further includes: If the noise in the fused image is greater than the second noise, the fused image is used as the first noisy image in the next iteration, and the step of denoising the first noisy image is returned and execution continues.

8. The method according to claim 1, characterized in that, The step of denoising the target image to obtain the cleaned image corresponding to the adversarial image includes: Determine the neighborhood of feature points in the target denoised image, and determine the feature points within the neighborhood; The pixel values ​​of the feature points in the target denoised image are updated based on the pixel values ​​of the feature points within the neighborhood to obtain the purified image corresponding to the adversarial image.

9. The method according to claim 1, characterized in that, The method further includes: Obtain a first descriptive text for a first region and a second descriptive text for a second region in the adversarial image; Noise is filtered based on the first descriptive text and the attention mask of feature points in the first region to obtain the first noise for the first region; Noise is filtered based on the second descriptive text and the attention mask of feature points in the second region to obtain the second noise for the second region.

10. The method according to claim 9, characterized in that, The step of denoising the first region in the target noisy image to the second noise to obtain the target denoised image includes: Based on the first descriptive text, the first region in the target noisy image is guided to be denoised into the second noise, thereby obtaining the target denoised image; The step of denoising the target image to obtain the cleaned image corresponding to the adversarial image includes: Based on the first descriptive text and the second descriptive text, the target denoised image is guided to undergo denoising to obtain a cleaned image corresponding to the adversarial image.

11. The method according to any one of claims 1 to 10, characterized in that, The step of adding first noise to the first region and second noise to the second region in the adversarial image to obtain a target noisy image includes: Add first noise to the adversarial image to obtain a first noisy image; A second noise is added to the adversarial image to obtain a second noisy image; The first region in the first noisy image and the second region in the second noisy image are fused together to obtain the target noisy image.

12. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire an adversarial image and determine the attention mask of feature points in the adversarial image; The recognition module is used to identify a first region and a second region in the adversarial image based on the attention mask of the feature points, wherein the attention mask of the feature points in the first region is greater than the attention mask of the feature points in the second region. A noise-adding module is used to add a first noise to the first region and a second noise to the second region in the adversarial image to obtain a target noise-adding image; the intensity of the first noise is greater than the intensity of the second noise. The first noise reduction module is used to reduce the noise of the first region of the target noisy image to the second noise, thereby obtaining the target noisy image; The second noise reduction module is used to denoise the target denoised image to obtain a cleaned image corresponding to the adversarial image.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images