An image processing method, apparatus, computer, and readable storage medium

Abstract

An embodiment of the present application discloses an image processing method, apparatus, computer, and readable storage medium. The method includes: obtaining an image to be processed, extracting a first initial image feature and a second initial image feature of the image to be processed, performing feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature; the resolution of the first initial image feature is greater than the resolution of the second initial image feature; performing convolution processing on the image fusion feature to obtain a target image feature corresponding to the image to be processed, and determining a target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature; performing feature enhancement on the target image feature through the target noise feature to obtain a denoised image feature, and converting the denoised image feature into a denoised and optimized image corresponding to the image to be processed. By adopting the present application, the efficiency and generalization of image denoising processing can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to an image processing method, apparatus, computer, and readable storage medium. Background Art

[0002] In daily life, the collected images often have foreground noise, which usually affects the quality of the corresponding images. Therefore, it is necessary to repair the images to remove the noise in the images. Currently, generally, the noise is modeled based on the prior knowledge of the noise, and the image after denoising the image is obtained by comparing with the noise layer. In this way, it is necessary to perform corresponding modeling for specific noise, that is, the images under different noise types cannot be processed, resulting in low generalization and low efficiency of image denoising. Summary of the Invention

[0003] Embodiments of this application provide an image processing method, apparatus, computer, and readable storage medium, which can improve the accuracy and detection efficiency of image processing.

[0004] On the one hand, an embodiment of this application provides an image processing method, which includes:

[0005] Obtain an image to be processed, extract a first initial image feature and a second initial image feature of the image to be processed, and perform feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature; the resolution of the first initial image feature is greater than the resolution of the second initial image feature;

[0006] Perform convolution processing on the image fusion feature to obtain a target image feature corresponding to the image to be processed, and determine a target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature;

[0007] Enhance the target image feature through the target noise feature to obtain a denoised image feature, and convert the denoised image feature into a denoised and optimized image corresponding to the image to be processed.

[0008] On the one hand, an embodiment of this application provides an image processing method, which includes:

[0009] Obtain a noisy image sample and a ground truth denoised image corresponding to the noisy image sample;

[0010] In the initial feature extraction network of the initial image denoising model, extract a first initial sample image feature and a second initial sample image feature of the noisy image sample, and perform feature fusion on the first initial sample image feature and the second initial sample image feature to obtain a sample fusion feature; the resolution of the first initial sample image feature is greater than the resolution of the second initial sample image feature;

[0011] In the initial feature separation network of the initial image denoising model, the sample fusion feature is subjected to convolution processing to obtain the target sample image feature corresponding to the noisy image sample, and the sample noise feature corresponding to the noisy image sample is determined according to the sample fusion feature and the target sample image feature;

[0012] In the initial image optimization network of the initial image denoising model, the target sample image feature is enhanced by the sample noise feature to obtain the sample denoised image feature, and the sample denoised image feature is converted into the denoised optimized image sample corresponding to the noisy image sample;

[0013] The parameters of the initial image denoising model are adjusted according to the denoised optimized image sample, the sample noise feature and the denoising ground truth map to obtain the image denoising model.

[0014] One aspect of the embodiments of the present application provides an image processing apparatus, and the apparatus includes:

[0015] An image acquisition module, configured to acquire an image to be processed;

[0016] A feature extraction module, configured to extract a first initial image feature and a second initial image feature of the image to be processed;

[0017] A feature fusion module, configured to perform feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature; the resolution of the first initial image feature is greater than the resolution of the second initial image feature;

[0018] An image extraction module, configured to perform convolution processing on the image fusion feature to obtain a target image feature corresponding to the image to be processed;

[0019] A noise determination module, configured to determine a target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature;

[0020] An image denoising module, configured to enhance the target image feature by the target noise feature to obtain a denoised image feature, and convert the denoised image feature into a denoised optimized image corresponding to the image to be processed.

[0021] Wherein, the feature extraction module includes:

[0022] A convolution layer extraction unit, configured to input the image to be processed into a feature extraction network, and extract the convolution features corresponding to the image to be processed in each of the f feature extraction convolution layers in the feature extraction network; f is a positive integer;

[0023] A convolution processing unit, configured to perform convolution processing on the f convolution features to obtain a first initial image feature and a second initial image feature of the image to be processed.

[0024] Among them, the convolution processing unit includes:

[0025] A convolution division sub-unit, configured to divide f feature extraction convolution layers into a first convolution layer and a second convolution layer; the depth of the first convolution layer in the feature extraction network is less than the depth of the second convolution layer in the feature extraction network;

[0026] A first convolution sub-unit, configured to perform convolution processing on the convolution features corresponding to the first convolution layer to obtain a first initial image feature of the image to be processed;

[0027] A second convolution sub-unit, configured to perform convolution processing on the convolution features corresponding to the second convolution layer to obtain a second initial image feature of the image to be processed.

[0028] Among them, the feature fusion module includes:

[0029] A first sampling unit, configured to perform upsampling processing on the first initial image feature to obtain a first sampling feature corresponding to the first initial image feature;

[0030] A second sampling unit, configured to perform upsampling processing on the second initial image feature to obtain a second sampling feature corresponding to the second initial image feature; the first sampling feature and the second sampling feature have the same feature resolution;

[0031] A sampling splicing unit, configured to perform feature splicing on the first sampling feature and the second sampling feature to obtain a spliced feature;

[0032] An information fusion unit, configured to perform feature information fusion on the spliced feature through a feature fusion convolution layer to obtain an image fusion feature corresponding to the spliced feature.

[0033] Among them, the image extraction module includes:

[0034] A feature separation unit, configured to perform convolution processing on the image fusion feature through a feature separation convolution layer to obtain an image separation feature, and perform activation processing on the image separation feature to obtain a feature distribution map;

[0035] A feature weighting unit, configured to perform weighting processing on the image fusion feature based on the feature distribution map to obtain a target image feature corresponding to the image to be processed.

[0036] Among them, the noise determination module includes:

[0037] A difference processing unit, configured to determine the feature difference between the image fusion feature and the target image feature as an initial noise feature;

[0038] A noise merging unit, configured to obtain N noise feature channels included in the initial noise feature, and merge the noise sub-features at the same feature position among the d noise sub-features respectively corresponding to the N noise feature channels, so as to obtain a target noise feature corresponding to the image to be processed; N is a positive integer, and d is a positive integer.

[0039] Wherein, the image denoising module includes:

[0040] A feature determination unit, configured to obtain the noise distribution information corresponding to the target noise feature, and determine the feature to be enhanced in the target image feature according to the noise distribution information;

[0041] A denoising processing unit, configured to enhance the feature to be enhanced to obtain a denoised image feature, and perform decoding processing on the denoised image feature to obtain a denoised optimized image corresponding to the image to be processed.

[0042] Wherein, the image denoising module includes:

[0043] A distribution acquisition unit, configured to input the target noise feature and the target image feature into an image optimization network, and obtain the noise distribution information of the target noise feature through the image optimization network;

[0044] An enhancement generation unit, configured to obtain an image mask feature of the target image feature based on the noise distribution information in the mask sub-network of the image optimization network, and generate an image enhancement feature of the target image feature according to the image mask feature;

[0045] A feature enhancement unit, configured to enhance the target image feature based on the image enhancement feature to obtain a denoised image feature;

[0046] A feature conversion unit, configured to convert the denoised image feature into a denoised optimized image corresponding to the image to be processed.

[0047] Wherein, the image denoising module includes:

[0048] A feature input unit, configured to input the target noise feature and the target image feature into an image optimization network; the image optimization network includes k optimization sub-networks, the k optimization sub-networks include a first optimization sub-network and a second optimization sub-network, and k is a positive integer;

[0049] A first enhancement unit, configured to enhance the target image feature based on the target noise feature in the first optimization sub-network to obtain a first enhancement feature, and update the target noise feature based on the feature enhancement result to obtain a first noise feature;

[0050] A second enhancement unit, configured to enhance the first enhancement feature based on the first noise feature in the second optimization sub-network to obtain a denoised image feature;

[0051] The feature conversion unit is configured to convert the denoised image feature into a denoised and optimized image corresponding to the image to be processed.

[0052] On the one hand, an embodiment of the present application provides an image processing device, which includes:

[0053] A sample acquisition module, configured to acquire a noisy image sample and a denoising ground truth map corresponding to the noisy image sample;

[0054] A sample extraction module, configured to extract a first initial sample image feature and a second initial sample image feature of the noisy image sample in the initial feature extraction network of the initial image denoising model;

[0055] A sample fusion module, configured to perform feature fusion on the first initial sample image feature and the second initial sample image feature to obtain a sample fusion feature; the resolution of the first initial sample image feature is greater than the resolution of the second initial sample image feature;

[0056] A feature processing module, configured to perform convolution processing on the sample fusion feature in the initial feature separation network of the initial image denoising model to obtain a target sample image feature corresponding to the noisy image sample, and determine a sample noise feature corresponding to the noisy image sample according to the sample fusion feature and the target sample image feature;

[0057] A sample optimization module, configured to perform feature enhancement on the target sample image feature through the sample noise feature in the initial image optimization network of the initial image denoising model to obtain a sample denoised image feature, and convert the sample denoised image feature into a denoised and optimized image sample corresponding to the noisy image sample;

[0058] A model adjustment module, configured to adjust the parameters of the initial image denoising model according to the denoised and optimized image sample, the sample noise feature, and the denoising ground truth map to obtain an image denoising model.

[0059] Wherein, the model adjustment module includes:

[0060] A sample generation unit, configured to generate a noise image sample according to the noisy image sample and the denoising ground truth map;

[0061] A loss generation unit, configured to generate a first loss function according to the sample noise feature and the noise image sample, and generate a second loss function according to the denoised and optimized image sample and the denoising ground truth map;

[0062] A parameter adjustment unit, configured to adjust the parameters of the initial image denoising model based on the first loss function and the second loss function to obtain an image denoising model.

[0063] On the one hand, an embodiment of the present application provides a computer device, including a processor, a memory, and an input / output interface;

[0064] The processor is respectively connected to the memory and the input / output interface. Among them, the input / output interface is used to receive and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that a computer device including the processor executes the image processing method in one aspect of the embodiments of the present application.

[0065] One aspect of the embodiments of the present application provides a computer-readable storage medium storing a computer program, which is adapted to be loaded and executed by a processor so that a computer device having the processor executes the image processing method in one aspect of the embodiments of the present application.

[0066] One aspect of the embodiments of the present application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device executes the methods provided in various alternative manners in one aspect of the embodiments of the present application.

[0067] Implementing the embodiments of the present application will have the following beneficial effects:

[0068] In an embodiment of the present application, a to-be-processed image is obtained, a first initial image feature and a second initial image feature of the to-be-processed image are extracted, the first initial image feature and the second initial image feature are feature-fused to obtain an image fusion feature; the resolution of the first initial image feature is greater than the resolution of the second initial image feature; the image fusion feature is subjected to convolution processing to obtain a target image feature corresponding to the to-be-processed image, and based on the image fusion feature and the target image feature, a target noise feature corresponding to the to-be-processed image is determined; the target image feature is feature-enhanced through the target noise feature to obtain a denoised image feature, and the denoised image feature is converted into a denoised and optimized image corresponding to the to-be-processed image. Through the above process, the computer device can extract the intermediate features (i.e., the first initial image feature and the second initial image feature) of the to-be-processed image, rather than directly obtaining the final features of the to-be-processed image, so that the computer device can obtain the features of the to-be-processed image in different dimensions, that is, various information of the to-be-processed image from local to global can be obtained, improving the comprehensiveness of data acquisition. Moreover, based on the extracted first initial image feature and second initial image feature, the to-be-processed image can be preliminarily denoised, and based on the obtained target noise feature, the target image feature obtained by preliminary denoising can be optimized, so as to realize the denoising process of the to-be-processed image, thereby improving the efficiency of image denoising. Since the above process realizes the image denoising process based on the features of the image itself, the denoising process of images with different noises can be realized based on the above process, improving the generalization of image denoising. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0070] Figure 1 is a network interaction architecture diagram of an image processing provided by an embodiment of the present application;

[0071] Figure 2 is a schematic diagram of an image processing scenario provided by an embodiment of the present application;

[0072] Figure 3 is a flowchart of a method for image processing provided by an embodiment of the present application;

[0073] Figure 4 is a schematic diagram of an image feature extraction scenario provided by an embodiment of the present application;

[0074] Figure 5This is a schematic diagram of an image optimization network architecture provided by an embodiment of the present application;

[0075] Figure 6 This is another schematic diagram of an image optimization network architecture provided by an embodiment of the present application;

[0076] Figure 7 This is a schematic diagram of an image optimization method flow provided by an embodiment of the present application;

[0077] Figure 8 This is a schematic diagram of an image denoising network architecture provided by an embodiment of the present application;

[0078] Figure 9 This is a flowchart of an image denoising model training method provided by an embodiment of the present application;

[0079] Figure 10 This is a schematic diagram of an image processing device provided by an embodiment of the present application;

[0080] Figure 11 This is a schematic diagram of an image processing device provided by an embodiment of the present application;

[0081] Figure 12 This is a schematic diagram of the structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0082] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0083] In the embodiments of the present application, please refer to Figure 1 , Figure 1It is a network interaction architecture diagram for image processing provided by an embodiment of the present application. Among them, the computer device 101 can perform data interaction with one or at least two terminal devices (such as terminal devices 102a, 102b, 102c, etc.). Among them, the computer device 101 can obtain the image to be processed from the computer device 101, or can obtain the image to be processed from any one of the terminal devices, etc., and perform denoising processing on the image to be processed. For example, the computer device 101 obtains the image to be processed from the terminal device 102a, and performs denoising processing on the image to be processed based on the solution implemented in the present application to obtain the denoising optimized image corresponding to the image to be processed. Optionally, the computer device 101 can send the denoising optimized image to the terminal device 102a. Optionally, assuming that the computer device 101 is the device corresponding to the target application program, after the computer device 101 obtains the image to be processed, it performs denoising processing on the image to be processed to obtain the denoising optimized image corresponding to the image to be processed, and pushes the denoising optimized image to the terminal device associated with the target application program. Or, the computer device 101 can perform denoising processing on the image to be processed to obtain the denoising optimized image corresponding to the image to be processed. When the computer device 101 receives an image acquisition request for the image to be processed from the target terminal device, it sends the denoising optimized image to the target terminal device, or can send the image to be processed and the denoising optimized image to the target terminal device, etc., which is not limited here. Among them, the target terminal device can be any one of the one or at least two terminal devices.

[0084] Specifically, please refer to Figure 2 , Figure 2 It is a schematic diagram of an image processing scenario provided by an embodiment of the present application. As Figure 2As shown, the computer device obtains the image 101 to be processed, and extracts the first initial image feature and the second initial image feature of the image to be processed. Among them, the resolution of the first initial image feature is greater than that of the second initial image feature. The computer device can perform feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature, and perform convolution processing on the image fusion feature to obtain the target image feature corresponding to the image to be processed. The target image feature refers to the image feature obtained after preliminary denoising processing of the image to be processed. Further, the computer device can determine the target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature. The target noise feature is used to represent the feature of the image noise included in the image to be processed. Further, the computer device can perform feature enhancement on the target image feature based on the target noise feature to obtain a denoised image feature, that is, further optimize the target image feature, and convert the denoised image feature into the denoised and optimized image 202 corresponding to the image to be processed. That is to say, the denoising process of the image to be processed in this application is realized based on the features of the image to be processed at different scales, and it is a process of processing the features of the image to be processed. When the image to be processed carries different types of image noise, the denoising process of the image to be processed can be realized through the above process, which can improve the generalization of image denoising.

[0085] It can be understood that the computer device mentioned in the embodiments of this application includes, but is not limited to, a terminal device or a server. In other words, the computer device can be a server or a terminal device, or a system composed of a server and a terminal device. Among them, the terminal device mentioned above can be an electronic device, including but not limited to mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, in-vehicle devices, augmented reality / virtual reality (AR / VR) devices, head-mounted displays, smart TVs, wearable devices, smart speakers, digital cameras, cameras, and other mobile internet devices (MID) with network access capabilities, or terminal devices in scenarios such as trains, ships, and flights. Among them, the server mentioned above can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, vehicle-road coordination, content delivery network (CDN), and big data and artificial intelligence platforms.

[0086] Optionally, the data involved in the embodiments of the present application can be stored in a computer device, or the data can be stored based on cloud storage technology or a blockchain network, which is not limited herein. Optionally, the computer device can generate a smart contract based on the image processing method implemented by the present application (specifically, refer to the following Figure 3 specific description shown). When the computer device obtains an image to be processed, it calls the smart contract to perform denoising processing on the image to be processed, and obtains a denoised and optimized image corresponding to the image to be processed, etc., which is not limited herein.

[0087] Optionally, the present application can adopt machine learning technology in the field of artificial intelligence to implement the image denoising process.

[0088] Among them, Artificial Intelligence (AI) is to use a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, a theory, method, technology, and application system that perceives the environment, acquires knowledge, and uses knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, enabling the machines to have the functions of perception, reasoning, and decision-making.

[0089] Artificial intelligence technology is an interdisciplinary subject, involving a wide range of fields, including both hardware-level technologies and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, autonomous driving, and intelligent transportation.

[0090] Further, please refer to Figure 3 , Figure 3 which is a flowchart of a method for image processing provided by an embodiment of the present application. As Figure 3 shown, taking an original image as an example for description. In other words, in the Figure 3 method embodiment described, the image processing process includes the following steps:

[0091] Step S301, obtain an image to be processed, extract the first initial image feature and the second initial image feature of the image to be processed, and perform feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature.

[0092] In an embodiment of the present application, a computer device may obtain a to-be-processed image that needs to be denoised, and extract a first initial image feature and a second initial image feature of the to-be-processed image. The resolution of the first initial image feature is greater than that of the second initial image feature. Among them, the first initial image feature refers to a low-dimensional high-resolution low-level feature, and the second initial image feature refers to a high-dimensional low-resolution high-level feature. Here, "high" and "low" are relative to the first initial image feature and the second initial image feature. That is to say, the feature dimension indicated by the first initial image feature is lower than that indicated by the second initial image feature, and the resolution of the first initial image feature is greater than that of the second initial image feature. Among them, the first initial image feature contains more position and detail information in the to-be-processed image, etc. It can be considered that the first initial image feature can be used to represent the local feature of the to-be-processed image; the second initial image feature can be considered to be used to represent the global feature of the to-be-processed image, etc. The number of convolutional layers passed by the first initial image feature is less than the number of convolutional layers passed by the second initial image feature.

[0093] Specifically, the computer device may input the to-be-processed image into a feature extraction network, and through f feature extraction convolutional layers in the feature extraction network, extract the corresponding convolutional features of the to-be-processed image in each feature extraction convolutional layer, where f is a positive integer. Among them, when the computer device inputs the to-be-processed image into the feature extraction network, it will sequentially pass through f feature extraction convolutional layers and extract the corresponding convolutional features in each feature extraction convolutional layer; for example, assuming f is 3, input the to-be-processed image into the feature extraction network, obtain the convolutional feature 1 corresponding to the first feature extraction convolutional layer, input the convolutional feature 1 into the second feature extraction convolutional layer to get the convolutional feature 2, and input the convolutional feature 2 into the third feature extraction convolutional layer to get the convolutional feature 3, so as to obtain the corresponding convolutional features of the to-be-processed image in each feature extraction convolutional layer. Perform convolutional processing on the f convolutional features to obtain the first initial image feature and the second initial image feature of the to-be-processed image.

[0094] Specifically, divide the f feature extraction convolutional layers into a first convolutional layer and a second convolutional layer; the depth of the first convolutional layer in the feature extraction network is less than the depth of the second convolutional layer in the feature extraction network. Perform convolutional processing on the convolutional features corresponding to the first convolutional layer to obtain the first initial image feature of the to-be-processed image; perform convolutional processing on the convolutional features corresponding to the second convolutional layer to obtain the second initial image feature of the to-be-processed image.

[0095] For example, please refer to Figure 4 , Figure 4 which is a schematic diagram of an image feature extraction scenario provided by an embodiment of the present application. As Figure 4As shown in the figure, the feature extraction network includes f feature extraction convolutional layers. The computer device may divide the f feature extraction convolutional layers into a first convolutional layer 401 and a candidate convolutional layer 402. Among them, the candidate convolutional layer 402 refers to the feature extraction convolutional layers other than the first convolutional layer 401 in the f feature extraction convolutional layers. Optionally, the computer device may determine the candidate convolutional layer 402 as the second convolutional layer; or, it may determine the first convolutional layer 401 and the candidate convolutional layer 402 together as the second convolutional layer, that is, determine the f feature extraction convolutional layers as the second convolutional layer; that is to say, the depth of the first convolutional layer 401 in the feature extraction network is less than the depth of the second convolutional layer in the feature extraction network. Assume that the first convolutional layer 401 includes feature extraction convolutional layer 1, feature extraction convolutional layer 2,..., and feature extraction convolutional layer f 1 , and the second convolutional layer includes feature extraction convolutional layer f 2 ,..., and feature extraction convolutional layer f, or the second convolutional layer may include feature extraction convolutional layer 1, feature extraction convolutional layer 2,..., and feature extraction convolutional layer f, etc. Among them, f 1 is a positive integer less than or equal to f, and f 2 is a positive integer greater than f 1 and less than f. The computer device may perform convolution processing on the convolution features corresponding to the first convolutional layer to obtain the first initial image feature of the image to be processed; perform convolution processing on the convolution features corresponding to the second convolutional layer to obtain the second initial image feature of the image to be processed. Among them, Figure 4 the dotted line shown in the figure is used to indicate an optional method. For example, the convolution features for generating the second initial image feature may or may not include convolution feature 1, which is determined by the determination method of the second convolutional layer. That is to say, if the second convolutional layer includes feature extraction convolutional layer 1, the convolution features for generating the second initial image feature include convolution feature 1; if the second convolutional layer does not include feature extraction convolutional layer 1, the convolution features for generating the second initial image feature do not include convolution feature 1.

[0096] Furthermore, the computer device may perform feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature. Specifically, the computer device may perform scale change on the first initial image feature to obtain a first sampling feature corresponding to the first initial image feature; perform scale change on the second initial image feature to obtain a second sampling feature corresponding to the second initial image feature; perform feature fusion on the first sampling feature and the second sampling feature to obtain an image fusion feature. Among them, the scale change may be upsampling processing or direct interpolation processing, etc., which is not limited here.

[0097] For example, a computer device can perform upsampling on the first initial image feature to obtain a first sampled feature corresponding to the first initial image feature. Perform upsampling on the second initial image feature to obtain a second sampled feature corresponding to the second initial image feature; the first sampled feature and the second sampled feature have the same feature resolution. Assume that the feature resolutions of the first sampled feature and the second sampled feature are both h*w*c, that is, the width of the first sampled feature is h, the height is w, and the number of channels is c, and the width of the second sampled feature is h, the height is w, and the number of channels is c, where h is a positive integer, w is a positive integer, and c is a positive integer. Among them, obtain the first resolution of the first initial image feature and the second resolution of the second initial image feature, determine the feature resolution based on the first resolution and the second resolution. The computer device can perform upsampling on the first initial image feature based on the feature resolution, and can perform upsampling on the second initial image feature based on the feature resolution. Optionally, the computer device can determine the larger resolution among the first resolution and the second resolution as the feature resolution. For example, if the first resolution is greater than the second resolution, the first resolution is determined as the feature resolution; if the second resolution is greater than the first resolution, the second resolution is determined as the feature resolution; or, the computer device can determine the least common multiple of the first resolution and the second resolution as the feature resolution. Optionally, the least common multiple can be a default least common multiple, including but not limited to the least common multiple, twice the least common multiple, etc. Perform feature concatenation on the first sampled feature and the second sampled feature to obtain a concatenated feature. Optionally, the computer device can perform feature concatenation on the first sampled feature and the second sampled feature based on the feature channels to obtain a concatenated feature. Simply put, the feature channels of the first sampled feature and the second sampled feature can be concatenated to obtain a concatenated feature, and the resolution of the concatenated feature is h*w*2c. Perform feature information fusion on the concatenated feature through a feature fusion convolutional layer to obtain an image fusion feature corresponding to the concatenated feature. Specifically, perform convolutional processing on the concatenated feature through a feature fusion convolutional layer to achieve feature information fusion of the concatenated feature, and then obtain an image fusion feature corresponding to the concatenated feature. Among them, the image fusion feature carries semantic information and detail information of the image to be processed, that is, it fuses the high-level features and low-level features of the image to be processed.

[0098] Step S302: Perform convolutional processing on the image fusion feature to obtain a target image feature corresponding to the image to be processed, and determine a target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature.

[0099] In an embodiment of the present application, a computer device may perform convolution processing on an image fusion feature by using a feature separation convolution layer to obtain an image separation feature, and perform activation processing on the image separation feature to obtain a feature distribution map, where the resolution of the feature distribution map is the same as that of the image fusion feature. Based on the feature distribution map, weighted processing is performed on the image fusion feature to obtain a target image feature corresponding to the image to be processed. Specifically, assuming that the resolutions of the feature distribution map and the image fusion feature are both h1*w1*c1, the i-th first eigenvalue in the feature distribution map is used as the weight of the i-th second eigenvalue in the image fusion feature, and weighted processing is performed on the image fusion feature to obtain a target image feature corresponding to the image to be processed. This target image feature can be considered as the image feature obtained after preliminary denoising processing of the image to be processed, and i is a positive integer. Further, the computer device may determine the feature difference between the image fusion feature and the target image feature as the initial noise feature, that is, the image fusion feature may be subtracted from the target image feature to obtain the initial noise feature.

[0100] Further, the computer device may perform channel merging on the initial noise feature to obtain a single-channel target noise feature. Specifically, the computer device may obtain N noise feature channels included in the initial noise feature, and perform feature merging on the noise sub-features located at the same feature position among the d noise sub-features corresponding to the N noise feature channels respectively to obtain a target noise feature corresponding to the image to be processed; N is a positive integer, and d is a positive integer. Briefly, the initial noise feature includes N noise feature channels, and each noise feature channel corresponds to d noise sub-features. Taking the j-th noise sub-feature among the d noise sub-features corresponding to each noise feature channel as an example, feature merging may be performed on the j-th noise sub-features corresponding to the N noise feature channels respectively to obtain the j-th noise eigenvalue, so that d noise eigenvalues can be obtained, and the d noise eigenvalues form the target noise feature, where j is a positive integer less than or equal to d.

[0101] Step S303, perform feature enhancement on the target image feature through the target noise feature to obtain a denoised image feature, and convert the denoised image feature into a denoised and optimized image corresponding to the image to be processed.

[0102] In an embodiment of the present application, the computer device may determine noise distribution information through the target noise feature, and perform feature enhancement on the target image feature based on the noise distribution information to obtain a denoised image feature.

[0103] Specifically, the computer device can obtain the noise distribution information corresponding to the target noise feature, and determine the feature to be enhanced in the target image feature according to the noise distribution information. Among them, the larger the noise value in the target noise feature, the stronger the noise at the position corresponding to the noise value, and the feature at the position corresponding to the noise value in the target image feature should be enhanced. That is to say, the noise distribution information can be determined based on the target noise feature, and the feature to be enhanced that needs to be feature-enhanced in the target image feature can be determined based on the noise distribution information. Feature enhancement is performed on the feature to be enhanced to obtain a denoised image feature, and the denoised image feature is decoded to obtain a denoised and optimized image corresponding to the image to be processed.

[0104] Optionally, the computer device can input the target noise feature and the target image feature into an image optimization network, and obtain the noise distribution information of the target noise feature through the image optimization network; in the mask sub-network of the image optimization network, the image mask feature of the target image feature is obtained based on the noise distribution information, and the image enhancement feature of the target image feature is generated according to the image mask feature; feature enhancement is performed on the target image feature based on the image enhancement feature to obtain a denoised image feature; the denoised image feature is converted into a denoised and optimized image corresponding to the image to be processed. For example, reference can be made to Figure 5 , Figure 5 which is a schematic diagram of an image optimization network architecture provided by an embodiment of the present application. As Figure 5As shown, the image optimization network includes a backbone sub-network and a mask sub-network. A computer device can input the target noise feature and the target image feature into the image optimization network. Herein, the target image feature is denoted as x, and the target noise feature is used as a parameter of the mask sub-network. The noise distribution information of the target noise feature is obtained through the image optimization network. In the mask sub-network of the image optimization network, the image mask feature of the target image feature is obtained based on the noise distribution information, and the image mask feature is denoted as M(x), where M() represents the mask sub-network. The image enhancement feature of the target image feature is generated according to the image mask feature M(x). Optionally, the computer device can use the image mask feature as the weight of the target image feature and perform weighted processing on the target image feature to obtain the image enhancement feature; or the image mask feature can be determined as the image enhancement feature of the target image feature; or the image mask feature can be normalized, and the normalized image mask feature is used to perform weighted processing on the target image feature to obtain the image enhancement feature, etc., which are not limited herein. Further, the target image feature is enhanced based on the image enhancement feature to obtain the denoised image feature. Optionally, the computer device can perform feature processing on the target image feature in the backbone sub-network of the image optimization network to obtain the backbone image feature; generate the image enhancement feature of the target image feature according to the image mask feature; and enhance the backbone image feature based on the image enhancement feature to obtain the denoised image feature.

[0105] For example Figure 5 As shown in, the target image feature is processed in the backbone sub-network to obtain the backbone image feature, which is denoted as T(x), where T() represents the backbone sub-network. The computer device can also generate the image enhancement feature of the target image feature based on the image mask feature and the backbone image feature. For example Figure 5 As shown in, the image mask feature and the backbone image feature are multiplied in terms of features to obtain the image enhancement feature, which is denoted as M(x)T(x). denotes feature multiplication; the backbone image feature T(x) is enhanced based on the image enhancement feature to obtain the denoised image feature. Specifically, the image enhancement feature and the backbone image feature can be added in terms of features to obtain the denoised image feature. For example, the denoised image feature X = T(x) + M(x)T(x) = (1 + M(x))T(x). Herein denotes feature addition.

[0106] Further optionally, the computer device may input the target noise feature and the target image feature into an image optimization network; the image optimization network includes k optimization sub-networks, and each optimization sub-network includes a mask sub-network and a backbone sub-network. The k optimization sub-networks include a first optimization sub-network and a second optimization sub-network, and k is a positive integer. In the first optimization sub-network, the target image feature is enhanced based on the target noise feature to obtain a first enhanced feature, and the target noise feature is updated based on the feature enhancement result to obtain a first noise feature; in the second optimization sub-network, the first enhanced feature is enhanced based on the first noise feature to obtain a denoised image feature; the denoised image feature is converted into a denoised optimized image corresponding to the image to be processed.

[0107] For example, please refer to Figure 6 , Figure 6 which is another schematic diagram of the image optimization network architecture provided by the embodiments of the present application. As Figure 6 shown, the image optimization network includes k optimization sub-networks, namely optimization sub-network 1, optimization sub-network 2, …, and optimization sub-network k. The computer device may input the target noise feature and the target image feature into the image optimization network. In optimization sub-network 1, the target image feature is enhanced based on the target noise feature to obtain a first enhanced feature, and the target noise feature is updated based on the feature enhancement result to obtain a first noise feature. The generation process of the first enhanced feature may refer to Figure 5 the generation process of the denoised image feature in. That is to say, in the mask sub-network of optimization sub-network 1, the image mask feature 1 of the target image feature is obtained, the image enhancement feature 1 of the target image feature is generated according to the image mask feature 1, the target image feature is enhanced based on the image enhancement feature 1 to obtain a first enhanced feature, and the target noise feature is updated based on the feature enhancement result to obtain a first noise feature. The feature enhancement result refers to the relevant information of the feature enhancement of the target image feature. After the target image feature is enhanced, it means that the target image feature has been repaired. Therefore, when further optimizing the target image feature subsequently, the degree of feature enhancement required can be reduced accordingly. In other words, the computer device may dynamically update the target noise feature based on the feature enhancement result. Similarly, in optimization sub-network 2, the first enhanced feature is enhanced based on the first noise feature to obtain a second enhanced feature, and the first noise feature is updated according to the feature enhancement result of optimization sub-network 2 to obtain a second noise feature; …; in optimization sub-network k, the (k-1)th enhanced feature is enhanced based on the (k-1)th noise feature to obtain a denoised image feature.

[0108] Specifically, it may refer to Figure 7 , Figure 7It is a schematic flowchart of an image optimization method provided by an embodiment of the present application. As Figure 7 shown, the method includes the following steps:

[0109] Step S701, input the target noise feature and the target image feature into the image optimization network.

[0110] Step S702, u = 1.

[0111] In the embodiment of the present application, u is set, where u is a positive integer less than or equal to k, used to represent the layer number of the current optimization sub-network being processed, and the initial value of u is set to 1.

[0112] Step S703, in the first optimization sub-network, perform feature enhancement on the target image feature based on the target noise feature to obtain the first enhanced feature, and update the target noise feature based on the feature enhancement result to obtain the first noise feature.

[0113] In the embodiment of the present application, it can also be considered that in the u-th optimization sub-network, where u is 1 at this time.

[0114] Step S704, u++.

[0115] In the embodiment of the present application, the value of u is incremented by 1, used to represent processing based on the next optimization sub-network.

[0116] Step S705, in the u-th optimization sub-network, perform feature enhancement on the (u - 1)-th enhanced feature based on the (u - 1)-th noise feature to obtain the u-th enhanced feature, and update the (u - 1)-th noise feature based on the feature enhancement result to obtain the u-th noise feature.

[0117] Step S706, u = k?

[0118] In the embodiment of the present application, it is detected whether u is equal to k. If u is equal to k, it means that the current optimization sub-network being processed is the k-th optimization sub-network, that is, the last optimization sub-network in the image optimization network, and step S707 is executed; if u is not equal to k, it means that the current optimization sub-network being processed is not the k-th optimization sub-network, and return to execute step S704 to call the next optimization sub-network.

[0119] Step S707, determine the u-th enhanced feature as the denoised image feature.

[0120] In an embodiment of the present application, a to-be-processed image is obtained, a first initial image feature and a second initial image feature of the to-be-processed image are extracted, the first initial image feature and the second initial image feature are feature-fused to obtain an image fusion feature; the resolution of the first initial image feature is greater than that of the second initial image feature; the image fusion feature is subjected to convolution processing to obtain a target image feature corresponding to the to-be-processed image, and based on the image fusion feature and the target image feature, a target noise feature corresponding to the to-be-processed image is determined; the target image feature is feature-enhanced by the target noise feature to obtain a denoised image feature, and the denoised image feature is converted into a denoised and optimized image corresponding to the to-be-processed image. Through the above process, the computer device can extract the intermediate features (i.e., the first initial image feature and the second initial image feature) of the to-be-processed image, rather than directly obtaining the final features of the to-be-processed image, so that the computer device can obtain the features of the to-be-processed image in different dimensions, that is, various information of the to-be-processed image from local to global can be obtained, improving the comprehensiveness of data acquisition. Moreover, based on the extracted first initial image feature and second initial image feature, the to-be-processed image can be preliminarily denoised, and based on the obtained target noise feature, the target image feature obtained by the preliminary denoising can be optimized, so as to realize the denoising process of the to-be-processed image, and further improve the efficiency of image denoising. Since the above process realizes the denoising process of the image based on the features of the image itself, the denoising process of images with different noises can be realized based on the above process, improving the generalization of image denoising.

[0121] Further, please refer to Figure 8 , Figure 8 which is an image denoising network architecture diagram provided by an embodiment of the present application. As Figure 8 shown, the computer device can input the to-be-processed image into the image denoising model. In the feature extraction network of the image denoising model, the first initial image feature and the second initial image feature of the to-be-processed image are extracted, and the first initial image feature and the second initial image feature are feature-fused to obtain an image fusion feature; in the feature separation network of the image denoising model, the image fusion feature is subjected to convolution processing to obtain a target image feature corresponding to the to-be-processed image, and based on the image fusion feature and the target image feature, an initial noise feature is determined; in the noise sub-network of the image denoising model, the initial noise feature is subjected to channel feature fusion to obtain a target noise feature; in the image optimization network of the image denoising model, the target image feature is feature-enhanced based on the target noise feature to obtain a denoised image feature, and the denoised image feature is converted into a denoised and optimized image corresponding to the to-be-processed image.

[0122] Further, it can be referred to Figure 9 , Figure 9It is a flowchart of a method for training an image denoising model provided by an embodiment of the present application. As Figure 9 shown, the method includes the following steps:

[0123] Step S901: Obtain a noisy image sample and a denoising ground truth map corresponding to the noisy image sample.

[0124] In an embodiment of the present application, a computer device can collect a denoising ground truth map, obtain noise parameters, and add the noise parameters to the denoising ground truth map to obtain a noisy image sample corresponding to the denoising ground truth map. Optionally, the number of the noise parameters can be p, and the noise types corresponding to the p noise parameters are not completely the same. p is a positive integer, and the noise types include but are not limited to reflection noise type, rain noise type, fog noise type, or light noise type, etc. Optionally, the computer device can also directly obtain a noisy image sample and a denoising ground truth map corresponding to the noisy image sample from an existing image library. Among them, the obtaining methods of the noisy image sample and the denoising ground truth map corresponding to the noisy image sample are not limited to the above methods, and can also be obtained by other methods, which are not limited here.

[0125] Step S902: In the initial feature extraction network of the initial image denoising model, extract the first initial sample image feature and the second initial sample image feature of the noisy image sample, and perform feature fusion on the first initial sample image feature and the second initial sample image feature to obtain a sample fusion feature.

[0126] In an embodiment of the present application, reference can be made to Figure 3 the specific description shown in step S301 in Figure 3 which is not limited here. In other words, the usage process and the training process of the model are the same in the way of processing the data input into the model. Therefore, reference can be directly made to

[0127] the specific description shown in step S301 in

[0128] In an embodiment of the present application, reference can be made to Figure 3 the specific description of step S302 in

[0129] Step S904: In the initial image optimization network of the initial image denoising model, perform feature enhancement on the target sample image feature through the sample noise feature to obtain a sample denoised image feature, and convert the sample denoised image feature into a denoising optimized image sample corresponding to the noisy image sample.

[0130] In the embodiments of the present application, reference may be made to Figure 3 the specific description of step S303 in

[0131] Step S905: Adjust the parameters of the initial image denoising model according to the denoising-optimized image samples, sample noise features, and denoising ground truth maps to obtain an image denoising model.

[0132] In the embodiments of the present application, a computer device may generate noise image samples based on the noisy image samples and denoising ground truth maps. Specifically, the computer device may generate noise image samples according to the feature differences between the noisy image samples and the denoising ground truth maps. Generate a first loss function based on the sample noise features and the noise image samples, and generate a second loss function based on the denoising-optimized image samples and the denoising ground truth maps; wherein, the first loss function may be the feature difference between the sample noise features and the noise image samples, or the mean square error between the two, etc., which is not limited herein. The first loss function may be, but is not limited to, a logarithmic loss function, a square loss function, a perceptual loss function, a cross-entropy loss function, or an L1 reconstruction loss function, etc.; the second loss function may be the feature difference between the denoising-optimized image samples and the denoising ground truth maps, or the mean square error between the two, etc., which is not limited herein. The second loss function may be, but is not limited to, a logarithmic loss function, a square loss function, a perceptual loss function, a cross-entropy loss function, or an L1 reconstruction loss function, etc. Optionally, the first loss function may be obtained by combining one or at least two loss functions, and the second loss function may be obtained by combining one or at least two loss functions. Adjust the parameters of the initial image denoising model based on the first loss function and the second loss function to obtain an image denoising model.

[0133] Optionally, the L1 reconstruction loss function may be as shown in Formula ①.

[0134] <![CDATA[L l1 = ||Y - Y gt ||]]> ①

[0135] where Y is the network prediction map, that is, the map output by the model, and Y gt is the ground truth map.

[0136] Optionally, the perceptual loss function may be as shown in Formula ②.

[0137]

[0138] where C is the number of channels, H is the feature height, W is the feature width, l is the current feature level, L is the total number of feature levels, and φ l (Y) is the feature map corresponding to the network prediction map, and φ l (Y gt ) is the feature map corresponding to the ground truth map.

[0139] Among them, the conditional adversarial loss function can be seen in Formula ③ as follows.

[0140]

[0141] Among them, B is the denoised ground truth map, I is the noisy image sample, D() is used to obtain the discriminative features, which is used here to obtain the discriminative features between the denoised ground truth map and the noisy image sample. Through this conditional adversarial loss function, the difference between the denoised ground truth map and the denoised optimized image sample is reduced.

[0142] Optionally, the computer device can obtain the first L1 reconstruction loss function between the sample noise features and the noisy image sample, obtain the first loss weight, and generate the first loss function according to the first loss function and the first L1 reconstruction loss function. The first loss function can be seen in Formula ④ as follows.

[0143]

[0144] Among them, refers to the first loss weight. The superscript n is used to indicate that this parameter belongs to the first loss function and is used to optimize and adjust the prediction result of the noise. The subscript l1 is used to indicate that this parameter is related to the L1 reconstruction loss function. Among them, Y in Formula ① is determined as the sample noise features, and Y gt in Formula ① is determined as the noisy image sample, and the first L1 reconstruction loss function can be obtained L noise is used to represent the first loss function.

[0145] Optionally, the computer device can obtain the second L1 reconstruction loss function, the perceptual loss function, and the conditional adversarial loss function between the denoised optimized image sample and the denoised ground truth map, obtain the second loss weight corresponding to the second L1 reconstruction loss function, the third loss weight corresponding to the perceptual loss function, and the fourth loss weight corresponding to the conditional adversarial loss function, and generate the second loss function based on the second loss weight, the second L1 reconstruction loss function, the third loss weight, the perceptual loss function, the fourth loss weight, and the conditional adversarial loss function. The second loss function can be seen in Formula ⑤ as follows.

[0146]

[0147] Among them, refers to the second loss weight, refers to the third loss weight, Refers to the fourth loss weight. The superscript r is used to indicate that this parameter belongs to the second loss function and is used to adjust the optimization of the image. The subscript per is used to indicate that this parameter is related to the perceptual loss function, and the subscript adv is used to indicate that this parameter is related to the conditional adversarial loss function. Among them, Y in formula ① is determined as the denoised optimized image sample, and Y gt is determined as the denoised ground truth map, and the second L1 reconstruction loss function can be obtained Determine Y in formula ② as the denoised optimized image sample, and Y in formula ② gt is determined as the denoised ground truth map, and the perceptual loss function can be obtained Based on formula ③, the conditional adversarial loss function is obtained Among them, L refine is used to represent the second loss function.

[0148] Furthermore, please refer to Figure 10 , Figure 10 which is a schematic diagram of an image processing device provided by an embodiment of the present application. The image processing device may be a computer program (including program codes, etc.) running in a computer device. For example, the image processing device may be an application software; the device may be used to execute the corresponding steps in the method provided by the embodiment of the present application. As Figure 10 shown, the image processing device 1000 may be used for Figure 3 the computer device corresponding to the corresponding embodiment. Specifically, the device may include: an image acquisition module 11, a feature extraction module 12, a feature fusion module 13, an image extraction module 14, a noise determination module 15, and an image denoising module 16.

[0149] The image acquisition module 11 is used to acquire the image to be processed;

[0150] The feature extraction module 12 is used to extract the first initial image feature and the second initial image feature of the image to be processed;

[0151] The feature fusion module 13 is used to perform feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature; the resolution of the first initial image feature is greater than the resolution of the second initial image feature;

[0152] The image extraction module 14 is used to perform convolution processing on the image fusion feature to obtain the target image feature corresponding to the image to be processed;

[0153] The noise determination module 15 is used to determine the target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature;

[0154] An image denoising module 16, which is used to enhance the target image features through the target noise features to obtain denoised image features, and convert the denoised image features into a denoised and optimized image corresponding to the image to be processed.

[0155] Among them, the feature extraction module 12 includes:

[0156] A convolutional layer extraction unit 121, which is used to input the image to be processed into a feature extraction network, and extract the convolutional features corresponding to the image to be processed in each of the f feature extraction convolutional layers in the feature extraction network; f is a positive integer;

[0157] A convolution processing unit 122, which is used to perform convolution processing on the f convolutional features to obtain a first initial image feature and a second initial image feature of the image to be processed.

[0158] Among them, the convolution processing unit 122 includes:

[0159] A convolution division sub-unit 1221, which is used to divide the f feature extraction convolutional layers into a first convolutional layer and a second convolutional layer; the depth of the first convolutional layer in the feature extraction network is less than the depth of the second convolutional layer in the feature extraction network;

[0160] A first convolution sub-unit 1222, which is used to perform convolution processing on the convolutional features corresponding to the first convolutional layer to obtain a first initial image feature of the image to be processed;

[0161] A second convolution sub-unit 1223, which is used to perform convolution processing on the convolutional features corresponding to the second convolutional layer to obtain a second initial image feature of the image to be processed.

[0162] Among them, the feature fusion module 13 includes:

[0163] A first sampling unit 131, which is used to perform upsampling processing on the first initial image feature to obtain a first sampling feature corresponding to the first initial image feature;

[0164] A second sampling unit 132, which is used to perform upsampling processing on the second initial image feature to obtain a second sampling feature corresponding to the second initial image feature; the first sampling feature and the second sampling feature have the same feature resolution;

[0165] A sampling splicing unit 133, which is used to perform feature splicing on the first sampling feature and the second sampling feature to obtain a spliced feature;

[0166] An information fusion unit 134, which is used to perform feature information fusion on the spliced feature through a feature fusion convolutional layer to obtain an image fusion feature corresponding to the spliced feature.

[0167] Among them, the image extraction module 14 includes:

[0168] The feature separation unit 141 is configured to perform convolution processing on the image fusion feature by using a feature separation convolutional layer to obtain an image separation feature, and perform activation processing on the image separation feature to obtain a feature distribution map;

[0169] The feature weighting unit 142 is configured to perform weighting processing on the image fusion feature based on the feature distribution map to obtain a target image feature corresponding to the image to be processed.

[0170] Wherein, the noise determination module 15 includes:

[0171] The difference processing unit 151 is configured to determine the feature difference between the image fusion feature and the target image feature as an initial noise feature;

[0172] The noise merging unit 152 is configured to obtain N noise feature channels included in the initial noise feature, and perform feature merging on the noise sub-features located at the same feature position among the d noise sub-features respectively corresponding to the N noise feature channels to obtain a target noise feature corresponding to the image to be processed; N is a positive integer, and d is a positive integer.

[0173] Wherein, the image denoising module 16 includes:

[0174] The feature determination unit 161 is configured to obtain the noise distribution information corresponding to the target noise feature, and determine the feature to be enhanced in the target image feature according to the noise distribution information;

[0175] The denoising processing unit 162 is configured to perform feature enhancement on the feature to be enhanced to obtain a denoised image feature, and perform decoding processing on the denoised image feature to obtain a denoised optimized image corresponding to the image to be processed.

[0176] Wherein, the image denoising module 16 includes:

[0177] The distribution acquisition unit 163 is configured to input the target noise feature and the target image feature into an image optimization network, and obtain the noise distribution information of the target noise feature through the image optimization network;

[0178] The enhancement generation unit 164 is configured to obtain an image mask feature of the target image feature based on the noise distribution information in the mask sub-network of the image optimization network, and generate an image enhancement feature of the target image feature according to the image mask feature;

[0179] The feature enhancement unit 165 is configured to perform feature enhancement on the target image feature based on the image enhancement feature to obtain a denoised image feature;

[0180] The feature conversion unit 166 is configured to convert the denoised image feature into a denoised optimized image corresponding to the image to be processed.

[0181] Among them, the image denoising module 16 includes:

[0182] A feature input unit 167, configured to input a target noise feature and a target image feature into an image optimization network; the image optimization network includes k optimization sub-networks, the k optimization sub-networks include a first optimization sub-network and a second optimization sub-network, and k is a positive integer;

[0183] A first enhancement unit 168, configured to perform feature enhancement on the target image feature based on the target noise feature in the first optimization sub-network to obtain a first enhanced feature, and update the target noise feature based on the feature enhancement result to obtain a first noise feature;

[0184] A second enhancement unit 169, configured to perform feature enhancement on the first enhanced feature based on the first noise feature in the second optimization sub-network to obtain a denoised image feature;

[0185] The feature conversion unit 166 is configured to convert the denoised image feature into a denoised and optimized image corresponding to the image to be processed.

[0186] An embodiment of the present application provides an image processing apparatus, which can acquire an image to be processed, extract a first initial image feature and a second initial image feature of the image to be processed, perform feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature; the resolution of the first initial image feature is greater than the resolution of the second initial image feature; perform convolution processing on the image fusion feature to obtain a target image feature corresponding to the image to be processed, determine a target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature; perform feature enhancement on the target image feature through the target noise feature to obtain a denoised image feature, and convert the denoised image feature into a denoised and optimized image corresponding to the image to be processed. Through the above process, the computer device can extract the intermediate features (i.e., the first initial image feature and the second initial image feature) of the image to be processed, rather than directly obtaining the final features of the image to be processed, so that the computer device can obtain the features of the image to be processed in different dimensions, that is, it can obtain various information of the image to be processed from local to global, improving the comprehensiveness of data acquisition. Moreover, based on the extracted first initial image feature and second initial image feature, the image to be processed can be preliminarily denoised, and based on the obtained target noise feature, the target image feature obtained by preliminary denoising can be optimized, so as to realize the denoising process of the image to be processed, and further improve the efficiency of image denoising. Since the above process realizes the image denoising process based on the features of the image itself, the denoising process of images carrying different noises can be realized based on the above process, improving the generalization of image denoising.

[0187] Further, please refer toFigure 11 , Figure 11 is a schematic diagram of an image processing device provided by an embodiment of the present application. The image processing device may be a computer program (including program codes, etc.) running on a computer device. For example, the image processing device may be an application software; the device may be used to execute corresponding steps in the method provided by the embodiment of the present application. As Figure 11 shown, the image processing device 1100 may be used for Figure 3 the computer device in the corresponding embodiment. Specifically, the device may include: a sample acquisition module 21, a sample extraction module 22, a sample fusion module 23, a feature processing module 24, a sample optimization module 25, and a model adjustment module 26.

[0188] The sample acquisition module 21 is used to acquire a noisy image sample and a denoising ground truth map corresponding to the noisy image sample;

[0189] The sample extraction module 22 is used to extract a first initial sample image feature and a second initial sample image feature of the noisy image sample in the initial feature extraction network of the initial image denoising model;

[0190] The sample fusion module 23 is used to perform feature fusion on the first initial sample image feature and the second initial sample image feature to obtain a sample fusion feature; the resolution of the first initial sample image feature is greater than the resolution of the second initial sample image feature;

[0191] The feature processing module 24 is used to perform convolution processing on the sample fusion feature in the initial feature separation network of the initial image denoising model to obtain a target sample image feature corresponding to the noisy image sample, and determine a sample noise feature corresponding to the noisy image sample according to the sample fusion feature and the target sample image feature;

[0192] The sample optimization module 25 is used to perform feature enhancement on the target sample image feature through the sample noise feature in the initial image optimization network of the initial image denoising model to obtain a sample denoised image feature, and convert the sample denoised image feature into a denoising optimized image sample corresponding to the noisy image sample;

[0193] The model adjustment module 26 is used to adjust the parameters of the initial image denoising model according to the denoising optimized image sample, the sample noise feature, and the denoising ground truth map to obtain an image denoising model.

[0194] Among them, the model adjustment module 26 includes:

[0195] A sample generation unit 264, which is used to generate a noise image sample according to the noisy image sample and the denoising ground truth map;

[0196] A loss generation unit 262 is configured to generate a first loss function according to the sample noise features and the noise image samples, and generate a second loss function according to the denoised optimized image samples and the denoised ground truth maps;

[0197] A parameter adjustment unit 263 is configured to adjust the parameters of the initial image denoising model based on the first loss function and the second loss function to obtain an image denoising model.

[0198] Optionally, Figure 10 the computer device for image processing in Figure 11 and the computer device for model training in

[0199] See Figure 12 , Figure 12 which is a schematic structural diagram of a computer device provided by an embodiment of the present application. As Figure 12 shown, the computer device in the embodiment of the present application may include: one or more processors 1201, a memory 1202, and an input / output interface 1203. The processor 1201, the memory 1202, and the input / output interface 1203 are connected through a bus 1204. The memory 1202 is used to store a computer program, the computer program includes program instructions, the input / output interface 1203 is used to receive data and output data, such as for data interaction between the computer device and a terminal device; the processor 1201 is used to execute the program instructions stored in the memory 1202.

[0200] Wherein, when the processor 1201 is located in the computer device for image processing, the following operations may be performed:

[0201] Obtain an image to be processed, extract a first initial image feature and a second initial image feature of the image to be processed, perform feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature; the resolution of the first initial image feature is greater than the resolution of the second initial image feature;

[0202] Perform convolutional processing on the image fusion feature to obtain a target image feature corresponding to the image to be processed, and determine a target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature;

[0203] Enhance the target image feature through the target noise feature to obtain a denoised image feature, and convert the denoised image feature into a denoised optimized image corresponding to the image to be processed.

[0204] Wherein, when the processor 1201 is located in the computer device for model training, the following operations may be performed:

[0205] Obtain a noisy image sample and a denoising ground truth map corresponding to the noisy image sample;

[0206] In the initial feature extraction network of the initial image denoising model, extract the first initial sample image feature and the second initial sample image feature of the noisy image sample, and perform feature fusion on the first initial sample image feature and the second initial sample image feature to obtain a sample fusion feature; the resolution of the first initial sample image feature is greater than the resolution of the second initial sample image feature;

[0207] In the initial feature separation network of the initial image denoising model, perform convolution processing on the sample fusion feature to obtain a target sample image feature corresponding to the noisy image sample, and determine a sample noise feature corresponding to the noisy image sample according to the sample fusion feature and the target sample image feature;

[0208] In the initial image optimization network of the initial image denoising model, perform feature enhancement on the target sample image feature through the sample noise feature to obtain a sample denoised image feature, and convert the sample denoised image feature into a denoising optimized image sample corresponding to the noisy image sample;

[0209] According to the denoising optimized image sample, the sample noise feature, and the denoising ground truth map, adjust the parameters of the initial image denoising model to obtain an image denoising model.

[0210] In some feasible implementation manners, the processor 1201 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0211] The memory 1202 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1201 and the input / output interface 1203. A part of the memory 1202 may also include a non-volatile random access memory. For example, the memory 1202 may also store information about the device type.

[0212] In specific implementation, the computer device may execute through its built-in various functional modules as the Figure 3 or Figure 9The implementation manners provided in each step can be specifically referred to in this Figure 3 or Figure 9 The implementation manners provided in each step, which will not be elaborated here.

[0213] An embodiment of the present application provides a computer device, including: a processor, an input / output interface, and a memory. The processor obtains a computer program in the memory and executes each step of the method shown in this Figure 3 to perform an image processing operation. The embodiment of the present application realizes obtaining an image to be processed, extracting a first initial image feature and a second initial image feature of the image to be processed, performing feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature; the resolution of the first initial image feature is greater than that of the second initial image feature; performing convolution processing on the image fusion feature to obtain a target image feature corresponding to the image to be processed, and determining a target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature; enhancing the target image feature through the target noise feature to obtain a denoised image feature, and converting the denoised image feature into a denoised and optimized image corresponding to the image to be processed. Through the above process, the computer device can extract intermediate features (i.e., the first initial image feature and the second initial image feature) of the image to be processed, rather than directly obtaining the final features of the image to be processed, so that the computer device can obtain features of the image to be processed in different dimensions, that is, it can obtain various information of the image to be processed from local to global, improving the comprehensiveness of data acquisition. Moreover, based on the extracted first initial image feature and second initial image feature, the image to be processed can be preliminarily denoised, and the target image feature obtained by preliminary denoising can be optimized based on the obtained target noise feature, so as to realize the denoising process of the image to be processed, thereby improving the efficiency of image denoising. Since the above process realizes the denoising process of the image based on the features of the image itself, the denoising process of images with different noises can be realized based on the above process, improving the generalization of image denoising.

[0214] The embodiment of the present application further provides a computer-readable storage medium storing a computer program, which is suitable for being loaded and executed by the processor Figure 3 or Figure 9 The image processing method provided in each step, which can be specifically referred to in this Figure 3 or Figure 9The implementation manners provided in each step are not described herein again. In addition, the beneficial effects of adopting the same method are not described again. For the technical details not disclosed in the embodiments of the computer-readable storage medium involved in the present application, please refer to the description of the method embodiments of the present application. As an example, the computer program can be deployed to be executed on a computer device, or on multiple computer devices located at one place, or on multiple computer devices distributed at multiple places and interconnected through a communication network.

[0215] The computer-readable storage medium can be the image processing device provided in any of the foregoing embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store the data that has been output or will be output.

[0216] The embodiments of the present application also provide a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 3 or Figure 9 the methods provided in the various alternative manners in, and realizes the extraction of the intermediate features (i.e., the first initial image feature and the second initial image feature) of the image to be processed, rather than directly obtaining the final feature of the image to be processed, so that the computer device can obtain the features of the image to be processed in different dimensions, that is, can obtain various information of the image to be processed from local to global, improving the comprehensiveness of data acquisition. Moreover, based on the extracted first initial image feature and second initial image feature, the image to be processed can be preliminarily denoised, and based on the obtained target noise feature, the target image feature obtained by the preliminary denoising can be optimized, so as to realize the denoising process of the image to be processed, and further improve the efficiency of image denoising. Since the above process realizes the denoising process of the image based on the features of the image itself, the denoising process of the image carrying different noises can be realized based on the above process, improving the generalization of image denoising.

[0217] In the description, claims, and drawings of the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally further include steps or modules not listed, or may optionally further include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0218] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in this description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0219] The methods and related apparatuses provided by the embodiments of the present application are described with reference to the method flowcharts and / or structural schematic diagrams provided by the embodiments of the present application. Specifically, each process and / or block of the method flowchart and / or structural schematic diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable image processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable image processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or structural schematic Figure 1 one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable image processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or structural schematic Figure 1 one block or multiple blocks. These computer program instructions can also be loaded onto a computer or other programmable image processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or structural schematic one block or multiple blocks.

[0220] The steps in the method of the embodiment of the present application can be adjusted in sequence, combined, and deleted according to actual needs.

[0221] The modules in the device of the embodiment of the present application can be combined, divided, and deleted according to actual needs.

[0222] The above-disclosed are only the preferred embodiments of the present application. Of course, the scope of the rights of the present application cannot be limited thereby. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. An image processing method, characterized in that, the method includes: obtaining an image to be processed, extracting a first initial image feature and a second initial image feature of the image to be processed, and performing feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature; the resolution of the first initial image feature is greater than the resolution of the second initial image feature; performing convolution processing on the image fusion feature by using a feature separation convolutional layer to obtain an image separation feature, and performing activation processing on the image separation feature to obtain a feature distribution map; performing weighted processing on the image fusion feature based on the feature distribution map to obtain a target image feature corresponding to the image to be processed; determining a target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature; performing feature enhancement on the target image feature by using the target noise feature to obtain a denoised image feature, and converting the denoised image feature into a denoised and optimized image corresponding to the image to be processed.

2. The method according to claim 1, characterized in that, the extracting the first initial image feature and the second initial image feature of the image to be processed includes: inputting the image to be processed into a feature extraction network, and passing through f feature extraction convolutional layers in the feature extraction network to extract convolutional features corresponding to the image to be processed in each feature extraction convolutional layer; f is a positive integer; performing convolution processing on the f convolutional features to obtain the first initial image feature and the second initial image feature of the image to be processed.

3. The method according to claim 2, characterized in that, the performing convolution processing on the f convolutional features to obtain the first initial image feature and the second initial image feature of the image to be processed includes: dividing the f feature extraction convolutional layers into a first convolutional layer and a second convolutional layer; the depth of the first convolutional layer in the feature extraction network is less than the depth of the second convolutional layer in the feature extraction network; performing convolution processing on the convolutional features corresponding to the first convolutional layer to obtain the first initial image feature of the image to be processed; performing convolution processing on the convolutional features corresponding to the second convolutional layer to obtain the second initial image feature of the image to be processed.

4. The method according to claim 1, characterized in that, the performing feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature includes: performing upsampling processing on the first initial image feature to obtain a first sampling feature corresponding to the first initial image feature; performing upsampling processing on the second initial image feature to obtain a second sampling feature corresponding to the second initial image feature; the first sampling feature and the second sampling feature have the same feature resolution; performing feature splicing on the first sampling feature and the second sampling feature to obtain a spliced feature; performing feature information fusion on the spliced feature by using a feature fusion convolutional layer to obtain an image fusion feature corresponding to the spliced feature.

5. The method according to claim 1, characterized in that, Determining the target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature includes: Determining the feature difference between the image fusion feature and the target image feature as the initial noise feature; Obtaining N noise feature channels included in the initial noise feature, and performing feature merging on the noise sub-features located at the same feature position among the d noise sub-features corresponding to the N noise feature channels, to obtain the target noise feature corresponding to the image to be processed; N is a positive integer, and d is a positive integer.

6. The method according to claim 1, wherein, Performing feature enhancement on the target image feature through the target noise feature to obtain a denoised image feature, and converting the denoised image feature into a denoised and optimized image corresponding to the image to be processed, includes: Obtaining the noise distribution information corresponding to the target noise feature, and determining the feature to be enhanced in the target image feature according to the noise distribution information; Performing feature enhancement on the feature to be enhanced to obtain a denoised image feature, and performing decoding processing on the denoised image feature to obtain the denoised and optimized image corresponding to the image to be processed.

7. The method according to claim 1, wherein, Performing feature enhancement on the target image feature through the target noise feature to obtain a denoised image feature, and converting the denoised image feature into a denoised and optimized image corresponding to the image to be processed, includes: Inputting the target noise feature and the target image feature into an image optimization network, and obtaining the noise distribution information of the target noise feature through the image optimization network; In the mask sub-network of the image optimization network, obtaining the image mask feature of the target image feature based on the noise distribution information, and generating an image enhancement feature of the target image feature according to the image mask feature; Performing feature enhancement on the target image feature based on the image enhancement feature to obtain a denoised image feature; Converting the denoised image feature into a denoised and optimized image corresponding to the image to be processed.

8. The method according to claim 1, wherein, Performing feature enhancement on the target image feature through the target noise feature to obtain a denoised image feature, and converting the denoised image feature into a denoised and optimized image corresponding to the image to be processed, includes: Inputting the target noise feature and the target image feature into an image optimization network; the image optimization network includes k optimization sub-networks, the k optimization sub-networks include a first optimization sub-network and a second optimization sub-network, and k is a positive integer; In the first optimization sub-network, performing feature enhancement on the target image feature based on the target noise feature to obtain a first enhancement feature, and updating the target noise feature based on the feature enhancement result to obtain a first noise feature; In the second optimization sub-network, performing feature enhancement on the first enhancement feature based on the first noise feature to obtain a denoised image feature; Converting the denoised image feature into a denoised and optimized image corresponding to the image to be processed.

9. An image processing method, wherein, The method includes: Obtaining a noisy image sample and a denoising ground truth map corresponding to the noisy image sample; In an initial feature extraction network of an initial image denoising model, extracting a first initial sample image feature and a second initial sample image feature of the noisy image sample, and performing feature fusion on the first initial sample image feature and the second initial sample image feature to obtain a sample fusion feature; the resolution of the first initial sample image feature is greater than that of the second initial sample image feature; In an initial feature separation network of the initial image denoising model, performing convolution processing on the sample fusion feature to obtain an image separation feature for the sample fusion feature, and performing activation processing on the image separation feature for the sample fusion feature to obtain a sample feature distribution map; Performing weighted processing on the sample fusion feature based on the sample feature distribution map to obtain a target sample image feature corresponding to the noisy image sample; Determining a sample noise feature corresponding to the noisy image sample according to the sample fusion feature and the target sample image feature; In an initial image optimization network of the initial image denoising model, enhancing the target sample image feature through the sample noise feature to obtain a sample denoised image feature, and converting the sample denoised image feature into a denoising optimized image sample corresponding to the noisy image sample; Adjusting parameters of the initial image denoising model according to the denoising optimized image sample, the sample noise feature, and the denoising ground truth map to obtain an image denoising model.

10. The method according to claim 9, wherein, the adjusting parameters of the initial image denoising model according to the denoising optimized image sample, the sample noise feature, and the denoising ground truth map to obtain an image denoising model includes: Generating a noise image sample according to the noisy image sample and the denoising ground truth map; Generating a first loss function according to the sample noise feature and the noise image sample, and generating a second loss function according to the denoising optimized image sample and the denoising ground truth map; Adjusting parameters of the initial image denoising model based on the first loss function and the second loss function to obtain an image denoising model.

11. An image processing apparatus, wherein, the apparatus includes: An image acquisition module for acquiring an image to be processed; A feature extraction module for extracting a first initial image feature and a second initial image feature of the image to be processed; A feature fusion module for performing feature fusion on the first initial image feature and the second initial image feature to obtain an image fusion feature; the resolution of the first initial image feature is greater than that of the second initial image feature; An image extraction module for performing convolution processing on the image fusion feature by using a feature separation convolutional layer to obtain an image separation feature, and performing activation processing on the image separation feature to obtain a feature distribution map; The image extraction module is further configured to perform weighted processing on the image fusion feature based on the feature distribution map to obtain a target image feature corresponding to the image to be processed; A noise determination module, configured to determine a target noise feature corresponding to the image to be processed according to the image fusion feature and the target image feature; An image denoising module, configured to perform feature enhancement on the target image feature through the target noise feature to obtain a denoised image feature, and convert the denoised image feature into a denoised and optimized image corresponding to the image to be processed.

12. An image processing apparatus, characterized in that the apparatus includes: A sample acquisition module, configured to acquire a noisy image sample and a denoising ground truth map corresponding to the noisy image sample; A sample extraction module, configured to extract a first initial sample image feature and a second initial sample image feature of the noisy image sample in an initial feature extraction network of an initial image denoising model; A sample fusion module, configured to perform feature fusion on the first initial sample image feature and the second initial sample image feature to obtain a sample fusion feature; the resolution of the first initial sample image feature is greater than the resolution of the second initial sample image feature; A feature processing module, configured to perform convolution processing on the sample fusion feature in an initial feature separation network of the initial image denoising model to obtain an image separation feature for the sample fusion feature, and perform activation processing on the image separation feature for the sample fusion feature to obtain a sample feature distribution map; The feature processing module is further configured to perform weighted processing on the sample fusion feature based on the sample feature distribution map to obtain a target sample image feature corresponding to the noisy image sample, and determine a sample noise feature corresponding to the noisy image sample according to the sample fusion feature and the target sample image feature; A sample optimization module, configured to perform feature enhancement on the target sample image feature through the sample noise feature in an initial image optimization network of the initial image denoising model to obtain a sample denoised image feature, and convert the sample denoised image feature into a denoised and optimized image sample corresponding to the noisy image sample; A model adjustment module, configured to adjust parameters of the initial image denoising model according to the denoised and optimized image sample, the sample noise feature, and the denoising ground truth map to obtain an image denoising model.

13. A computer device, characterized in that it includes a processor, a memory, and an input / output interface; the processor is respectively connected to the memory and the input / output interface, wherein the input / output interface is used to receive data and output data, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1-8, or executes the method according to any one of claims 9-10.

14. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the method according to any one of claims 1-8, or executes the method according to any one of claims 9-10.

Images