Image filtering method and device based on convolutional neural network, equipment and medium

Abstract

The application belongs to the technical field of image processing, and specifically discloses an image filtering method and device based on a convolutional neural network, equipment and a medium. Through the application, the target value of the noise loss function and the noise network parameter is determined, and a target noise extraction network is constructed in combination with the noise data type; noise extraction is performed on the to-be-filtered image based on the target noise extraction network; the target noise data and the to-be-filtered image are fused, and the fused double-channel image is filtered based on the target denoising network. In the foregoing manner, the characteristics of the noise data are fully considered, the target noise extraction network and the target denoising network with the advantages of low computational complexity, feature learning and nonlinearity are constructed based on the image noise characteristics and the convolutional neural network theory, noise extraction is performed based on the target noise extraction network, and the double-channel image is filtered based on the target denoising network, so that the accuracy of image filtering can be effectively improved and the complexity of image filtering can be reduced.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and more specifically, relates to image filtering methods, apparatus, devices and media based on convolutional neural networks. Background Technology

[0002] Sufficient image clarity and information content are crucial for image analysis. After image acquisition, preprocessing, such as filtering, is necessary. Currently, common methods for image filtering rely on traditional transform domain methods and deep learning algorithms. Traditional transform domain methods transform the acquired image data into an appropriate domain to separate image data from noisy data. However, this requires manually abstracting an ideal model from the actual problem for processing, which inevitably introduces some degree of bias. Deep learning algorithms require constructing complex network structures (i.e., complex nonlinear functions), relying on data-driven fitting of complex real-world problems, and demanding powerful GPUs during training. Therefore, the accuracy and complexity of image filtering using these methods are relatively low. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this application aims to provide an image filtering method, apparatus, device, and medium based on convolutional neural networks. The goal is to solve the problems of low accuracy and high complexity in image filtering caused by human-induced abstraction biases, complex network structures, and the need for powerful GPUs during training.

[0004] To achieve the above objectives, in a first aspect, this application provides an image filtering method based on a convolutional neural network, comprising: Obtain the image to be filtered, and determine the noise data type and image data type of the image to be filtered respectively; The target values ​​of the noise loss function and the noise network parameters are determined based on the image data type, and a target noise extraction network is constructed based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters. Based on the target noise extraction network, noise is extracted from the image to be filtered to obtain target noise data; The target noise data and the image to be filtered are fused together, and the fused dual-channel image is filtered based on the target denoising network.

[0005] In one embodiment, the step of determining the target values ​​of the noise loss function and the noise network parameters based on the image data type, and constructing a target noise extraction network based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters, includes: The asymmetric loss function and total variation constraint term of the noise extraction network are determined according to the image data type, respectively. The noise loss function is calculated based on the asymmetric loss function, the trade-off parameter of the asymmetric loss function, the total variation constraint term, and the trade-off parameter of the total variation constraint term. Obtain the parameters of the noisy network and the simulation test values ​​of the network parameters; Based on the controlled variable algorithm, the target values ​​of the noise network parameters are determined according to the simulation test values. A target noise extraction network is constructed based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters.

[0006] In one embodiment, the step of constructing a target noise extraction network based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters includes: Calculate the complexity feature value of the noise data type, and determine the network architecture based on the complexity feature value, wherein the network architecture includes the number of fully convolutional layers, the size of each convolutional kernel, and the number of feature channels; An initial noise extraction network is constructed based on the described network architecture; The sample training images are input into the initial noise extraction network, and the estimated noise data output by the initial noise extraction network is obtained. The difference between the estimated noise data and the actual noise data is determined, and the initial network is backpropagated and optimized based on the difference, the noise loss function, and the target values ​​of the noise network parameters to obtain the target noise extraction network.

[0007] In one embodiment, the step of determining the asymmetric loss function and total variation constraint term of the noise extraction network according to the image data type includes: Based on the image data type, obtain the actual noise data, estimated noise data, horizontal gradient operator, and vertical gradient operator at each pixel. The actual noise value is calculated based on the actual noise data, and the estimated noise value is calculated based on the estimated noise data; Based on the noise level estimation function, the asymmetric loss function of the noise extraction network is determined according to the actual noise value, the estimated noise value, and the target value of the noise network parameters. The total variation constraint term is determined based on the gradient operator in the horizontal direction, the gradient operator in the vertical direction, and the estimated noise value.

[0008] In one embodiment, the step of fusing the target noise data and the image to be filtered, and filtering the fused dual-channel image based on a target denoising network, includes: The target noise data and the image to be filtered are fused together; Obtain noise-free target data and denoising results; The denoising loss function is determined based on the noise-free target data and the denoising result, and the initial denoising network is iteratively updated based on the denoising loss function and the target values ​​of the denoising network parameters to obtain the target denoising network. The fused dual-channel image is filtered based on the target denoising network.

[0009] In one embodiment, the step of iteratively updating the initial denoising network based on the denoising loss function and the target values ​​of the denoising network parameters to obtain the target denoising network includes: Acquire sample training images and sample training noise data; The encoder in the initial denoising network is used to downsample the sample training image and the sample training noise data, and features are extracted from the downsampled data. The extracted features are then compiled into denoising codes. The denoised code is upsampled based on the decoder in the initial denoising network, the upsampled data is reconstructed for features, and the predicted filtered image is determined based on the reconstructed features. Calculate the mean square error between the predicted filtered image and the actual filtered image; With the goal of minimizing the mean square error, the initial denoising network is iteratively updated based on the parameters of the denoising network using a gradient backpropagation strategy to obtain the target denoising network.

[0010] Secondly, this application provides an image filtering device based on a convolutional neural network, comprising: A determination module is used to acquire the image to be filtered and determine the noise data type and image data type of the image to be filtered, respectively. The determining module is further configured to determine the target values ​​of the noise loss function and the noise network parameters according to the image data type, and construct a target noise extraction network according to the noise data type, the noise loss function and the target values ​​of the noise network parameters; The extraction module is used to extract noise from the image to be filtered based on the target noise extraction network to obtain target noise data; The filtering module is used to fuse the target noise data and the image to be filtered, and to filter the fused dual-channel image based on the target denoising network.

[0011] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.

[0012] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0013] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0014] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0015] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) This application fully considers the characteristics of noise data. Based on the characteristics of image noise and the theory of convolutional neural networks, it constructs a target denoising network according to the noise data type, noise loss function and target value of noise network parameters. It iteratively updates the initial denoising network according to the denoising loss function and target value of denoising network parameters to obtain the target denoising network. Both the target noise extraction network and the target denoising network have advantages such as low computational complexity, feature learning and nonlinearity. After extracting the target noise data based on the target noise extraction network, the fused dual-channel image can be filtered based on the target denoising network, thereby effectively improving the accuracy of image filtering and reducing the complexity of image filtering.

[0016] (2) This application also introduces a total variation constraint term to make the estimated noise data smoother. The noise loss function is calculated based on the asymmetric loss function, the trade-off parameter of the asymmetric loss function, the total variation constraint term, and the trade-off parameter of the total variation constraint term. In addition, to make the noise extraction network more robust, this application uses ReLU as the activation function after each convolutional layer. Its advantages are low computational complexity, which reduces the complexity of image filtering, and can effectively avoid the occurrence of gradient vanishing and gradient exploding.

[0017] In summary, this application acquires an image to be filtered and determines the noise data type and image data type of the image to be filtered; it determines the target values ​​of the noise loss function and noise network parameters based on the image data type, and constructs a target noise extraction network based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters; it extracts noise from the image to be filtered based on the target noise extraction network to obtain target noise data; it fuses the target noise data with the image to be filtered, and filters the fused dual-channel image based on the target denoising network. Through this method, fully considering the characteristics of the noise data, and based on the characteristics of image noise and convolutional neural network theory, a target noise extraction network and a target denoising network with advantages such as low computational complexity, feature learning, and nonlinearity are constructed respectively. Noise extraction is performed based on the target noise extraction network, and filtering is performed on the fused dual-channel image based on the target denoising network, thereby effectively improving the accuracy of image filtering and reducing the complexity of image filtering. Attached Figure Description

[0018] Figure 1 This is one of the flowcharts illustrating the image filtering method based on a convolutional neural network provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the application target denoising network provided in an embodiment of this application; Figure 3 This is the second flowchart illustrating the image filtering method based on a convolutional neural network provided in this application embodiment; Figure 4 This is a flowchart illustrating the application target noise extraction network provided in an embodiment of this application; Figure 5 This is a schematic diagram of the module structure of the image filtering device based on a convolutional neural network provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0021] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0022] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0023] Based on this, embodiments of this application provide an image filtering method based on a convolutional neural network, referring to... Figure 1 , Figure 1 This is one of the flowcharts illustrating an image filtering method based on a convolutional neural network provided in this application. In this embodiment, the image filtering method based on a convolutional neural network includes steps S10 to S40: Step S10: Obtain the image to be filtered, and determine the noise data type and image data type of the image to be filtered.

[0024] It should be noted that the image to be filtered includes both noise data and image data. Noise data type refers to the type of noise data in the image to be filtered, such as Gaussian noise, salt-and-pepper noise, etc. Image data type refers to the type of image data in the image to be filtered, such as natural images, remote sensing images, etc.

[0025] Step S20: Determine the target values ​​of the noise loss function and the noise network parameters according to the image data type, and construct the target noise extraction network according to the noise data type, the noise loss function, and the target values ​​of the noise network parameters.

[0026] It is understandable that selecting appropriate loss functions and network parameters is crucial for achieving optimal filtering results for different types of image noise. Therefore, the target values ​​of the noise loss function and noise network parameters are determined based on the image data type. The noise loss function refers to the loss function used to train the target noise extraction network. Different noise data types require different network structures. Therefore, a target noise extraction network needs to be constructed based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters. The noise network parameters refer to the network parameters used to construct the noise loss function, and these parameters include, but are not limited to, tradeoff parameters and preset baseline parameters.

[0027] Step S30: Based on the target noise extraction network, noise is extracted from the image to be filtered to obtain target noise data.

[0028] It should be understood that target noise data refers to the noise data contained in the noise data to be filtered. After training the target noise extraction network, the target noise data can be extracted from the image to be filtered based on the target noise extraction network, thereby achieving the separation of target noise data from image data.

[0029] Step S40: The target noise data and the image to be filtered are fused, and the fused dual-channel image is filtered based on the target denoising network.

[0030] It should be noted that the target denoising network refers to a denoising network that performs image filtering. This target denoising network can have a U-Net structure, including an encoder and a decoder. After extracting the target noise data, it can be fused with the image to be filtered to form a dual-channel image. Then, filtering is performed on the fused dual-channel image based on the target denoising network. Compared to networks trained using deep learning algorithms in existing technologies, this method has the advantages of high efficiency and no need for complex parameter tuning, resulting in higher overall denoising efficiency.

[0031] Further, step S40 includes: fusing the target noise data with the image to be filtered; obtaining noise-free target data and denoising results; determining a denoising loss function based on the noise-free target data and the denoising results, and iteratively updating the initial denoising network based on the denoising loss function and the target values ​​of the denoising network parameters to obtain a target denoising network; and filtering the fused dual-channel image based on the target denoising network.

[0032] It is understandable that the denoising loss function refers to the loss function used to train the target denoising network. After obtaining the noise-free target data and the denoising result, the denoising loss function is determined. Specifically: .

[0033] in, This represents target data that is free of noise. This indicates the denoising result.

[0034] It should be noted that the reference Figure 2 , Figure 2The flowchart illustrates the application of the target denoising network. Specifically, the target denoising network performs an image filtering task. The input is a fused dual-channel image, and the output is the filtered image. The network structure consists of 24 convolutional layers with a kernel size of 33. In the first 6 layers, downsampling is performed after every 3 convolutional layers, and in the last 6 layers, upsampling is performed after every 3 convolutional layers. The concatenation operation occurs after every 3 layers in the first 6 layers. This not only maintains a consistent number of channels but also allows for further learning of previous effective information. Furthermore, the denoising loss function in the target denoising network can be determined based on the reconstruction error of the target denoising network to improve consistency.

[0035] Further, the step of iteratively updating the initial denoising network according to the denoising loss function and the target values ​​of the denoising network parameters to obtain the target denoising network includes: acquiring sample training images and sample training noise data; downsampling the sample training images and sample training noise data based on the encoder in the initial denoising network, extracting features from the downsampled data, and compiling the extracted features into denoising codes; upsampling the denoising codes based on the decoder in the initial denoising network, reconstructing features from the upsampled data, and determining a predicted filtered image based on the reconstructed features; calculating the mean square error between the predicted filtered image and the actual filtered image; and iteratively updating the initial denoising network according to the denoising network parameters using a gradient backpropagation strategy with the goal of minimizing the mean square error to obtain the target denoising network.

[0036] It should be understood that before performing denoising based on the target denoising network, the target denoising network needs to be trained. The prepared training images and training noise data are input into the initial denoising network, which includes an encoder and a decoder. The encoder downsamples and extracts features from the training images and training noise data, compiling the extracted features into denoised codes to complete the feature representation. The decoder upsamples and reconstructs features from the denoised codes, determining the predicted filtered image based on the reconstructed features; this is the network output. To effectively improve the accuracy of training the target denoising network, the mean square error between the output predicted filtered image and the actual filtered image is calculated. This error measures the pixel-level difference between the network output and the real target. Using a gradient backpropagation strategy, the initial denoising network is iteratively updated based on the denoising network parameters, achieving iterative optimization of the denoising network.

[0037] This embodiment acquires an image to be filtered and determines the noise data type and image data type of the image to be filtered. Based on the image data type, it determines the target values ​​of the noise loss function and noise network parameters, and constructs a target noise extraction network based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters. Noise is extracted from the image to be filtered using the target noise extraction network to obtain target noise data. The target noise data is then fused with the image to be filtered, and the fused dual-channel image is filtered using a target denoising network. By fully considering the characteristics of the noise data and based on image noise characteristics and convolutional neural network theory, a target noise extraction network and a target denoising network with advantages such as low computational complexity, feature learning, and nonlinearity are constructed. Noise extraction is performed based on the target noise extraction network, and the fused dual-channel image is filtered based on the target denoising network, thereby effectively improving the accuracy and reducing the complexity of image filtering.

[0038] In one specific implementation, this application provides steps for constructing a target noise extraction network. Please refer to... Figure 3 , Figure 3 This is the second flowchart illustrating the image filtering method based on a convolutional neural network provided in this application embodiment. Step S20 includes steps S201 to S205: Step S201: Determine the asymmetric loss function and total variation constraint term of the noise extraction network according to the image data type.

[0039] It should be noted that the asymmetric loss function is more sensitive to underestimation of noise due to non-blind denoising, but is robust to overestimation. Therefore, this characteristic can be used to adjust the magnitude of the estimated noise data to achieve better denoising effect. The total variation constraint term refers to the constraint term that achieves effective noise reduction in smooth areas and sharp edge effect at the boundaries through the total variation constraint method of the image. This total variation constraint term can be the TV constraint term.

[0040] Further, step S201 includes: acquiring real noise data, estimated noise data, horizontal gradient operators, and vertical gradient operators at each pixel point according to the image data type; calculating real noise values ​​based on the real noise data, and calculating estimated noise values ​​based on the estimated noise data; determining the asymmetric loss function of the noise extraction network based on the noise level estimation function, the real noise values, the estimated noise values, and the target values ​​of the noise network parameters; and determining the total variation constraint term based on the horizontal gradient operators, the vertical gradient operators, and the estimated noise values.

[0041] It should be understood that the true noise value represents the magnitude of the actual noise at each pixel, and similarly, the estimated noise value represents the magnitude of the estimated noise at each pixel. After calculating the true noise value based on the true noise data and the estimated noise value based on the estimated noise data, the asymmetric loss function of the noise extraction network is determined based on the noise level estimation function. Specifically: .

[0042] in, This indicates the preset baseline parameters. This represents the actual noise value. This represents the estimated noise value. This represents a noise level estimation function, which can estimate the magnitude of the noise level. Specifically: .

[0043] It should be noted that for the preset baseline parameters The value can be in the range of [0, 0.5]. It can impose a greater penalty on the underestimation of noise error, enabling the network to better estimate image noise. At the same time, it can provide the denoising network with noise information that is not underestimated, further improving the overall denoising effect.

[0044] Understandably, to make the estimated noise data smoother, this embodiment introduces a total variation constraint term. After obtaining the gradient operators in the horizontal and vertical directions, the total variation constraint term can be determined by combining the estimated noise values. This total variation constraint term can be a TV constraint term. Specifically: .

[0045] in, This represents the gradient operator in the horizontal direction. This represents the gradient operator in the vertical direction. This indicates the estimated noise level.

[0046] Step S202: Calculate the noise loss function based on the asymmetric loss function, the trade-off parameter of the asymmetric loss function, the total variation constraint term, and the trade-off parameter of the total variation constraint term.

[0047] It is understandable that, for the parameters of the noisy network, including but not limited to trade-off parameters and preset baseline parameters, after determining the asymmetric loss function and the total variation constraint term respectively, the noise loss function can be calculated in combination with their respective trade-off parameters. Specifically: .

[0048] in, The parameters representing the trade-offs in the asymmetric loss function, This represents the trade-off parameter for the total variation constraint term. This represents the asymmetric loss function of the noise extraction network. This indicates the TV constraint.

[0049] Step S203: Obtain the noise network parameters and the simulation test values ​​of the network parameters.

[0050] Step S204: Based on the control variable algorithm, determine the target values ​​of the noise network parameters according to the simulation test values.

[0051] Understandably, the controlled variable algorithm is used to investigate the impact of one (or several) factors on the outcome of a complex system while keeping all other potentially influential factors constant, thereby clearly analyzing the causal relationship between the target factor and the outcome. In this embodiment, the controlled variable algorithm is introduced to determine the target values ​​of the noise network parameters based on simulation experimental data. These target values ​​can be optimal parameter values, such as preset baseline parameters. Trade-off parameters of the asymmetric loss function The trade-off parameters of the total variation constraint term .

[0052] Step S205: Construct a target noise extraction network based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters.

[0053] Further, step S205 includes: calculating the complexity feature value of the noise data type, and determining the network construction architecture based on the complexity feature value, wherein the network construction architecture includes the number of fully convolutional layers, the size of each convolutional kernel, and the number of feature channels; constructing an initial noise extraction network based on the network construction architecture; inputting sample training images into the initial noise extraction network and obtaining the estimated noise data output by the initial noise extraction network; determining the difference between the estimated noise data and the actual noise data, and performing backpropagation and optimization on the initial network based on the difference, the noise loss function, and the target value of the noise network parameters to obtain the target noise extraction network.

[0054] It should be noted that for the network architecture of noise extraction networks, the larger the complexity feature value of the noise data type, the more complex the noise data type, and the more fully convolutional layers are required. That is, as the complexity feature value increases, the number of fully convolutional layers needs to be increased adaptively. In addition to the number of fully convolutional layers, the network architecture also includes the size of the convolutional kernel and the number of feature channels in each layer. The convolutional kernel acts on the local matrix region to obtain local information of the matrix. To make the noise extraction network more robust, ReLU is used as the activation function after each convolutional layer. Its advantages are low computational complexity and effective avoidance of gradient vanishing and gradient exploding.

[0055] Understandably, for the initial noise extraction network, the input is the sample training image, and the output is the estimated noise data. At this point, it's necessary to determine the difference between the estimated noise data and the actual noise data. Based on this difference, the noise loss function, and the target values ​​of the noise network parameters, the initial network is backpropagated and optimized to obtain the best-performing target noise extraction network, which is then used to extract the target noise data from the image to be filtered. (Reference) Figure 4 , Figure 4 The flowchart of the target noise extraction network is as follows: The target noise extraction network performs a noise extraction task. The input is the image to be filtered and the output is the target noise data. The network structure consists of 8 fully convolutional layers. Each layer does not contain pooling and batch normalization operations. The size of each convolutional kernel is 3×3 and the number of feature channels is 32.

[0056] This embodiment determines the asymmetric loss function and total variation constraint term of the noise extraction network according to the image data type; calculates the noise loss function based on the asymmetric loss function, its trade-off parameter, the total variation constraint term, and its trade-off parameter; obtains the noise network parameters and their simulation test values; determines the target values ​​of the noise network parameters based on the simulation test values ​​using a control variable algorithm; and constructs the target noise extraction network based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters. By introducing the total variation constraint term, the estimated noise data becomes smoother. After determining the asymmetric loss function and total variation constraint term of the noise extraction network, the noise loss function is calculated using their respective trade-off parameters, and the target values ​​of the noise network parameters are determined based on a control variable algorithm. Finally, the target noise extraction network is constructed by combining the noise data type and the noise loss function, thereby effectively improving the accuracy and efficiency of the target noise extraction network.

[0057] The image filtering device based on a convolutional neural network provided in this application is described below. The image filtering device based on a convolutional neural network described below corresponds to the image filtering method based on a convolutional neural network described above. Please refer to... Figure 5 , Figure 5 This is a schematic diagram of the module structure of the image filtering device based on a convolutional neural network provided in an embodiment of this application, including: The determination module T10 is used to acquire the image to be filtered and determine the noise data type and image data type of the image to be filtered.

[0058] The determining module T10 is further configured to determine the target values ​​of the noise loss function and the noise network parameters according to the image data type, and to construct a target noise extraction network according to the noise data type, the noise loss function and the target values ​​of the noise network parameters.

[0059] The extraction module T20 is used to extract noise from the image to be filtered based on the target noise extraction network to obtain target noise data.

[0060] The filtering module T30 is used to fuse the target noise data and the image to be filtered, and to filter the fused dual-channel image based on the target denoising network.

[0061] This embodiment acquires an image to be filtered and determines the noise data type and image data type of the image to be filtered. Based on the image data type, it determines the target values ​​of the noise loss function and noise network parameters, and constructs a target noise extraction network based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters. Noise is extracted from the image to be filtered using the target noise extraction network to obtain target noise data. The target noise data is then fused with the image to be filtered, and the fused dual-channel image is filtered using a target denoising network. By fully considering the characteristics of the noise data and based on image noise characteristics and convolutional neural network theory, a target noise extraction network and a target denoising network with advantages such as low computational complexity, feature learning, and nonlinearity are constructed. Noise extraction is performed based on the target noise extraction network, and the fused dual-channel image is filtered based on the target denoising network, thereby effectively improving the accuracy and reducing the complexity of image filtering.

[0062] It is understood that the detailed functional implementation of each of the above modules can be found in the description of the aforementioned method embodiments, and will not be repeated here.

[0063] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0064] Based on the methods in the above embodiments, this application provides an electronic device, please refer to... Figure 6 , Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application.

[0065] It should be noted that the system may include: a processor 10, a communications interface 20, a memory 30, and a communication bus 40. The processor 10, communications interface 20, and memory 30 communicate with each other via the communication bus 40. The processor 10 can invoke logical instructions stored in the memory 30 to execute the methods described in the above embodiments.

[0066] Furthermore, the logical instructions in the aforementioned memory 30 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0067] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0068] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0069] It is understood that the processor in the embodiments of this application can be a central processing unit, or it can be other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0070] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory, flash memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, registers, hard disks, portable hard disks, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor.

[0071] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. Those skilled in the art will readily understand that the above descriptions are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image filtering method based on convolutional neural networks, characterized in that, include: Obtain the image to be filtered, and determine the noise data type and image data type of the image to be filtered respectively; The target values ​​of the noise loss function and the noise network parameters are determined based on the image data type, and a target noise extraction network is constructed based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters. Based on the target noise extraction network, noise is extracted from the image to be filtered to obtain target noise data; The target noise data and the image to be filtered are fused together, and the fused dual-channel image is filtered based on the target denoising network.

2. The method as described in claim 1, characterized in that, The step of determining the target values ​​of the noise loss function and the noise network parameters based on the image data type, and constructing the target noise extraction network based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters, includes: The asymmetric loss function and total variation constraint term of the noise extraction network are determined according to the image data type, respectively. The noise loss function is calculated based on the asymmetric loss function, the trade-off parameter of the asymmetric loss function, the total variation constraint term, and the trade-off parameter of the total variation constraint term. Obtain the parameters of the noisy network and the simulation test values ​​of the network parameters; Based on the controlled variable algorithm, the target values ​​of the noise network parameters are determined according to the simulation test values. A target noise extraction network is constructed based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters.

3. The method as described in claim 2, characterized in that, The step of constructing a target noise extraction network based on the noise data type, the noise loss function, and the target values ​​of the noise network parameters includes: Calculate the complexity feature value of the noise data type, and determine the network architecture based on the complexity feature value, wherein the network architecture includes the number of fully convolutional layers, the size of each convolutional kernel, and the number of feature channels; An initial noise extraction network is constructed based on the described network architecture; The sample training images are input into the initial noise extraction network, and the estimated noise data output by the initial noise extraction network is obtained. The difference between the estimated noise data and the actual noise data is determined, and the initial network is backpropagated and optimized based on the difference, the noise loss function, and the target values ​​of the noise network parameters to obtain the target noise extraction network.

4. The method as described in claim 2, characterized in that, The step of determining the asymmetric loss function and total variation constraint term of the noise extraction network according to the image data type includes: Based on the image data type, obtain the actual noise data, estimated noise data, horizontal gradient operator, and vertical gradient operator at each pixel. The actual noise value is calculated based on the actual noise data, and the estimated noise value is calculated based on the estimated noise data; Based on the noise level estimation function, the asymmetric loss function of the noise extraction network is determined according to the actual noise value, the estimated noise value, and the target value of the noise network parameters. The total variation constraint term is determined based on the gradient operator in the horizontal direction, the gradient operator in the vertical direction, and the estimated noise value.

5. The method according to any one of claims 1 to 4, characterized in that, The step of fusing the target noise data and the image to be filtered, and filtering the fused dual-channel image based on the target denoising network, includes: The target noise data and the image to be filtered are fused together; Obtain noise-free target data and denoising results; The denoising loss function is determined based on the noise-free target data and the denoising result, and the initial denoising network is iteratively updated based on the denoising loss function and the target values ​​of the denoising network parameters to obtain the target denoising network. The fused dual-channel image is filtered based on the target denoising network.

6. The method as described in claim 5, characterized in that, The step of iteratively updating the initial denoising network based on the denoising loss function and the target values ​​of the denoising network parameters to obtain the target denoising network includes: Acquire sample training images and sample training noise data; The encoder in the initial denoising network is used to downsample the sample training image and the sample training noise data, and features are extracted from the downsampled data. The extracted features are then compiled into denoising codes. The denoised code is upsampled based on the decoder in the initial denoising network, the upsampled data is reconstructed for features, and the predicted filtered image is determined based on the reconstructed features. Calculate the mean square error between the predicted filtered image and the actual filtered image; With the goal of minimizing the mean square error, the initial denoising network is iteratively updated based on the parameters of the denoising network using a gradient backpropagation strategy to obtain the target denoising network.

7. An image filtering device based on a convolutional neural network, characterized in that, include: A determination module is used to acquire the image to be filtered and determine the noise data type and image data type of the image to be filtered, respectively. The determining module is further configured to determine the target values ​​of the noise loss function and the noise network parameters according to the image data type, and construct a target noise extraction network according to the noise data type, the noise loss function and the target values ​​of the noise network parameters; The extraction module is used to extract noise from the image to be filtered based on the target noise extraction network to obtain target noise data; The filtering module is used to fuse the target noise data and the image to be filtered, and to filter the fused dual-channel image based on the target denoising network.

8. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is run on the processor, it causes the processor to perform the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, When the computer program product is run on a processor, the processor causes the processor to perform the method as described in any one of claims 1-6.

Images