Image denoising method based on semi-regular data structure feature extraction

Abstract

The invention discloses an image denoising method based on semi-regular data structure feature extraction, and belongs to the technical field of image processing and video enhancement. According to the method, through three steps of constructing a convolutional neural network (CNN) module for extracting regular data structure features, constructing a graph convolutional neural network (GCN) module for extracting irregular data structure features, and establishing an adaptive feature fusion mechanism, various structure information in an image is comprehensively captured and fused, more effective denoising is realized, and the image quality is improved. Therefore, the image quality is obviously improved and the method is suitable for various imaging systems. According to the method, the CNN module which is good at extracting regular data structure features and the GCN module which is good at capturing irregular data structure features are introduced for the first time to jointly extract the features of the thermal infrared image, and the weight distribution of the CNN and the GCN is dynamically adjusted according to the noise level and the structure features of the thermal infrared image by constructing an adaptive feature fusion mechanism. And the denoising effect is further improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and video enhancement, and proposes an image denoising method based on semi-regular data structure feature extraction, which can effectively improve the image denoising effect and is applicable to various imaging systems. Background Art

[0002] In modern imaging technologies, image data is often interfered by noise, resulting in a decline in image quality and affecting subsequent analysis and applications. Noise may originate from the hardware characteristics of imaging devices, environmental factors, or signal transmission processes, and is manifested as random pixel fluctuations or structural distortions. Traditional image denoising methods are mainly based on manually designed filters, such as median filtering, Gaussian filtering, etc. Although these methods are simple and efficient, they often struggle to adapt to complex noise environments and are prone to blurring image details.

[0003] In recent years, with the development of deep learning technologies, convolutional neural networks (CNNs) have made significant progress in the field of image denoising. By learning the features of a large number of noisy and clean image pairs, CNNs can automatically extract the regular structural features in images, thereby achieving effective denoising. However, CNNs have limitations in processing irregular structural features. For example, for image regions with complex local connectivity or irregular topological structures, their denoising effects are often unsatisfactory. On the other hand, graph convolutional neural networks (GCNs), as an emerging deep learning model, can effectively process irregular structure data. It can capture the complex connection relationships and topological information between pixels through graph structure modeling, thereby better processing irregular structural features. However, GCNs are less efficient in processing regular structural features and require a large amount of computing resources.

[0004] To overcome the limitations of the existing technologies, the present invention proposes an image denoising method based on semi-regular data structure feature extraction. This method combines the advantages of CNNs and GCNs. By using CNNs to extract the regular structural features in images and simultaneously using GCNs to extract irregular structural features, it can more comprehensively capture various structural information in images, and then fuse the features extracted by the two networks, thereby achieving more effective denoising. In addition, the present invention also introduces an adaptive feature fusion mechanism, which can dynamically adjust the weight allocation of CNNs and GCNs according to the noise level and structural characteristics of images, further improving the denoising effect. Through this method of combining semi-regular data structure feature extraction, the present invention can significantly improve the image quality in various noise environments and provide more reliable technical support for image analysis and applications. Summary of the Invention

[0005] The present invention designs an image denoising method based on semi-regular data structure feature extraction. It is achieved by constructing a CNN module for extracting regular data structure features, constructing a GCN module for extracting irregular data structure features, and establishing an adaptive feature fusion mechanism. It comprehensively captures various structural information in the image and fuses them to achieve more effective denoising, thereby significantly improving image quality.

[0006] The present invention is achieved through the following technical solution, comprising the following steps:

[0007] Step 1: Build a CNN module to extract regular data structure features;

[0008] Step 2: Build a GCN module to extract irregular data structure features;

[0009] Step 3: Establish an adaptive feature fusion mechanism.

[0010] The creativity of the present invention is mainly reflected in:

[0011] Different from the common visible light images regarded as regular data structures (with rich semantic information such as shape, texture, color, edge, etc. visible to the naked eye, and the pixel values have strong local correlation in space) and spectral imaging images regarded as irregular data structures (such as hyperspectral images, whose pixels are nonlinearly mixed and have high-dimensional characteristics), infrared thermal imager imaging images (referred to as thermal infrared images) have the characteristics of semantic information such as shape and edge possessed by visible light images, and the characteristics of nonlinear mixing of hyperspectral image pixels, and are regarded as imaging images with semi-regular data structures. The present invention first introduces a CNN module that is good at extracting regular data structure features and a GCN module that is good at capturing irregular data structure features to jointly extract the features of thermal infrared images, and by constructing an adaptive feature fusion mechanism, dynamically adjusts the weight distribution of CNN and GCN according to the noise level and structural characteristics of the thermal infrared images, and further improves the denoising effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flow chart of the image denoising network model designed by the present invention based on semi-regular data structure feature extraction. DETAILED DESCRIPTION

[0013] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0014] Example:

[0015] Step 1: Build a CNN module to extract regular data structure features;

[0016] In view of the fact that thermal infrared images possess semantic information such as shape and edge that visible light images have, the present invention utilizes the advantage of CNN in extracting local features of images to construct a CNN module for extracting regular data structure features. This module consists of "1×1 convolutional layer - ReLU activation layer - 3×3 convolutional layer - batch normalization (BN) layer - ReLU activation layer - 1×1 convolutional layer", as Figure 1 shown. This module outputs the regular data structure feature F C .

[0017] Step 2: Construct a GCN module for extracting irregular data structure features;

[0018] The similarity between pixels in thermal infrared images usually depends not only on their spatial positional relationship but also on their gray values and infrared radiation characteristics. In view of the fact that thermal infrared images possess the characteristic of non-linear mixing of pixels in spectral imaging images (such as hyperspectral images), the present invention utilizes the advantage of GCN in capturing complex connection relationships and topological information between pixels to construct a GCN module for extracting irregular data structure features. This module consists of a "graph processing" sub-module and a "feature transformation" sub-module. Among them, the "graph processing" sub-module is constructed by "downsampling module - linear mapping layer - graph processing layer - residual connection", and the "feature transformation" sub-module is constructed by "multi-layer perceptron (MLP) - residual connection", as Figure 1 shown. This module outputs the irregular data structure feature F G . Combining the above steps, this method can capture various structural information in the image more comprehensively.

[0019] Step 3: Establish an adaptive feature fusion mechanism;

[0020] In order to achieve more effective denoising, the present invention establishes an adaptive feature fusion mechanism to fuse the regular data structure feature F C extracted in the above two steps and the irregular data structure feature F G . Specifically, the present invention sets two learnable weights ω C and ω G , which are respectively set after the CNN module and after the GCN module, as Figure 1 shown. These two learnable weights can dynamically adjust the weight distribution of CNN and GCN according to the noise level and structural characteristics of the thermal infrared image, further improving the denoising effect.

[0021] The output of the denoising network proposed by the present invention is the predicted noise of the thermal infrared image, and the denoised image can be obtained by subtracting the predicted noise from the input image. Therefore, the loss function for network update using mean square error (MSE) during the training process can be expressed as:

[0022]

[0023] where N is the number of images in the training set; x i and y i are respectively the i-th input noisy image and the corresponding clean target image in the training set; θ is the network parameter of the denoising network f θ (·), which is updated as the loss is updated.

Claims

1. Construct a CNN module for extracting the feature of the rule data structure; In view of the characteristics that thermal infrared images have semantic information such as shape and edge possessed by visible light images, the present invention utilizes the advantage of CNN in extracting local features of images to construct a CNN module for extracting regular data structure features. This module consists of "1×1 convolutional layer - ReLU activation layer - 3×3 convolutional layer - batch normalization (BN) layer - ReLU activation layer - 1×1 convolutional layer". This module outputs regular data structure feature F C .

2. Construct a GCN module for extracting the feature of the non-rule data structure; The similarity between pixels in thermal infrared images usually depends not only on their spatial positional relationship, but also on their gray values and infrared radiation characteristics. In view of the characteristics of spectral imaging images (such as hyperspectral images) with non-linear mixing of pixels in thermal infrared images, the present invention utilizes the advantage of GCN in capturing complex connection relationships and topological information between pixels to construct a GCN module for extracting features of irregular data structures. This module consists of a "graph processing" sub-module and a "feature transformation" sub-module. Among them, The "graph processing" sub-module is constructed by "downsampling module - linear mapping layer - graph processing layer - residual connection", and the "feature transformation" sub-module is constructed by "multi-layer perceptron (MLP) - residual connection". This module outputs the features F of the irregular data structure. G Combined with the above steps, this method can capture various structural information in the image more comprehensively.

3. Establish an adaptive feature fusion mechanism; To achieve more effective denoising, the present invention establishes an adaptive feature fusion mechanism to fuse the regular data structure features F C extracted in the above two steps and the irregular data structure features F G . Specifically, the present invention sets two learnable weights ω C and ω G , which are respectively set after the CNN module and the GCN module. These two learnable weights can dynamically adjust the weight allocation of the CNN and the GCN according to the noise level and structural characteristics of the thermal infrared image, further improving the denoising effect. The output of the denoising network proposed by the present invention is the noise of the predicted thermal infrared image, and the denoised image can be obtained by subtracting the predicted noise from the input image. Therefore, the loss function for network update using the mean square error (MSE) during the training process can be expressed as: Among them, N is the number of images in the training set; x i and y i are the i-th input noisy image and the corresponding clean target image of the i-th input noisy image in the training set, respectively; θ is the network parameter of the denoising network f θ (·), which is updated as the loss is updated.

Images