Unsupervised endoscope image enhancement method

Abstract

The invention discloses an unsupervised endoscope image enhancement method, which comprises the following steps of: decomposing a low-quality endoscope image into a reflection image and an illumination image by utilizing a decomposition network according to the guidance of histogram equalization prior, and then inputting the reflection image into a noise decoupling module to obtain a clean reflection image and reflection image noise. Inputting two pairs of data of the clean reflection image and the reflection image noise as well as the noiseless natural image and the reflection image noise into a noise synthesis module to obtain a synthesized reflection image and a synthesized natural image, and finally inputting the synthesized natural image into a noise decoupling module to obtain a pseudo natural image; a final enhanced image is obtained by introducing a discriminator and an anti-loss continuously optimized noise decoupling module. According to the method, the contrast and definition between the tissue and the blood vessel of the input image can be greatly enhanced, and a large amount of spiced salt and Gaussian noise can be effectively removed. And powerful support is provided for subsequent medical diagnosis.

Description

Technical Field

[0001] The invention relates to the technical field of image enhancement, and in particular to an unsupervised endoscopic image enhancement method. Background Art

[0002] In the actual use of endoscopes, due to the complexity of the internal structure of the human body, the concentration of light sources and the limitations of imaging technology, medical endoscope images generally have serious degradation problems such as uneven illumination, low contrast and a lot of noise. These defects not only make it difficult to observe tiny blood vessels and early, inconspicuous lesions, but also seriously affect doctors' diagnosis and treatment, and also reduce the accuracy of some auxiliary diagnostic equipment. Therefore, how to enhance endoscopic images is a problem with great practical application value and research significance.

[0003] Some existing traditional methods include: histogram equalization (HE), gamma correction, unsharp masking technology and Retinex-based methods. Although they can improve some brightness and contrast, they require a series of complex operations to adjust parameters for different scenes and tasks, and the enhancement effect is limited.

[0004] In the field of deep learning, many image quality enhancement methods based on supervised learning of paired images have been proposed. These methods no longer use complex parameter adjustment operations, but usually require a large amount of paired training data. Endoscopic images involve patient privacy, which is difficult to obtain in actual scenarios. In addition, the network model trained using the paired dataset is only effective for endoscopic image enhancement in a certain specific environment, so it lacks robustness and versatility. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides an unsupervised endoscopic image enhancement method, which aims to solve the problems in the background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: an unsupervised endoscopic image enhancement method, comprising the following steps: Step S1: Collect low-quality endoscopic images and high-quality noise-free natural images to construct a dataset; Step S2: construct a decomposition network model, input the low-quality endoscopic image into the decomposition network model, and obtain a reflectance map and an illumination map; Step S3: performing histogram equalization HE on the low-quality endoscopic image to obtain a histogram equalization prior, and inputting the histogram equalization prior and the reflection map into a pre-trained ConvNeXt network model to obtain intermediate features; Step S4: Calculate the loss of the histogram equalization prior according to the intermediate features, and introduce the illumination smoothing loss and reconstruction loss as the total loss of the decomposition network model for training; Step S5: constructing a denoising network model, wherein the denoising network model consists of a noise decoupling module and a noise synthesis module; Step S6: input the reflection image into the noise decoupling module to obtain a clean reflection image and reflection image noise, input the clean reflection image and reflection image noise as a pair and the noise-free natural image and reflection image noise as a pair into the noise synthesis module respectively to obtain a synthesized reflection image and a synthesized natural image, input the synthesized natural image into the noise decoupling module to obtain a pseudo natural image; introduce content consistency loss and adversarial loss as the total loss of the denoising network model for training, input the pseudo natural image into the trained denoising network model to obtain the final clean reflection image; Step S7: construct a discriminator, which uses a generative adversarial network; input the pseudo-natural image and the noise-free natural image obtained in step S6 into the discriminator as a group of image pairs, identify the noise-free natural image and the pseudo-natural image through the discriminator, and introduce the least squares loss as the adversarial loss to optimize the parameters of the noise decoupling module.

[0007] Furthermore, the decomposition network model consists of two convolutional layers and five convolutional blocks; first, the low-quality endoscopic image is normalized, and then the normalized image is passed through the first convolutional layer 5×5, the output of the first convolutional layer 5×5 is subjected to the first wavelet transform to obtain the first high-frequency component and the first low-frequency component respectively, the first low-frequency component is subjected to the second wavelet transform after passing through the first convolutional block to obtain the second high-frequency component and the second low-frequency component, the second low-frequency component is subjected to the third wavelet transform after passing through the second convolutional block to obtain the third high-frequency component and the third low-frequency component, the third high-frequency component is element-wise multiplied with the attention map, and the third The output of the third convolution block of the three low-frequency components and the result of the third high-frequency component and the attention map are concatenated on the channel for the first inverse wavelet transform. The output of the first inverse wavelet transform is concatenated on the channel with the result of the second high-frequency component and the attention map after the fourth convolution block. The result is concatenated on the channel for the second inverse wavelet transform. The output of the second inverse wavelet transform is concatenated on the channel with the result of the first high-frequency component and the attention map after the fifth convolution block. The result is concatenated on the channel for the third inverse wavelet transform. The output of the third inverse wavelet transform is passed through the second convolution layer 5×5 to obtain the reflection map and the illumination map. The attention map is composed of the value 1 minus the normalized low-quality endoscopic image.

[0008] Furthermore, the noise decoupling module is composed of multiple convolution blocks; the reflection image first passes through the sixth convolution block, the output of the sixth convolution block is downsampled for the first time, the output of the first downsampling passes through the seventh convolution block, the output of the seventh convolution block is downsampled for the second time, the output of the second downsampling passes through the eighth convolution block, the output of the eighth convolution block is downsampled for the third time, the output of the third downsampling passes through the ninth convolution block, the output of the ninth convolution block is upsampled for the first time, and the output of the first upsampling and the output of the eighth convolution block are subjected to a self-attention mechanism on the channel. The rows are spliced ​​as the input of the tenth convolution block, the output of the tenth convolution block is upsampled for the second time, the output of the second upsampling and the output of the seventh convolution block are spliced ​​on the channel after a self-attention mechanism as the input of the eleventh convolution block, the output of the eleventh convolution block is upsampled for the third time, the output of the third upsampling and the output of the sixth convolution block are spliced ​​on the channel after a self-attention mechanism as the input of the twelfth convolution block, and finally the input of the twelfth convolution block and the reflection map are element-wise added to obtain the denoised image, i.e., the clean reflection map and the reflection map noise.

[0009] Furthermore, each convolution block consists of two 3×3 convolutions, and each convolution is followed by a ReLU activation function.

[0010] Furthermore, the noise synthesis module is composed of six convolutional layers and three residual blocks. First, the clean reflection map or noise-free natural image and the reflection map noise are spliced ​​on the channel as the input of the third convolutional layer 7×7, the output of the third convolutional layer passes through the fourth convolutional layer 3×3 and the fifth convolutional layer 3×3 in sequence, the output of the fifth convolutional layer passes through the first residual block, the output of the first residual block and the output of the fifth convolutional layer are element-wise added as the input of the second residual block, the output of the second residual block and the output of the first residual block are element-wise added as the input of the third residual block, the output of the third residual block passes through the sixth convolutional layer 3×3, the seventh convolutional layer 3×3 and the eighth convolutional layer 7×7 in sequence, and the output of the eighth convolutional layer is element-wise added to the clean reflection map or noise-free natural image to obtain the final synthesized noise image, i.e., the pseudo natural image.

[0011] Furthermore, the output ends of the third convolutional layer 7×7, the fourth convolutional layer 3×3, the fifth convolutional layer 3×3, the sixth convolutional layer 3×3, and the seventh convolutional layer 3×3 are all connected to a LeakyReLU activation function; each residual block is composed of two convolutional layers 3×3 connected in sequence, and the output end of each convolutional layer 3×3 is connected to a LeakyReLU activation function.

[0012] Furthermore, the histogram equalization prior loss , expressed as: ; In the formula, Indicates intermediate features; Represents a reflection graph; represents the histogram equalization prior; Indicates the calculation of the mean square error between the two; The histogram equalization prior loss, illumination smoothing loss and reconstruction loss are introduced as the total loss of the decomposition network model. The weight coefficient of the decomposition network model is set to 0.1, and the hyperparameters Set to 0.01, train the decomposition network model in an unsupervised learning manner, and use the Adam optimizer for optimization; the total loss of the decomposition network model It is expressed as: ; In the formula, and are the weight coefficients of the decomposition network; represents the reconstruction loss; Represents the lighting smoothing loss.

[0013] Furthermore, we introduce content consistency loss The noise synthesis module is optimized, and the least square loss is introduced as the adversarial loss to optimize the parameters of the noise decoupling module; the denoising network model is jointly trained in an unsupervised learning manner using an unpaired dataset, and the total loss function of the denoising network model is expressed as: ; In the formula, represents the least squares loss; The unpaired data set is formed by combining a public natural image data set and a public endoscopic image data set.

[0014] Furthermore, the specific process of step S1 is: first, a large number of different types of low-quality endoscopic images are collected from the existing publicly available endoscopic datasets, and then a large number of high-quality noise-free natural images are collected from the existing natural image datasets, and the low-quality endoscopic images and high-quality noise-free natural images constitute a dataset.

[0015] Compared with the existing technology, the present invention has the following beneficial effects:

[0016] (1) The present invention introduces a histogram equalization prior to replace the normal endoscopic image as the true value to guide the training of the decomposition network model, so that the decomposed reflectance map and illumination map are more natural than most supervised learning methods and will not produce artifacts.

[0017] (2) In the decomposition network model, the present invention uses wavelet transform and attention map instead of ordinary convolution downsampling operation, so that the detailed texture information of the decomposed endoscopic image is more concentrated in the reflectance map, and the illumination map contains only brightness information as much as possible, avoiding the influence of uneven illumination on the detailed texture information, and can significantly improve the contrast and color reproduction of tissues and blood vessels in the endoscope.

[0018] (3) The present invention transfers the noise in the reflection image to the natural image through the noise synthesis module, and then introduces the generative adversarial loss to effectively remove the salt and pepper noise and Gaussian noise in the reflection image, thereby greatly improving the clarity and naturalness of the blood vessels and tissues in the endoscopic image. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is the overall model framework diagram of the algorithm of the present invention.

[0020] Figure 2 This is a structural diagram of the decomposition network model of the algorithm of the present invention.

[0021] Figure 3 This is a structural diagram of the noise decoupling module of the algorithm of the present invention.

[0022] Figure 4 This is a structural diagram of the noise synthesis module of the algorithm of the present invention. DETAILED DESCRIPTION

[0023] like Figure 1 As shown, the present invention provides a technical solution: an unsupervised endoscopic image enhancement method, comprising the following steps:

[0024] Step S1: Collect low-quality endoscopic images and high-quality noise-free natural images to construct a dataset.

[0025] Step S2: construct a decomposition network model, input the low-quality endoscopic image into the decomposition network model, and obtain a reflectance map and an illumination map.

[0026] Step S3: Perform histogram equalization (HE) on the low-quality endoscopic image to obtain a histogram equalization prior, and input the histogram equalization prior and the reflection map into the pre-trained ConvNeXt network model to obtain intermediate features.

[0027] Step S4: Calculate the loss of the histogram equalization prior based on the intermediate features, and introduce the illumination smoothing loss and reconstruction loss as the total loss of the decomposition network model for training.

[0028] Step S5: construct a denoising network model, which consists of a noise decoupling module and a noise synthesis module.

[0029] Step S6: input the reflection image into the noise decoupling module to obtain a clean reflection image and reflection image noise, input the clean reflection image and reflection image noise as a pair and the noise-free natural image and reflection image noise as a pair into the noise synthesis module respectively to obtain a synthesized reflection image and a synthesized natural image, input the synthesized natural image into the noise decoupling module to obtain a pseudo natural image; introduce content consistency loss and adversarial loss as the total loss of the denoising network model for training, input the pseudo natural image into the trained denoising network model to obtain the final clean reflection image.

[0030] Step S7: Construct a discriminator. The discriminator uses a generative adversarial network (GAN). In order to optimize and judge the denoising ability of the noise decoupling module, the pseudo natural image obtained in step S6 and the input noise-free natural image are input into the discriminator as a set of image pairs. The discriminator is used to identify whether the input is a noise-free natural image or a pseudo natural image. The least squares loss is introduced as the adversarial loss to optimize the parameters of the noise decoupling module, so that the pseudo natural image obtained is closer and closer to the noise-free natural image, thereby deceiving the discriminator. Through the confrontation between the discriminator and the noise decoupling module, the denoising effect of the noise decoupling module is continuously improved.

[0031] The specific process of step S1 is as follows: first, a large number of different types of low-quality endoscopic images are collected from existing publicly available endoscopic datasets, where the low-quality endoscopic images include images of the intestines, abdominal cavity and other parts; then, a large number of high-quality noise-free natural images are collected from existing natural image datasets (LOL and DICM datasets); and the low-quality endoscopic images and high-quality noise-free natural images constitute a dataset.

[0032] Among them, before inputting the low-quality endoscopic images into the decomposition network model, the low-quality endoscopic images are preprocessed, including random cropping to 128*128 pixel size, random flipping and other data enhancement operations.

[0033] like Figure 2As shown, the decomposition network model consists of two convolutional layers and five convolutional blocks; first, the input image (low-quality endoscopic image) is normalized, and then the normalized image is passed through the first convolutional layer (5×5), the output of the first convolutional layer (5×5) is subjected to the first wavelet transform to obtain the first high-frequency component and the first low-frequency component respectively, the first low-frequency component is subjected to the second wavelet transform after passing through the first convolutional block to obtain the second high-frequency component and the second low-frequency component, the second low-frequency component is subjected to the third wavelet transform after passing through the second convolutional block to obtain the third high-frequency component and the third low-frequency component, and the third high-frequency component is element-wise multiplied with the attention map , the output of the third low-frequency component after the third convolution block and the result of the third high-frequency component and the attention map are element-wise multiplied and concatenated on the channel for the first inverse wavelet transform, the output of the first inverse wavelet transform is passed through the fourth convolution block and the result of the second high-frequency component and the attention map are element-wise multiplied and concatenated on the channel for the second inverse wavelet transform, the output of the second inverse wavelet transform is passed through the fifth convolution block and the result of the first high-frequency component and the attention map are element-wise multiplied and concatenated on the channel for the third inverse wavelet transform, and the output of the third inverse wavelet transform is passed through the second convolution layer (5×5) to obtain the reflectance map and the illumination map.

[0034] Among them, the attention map is composed of the value 1 minus the normalized input image (low-quality endoscopic image), and each convolution block is composed of two convolutional layers (3×3), and each convolutional layer is followed by a ReLU activation function.

[0035] Among them, the loss of histogram equalization prior It is expressed as: ; In the formula, Indicates intermediate features; Represents a reflection graph; represents the histogram equalization prior; It means calculating the mean square error between the two.

[0036] Since image decomposition is an ill-posed decomposition problem, some constraints need to be imposed on the decomposition network model. According to physical prior knowledge, the illumination changes of natural images are continuous and gradual, so the illumination smoothing loss is introduced to constrain the decomposed illumination map to be as smooth as possible in terms of texture details while retaining the structural boundaries. It is expressed as: ; In the formula, Indicates taking the absolute value; To avoid a small constant with a denominator of 0; It means taking the maximum value between the two. Represents gradient (including horizontal gradient and vertical gradient); represents an illumination diagram; Represents the input low-quality endoscopic image.

[0037] In order to further constrain the decomposition of the decomposition network model so that the decomposed illumination map and reflectance map can reproduce the original input image, the reconstruction loss is introduced. , expressed as: ; In the formula, Indicates calculation of L1 norm; Represents a reflection graph; Represents element-wise multiplication.

[0038] The histogram equalization prior loss, illumination smoothing loss and reconstruction loss are taken as the total loss of the decomposition network model. The weight coefficient of the decomposition network model is set to 0.1, and the hyperparameters Set to 0.01, train the decomposition network model in an unsupervised learning manner, use the Adam optimizer for optimization, the initial learning rate is 0.0001, and train for 200 epochs; the total loss of the decomposition network model It is expressed as: ; In the formula, and are the weight coefficients of the decomposition network.

[0039] like Figure 3As shown in FIG. 1 , the noise decoupling module is composed of multiple convolution blocks; the reflection image first passes through the sixth convolution block, the output of the sixth convolution block is downsampled for the first time, the output of the first downsampled is passed through the seventh convolution block, the output of the seventh convolution block is downsampled for the second time, the output of the second downsampled is passed through the eighth convolution block, the output of the eighth convolution block is downsampled for the third time, the output of the third downsampled is passed through the ninth convolution block, the output of the ninth convolution block is upsampled for the first time, and the output of the first upsampled and the output of the eighth convolution block are concatenated on the channel after a self-attention mechanism. The output of the tenth convolution block is then taken as the input of the tenth convolution block, the output of the tenth convolution block is upsampled for the second time, the output of the second upsampling and the output of the seventh convolution block are concatenated on the channel after a self-attention mechanism as the input of the eleventh convolution block, the output of the eleventh convolution block is upsampled for the third time, the output of the third upsampling and the output of the sixth convolution block are concatenated on the channel after a self-attention mechanism as the input of the twelfth convolution block, and finally the input of the twelfth convolution block and the reflection map are element-wise added to obtain the denoised image (clean reflection map and reflection map noise).

[0040] Among them, each convolution block consists of two 3×3 convolutions, and each convolution is followed by a ReLU activation function.

[0041] like Figure 4 As shown in the figure, the noise synthesis module consists of six convolutional layers and three residual blocks. First, the input image (clean reflection map or noise-free natural image) and the input noise (reflection map noise) are concatenated on the channel as the input of the third convolutional layer (7×7). The output of the third convolutional layer passes through the fourth convolutional layer (3×3) and the fifth convolutional layer (3×3) in sequence. The output of the fifth convolutional layer passes through the first residual block. The output of the first residual block is element-wise added to the output of the fifth convolutional layer as the input of the second residual block. The output of the second residual block is element-wise added to the output of the first residual block as the input of the third residual block. The output of the third residual block passes through the sixth convolutional layer (3×3), the seventh convolutional layer (3×3) and the eighth convolutional layer (7×7) in sequence. The output of the eighth convolutional layer is element-wise added to the input image (clean reflection map or noise-free natural image) to obtain the final synthesized noise image (pseudo natural image).

[0042] Among them, the output ends of the third convolutional layer (7×7), the fourth convolutional layer (3×3), the fifth convolutional layer (3×3), the sixth convolutional layer (3×3) and the seventh convolutional layer (3×3) are all connected to a LeakyReLU activation function; each residual block is composed of two convolutional layers (3×3) connected in sequence, and the output end of each convolutional layer (3×3) is connected to a LeakyReLU activation function.

[0043] The noise synthesis module plays the role of noise migration, migrating the noise in the original reflection map to the original natural image. In order to avoid migrating the content and details in the original reflection map to the original natural image, and to make the synthesized reflection map and the synthesized noisy natural image consistent with the original reflection map and the original natural image content structure, the content consistency loss is introduced. Optimize the noise synthesis module, and its calculation formula is as follows: ; In the formula, Represents a blur kernel size of Gaussian filter operator; Represents SSIM loss, which is used to constrain the structural information of the image; and Represent the original reflection map and the synthetic reflection map respectively; and is the weight coefficient of content consistency loss. Content consistency loss consists of two parts: L1 loss of Gaussian filtering and structural similarity loss. The former constrains the low-frequency information of the image, and the latter constrains the structural information of the image. The parameters of the noise synthesis module are continuously optimized through content consistency loss to generate high-quality synthetic noise images.

[0044] Introducing least squares loss as adversarial loss It is expressed as: ; In the formula, is a noise-free natural image; It is a pseudo-natural image; is the discriminator; is the generator. The parameters of the noise decoupling module can be continuously optimized through adversarial loss, so that the denoising effect is optimal.

[0045] Among them, the denoising network model uses an unpaired dataset for joint training in an unsupervised learning manner, and uses the total loss function of the denoising network model: , where the weight coefficient of content consistency loss is and are set to 0.1 and 1 respectively, and the kernel size The training was set to 5, 200 epochs were trained, the learning rate was 0.0001, the stochastic gradient descent algorithm (SGD) was used for optimization, and the optimal model parameters were saved after all training was completed.

[0046] The unpaired dataset is formed by combining a public natural image dataset (LOL natural image dataset, DICM natural image dataset) with a public endoscopic image dataset (SUN endoscopic image dataset).

[0047] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An unsupervised endoscopic image enhancement method, characterized in that: The steps include: Step S1: Collect low-quality endoscopic images and high-quality noise-free natural images to construct a dataset; Step S2: construct a decomposition network model, input the low-quality endoscopic image into the decomposition network model, and obtain a reflectance map and an illumination map; Step S3: performing histogram equalization HE on the low-quality endoscopic image to obtain a histogram equalization prior, and inputting the histogram equalization prior and the reflection map into a pre-trained ConvNeXt network model to obtain intermediate features; Step S4: Calculate the loss of the histogram equalization prior according to the intermediate features, and introduce the illumination smoothing loss and reconstruction loss as the total loss of the decomposition network model for training; Step S5: constructing a denoising network model, wherein the denoising network model consists of a noise decoupling module and a noise synthesis module; Step S6: input the reflection image into the noise decoupling module to obtain a clean reflection image and reflection image noise, input the clean reflection image and reflection image noise as a pair and the noise-free natural image and reflection image noise as a pair into the noise synthesis module respectively to obtain a synthesized reflection image and a synthesized natural image, input the synthesized natural image into the noise decoupling module to obtain a pseudo natural image; introduce content consistency loss and adversarial loss as the total loss of the denoising network model for training, input the pseudo natural image into the trained denoising network model to obtain the final clean reflection image; Step S7: construct a discriminator, which uses a generative adversarial network; The pseudo natural image and the noise-free natural image obtained in step S6 are input into the discriminator as a group of image pairs. The noise-free natural image and the pseudo natural image are identified by the discriminator, and the least squares loss is introduced as the adversarial loss to optimize the parameters of the noise decoupling module.

2. The unsupervised endoscopic image enhancement method according to claim 1, characterized in that: The decomposition network model consists of two convolutional layers and five convolutional blocks; first, the low-quality endoscopic image is normalized, and then the normalized image is passed through the first convolutional layer 5×5, the output of the first convolutional layer 5×5 is subjected to the first wavelet transform to obtain the first high-frequency component and the first low-frequency component respectively, the first low-frequency component is subjected to the second wavelet transform after passing through the first convolutional block to obtain the second high-frequency component and the second low-frequency component, the second low-frequency component is subjected to the third wavelet transform after passing through the second convolutional block to obtain the third high-frequency component and the third low-frequency component, the third high-frequency component is element-wise multiplied with the attention map, and the third low-frequency component is converted into the third high-frequency component. The output of the third convolution block and the result of the third element-wise multiplication of the high-frequency component and the attention map are concatenated on the channel for the first inverse wavelet transform. The output of the first inverse wavelet transform is concatenated on the channel with the result obtained by passing the fourth convolution block and the result obtained by element-wise multiplication of the second high-frequency component and the attention map for the second inverse wavelet transform. The output of the second inverse wavelet transform is concatenated on the channel with the result obtained by passing the fifth convolution block and the result obtained by element-wise multiplication of the first high-frequency component and the attention map for the third inverse wavelet transform. The output of the third inverse wavelet transform is passed through the second convolution layer 5×5 to obtain a reflectance map and an illumination map. The attention map is composed of the value 1 minus the normalized low-quality endoscopic image.

3. The unsupervised endoscopic image enhancement method according to claim 2, characterized in that: The noise decoupling module consists of multiple convolution blocks; the reflection map first passes through the sixth convolution block, the output of the sixth convolution block is downsampled for the first time, the output of the first downsampling passes through the seventh convolution block, the output of the seventh convolution block is downsampled for the second time, the output of the second downsampling passes through the eighth convolution block, the output of the eighth convolution block is downsampled for the third time, the output of the third downsampling passes through the ninth convolution block, the output of the ninth convolution block is upsampled for the first time, and the output of the first upsampling and the output of the eighth convolution block are spliced ​​on the channel after a self-attention mechanism. As the input of the tenth convolution block, the output of the tenth convolution block is upsampled for the second time, and the output of the second upsampled image and the output of the seventh convolution block are concatenated on the channel after a self-attention mechanism as the input of the eleventh convolution block. The output of the eleventh convolution block is upsampled for the third time, and the output of the third upsampled image and the output of the sixth convolution block are concatenated on the channel after a self-attention mechanism as the input of the twelfth convolution block. Finally, the input of the twelfth convolution block and the reflection map are element-wise added to obtain the denoised image, i.e., the clean reflection map and the reflection map noise.

4. The unsupervised endoscopic image enhancement method according to claim 3, characterized in that: Each convolution block consists of two 3×3 convolutions, and each convolution is followed by a ReLU activation function.

5. The unsupervised endoscopic image enhancement method according to claim 4, characterized in that: The noise synthesis module consists of six convolutional layers and three residual blocks. First, the clean reflection map or noise-free natural image and the reflection map noise are spliced ​​on the channel as the input of the third convolutional layer 7×7, the output of the third convolutional layer passes through the fourth convolutional layer 3×3 and the fifth convolutional layer 3×3 in sequence, the output of the fifth convolutional layer passes through the first residual block, the output of the first residual block and the output of the fifth convolutional layer are element-wise added as the input of the second residual block, the output of the second residual block and the output of the first residual block are element-wise added as the input of the third residual block, the output of the third residual block passes through the sixth convolutional layer 3×3, the seventh convolutional layer 3×3 and the eighth convolutional layer 7×7 in sequence, the output of the eighth convolutional layer is element-wise added to the clean reflection map or noise-free natural image to obtain the final synthesized noise image, i.e., the pseudo natural image.

6. The unsupervised endoscopic image enhancement method according to claim 5, characterized in that: The output ends of the third convolutional layer 7×7, the fourth convolutional layer 3×3, the fifth convolutional layer 3×3, the sixth convolutional layer 3×3 and the seventh convolutional layer 3×3 are all connected to a LeakyReLU activation function; each residual block is composed of two convolutional layers 3×3 connected in sequence, and the output end of each convolutional layer 3×3 is connected to a LeakyReLU activation function.

7. The unsupervised endoscopic image enhancement method according to claim 6, characterized in that: The histogram equalization prior loss , expressed as: ； In the formula, Indicates intermediate features; Represents a reflection graph; represents the histogram equalization prior; Indicates the calculation of the mean square error between the two; The histogram equalization prior loss, illumination smoothing loss and reconstruction loss are introduced as the total loss of the decomposition network model. The weight coefficient of the decomposition network model is set to 0.1, and the hyperparameters Set to 0.01, train the decomposition network model in an unsupervised learning manner, and use the Adam optimizer for optimization; Decomposition of the total loss of the network model It is expressed as: ； In the formula, and are the weight coefficients of the decomposition network; represents the reconstruction loss; Represents the lighting smoothing loss.

8. The unsupervised endoscopic image enhancement method according to claim 7, characterized in that: Introducing content consistency loss The noise synthesis module is optimized, and the least square loss is introduced as the adversarial loss to optimize the parameters of the noise decoupling module; the denoising network model is jointly trained in an unsupervised learning manner using an unpaired dataset, and the total loss function of the denoising network model is expressed as: ； In the formula, represents the least squares loss; The unpaired data set is formed by combining a public natural image data set and a public endoscopic image data set.

9. The unsupervised endoscopic image enhancement method according to claim 1, characterized in that: The specific process of step S1 is: first, a large number of different types of low-quality endoscopic images are collected from the existing public endoscopic datasets, and then a large number of high-quality noise-free natural images are collected from the existing natural image datasets, and the low-quality endoscopic images and high-quality noise-free natural images constitute a dataset.

Images