Low-light medical image enhancement method, system and device based on brightness guidance and detail restoration and medium

Abstract

The application belongs to the technical field of image processing and the technical field of computer-aided medical diagnosis, and discloses a low-light medical image enhancement method, system, device and medium based on brightness guidance and detail restoration. The method comprises the following steps: processing an input low-light image through an initial brightness adjuster to obtain a preliminary brightness image and corresponding image features; inputting the preliminary brightness image and the corresponding image features into a dynamic brightness-guided detail restoration device, optimizing the initial brightness image through a dynamic brightness refinement module, simultaneously restoring image structure texture and detail information by using a two-stage frequency domain perception attention unit, outputting an enhanced image, and constraining the enhanced image and a paired reference image by using a multi-frequency consistency loss. The application realizes more natural and better medical image enhancement, and provides high-quality image input for intelligent analysis and clinical interpretation of medical images.

Description

Technical Field

[0001] This application relates to the fields of image processing technology and computer-aided medical diagnosis technology, and in particular to a method, system, device and medium for enhancing low-light medical images based on brightness guidance and detail restoration. Background Technology

[0002] Medical images play a crucial role in disease identification, diagnosis, and clinical decision-making. However, due to limitations in equipment performance, operational errors, and patient factors, images are often acquired in low-light environments, resulting in insufficient brightness, uneven illumination, and low contrast. This severely weakens the expression of key diagnostic information and affects the accuracy and reliability of clinical judgment. Therefore, there is an urgent need to develop an efficient and practical low-light medical image enhancement method that can improve brightness and structural contrast while maintaining the authenticity of anatomical structures and restoring color fidelity, providing high-quality image input for intelligent analysis and clinical interpretation.

[0003] Currently, many low-light image enhancement methods utilize static illumination priors to identify underexposed areas, such as manually designed strategies like inverse brightness maps or RGB channel mean estimation. While these methods are computationally simple and intuitive, in practical applications they often neglect image structural information and are susceptible to noise interference, leading to inaccurate illumination estimation and limited enhancement effects. To improve the quality of low-light images, numerous studies in recent years have introduced attention mechanisms, aiming to adaptively focus on information-rich regions to enhance local contrast and highlight key structures. However, most existing methods only model in the spatial domain, failing to fully exploit supplementary information in the frequency domain. Ignoring frequency features affects the model's ability to comprehensively model both local and global aspects, reducing enhancement quality. Therefore, some studies have begun to attempt to introduce frequency domain modeling to compensate for the shortcomings of spatial modeling, but problems remain, such as separate processing of spatial and frequency features and a lack of a unified modeling framework with cross-domain consistency, limiting their further application in medical scenarios. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a method, system, device, and medium for enhancing low-light medical images based on brightness guidance and detail restoration, thereby resolving the problems of insufficient brightness and blurred details in medical images under low-light environments.

[0005] According to the first aspect of this application, a low-light medical image enhancement method based on brightness guidance and detail restoration is provided, the method comprising:

[0006] The input low-light image is processed by an initial brightness adjuster to obtain a preliminary brightness map and its corresponding image features;

[0007] The initial brightness map and its corresponding image features are input into the dynamic brightness-guided detail restorer. The initial brightness map is optimized by the dynamic brightness refinement module. At the same time, the image structure, texture and detail information are restored by the dual-stage frequency domain perception attention unit, and the enhanced image is output.

[0008] The initial brightness adjuster and the dynamic brightness-guided detail restorer are combined to form an image enhancement model. The image enhancement model is trained to enhance low-light medical images. During the training process, the enhanced image and the paired reference image are constrained by multi-domain consistency loss.

[0009] Furthermore, the dynamic brightness refinement module is integrated into each stage of the encoder and decoder, and the inputs of each stage are different. The initial brightness map is optimized through the dynamic brightness refinement module, including:

[0010] The brightness map of layer i-1 and its corresponding image features F (i-1) As input to the dynamic brightness refinement module;

[0011] Brightness map at layer i-1 Under adaptive guidance, updated image features F are extracted through a two-stage frequency domain perception attention unit. (i) ;

[0012] Calculate F (i) With F (i-1) The difference is used to generate a difference map;

[0013] Perform average pooling on the difference map to extract regional brightness variations.

[0014] In the encoder, Directly used to guide feature enhancement;

[0015] In the decoder, Brightness diagram connected to jump The enhanced brightness map is obtained by fusion and then combined with... Add them together to generate a refined brightness map.

[0016] Furthermore, the dual-stage frequency domain sensing attention unit includes a spatial guidance enhancement module and a frequency sensing enhancement module, with a frequency sensing feedforward network set between the spatial guidance enhancement module and the frequency sensing enhancement module, and a frequency sensing feedforward network connected after the frequency sensing enhancement module;

[0017] Methods for recovering image structure, texture, and detail information using a two-stage frequency domain perceptual attention unit include:

[0018] Image features F (i) and brightness diagram As input to the space guidance enhancement module; for F (i) Perform convolution processing to generate the first query tensor Q. s First key tensor K s and the first value tensor V s ; through the first-valued tensor V s With brightness diagram Element-wise multiplication generates brightness-perceived features, and the brightness-perceived features are then passed through convolutional projection and residual connection to obtain the first feature output by the spatially guided enhancement module.

[0019] Using the first feature as the input to the frequency-aware feedforward network, the first feature Y is encoded through convolution. s Obtain the second feature Y f ; the second feature Y f It is divided into two parallel branches, namely the first branch Y f1 Second branch Y f2 The first branch Y f1 The third feature G is calculated using a squeeze-excitation mechanism in the frequency domain, and then the third feature G is compared with the first branch Y. f1 The fourth feature Z' is obtained by concatenating the components and fusing them through a 1×1 convolution; the second branch Y f2 Preserving complementary spatial details not explicitly encoded in the frequency enhancement path, through the fourth feature Z' and the second branch Y f2 The fifth feature Z is obtained by gating modulation between them;

[0020] Using the fifth feature Z as the input to the frequency-aware enhancement module, the second query tensor Q is generated through convolution. f Second key tensor K f Second-valued tensor V f The second query tensor Q f Second bond tensor K f The tensor is reshaped and divided into blocks, and each block is transformed to the frequency domain to calculate frequency domain attention. Then, spatial alignment is restored through inverse FFT to obtain the adjusted second query tensor and second key tensor. This is then analyzed using a brightness map. Modulation V f Based on the modulated second query tensor Q f Second key tensor K f Second-valued tensor V f The output characteristics of the frequency sensing enhancement module are obtained.

[0021] Furthermore, in the spatial guidance enhancement module, the first query tensor Q is generated using the following formula. s First key tensor Ks and the first value tensor V s :

[0022] Q s =DConv 3×3 (Conv 1×1 (F (i) )) (1)

[0023] K s V s =Split(DConv) 3×3 (Conv 1×1 (F (i) (2)

[0024] In the formula, DConv 3×3 Represents a 3×3 depthwise convolution, Conv 1×1 This represents a 1×1 convolution, and Split represents a splitting operation;

[0025] Through the first-valued tensor V s With brightness diagram The element-wise multiplication generates brightness-perceived features, and these features are then processed through convolutional projection and residual concatenation to obtain the first feature output by the spatially guided enhancement module. The calculation formula is as follows:

[0026]

[0027] In the formula, As represents the attention weight, Softmax represents the normalized exponential function, T represents the weight transpose, ⊙ represents element-wise multiplication, and α represents the scaling factor.

[0028] Furthermore, the first branch Y f1 The formula for calculating the third feature G using the squeeze-excitation mechanism in the frequency domain is as follows:

[0029] G=SE(ζ(Y f1 ))) (5)

[0030] In the formula, SE represents the squeeze-excitation module, and ζ(·) represents the Fourier transform.

[0031] Furthermore, the modulation of the second bond tensor is achieved through the following formula:

[0032] E = ζ -1 (ζ(K f-patch )⊙ζ(K f-patch (6)

[0033] In the formula, ζ -1 Represents the inverse FFT, ⊙ represents element-wise multiplication, E represents the modulated bond tensor, ζ(·) represents the Fourier transform, and K f-patchThis represents the frequency domain key feature block obtained after reshaping and segmenting the second key tensor in the frequency sensing enhancement module.

[0034] Furthermore, the multi-domain consistency loss includes spatial consistency loss and frequency domain consistency loss; wherein, the spatial consistency loss is expressed as:

[0035]

[0036] In the formula, L SF This is the spatial consistency loss, where m is the number of images and t is the image index. It is the enhanced image of the t-th image. It is the reference image for the t-th image;

[0037] The frequency domain consistency loss includes amplitude loss, phase loss, and spectral structure similarity loss, as shown in formulas (8)-(10) below:

[0038]

[0039] In the formula, L Amp L Pha and L Spe-SSIM Let represent amplitude loss, phase loss, and spectral structure similarity loss, respectively. Let C represent the number of image channels, and l, h, and s represent the indices of the image channels. It enhances the amplitude components of the image and the reference image in the l-th channel. It enhances the phase components of the image and the reference image in the h-th channel. SSIM represents the amplitude components of the enhanced image and the reference image in the s-th channel, and SSIM denotes the similarity function.

[0040] According to the second technical solution of this application, a low-light medical image enhancement system based on brightness guidance and detail restoration is provided, the system comprising:

[0041] The preliminary processing module is configured to process the input low-light image through an initial brightness adjuster to obtain a preliminary brightness map and its corresponding image features;

[0042] The image enhancement module is configured to input the initial brightness map and its corresponding image features into a dynamic brightness-guided detail restorer, optimize the initial brightness map through a dynamic brightness refinement module, and simultaneously restore the image structure, texture and detail information using a dual-stage frequency domain perceptual attention unit to output an enhanced image.

[0043] The model training module is configured to combine the initial brightness adjuster with the dynamic brightness-guided detail restorer to form an image enhancement model, and train the image enhancement model to achieve low-light medical image enhancement; wherein, during the training process, the enhanced image and the paired reference image are constrained by multi-domain consistency loss.

[0044] According to the third technical solution of this application, an electronic device is provided, the electronic device comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the method described above.

[0045] According to the fourth technical solution of this application, a non-transitory computer-readable storage medium storing instructions is provided, which, when executed by a processor, performs the method described above.

[0046] The low-light medical image enhancement methods, systems, devices, and media based on brightness guidance and detail restoration according to the various schemes of this application have at least the following technical effects:

[0047] The core of this application lies in the design of a dynamic brightness refinement module and a two-stage frequency domain awareness attention mechanism. The dynamic brightness refinement module further refines the brightness, while the two-stage frequency domain awareness attention mechanism restores image details under the guidance of dynamic brightness. Through the synergistic effect of these two components, this application achieves more natural and effective medical image enhancement, providing high-quality image input for intelligent analysis and clinical interpretation of medical images.

[0048] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0049] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0050] Figure 1 A flowchart illustrating a low-light medical image enhancement method based on brightness guidance and detail restoration, provided in this application embodiment;

[0051] Figure 2 This application provides a network framework diagram for implementing a low-light medical image enhancement method.

[0052] Figure 3 This is a data processing flowchart of the dynamic brightness refinement module provided in an embodiment of this application;

[0053] Figure 4 A framework diagram of the dynamic brightness refinement module provided in an embodiment of this application;

[0054] Figure 5 A framework diagram of a two-stage frequency domain sensing attention unit provided in an embodiment of this application;

[0055] Figure 6 This is a data processing flowchart of a two-stage frequency domain sensing attention unit provided in an embodiment of this application;

[0056] Figure 7 This is a structural diagram of a low-light medical image enhancement system based on brightness guidance and detail restoration, provided in an embodiment of this application. Detailed Implementation

[0057] To enable those skilled in the art to better understand the technical solution of this application, the application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] This application provides a low-light medical image enhancement method based on brightness guidance and detail restoration. Please refer to... Figure 1 This is a flowchart illustrating a low-light medical image enhancement method based on brightness guidance and detail restoration, provided in an embodiment of this application. The method includes steps S10-S30. Please refer to... Figure 2 This is a network framework diagram for implementing a low-light medical image enhancement method provided in this application embodiment. This network framework is named the Image Enhancement Model (LGDENet model) in this application. The LGDENet model mainly consists of two parts: an initial brightness adjuster and a dynamically brightness-guided detail restorer. The initial brightness adjuster is used to perform... Figure 1 S10, the dynamic brightness-guided detail restorer, is used to perform... Figure 1 The following section will detail the network framework, its data processing mechanism, and the corresponding effects of its related modules, based on the specific implementation process of S20.

[0059] Please see Figure 2 The LGDENet model proposed in this application first uses an initial brightness adjuster. This initial brightness adjuster processes the input low-light image I from the perspective of brightness adjustment to obtain an initial brightness map and generate a preliminary enhanced image I. lu Due to imperfections in illumination estimation and the loss of structural details, image sharpness is still somewhat affected. LGDENet further inputs the initial enhanced image I into a Dynamic Luminance-Guided Detail Restorer (DLDR). Within the DLDR, the initial luminance map is optimized by a Dynamic Luminance Refinement Module (DLRM), while selectively restoring the image's structural texture and detail information using Two-Stage Frequency Domain Aware Attention (TFAA). The DLDR module, based on a U-shaped structure design, effectively integrates multi-scale features, ultimately outputting an enhanced image I with uniform illumination and rich structure.en During the network training phase, multi-domain consistency loss is used to enhance image I. en Constrained with the paired reference image.

[0060] Based on the data processing flow of the LGDENet model described above, this low-light medical image enhancement method based on brightness guidance and detail restoration begins in step S10, where the input low-light image is processed by an initial brightness adjuster to obtain a preliminary brightness map and its corresponding image features.

[0061] For example, the operation within the initial brightness adjuster is to perform average grayscale conversion on the low-light input to obtain the illumination prior. The illumination prior and the low-light input are merged in the channel dimension and then passed through three convolutional blocks to obtain the preliminary brightness map and the corresponding feature map.

[0062] Next, in step S20, the initial brightness map and its corresponding image features are input into the dynamic brightness-guided detail restorer. The initial brightness map is optimized by the dynamic brightness refinement module, and the image structure, texture and detail information are restored by the dual-stage frequency domain perception attention unit, and the enhanced image is output.

[0063] In some embodiments, such as Figure 3 The diagram shows the data processing flowchart of the dynamic brightness refinement module. This module is integrated into various stages of the encoder and decoder, and the inputs of each stage are different. The dynamic brightness refinement module optimizes the initial brightness map, including:

[0064] S201, using the brightness diagram of the (i-1)th layer and its corresponding image features F (i-1) As input to the dynamic brightness refinement module;

[0065] S202, Brightness diagram at layer i-1 Under adaptive guidance, updated image features F are extracted through a two-stage frequency domain perception attention unit. (i) ;

[0066] S203. Calculate the updated image features F (i) With the input image features F (i-1) The difference is used to generate a difference map;

[0067] S204. Perform average pooling on the difference map to extract the changes in regional brightness.

[0068] S205. In the encoder, the area brightness change Directly used to guide feature enhancement;

[0069] S206. In the decoder, the area brightness change Brightness diagram connected to jump The enhanced brightness map is obtained by fusion and then combined with... Add them together to generate a refined brightness map.

[0070] In some embodiments, such as Figure 4 As shown, Figure 4 This is a framework diagram of the Dynamic Luminance Refinement Module (DLRM). To adaptively optimize the initial luminance and improve the expressive power of spatial features, this application designs a Dynamic Luminance Refinement Module (DLRM). The DLRM is integrated into various stages of the encoder and decoder, with each stage receiving a different input. Specifically, the DLRM uses the luminance map of the (i-1)th layer... and its corresponding feature F (i-1) As input. In the brightness map Under adaptive guidance, the updated feature F is extracted through TAFF. (i) By calculating F (i) With F (i-1) The difference is used to generate a difference map to highlight underexposed or overexposed areas. Average pooling is then applied to the difference map to extract the brightness variations in the regions. As a guiding signal for subsequent feature enhancement. In the encoder, It is used directly for guiding feature enhancement; in the decoder, it is further connected to the luminance map of skip connections. This process involves fusion to achieve refined and context-aware brightness modulation. Finally, the enhanced brightness map is then combined with... Add them together to generate a refined brightness map.

[0071] In some embodiments, such as Figure 5 The diagram shows the framework of the two-stage frequency-domain aware attention unit (TAFF). The TAFF is used to enhance structural information. It includes a spatially guided enhancement module (SGEA) and a frequency-aware enhancement module (FSEA), with a frequency-aware feedforward network between them. Each SGEA module is followed by a frequency-aware feedforward network (FAFN) to modulate feature learning using frequency domain information.

[0072] like Figure 6 The diagram shows the data processing flowchart of the dual-stage frequency domain perceptual attention unit. The method of restoring image structure texture and detail information using the dual-stage frequency domain perceptual attention unit includes the following steps S210-S230.

[0073] S210, using image features F (i) and brightness diagram As input to the space guidance enhancement module; for F (i) Perform convolution processing to generate the first query tensor Q. s First key tensor K s and the first value tensor V s ; through the first-valued tensor V s With brightness diagram The element-wise multiplication generates the brightness-perceived features, and the brightness-perceived features are then passed through convolutional projection and residual connection to obtain the first feature output by the spatially guided enhancement module.

[0074] In this embodiment, step S210 is executed through the Spatial Guidance Enhancement Module (SGEA). SGEA uses image features F (i) and brightness diagram As input. First, for F (i) Perform 1×1 and 3×3 convolutions to generate the first query tensor Q. s First key tensor K s and the first value tensor V s :

[0075] Q s =DConv 3×3 (Conv 1×1 (F (i) )) (1)

[0076] K s V s =Split(DConv) 3×3 (Conv 1×1 (F (i) (2)

[0077] In the formula, DConv 3×3 Represents a 3×3 depthwise convolution, Conv 1×1 represents a 1×1 convolution, and Split represents a splitting operation.

[0078] Subsequently, through the first-valued tensor V s With brightness diagram Element-wise multiplication generates brightness-perceived features to enhance information-rich regions and suppress irrelevant responses. Finally, the output features are reshaped and projected into a unified representation via a 1×1 convolution, and residual connections are added to obtain the first feature Y output by the spatially guided enhancement module. s To ensure stable training.

[0079]

[0080] In the formula, As represents the attention weight, Softmax represents the normalized exponential function, T represents the weight transpose, ⊙ represents element-wise multiplication, and α represents the scaling factor.

[0081] S220. Using the first feature as the input to the frequency-aware feedforward network, the first feature Y is encoded through convolution. s Obtain the second feature Y f ; the second feature Y f It is divided into two parallel branches, namely the first branch Y f1 Second branch Y f2 The first branch Y f1 The third feature G is calculated using a squeeze-excitation mechanism in the frequency domain, and then the third feature G is compared with the first branch Y. f1 The fourth feature Z' is obtained by concatenating the components and fusing them through a 1×1 convolution; the second branch Y f2 Preserving complementary spatial details not explicitly encoded in the frequency enhancement path, through the fourth feature Z' and the second branch Y f2 The fifth feature Z is obtained by gating modulation between them.

[0082] To establish semantic and structural connections between the spatial and frequency domains, this application designs a Frequency-Aware Feedforward Network (FAFN) to perform step S220 above. The FAFN outputs SGEA as the first feature Y. s As input, the first feature Y is first encoded using 1×1 and 3×3 convolutions. s Obtain the second feature Y f Subsequently, the second feature Y... f It is divided into two parallel branches, namely the first branch Y f1 Second branch Y f2 First branch Y f1 The third feature G is calculated using a squeeze-excitation mechanism in the frequency domain.

[0083] G=SE(ζ(Y f1 ))) (5)

[0084] In the formula, SE represents the squeeze-excitation module, and ζ(·) represents the Fourier transform.

[0085] Then combine the third feature G with the first branch Y f1 The fourth feature Z' is obtained by concatenating the components and fusing them through a 1×1 convolution; the second branch Y f2 Complementary spatial details not explicitly encoded in the frequency enhancement path are preserved. This is achieved through the fourth feature Z' and the second branch Y. f2 The fifth feature Z is obtained by gating modulation between the two sides. FAFN can adaptively fuse frequency-aware information and rich spatial information.

[0086] S230. Using the fifth feature Z as the input to the frequency-aware enhancement module, the second query tensor Q is generated through convolution. f Second key tensor K f Second-valued tensor V f The second query tensor Q f Second bond tensor K f The tensor is reshaped and divided into blocks, and each block is transformed to the frequency domain to calculate frequency domain attention. Then, spatial alignment is restored through inverse FFT to obtain the adjusted second query tensor and second key tensor. This is then analyzed using a brightness map. Modulation V f Based on the modulated second query tensor Q f Second key tensor K f Second-valued tensor V f The output characteristics of the frequency sensing enhancement module are obtained.

[0087] To better model diverse frequency-aware textures and spatial patterns, a Frequency-Aware Enhancement Module (FSEA) was designed to perform step S230 above. The FSEA takes the fifth feature Z as input and generates a second query tensor Q through 1×1 and 3×3 convolutions. f Second key tensor K f Second-valued tensor V f Subsequently, the second query tensor Q... f Second bond tensor K f The tensor is reshaped and divided into blocks, and each block is transformed to the frequency domain to calculate frequency domain attention. Spatial alignment is then restored using inverse FFT to obtain the adjusted second query tensor and second key tensor. Taking the second key tensor as an example, modulation of the second key tensor is achieved using the following formula:

[0088] E = ζ -1 (ζ(K f-patch )⊙ζ(K f-patch (6)

[0089] In the formula, ζ -1 Represents the inverse FFT, ⊙ represents element-wise multiplication, E represents the modulated bond tensor, ζ(·) represents the Fourier transform, and K f-patch This represents the frequency domain key feature block obtained after reshaping and segmenting the second key tensor in the frequency sensing enhancement module.

[0090] Furthermore, through the brightness map Modulation V f To enhance brightness-related features and suppress insignificant responses.

[0091] Based on the modulated second query tensor Q f Second key tensor K f Second-valued tensor Vf The output features of the frequency-aware enhancement module can be obtained. These output features are then fused across domains via a frequency-aware feedforward network (the processing flow of the frequency-aware feedforward network has been described previously and will not be repeated here). The resulting fused features are then residually concatenated with the image features obtained from the initial brightness adjuster to finally obtain the enhanced image I. en .

[0092] Finally, in step S30, the initial brightness adjuster and the dynamic brightness-guided detail restorer are combined to form an image enhancement model. The image enhancement model is trained to achieve low-light medical image enhancement. During the training process, the enhanced image and the paired reference image are constrained by multi-domain consistency loss.

[0093] In some embodiments, multi-domain consistency loss is employed for optimization, including spatial consistency (SF) loss and frequency domain consistency (FC) loss. The SF loss uses L1 loss to enhance the image I. en and reference image I re Pixel-wise similarity between them:

[0094]

[0095] In the formula, L SF This is the spatial consistency loss, where m is the number of images and t is the image index. It is the enhanced image of the t-th image. It is the reference image for the t-th image.

[0096] FC loss evaluates global similarity in the frequency domain by combining amplitude and phase information from the augmented and reference images, including Amp loss, Phaloss loss, and Spe-SSIM loss.

[0097] Amp loss uses logarithmic transform L1 distance to capture perceived contrast and intensity differences:

[0098]

[0099] To maintain good structural alignment, the Pha loss is defined using cosine similarity:

[0100]

[0101] To quantify the structural similarity in the frequency domain, this embodiment introduces the Spe-SSIM loss:

[0102]

[0103] In equations (8)-(10), L Amp LPha and L Spe-SSIM Let represent amplitude loss, phase loss, and spectral structure similarity loss, respectively. Let C represent the number of image channels, and l, h, and s represent the indices of the image channels. It enhances the amplitude components of the image and the reference image in the l-th channel. It enhances the phase components of the image and the reference image in the h-th channel. SSIM represents the amplitude components of the enhanced image and the reference image in the s-th channel, and SSIM denotes the similarity function.

[0104] The following section will introduce the dataset source, training environment, optimizer settings, batch size, and number of iterations during the training phase.

[0105] This embodiment uses the publicly available medical low-light dataset RLE, which consists of 1280 pairs of capsule endoscopy images with low brightness, low contrast, and color distortion, and their paired normal-light images. For experimental convenience, all images in this embodiment are uniformly resized to 128×128 for model training.

[0106] The network model (image enhancement model) in this embodiment is implemented using the PyTorch deep learning framework and runs on the Ubuntu 18.04 operating system. The network model uses the Adam optimizer with an initial learning rate of 2×10⁻⁶. -4 Following the cosine annealing method, at 9.2 × 10 4 Step decay is 3×10 -4 , at 2.08×10 5 Step decay is 1×10 -6 The batch size of the training data during training is 8.3 × 10⁻⁶. 5 The next iteration.

[0107] This embodiment uses Peak Signal-to-Noise Ratio (PSNR), a widely accepted metric in the field of medical image and video enhancement, to measure network performance. Generally, a higher PSNR indicates better performance.

[0108] The experimental results presented in this embodiment show that the PSNR of the method proposed in this application is 35.0922 on the publicly available RLE dataset, demonstrating the feasibility and progressiveness of this application.

[0109] Another aspect of this application provides a low-light medical image enhancement system based on brightness guidance and detail restoration, such as... Figure 7 The diagram shown is a structural diagram of a low-light medical image enhancement system based on brightness guidance and detail restoration provided in an embodiment of this application. The low-light medical image enhancement system based on brightness guidance and detail restoration includes:

[0110] The preliminary processing module 701 is configured to process the input low-light image through an initial brightness adjuster to obtain a preliminary brightness map and its corresponding image features.

[0111] Image enhancement module 702 is configured to input the preliminary brightness map and its corresponding image features into a dynamic brightness-guided detail restorer, optimize the initial brightness map through a dynamic brightness refinement module, and simultaneously restore the image structure, texture and detail information using a dual-stage frequency domain perceptual attention unit to output an enhanced image.

[0112] The model training module 703 is configured to combine the initial brightness adjuster and the dynamic brightness-guided detail restorer to form an image enhancement model, train the image enhancement model, and use the trained image enhancement model to enhance low-light medical images; wherein, during the training process, the enhanced image and the paired reference image are constrained by multi-domain consistency loss.

[0113] In some embodiments, the dynamic brightness refinement module is integrated into various stages of the encoder and decoder, and the input to each stage is different. The image enhancement module is further configured to:

[0114] The brightness map of layer i-1 and its corresponding image features F (i-1) As input to the dynamic brightness refinement module;

[0115] Brightness map at layer i-1 Under adaptive guidance, updated image features F are extracted through a two-stage frequency domain perception attention unit. (i) ;

[0116] Calculate F (i) With F (i-1) The difference is used to generate a difference map;

[0117] Perform average pooling on the difference map to extract regional brightness variations.

[0118] In the encoder, Directly used to guide feature enhancement;

[0119] In the decoder, Brightness diagram connected to jump The enhanced brightness map is obtained by fusion and then combined with... Add them together to generate a refined brightness map.

[0120] In some embodiments, the dual-stage frequency domain sensing attention unit includes a spatial guidance enhancement module and a frequency sensing enhancement module, a frequency sensing feedforward network is provided between the spatial guidance enhancement module and the frequency sensing enhancement module, and a frequency sensing feedforward network is connected after the frequency sensing enhancement module.

[0121] The image enhancement module is further configured as follows:

[0122] Image features F (i) and brightness diagram As input to the space guidance enhancement module; for F (i) Perform convolution processing to generate the first query tensor Q. s First key tensor K s and the first value tensor V s ; through the first-valued tensor V s With brightness diagram Element-wise multiplication generates brightness-perceived features, and the brightness-perceived features are then passed through convolutional projection and residual connection to obtain the first feature output by the spatially guided enhancement module.

[0123] Using the first feature as the input to the frequency-aware feedforward network, the first feature Y is encoded through convolution. s Obtain the second feature Y f ; the second feature Y f It is divided into two parallel branches, namely the first branch Y f1 Second branch Y f2 The first branch Y f1 The third feature G is calculated using a squeeze-excitation mechanism in the frequency domain, and then the third feature G is compared with the first branch Y. f1 The fourth feature Z' is obtained by concatenating the components and fusing them through a 1×1 convolution; the second branch Y f2 Preserving complementary spatial details not explicitly encoded in the frequency enhancement path, through the fourth feature Z' and the second branch Y f2 The fifth feature Z is obtained by gating modulation between them;

[0124] Using the fifth feature Z as the input to the frequency-aware enhancement module, the second query tensor Q is generated through convolution. f Second key tensor K f Second-valued tensor V f The second query tensor Q f Second bond tensor K f The tensor is reshaped and divided into blocks, and each block is transformed to the frequency domain to calculate frequency domain attention. Then, spatial alignment is restored through inverse FFT to obtain the adjusted second query tensor and second key tensor. This is then analyzed using a brightness map. Modulation V f Based on the modulated second query tensor Qf Second key tensor K f Second-valued tensor V f The output characteristics of the frequency sensing enhancement module are obtained.

[0125] In some embodiments, the image enhancement module is further configured to, within the spatially guided enhancement module, generate a first query tensor Q using the following formula. s First key tensor K s and the first value tensor V s :

[0126] Q s =DConv 3×3 (Conv 1×1 (F (i) )) (1)

[0127] K s V s =Split(DConv) 3×3 (Conv 1×1 (F (i) (2)

[0128] In the formula, DConv 3×3 Represents a 3×3 depthwise convolution, Conv 1×1 This represents a 1×1 convolution, and Split represents a splitting operation;

[0129] Through the first-valued tensor V s With brightness diagram The element-wise multiplication generates brightness-perceived features, and these features are then processed through convolutional projection and residual concatenation to obtain the first feature output by the spatially guided enhancement module. The calculation formula is as follows:

[0130]

[0131] In the formula, As represents the attention weight, Softmax represents the normalized exponential function, T represents the weight transpose, ⊙ represents element-wise multiplication, and α represents the scaling factor.

[0132] In some embodiments, the first branch Y f1 The formula for calculating the third feature G using the squeeze-excitation mechanism in the frequency domain is as follows:

[0133] G=SE(ζ(Y f1 ))) (5)

[0134] In the formula, SE represents the squeeze-excitation module, and ζ(·) represents the Fourier transform.

[0135] In some embodiments, the image enhancement module is further configured to modulate the second key tensor using the following formula:

[0136] E = ζ -1 (ζ(K f-patch )⊙ζ(K f-patch (6)

[0137] In the formula, ζ -1 Represents the inverse FFT, ⊙ represents element-wise multiplication, E represents the modulated bond tensor, ζ(·) represents the Fourier transform, and K f-patch This represents the frequency domain key feature block obtained after reshaping and segmenting the second key tensor in the frequency sensing enhancement module.

[0138] In some embodiments, the multi-domain consistency loss includes spatial consistency loss and frequency domain consistency loss; wherein, the spatial consistency loss is expressed as:

[0139]

[0140] In the formula, L SF This is the spatial consistency loss, where m is the number of images and t is the image index. It is the enhanced image of the t-th image. It is the reference image for the t-th image;

[0141] The frequency domain consistency loss includes amplitude loss, phase loss, and spectral structure similarity loss, as shown in formulas (8)-(10) below:

[0142]

[0143] In the formula, L Amp L Pha and L Spe-SSIM Let represent amplitude loss, phase loss, and spectral structure similarity loss, respectively. Let C represent the number of image channels, and l, h, and s represent the indices of the image channels. It enhances the amplitude components of the image and the reference image in the l-th channel. It enhances the phase components of the image and the reference image in the h-th channel. SSIM represents the amplitude components of the enhanced image and the reference image in the s-th channel, and SSIM denotes the similarity function.

[0144] It should be noted that the low-light medical image enhancement device based on brightness guidance and detail restoration provided in the above embodiments and the low-light medical image enhancement method based on brightness guidance and detail restoration provided in the aforementioned embodiments belong to the same concept. The specific way in which each module and unit performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0145] Another aspect of this application provides an electronic device, including: a controller; and a memory for storing one or more programs, which, when executed by the controller, perform the methods described in the various embodiments above.

[0146] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0147] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various embodiments described above.

[0148] According to one aspect of the embodiments of this application, a computer system is also provided, including a Central Processing Unit (CPU), which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from storage into random access memory (RAM), such as performing the methods described above. Various programs and data required for system operation are also stored in the RAM. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0149] For example, a computer system includes a Central Processing Unit (CPU), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or loaded from storage into random access memory (RAM), such as executing the methods described in the above embodiments. The RAM also stores various programs and data required for system operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0150] The following components are connected to the I / O interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard drives; and communication sections including network interface cards such as LAN (Local Area Network) cards and modems. The communication sections perform communication processing via networks such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0151] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs various functions defined in the system of this application.

[0152] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0153] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0154] The module units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0155] The above embodiments are only used to illustrate this application and are not intended to limit this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of this application. Therefore, all equivalent technical solutions also fall within the scope of this application, and the patent protection scope of this application should be defined by the claims.

Claims

1. A low-light medical image enhancement method based on brightness guidance and detail restoration, characterized in that, The method includes: The input low-light image is processed by an initial brightness adjuster to obtain a preliminary brightness map and its corresponding image features; The initial brightness map and its corresponding image features are input into the dynamic brightness-guided detail restorer. The initial brightness map is optimized by the dynamic brightness refinement module. At the same time, the image structure, texture and detail information are restored by the dual-stage frequency domain perception attention unit, and the enhanced image is output. The initial brightness adjuster and the dynamic brightness-guided detail restorer are combined to form an image enhancement model. The image enhancement model is trained to enhance low-light medical images. During the training process, the enhanced image and the paired reference image are constrained by multi-domain consistency loss. The dynamic brightness refinement module is integrated into each stage of the encoder and decoder, and the input of each stage is different. The initial brightness map is optimized through the dynamic brightness refinement module, including: With the first Brightness map of the layer and their corresponding image features As input to the dynamic brightness refinement module; In the Brightness map of the layer Under adaptive guidance, updated image features are extracted through a two-stage frequency domain perception attention unit. ; calculate and The difference is used to generate a difference map; Perform average pooling on the difference map to extract regional brightness variations. ; In the encoder, Directly used to guide feature enhancement; In the decoder, Brightness diagram connected to jump The enhanced brightness map is obtained by fusion and then combined with... Add them together to generate a refined brightness map. .

2. The method according to claim 1, characterized in that, The dual-stage frequency domain sensing attention unit includes a spatial guidance enhancement module and a frequency sensing enhancement module. A frequency sensing feedforward network is set between the spatial guidance enhancement module and the frequency sensing enhancement module. A frequency sensing feedforward network is connected after the frequency sensing enhancement module. Methods for recovering image structure, texture, and detail information using a two-stage frequency domain perceptual attention unit include: Image features and brightness diagram As input to the space guidance enhancement module; for Perform convolution processing to generate the first query tensor. First key tensor and the first value tensor ; through the first-value tensor With brightness diagram Element-wise multiplication generates brightness-perceived features, and the brightness-perceived features are then passed through convolutional projection and residual connection to obtain the first feature output by the spatially guided enhancement module. The first feature is used as the input to the frequency-aware feedforward network, and the first feature is encoded through convolution. Obtain the second feature ; the second feature It is divided into two parallel branches, namely the first branch. Second branch First branch The third feature is calculated using a squeeze-excitation mechanism in the frequency domain. G， The third feature G With the first branch The fourth feature is obtained by splicing and fusing the data through 1×1 convolution. Second branch Preserving complementary spatial details not explicitly encoded in the frequency enhancement path, through the fourth feature Second branch The fifth feature is obtained by gating modulation between them. ; With the fifth feature Z As input to the frequency-aware enhancement module, a second query tensor is generated through convolution. Second key tensor Second-valued tensor , the second query tensor Second bond tensor The tensor is reshaped and divided into blocks, and each block is transformed to the frequency domain to calculate frequency domain attention. Then, spatial alignment is restored through inverse FFT to obtain the adjusted second query tensor and second key tensor. This is then analyzed using a brightness map. modulation Based on the modulated second query tensor Second key tensor Second-valued tensor The output characteristics of the frequency sensing enhancement module are obtained.

3. The method according to claim 2, characterized in that, In the spatial guidance enhancement module, the first query tensor is generated using the following formula. First key tensor and the first value tensor : （1） （2） In the formula, This represents a 3×3 depthwise convolution. Represents a 1×1 convolution. Split Indicates a splitting operation; Through the first value tensor With brightness diagram The element-wise multiplication generates brightness-perceived features, and these features are then processed through convolutional projection and residual concatenation to obtain the first feature output by the spatially guided enhancement module. The calculation formula is as follows: （3） （4） In the formula, As represents the attention weight, Softmax represents the normalized exponential function, and T represents the weight transpose. This indicates element-wise multiplication. This represents the scaling factor.

4. The method according to claim 2, characterized in that, First Branch The third feature is calculated using a squeeze-excitation mechanism in the frequency domain. G The calculation formula is: （5） In the formula, SE represents the extrusion-excitation module. This represents the Fourier transform.

5. The method according to claim 2, characterized in that, The modulation of the second bond tensor is achieved using the following formula: （6） In the formula, Indicates the inverse FFT. This indicates element-wise multiplication, and E represents the modulated bond tensor. Indicates Fourier transform, This represents the frequency domain key feature block obtained after reshaping and segmenting the second key tensor in the frequency sensing enhancement module.

6. The method according to any one of claims 1 to 5, characterized in that, The multi-domain consistency loss includes spatial consistency loss and frequency domain consistency loss; wherein, the spatial consistency loss is expressed as: （7） In the formula, L SF It is a spatial consistency loss. It refers to the number of images. t It is an index of images. It is the enhanced image of the t-th image. It is the reference image for the t-th image; The frequency domain consistency loss includes amplitude loss, phase loss, and spectral structure similarity loss, as shown in formulas (8)-(10) below: （8） （9） （10） In the formula, L Amp , L Pha and L Spe-SSIM These represent amplitude loss, phase loss, and spectral structure similarity loss, respectively. This represents the number of channels in an image. l , h and s Both represent the indexes of image channels. , It is in the l The amplitude components of the enhanced image and the reference image in each channel, , It is in the h Phase components of the enhanced image and reference image in each channel, , It is in the s The amplitude components of the enhanced image and the reference image in each channel, SSIM Represents the similarity function.

7. A low-light medical image enhancement system based on brightness guidance and detail restoration, characterized in that, The system includes: The preliminary processing module is configured to process the input low-light image through an initial brightness adjuster to obtain a preliminary brightness map and its corresponding image features; The image enhancement module is configured to input the initial brightness map and its corresponding image features into a dynamic brightness-guided detail restorer, optimize the initial brightness map through a dynamic brightness refinement module, and simultaneously restore the image structure, texture and detail information using a dual-stage frequency domain perceptual attention unit to output an enhanced image. The model training module is configured to combine the initial brightness adjuster with the dynamic brightness-guided detail restorer to form an image enhancement model, train the image enhancement model, and use the trained image enhancement model to enhance low-light medical images; wherein, during the training process, the enhanced image and the paired reference image are constrained by multi-domain consistency loss. The dynamic brightness refinement module is integrated into each stage of the encoder and decoder, and the input to each stage is different. The image enhancement module is further configured as follows: With the first Brightness map of the layer and their corresponding image features As input to the dynamic brightness refinement module; In the Brightness map of the layer Under adaptive guidance, updated image features are extracted through a two-stage frequency domain perception attention unit. ; calculate and The difference is used to generate a difference map; Perform average pooling on the difference map to extract regional brightness variations. ; In the encoder, Directly used to guide feature enhancement; In the decoder, Brightness diagram connected to jump The enhanced brightness map is obtained by fusion and then combined with... Add them together to generate a refined brightness map. .

8. An electronic device, characterized in that, The electronic device includes: Memory, used to store computer programs; A processor for executing the computer program to implement the method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing instructions, characterized in that, When the instructions are executed by the processor, the method according to any one of claims 1 to 6 is performed.

Images