An x-ray image hierarchical enhancement network method based on tone mapping fusion

Abstract

The application discloses an X-ray image hierarchical enhancement network method based on tone mapping fusion, and belongs to the technical field of image processing, and comprises the following steps: a two-stage cascaded depth curve estimation network is constructed and is used for performing hierarchical adaptive tone mapping on an X-ray image; an area weighting mechanism based on an adaptive mask is designed, dark area information is converted into a weight mask and is integrated into a structural similarity loss function, the model is focused on the enhancement of a low response area, and the output is optimized through mask guided structural similarity loss; and the application effectively improves the visual quality, detail performance and structural fidelity of the X-ray image in a complex scene, and is suitable for image enhancement and intelligent identification tasks in the fields of security check and medical treatment.

Description

Technical Field

[0001] This invention relates to a hierarchical enhancement network method for X-ray images based on tone mapping fusion, belonging to the field of image processing. Background Technology

[0002] In recent years, image enhancement technology has undergone a paradigm shift from traditional model-driven to deep learning data-driven approaches. Researchers have made significant progress in optimizing traditional methods and deploying lightweight deep learning models.

[0003] To improve image enhancement, researchers have proposed the Contrast-Limited Adaptive Histogram Equalization (CLAHE) technique. Traditional image processing methods based on CLAHE have shown certain advantages in different application scenarios, improving image contrast, highlighting details, and suppressing noise to a certain extent. However, these methods still have limitations. These methods mostly rely on fixed physical or mathematical models, making it difficult to adaptively adjust the enhancement effect according to the image content. In addition, traditional image enhancement methods have limited adaptability to complex situations such as noise, high dynamic range, and occlusion, resulting in unstable final enhancement effects.

[0004] Research on deep learning-based image enhancement methods focuses more on exploring innovations in deep network architectures, such as decomposition models inspired by Retinex theory, low-light image enhancement methods based on the U-net++ network architecture, and depth curve estimation networks. Although single-stage depth curve estimation networks (DCE-Net) can improve brightness, it is difficult to balance global grayscale balance and local weak detail restoration through a single enhancement, which can easily lead to insufficient enhancement of dark areas or overexposure of bright areas. Summary of the Invention

[0005] This invention aims to solve the technical problem of difficulty in balancing global grayscale balance and local weak detail restoration during X-ray image enhancement, resulting in insufficient enhancement of dark areas or overexposure of bright areas. It proposes a hierarchical enhancement network method for X-ray images based on tone mapping fusion.

[0006] The technical solution of the present invention:

[0007] A hierarchical enhancement network method for X-ray images based on tone mapping fusion includes the following steps:

[0008] S1: Input the preprocessed X-ray image into the first-level depth curve estimation network to perform global primary brightness and contrast compensation, and perform normalization processing to obtain the primary enhanced image;

[0009] S2: Generate an adaptive mask based on the primary enhanced image;

[0010] S3: The primary enhanced image is input into the second-level depth curve estimation network. Under the guidance of the adaptive mask, the local weak features, complex occlusions and microstructure regions in the image are finely adjusted in the second stage. After normalization, the secondary enhanced image is obtained.

[0011] S4: Perform dual-path fusion processing on the primary enhanced image and the secondary enhanced image:

[0012] The first approach involves directly adding the primary enhancement image and the secondary enhancement image, then normalizing the result to obtain the first fused image.

[0013] The second approach involves concatenating the primary and secondary enhancement images along the channel dimension, performing convolutional dimensionality reduction, and then normalizing the result to obtain the second fused image.

[0014] The first fused image is used as the final output image of the X-ray image hierarchical enhancement network method based on tone mapping fusion;

[0015] S5: Construct a multi-level structural similarity loss function and calculate the structural similarity index loss at each of the three levels:

[0016] The first level is the structural similarity index loss between the primary enhanced image and the normalized supervised image;

[0017] The second level is the mask-weighted structural similarity index loss between the first fused image and the normalized supervised image;

[0018] The third level is the structural similarity index loss between the second fused image and the normalized supervised image;

[0019] The structural similarity index losses at the three levels are weighted and summed to obtain the total loss. The network parameters are then updated based on backpropagation using the total loss.

[0020] Specifically, the first-level depth curve estimation network and the second-level depth curve estimation network constitute a two-level cascaded depth curve estimation network.

[0021] Specifically, the formula for calculating the adaptive mask is:

[0022] ,

[0023] in, For adaptive masking, For primary image enhancement, It is an adjustable power exponent.

[0024] Specifically, the value of the adaptive mask approaches 1 in dark areas of the image as the pixel value of the primary enhanced image decreases, and approaches 0 in bright areas of the image as the pixel value of the primary enhanced image increases. This is used to guide the network to focus on the dark areas that need enhancement during backpropagation.

[0025] Specifically, when calculating the mask-weighted structural similarity index loss of the second level, the pixels of the first fused image and the normalized supervision image are subjected to point-by-point power mask weighting, with the adaptive mask as the weighting index.

[0026] Specifically, the formula for calculating the mask-weighted structural similarity index loss is as follows:

[0027] ,

[0028] ,

[0029] in, For the first fused image, For normalized supervised images.

[0030] Specifically, in the process of constructing the multi-level structural similarity loss function, a region weighting strategy based on adaptive masks is introduced. An adaptive mask is generated by estimating the primary enhanced image output by the network based on the first-level depth curve. The total loss is obtained by weighting the structural similarity index losses of the three levels and summing them. The network parameters are updated by backpropagation based on the total loss.

[0031] The beneficial effects of this invention are:

[0032] This invention achieves hierarchical enhancement of X-ray images from global to local through a two-level cascaded network. Adaptive masking precisely focuses on dark areas to improve the enhancement's specificity, dual-path fusion fully integrates image features, and multi-level SSIM loss optimizes network training. This effectively improves the enhancement effect in areas with weak features and microstructures, making details in dark areas clearer and the overall enhancement effect superior. It is suitable for image enhancement and intelligent recognition tasks in security inspection and medical fields. Attached Figure Description

[0033] Figure 1 This is a flowchart of the procedure for this invention;

[0034] Figure 2 This is a schematic diagram of the parameter matrix generation module in the single-stage depth curve estimation network of the present invention;

[0035] Figure 3 This is a schematic diagram of the mask-guided hierarchical tone mapping network structure of the present invention. Detailed Implementation

[0036] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0037] This embodiment discloses a hierarchical enhancement network method for X-ray images based on tone mapping fusion, the program flow of which is as follows: Figure 1 As shown, the network structure is as follows Figure 3 As shown, it includes the following steps:

[0038] Step 1: Input the preprocessed X-ray image into the first-level depth curve estimation network (DCE-Net), such as... Figure 2 As shown, global primary brightness and contrast compensation is performed, followed by normalization to obtain the primary enhanced image. ;

[0039] The preprocessing specifically involves grayscale normalization to [0,1], linearly mapping the pixel values ​​of the X-ray image to the [0,1] interval, so that subsequent convolution operations focus on structure and contrast enhancement.

[0040] In this embodiment, a two-stage cascaded depth curve estimation network is used, with the first-stage and second-stage depth curve estimation networks connected sequentially. The single-stage depth curve estimation network adopts the existing DCE network structure.

[0041] In this embodiment, an adaptation is made to take into account the characteristic that X-ray images are single-channel grayscale images: the number of data channels input to the network is set to single channel, and the parameter mapping tensor output by the network is adjusted to 8 grayscale single-channel mapping images. That is, for each pixel of the image, the network outputs 8 enhancement curve parameters. These parameters are applied to the input image through a recursive nonlinear mapping function to achieve detailed adaptive enhancement of brightness and contrast.

[0042] Step 2: Based on the primary enhanced image Generate an adaptive mask M:

[0043] The formula for calculating the adaptive mask M is:

[0044] ,

[0045] Where γ is an adjustable power exponent, used for primary image enhancement in dark areas. A smaller value corresponds to a larger adaptive mask M value that is close to 1, resulting in primary image enhancement in bright areas. A larger value corresponds to a smaller adaptive mask M value that is close to 0. This can be understood as an attention mechanism that makes the network pay more attention to the dark areas that need to be enhanced (areas with mask values ​​close to 1) during backpropagation.

[0046] Step 3: The primary enhanced image The image is input into a second-level depth curve estimation network, where it undergoes secondary fine-tuning of weak local features, complex occlusions, and microstructure regions under mask guidance, followed by normalization to obtain the second-level enhanced image. ;

[0047] In this embodiment, the first-level depth curve estimation network and the second-level depth curve estimation network constitute a two-level cascaded depth curve estimation network structure, realizing hierarchical partitioning from global to detail and nonlinear adaptive mapping; the second-level depth curve estimation network, based on the first-level depth curve estimation network, enhances the feature extraction capability of weak signal regions and occluded edges, further optimizes the mapping curve parameters, and outputs a second-level enhanced image with richer details.

[0048] Step 4: Fuse the primary enhanced image and the secondary enhanced image:

[0049] The first approach is to directly add the primary enhanced image and the secondary enhanced image and then normalize them to obtain the first fused image. ,

[0050] The second approach involves concatenating the primary and secondary enhanced images along the channel dimension, performing convolutional dimensionality reduction, and then normalizing the result to obtain the second fused image. ;

[0051] The first fused image is used as the final output image of the X-ray image hierarchical enhancement network method based on tone mapping fusion;

[0052] Step 5: Construct a multi-level structural similarity loss function, calculate the structural similarity index loss at each of the three levels, assign weights to the structural similarity index losses at the three levels and sum them to obtain the total loss, and update the network parameters based on backpropagation of the total loss.

[0053] The structural similarity index losses at the three levels are as follows:

[0054] The first level is the structural similarity index loss between the primary enhanced image and the normalized supervised image;

[0055] The second level is the mask-weighted structural similarity index loss between the first fused image and the normalized supervised image;

[0056] The third level is the structural similarity index loss between the second fused image and the normalized supervised image;

[0057] The structural similarity index losses at the three levels are weighted and summed to obtain the total loss. The network parameters are then updated based on backpropagation using the total loss.

[0058] In the second-level mask-weighted structural similarity index loss calculation, the initial enhanced image is first used as the basis for the calculation. Generate adaptive mask :

[0059] ,

[0060] Then process the first fused image With normalized supervised images Pointwise power-law mask weighting is performed, and the mask-weighted structural similarity index loss is calculated using the following formula:

[0061] ,

[0062] The loss at each level is weighted. In this embodiment, the weight of the first-level SSIM is 1, the weight of the second-level SSIM is 10, and the weight of the third-level SSIM is 1. The SSIM losses of the three levels are added together to obtain the multi-level structural similarity loss function SSIMLoss. The network is updated by backpropagation, and the network parameters are updated based on the total loss.

Claims

1. A hierarchical enhancement network method for X-ray images based on tone mapping fusion, characterized in that, Includes the following steps: S1: Input the preprocessed X-ray image into the first-level depth curve estimation network to perform global primary brightness and contrast compensation, and perform normalization processing to obtain the primary enhanced image; S2: Generate an adaptive mask based on the primary enhanced image; S3: The primary enhanced image is input into the second-level depth curve estimation network. Under the guidance of the adaptive mask, the local weak features, complex occlusions and microstructure regions in the image are finely adjusted in the second stage. After normalization, the secondary enhanced image is obtained. S4: Perform dual-path fusion processing on the primary enhanced image and the secondary enhanced image: The first approach involves directly adding the primary enhancement image and the secondary enhancement image, then normalizing the result to obtain the first fused image. The second approach involves concatenating the primary and secondary enhancement images along the channel dimension, performing convolutional dimensionality reduction, and then normalizing the result to obtain the second fused image. The first fused image is used as the final output image of the X-ray image hierarchical enhancement network method based on tone mapping fusion; S5: Construct a multi-level structural similarity loss function and calculate the structural similarity index loss at each of the three levels: The first level is the structural similarity index loss between the primary enhanced image and the normalized supervised image; The second level is the mask-weighted structural similarity index loss between the first fused image and the normalized supervised image; The third level is the structural similarity index loss between the second fused image and the normalized supervised image; The structural similarity index losses at the three levels are weighted and summed to obtain the total loss. The network parameters are then updated based on backpropagation using the total loss.

2. The X-ray image hierarchical enhancement network method based on tone mapping fusion according to claim 1, characterized in that, The first-level depth curve estimation network and the second-level depth curve estimation network constitute a two-level cascaded depth curve estimation network.

3. The X-ray image hierarchical enhancement network method based on tone mapping fusion according to claim 1, characterized in that, The formula for calculating the adaptive mask is: ， in, For adaptive masking, For primary image enhancement, It is an adjustable power exponent.

4. The X-ray image hierarchical enhancement network method based on tone mapping fusion according to claim 1, characterized in that, The value of the adaptive mask approaches 1 in dark areas of the image as the pixel value of the primary enhanced image decreases, and approaches 0 in bright areas of the image as the pixel value of the primary enhanced image increases. This is used to guide the network to focus on the dark areas that need enhancement during backpropagation.

5. The X-ray image hierarchical enhancement network method based on tone mapping fusion according to claim 1, characterized in that, When calculating the mask-weighted structural similarity index loss of the second level, the pixels of the first fused image and the normalized supervision image are subjected to point-by-point power mask weighting, and the adaptive mask is used as the weighting index.

6. The X-ray image hierarchical enhancement network method based on tone mapping fusion according to claim 5, characterized in that, The formula for calculating the mask-weighted structural similarity index loss is as follows: ， ， in, For the first fused image, For normalized supervised images.

7. The X-ray image hierarchical enhancement network method based on tone mapping fusion according to claim 6, wherein a region weighting strategy based on adaptive mask is introduced in the construction process of the multi-level structural similarity loss function, an adaptive mask is generated by estimating the primary enhanced image output by the network based on the first-level depth curve, and the total loss is obtained by weighting the structural similarity index loss of the three levels and summing them, and the network parameters are updated by backpropagation based on the total loss.

Images