Image enhancement method based on multi-scale convolution and hierarchical radiation transmission ode

Abstract

This invention relates to an image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE, belonging to the field of computer vision. It includes the following steps: acquiring a publicly available dataset and constructing an image enhancement dataset; preprocessing the images in the image enhancement dataset to obtain blurred images and original sharp images; constructing an image enhancement model based on multi-scale convolution and hierarchical radiative transfer ODE; processing the blurred images through the image enhancement model based on multi-scale convolution and hierarchical radiative transfer ODE to obtain enhanced images; constructing a loss function to measure the difference between the enhanced image and the original sharp image, and optimizing the model parameters using the Adam optimization algorithm to obtain a trained model; inputting the blurred image to be processed into the trained model to obtain the final enhanced image. The image enhancement model of this invention has significant advantages in image depth restoration and can improve image enhancement effects.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to an image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE. Background Technology

[0002] With the widespread application of digital imaging equipment in mobile terminals, security monitoring, medical imaging, and remote sensing imaging, image degradation caused by complex imaging environments has become increasingly prominent. For example, noise amplification in low-light scenes, contrast degradation in overexposed scenes under backlight conditions, and uneven brightness caused by multiple scattering in the imaging process all significantly weaken the visibility of image details and visual quality. Therefore, how to effectively restore the brightness, contrast, and color consistency of an image while maintaining its naturalness has become an important research topic in the field of image enhancement.

[0003] Traditional image enhancement methods typically rely on histogram equalization, Retinex theory, or filtering models to improve image quality by simply transforming the brightness distribution or reflectance components. However, these methods have limited effectiveness when faced with complex, non-uniform degradation, and are prone to problems such as color distortion, local over-enhancement, or loss of detail. With the development of deep learning, convolutional neural networks (CNNs) have made significant progress in image enhancement tasks. However, conventional CNNs, due to the limited receptive field of convolution, struggle to simultaneously capture local details and global illumination structure. Furthermore, they often treat the degradation process as a black box mapping, lacking physical interpretability and exhibiting insufficient generalization ability across different scenes.

[0004] In complex lighting or participating medium environments, the image formation process can be described by the radiative transfer equation (RTE), which characterizes the absorption, scattering, and attenuation of light during propagation. However, solving the complete RTE is often computationally complex and difficult to apply directly to real-time image enhancement. In recent years, some studies have attempted to use the ordinary differential equation (ODE) framework to approximate the imaging model, improving physical consistency and network generalization. However, existing methods still have limitations in layered medium modeling, multi-scale illumination coupling, and efficient solution.

[0005] Therefore, there is an urgent need for a novel image enhancement method that combines the multi-scale feature extraction capabilities of deep convolutional networks with imaging physics modeling based on hierarchical radiative transfer (ODE) to simultaneously achieve efficient and accurate modeling of local details, global illumination structure, and physical degradation processes, thereby obtaining more natural and stable enhancement effects in various complex scenes. Summary of the Invention

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] This invention provides an image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE, comprising the following steps:

[0008] S1. Obtain a public dataset and construct an image augmentation dataset;

[0009] S2. Preprocess the images in the image enhancement dataset to obtain blurred images. and original clear image ;

[0010] S3. Construct an image enhancement model based on multi-scale convolution and hierarchical radiative transfer ODE, including a frequency sensing layer, a first-scale convolutional layer, a first radiative transfer modeling module, a second-scale convolutional layer, a second radiative transfer modeling module, a first ODE solving module, a second ODE solving module, and a dynamic illumination adjustment module; the blurred image The enhanced image O is obtained by processing the image enhancement model based on multi-scale convolution and hierarchical radiative transfer ODE.

[0011] S4. Construct a metric-enhanced image O and the original sharp image. The mean squared error loss function of the difference between them is used to optimize the model parameters using the Adam optimization algorithm to obtain the trained model;

[0012] S5. Input the blurred image to be processed into the trained model to obtain the final enhanced image.

[0013] Furthermore, a SURF-based image registration algorithm is used to ensure a one-to-one correspondence between the blurred image and the original sharp image; the blurred image is used to simulate the image degradation process, while the original sharp image serves as a reference image for training the image enhancement model.

[0014] Furthermore, blurred images The input is fed into the frequency sensing layer to obtain frequency sensing features. This allows subsequent enhancement models to better recover image details; the blurred image is then transformed using a two-dimensional discrete Fourier transform. Transforming from the spatial domain to the frequency domain yields the frequency domain representation. ; for frequency domain representation High-pass filtering is performed to suppress low-frequency components and preserve high-frequency details, resulting in a frequency domain image. For frequency domain images Standardization processing is performed to obtain a standardized image. Standardized images and blurry images The frequency-sensing features are obtained by adding them point by point in the frequency domain. .

[0015] Furthermore, the first-scale convolutional layer includes a first convolutional layer, a normalization layer, a second convolutional layer, a normalization layer, a third convolutional layer, and a normalization layer; the frequency-aware feature The input is fed into the first-scale convolutional layer to obtain the first convolutional feature. .

[0016] Furthermore, the first convolutional features The first radiation feature is obtained by inputting it into the first radiative transfer modeling module. The first radiation feature and the first convolutional features The first modeling feature is obtained by adding the features point by point in the frequency domain. :

[0017] The calculation formula for the first radiative transfer modeling module is expressed as follows:

[0018] ,

[0019] Where γ and δ represent two different adjustment coefficients; Represents the first convolutional feature exist The value of the position; Indicated in the first convolution feature In three-dimensional space Gradient of position; Indicates the first convolution feature Summing all points in the three-dimensional space Ω; Indicates point-by-point multiplication; integration variable Represents the first convolutional feature Coordinates in the integration space This indicates the position of the feature map in the horizontal dimension. This indicates the position of the feature map in the vertical dimension. This indicates the position of the feature map in the channel dimension; the denominator is processed by weighted summation of the convolutional features at different positions. Represents the coordinates within the weighted sum; Indicates the first radiation characteristic exist The results of location calculations, and the results obtained from the calculations at each location, constitute the first radiation feature. .

[0020] Furthermore, the second-scale convolutional layer includes a fourth convolutional layer, a normalization layer, a fifth convolutional layer, a normalization layer, a sixth convolutional layer, and a normalization layer; the frequency-aware features are then processed. The input is fed into the second-scale convolutional layer to obtain the second convolutional features. .

[0021] Furthermore, the second convolutional features The input is fed into the second radiative transfer modeling module, which performs the same calculation process as the first radiative transfer modeling module to obtain the second convolutional feature. exist Location calculation results ; to each position according to The combination of positions yields the second radiation feature. The second radiation feature Second convolution features The second modeling feature is obtained by adding the features point by point in the frequency domain. .

[0022] Furthermore, the first ODE solving module includes a first dimension adjustment layer, a first ODE solving formula, and a normalization layer; the first dimension adjustment layer includes a seventh convolutional layer and a normalization layer; the second modeling feature The input is fed into the first-dimensional adjustment layer to obtain the first-dimensional adjusted features. The calculation process for the first ODE solution formula is as follows:

[0023] ,

[0024] in, Indicates the first combination of features exist The result of the location calculation; This represents the first parameter controlling the diffusion term; Indicated in the first modeling feature In three-dimensional space Gradient of position; This represents the first adjustment coefficient; Represents the Euclidean distance between two points in space; This represents the first spatial attenuation factor; the calculated value for each position... according to Position combination yields the first combination feature. The first combination of features The input is fed into the normalization layer to obtain the first solution feature. .

[0025] Furthermore, the second ODE solving module consists of a second dimension adjustment layer, a second ODE solving formula, and a normalization layer; the dimension adjustment layer includes an eighth convolutional layer and a normalization layer; the first modeling features are... The input is fed into the dimension adjustment layer to obtain the second dimension adjusted features. The calculation process for the second ODE solution formula is as follows:

[0026] ,

[0027] in, Indicates the second combination feature exist The result of the location calculation; This represents the second parameter controlling the diffusion term; Indicated in the second modeling feature In three-dimensional space Gradient of position; This represents the second adjustment coefficient; This represents the second spatial attenuation factor; the calculated value at each position... according to Positional combinations yield the second set of features. The second combination of features The input is fed into the normalization layer to obtain the second solution feature. .

[0028] Furthermore, in the dynamic illumination adjustment module, the first solved feature Second solution characteristics The splicing is performed along the channel dimension to obtain the splicing features. splicing features After adjusting the brightness function, the illumination characteristics are obtained. The illumination features The enhanced image O is obtained by dimensionality reduction through an average pooling layer.

[0029] The advantages of this invention are:

[0030] This invention integrates the multi-scale feature extraction capabilities of deep convolutional networks with the imaging physics modeling of hierarchical radiative transfer (ODE). First, a frequency sensing layer accurately extracts high-frequency detail features from the image through Fourier transform and high-pass filtering. Then, dual-scale convolutional layers with different kernel configurations capture multi-dimensional image features. Combined with a radiative transfer modeling module, the imaging physics process is concretely depicted, overcoming the shortcomings of traditional methods in lacking physical interpretability. The ODE solving module efficiently handles the multi-scale feature coupling problem, and the dynamic illumination adjustment module further optimizes image brightness and natural transitions, effectively avoiding the pitfalls common in traditional methods. To address issues such as color distortion, local over-enhancement, or loss of detail, this invention employs a mean squared error loss function and the Adam optimization algorithm to ensure the stability and convergence of model training. Experimental results demonstrate that this invention outperforms the comparative model in key metrics such as PSNR, SSIM, MSE, and MAE. It can not only efficiently restore the brightness, contrast, and details of images in complex scenes such as low light and underwater environments, but also restore the depth information and spatial structure of images. It has strong cross-scene adaptability and is suitable for multiple fields with high requirements for image quality and spatial structure, such as autonomous driving, robot vision, and medical imaging, possessing both practicality and wide application value. Attached Figure Description

[0031] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0032] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0033] Figure 2 This is an architectural diagram of the model of the present invention;

[0034] Figure 3 This is the enhanced image contrast result of the present invention;

[0035] Figure 4 This is a comparison result of the depth maps of the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Example 1

[0038] In this embodiment, as Figure 1As shown, an image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE is provided, the specific steps of which include:

[0039] S1. Obtain a public dataset and construct an image augmentation dataset;

[0040] Specifically, the publicly available datasets include the I-HAZE image dehazing dataset, the UIEB underwater image augmentation dataset, and the ExDark image dataset. A training set is constructed by randomly selecting 25 images from the I-HAZE image dehazing dataset, 15 images from the UIEB underwater image augmentation dataset, and 10 images from the ExDark image dataset. The remaining data from the three datasets are then randomly selected in sets of 10, 6, and 4 to construct test sets. These test sets are primarily used for performance evaluation and comparative analysis after model training.

[0041] S2. Preprocess the images in the image enhancement dataset to obtain blurred images. and original clear image ;

[0042] Specifically, the SURF-based image registration algorithm is used to ensure a one-to-one correspondence between the blurred image and the original sharp image. Each pair of original sharp images and blurred images is stored in the same folder for easy loading during subsequent model training. Subsequently, in order to improve the generalization ability of the model, data augmentation is performed by randomly rotating the blurred image and the sharp image by 0°, 90°, 180° and 270° respectively. The brightness and contrast of the images are randomly adjusted to simulate images under different lighting conditions.

[0043] S3. Construct an image enhancement model based on multi-scale convolution and hierarchical radiative transfer ODE, such as Figure 2 As shown, it includes a frequency sensing layer, a first-scale convolutional layer, a first radiative transfer modeling module, a second-scale convolutional layer, a second radiative transfer modeling module, a first ODE solving module, a second ODE solving module, and a dynamic illumination adjustment module; the blurred image The enhanced image O is obtained by processing the image enhancement model based on multi-scale convolution and hierarchical radiative transfer ODE.

[0044] Specifically, blurred image The input is fed into the frequency sensing layer, whose purpose is to extract frequency sensing features from the image through frequency domain analysis. This is done so that subsequent enhancement models can better recover image details. The process in this layer includes Fourier transform and enhancement of high-frequency information. The blurred image is then transformed using a two-dimensional discrete Fourier transform. Transforming from the spatial domain to the frequency domain yields the frequency domain representation. Frequency domain representation Including amplitude spectrum and phase spectrum Amplitude spectrum It reflects the intensity of the frequency components of an image, helping to capture high-frequency details; phase spectrum Containing geometric and structural information of the image is crucial for detail restoration; frequency domain representation is used to highlight high-frequency information in the image. High-pass filtering is performed to suppress low-frequency components and preserve high-frequency details, thus obtaining a frequency domain image. The high-pass filtering formula is expressed as follows:

[0045] ,

[0046] in, Indicates a high-pass filter; This represents frequency domain dot product, where the filter function is directly multiplied by the corresponding position in the spectrum; for frequency domain images... Standardization processing is performed to obtain a standardized image. The formula is expressed as follows:

[0047] ,

[0048] in, Indicates amplitude spectrum The mean; Indicates amplitude spectrum Standard deviation; Represents the phase spectrum Standard deviation; standardize the image and blurry images The frequency-sensing features are obtained by adding them point by point in the frequency domain. .

[0049] Specifically, the first-scale convolutional layer includes a first convolutional layer, a normalization layer, a second convolutional layer, a normalization layer, a third convolutional layer, and a normalization layer; the first convolutional layer has a 3×3 kernel size, 64 kernels, a stride of 1, same padding, and ReLU activation function; the second convolutional layer has a 3×3 kernel size, 128 kernels, a stride of 1, same padding, and ReLU activation function; the third convolutional layer has a 3×3 kernel size, 256 kernels, a stride of 1, same padding, and ReLU activation function; the normalization layer performs batch normalization to normalize the feature map; the frequency-aware feature... The input is fed into the first-scale convolutional layer to obtain the first convolutional feature. .

[0050] Specifically, the first convolutional features The first radiation feature is obtained by inputting it into the first radiative transfer modeling module. The first radiation feature and the first convolutional features The first modeling feature is obtained by adding the features point by point in the frequency domain. :

[0051] The calculation formula for the first radiative transfer modeling module is expressed as follows:

[0052] ,

[0053] Wherein, γ and δ represent two different adjustment coefficients, γ=0.778 and δ=0.426; Represents the first convolutional feature exist The value of the position; Indicated in the first convolution feature In three-dimensional space The gradient at a location is used to describe the rate of change of the feature map in space; Indicates the first convolution feature Summing all points in the three-dimensional space Ω; This indicates pointwise multiplication, meaning multiplying the eigenvalues ​​by their corresponding gradients at the same location; the integral variable... Represents the first convolutional feature Coordinates in the integration space This indicates the position of the feature map in the horizontal (width) dimension. This indicates the position of the feature map in the vertical (height) dimension. This represents the position of the feature map along the channel dimension; the denominator processes the convolutional features at different positions through a weighted sum. Represents the coordinates within the weighted sum; Indicates the first radiation characteristic exist The results of location calculations, and the results obtained from the calculations at each location, constitute the first radiation feature. .

[0054] Specifically, the second-scale convolutional layer includes a fourth convolutional layer, a normalization layer, a fifth convolutional layer, a normalization layer, a sixth convolutional layer, and a normalization layer. The fourth convolutional layer has a 1×1 kernel size, 128 kernels, a stride of 3, same padding, and ReLU activation. The fifth convolutional layer has a 3×3 kernel size, 256 kernels, a stride of 1, same padding, and ReLU activation. The sixth convolutional layer has a 5×5 kernel size, 512 kernels, a stride of 1, same padding, and ReLU activation. The normalization layer performs batch normalization to normalize the feature map; the frequency-aware features... The input is fed into the second-scale convolutional layer to obtain the second convolutional features. .

[0055] Specifically, the second convolutional features The input is fed into the second radiative transfer modeling module, which performs the same calculation process as the first radiative transfer modeling module to obtain the second convolutional feature. exist Location calculation results ; to each position according to The combination of positions yields the second radiation feature. The second radiation feature Second convolution features The second modeling feature is obtained by adding the features point by point in the frequency domain. .

[0056] Specifically, the first ODE solving module includes a first dimension adjustment layer, a first ODE solving formula, and a normalization layer; the first dimension adjustment layer includes a seventh convolutional layer and a normalization layer; the seventh convolutional layer has a kernel size of 5×5, a kernel count of 256, a stride of 1, a padding method of "same", and an activation function of ReLU; the second modeling feature The input is fed into the first-dimensional adjustment layer to obtain the first-dimensional adjusted features. The first dimension adjusts the features. With the first modeling feature The dimensions are the same; the calculation process of the first ODE solution formula is as follows:

[0057] ,

[0058] in, Indicates the first combination of features exist The result of the location calculation; This represents the first parameter controlling the diffusion term. =0.63; Indicated in the first modeling feature In three-dimensional space Gradient of position; This represents the first adjustment coefficient, used to control the influence of the integral term. =0.29; The Euclidean distance between two points in space is denoted as Euclidean distance. point and point The distance between them point Indicates the first modeling feature In a specific coordinate location in the three-dimensional feature space, x represents the first modeling feature. The index y in the width direction represents the first modeling feature. In the height direction, z represents the first modeling feature. Index in the channel direction; Indicates the first spatial attenuation factor. =1.17; The calculated value for each position... according to Position combination yields the first combination feature. The first combination of features The input is fed into the normalization layer to obtain the first solution feature. .

[0059] Specifically, the second ODE solving module consists of a second dimension adjustment layer, a second ODE solving formula, and a normalization layer group; the dimension adjustment layer includes an eighth convolutional layer and a normalization layer; the eighth convolutional layer has a kernel size of 5×5, a kernel count of 512, a stride of 1, a padding method of "same", and an activation function of ReLU; the first modeling features are... The input is fed into the dimension adjustment layer to obtain the second dimension adjusted features. The second dimension adjusts the features. With the second modeling feature The dimensions are the same; the calculation process of the second ODE solution formula is as follows:

[0060] ,

[0061] in, Indicates the second combination feature exist The result of the location calculation; This represents the second parameter controlling the diffusion term. =0.68; Indicated in the second modeling feature In three-dimensional space Gradient of position; This represents the second adjustment coefficient. =0.34, used to control the influence of the integral term; This represents the second spatial attenuation factor. =1.19; The calculated value for each position... according to Positional combinations yield the second set of features. The second combination of features The input is fed into the normalization layer to obtain the second solution feature. .

[0062] Specifically, in the dynamic illumination adjustment module, the first solved feature Second solution characteristics The splicing is performed along the channel dimension to obtain the splicing features. splicing features After adjusting the brightness function, the illumination characteristics are obtained. The brightness adjustment function uses `kornia.enhance.adjust_brightness()`, a utility function in Kornia, a publicly available differentiable computer vision library designed specifically for PyTorch. It belongs to the Kornia library's enhancement tools for modifying image brightness. This function is used to enhance the brightness of the input image tensor, primarily performing a linear brightness transformation on the tensor to simulate different illumination intensities, improve feature brightness distribution, or serve as a data augmentation method to enhance the model's robustness to illumination changes. The illumination features... The enhanced image O is obtained by dimensionality reduction through an average pooling layer.

[0063] S4. Construct a metric-enhanced image O and the original sharp image. The mean squared error loss function of the difference between them is used to optimize the model parameters using the Adam optimization algorithm to obtain the trained model;

[0064] Specifically, the loss function The construction method is as follows:

[0065] ,

[0066] in, This represents the total number of pixels in the image; This represents the absolute value of the i-th pixel in the enhanced image O; Represents the original sharp image The absolute value of the i-th pixel.

[0067] S5. Input the blurred image to be processed into the trained model to obtain the final enhanced image.

[0068] Example 2

[0069] In this embodiment, the performance of different depth restoration models is compared using various image quality evaluation metrics. Image quality evaluation metrics such as PSNR (Peak Signal-to-Noise Ratio), MSE (Mean Squared Error), SSIM (Structural Similarity Index), and MAE (Mean Absolute Error) are commonly used evaluation standards in image restoration and depth estimation tasks.

[0070] This invention selects four methods for comparison:

[0071] Traditional image enhancement methods (T): improve image quality through simple brightness and contrast enhancement;

[0072] CNN-based depth estimation (Monodepth): Infers depth information from monocular images using a trained depth estimation network;

[0073] Structured light and motion estimation (SFM): Relies on the parallax between multi-view images to recover depth information, but performs poorly on a single image;

[0074] Multi-scale convolution and hierarchical radiative transfer ODE model (ODEM): Combines multi-scale convolutional networks and physical radiative transfer models to recover the depth information of images;

[0075] The dataset constructed using this invention was used as a reference for evaluation. PSNR (Peak Signal-to-Noise Ratio) measures the difference between the restored image and the sharp image. A higher PSNR value indicates better image quality and a restoration closer to the original image. MSE (Mean Squared Error) measures the error between the restored image and the sharp image; a smaller value indicates better restoration. SSIM (Structural Similarity Index) measures the structural similarity between the restored image and the sharp image. A higher SSIM value indicates better structural restoration. MAE (Mean Absolute Error) calculates the mean absolute error between the restored image and the sharp image. A smaller value indicates higher accuracy in restoration depth.

[0076] Table 1 Comparison of Experimental Results

[0077]

[0078] Based on these experimental results, the present invention can draw the following conclusions:

[0079] Traditional image enhancement methods perform poorly in restoring image details and spatial structure, exhibiting low PSNR and SSIM, but high MSE and MAE, indicating limited restoration effectiveness, especially in complex environments. Monodepth, using a convolutional neural network, enhances images effectively, particularly in low-light conditions, recovering more details and significantly improving PSNR and SSIM. SFM performs the worst, with large restoration errors due to insufficient disparity information, resulting in low PSNR and poor SSIM and MAE. The multi-scale convolutional and hierarchical radiative transfer (ODE) model performs best, achieving the highest PSNR, SSIM, and MAE, demonstrating that this model not only significantly improves image quality but also restores more details and spatial structure in harsh environments.

[0080] In summary, the image enhancement model based on multi-scale convolution and hierarchical radiative transfer ODE has significant advantages in image depth restoration, and is especially suitable for applications with high requirements for image depth and spatial structure in low-light complex environments, such as autonomous driving and robot vision.

[0081] Example 3

[0082] In this embodiment, an image enhancement model based on multi-scale convolution and hierarchical radiative transfer ODE is applied to process an image taken under low-light conditions. The image details and contrast are severely degraded due to insufficient illumination or the influence of the participating medium. Figure 3 As shown, details in a blurred image are affected by environmental factors, resulting in poor visibility; the original clear image has a better visual effect; and the enhanced image shows a significant improvement in visual effect compared to the original image. The enhanced image shows significant improvements in brightness, contrast, and detail recovery. Although the image is still affected by the environment, the enhancement algorithm has effectively recovered more details and lighting. Specifically, the brightness and contrast of the image are improved, and the originally blurred parts become brighter, especially in the billboard area in the background, where text and structure are more clearly visible. Compared to the original image, the enhanced image has recovered details in distant billboards, trees, and other elements, greatly improving the image's recognizability. Although low-light environments still have some impact on the image, the color representation is more natural, the transition of lighting between different areas is smoother, and the naturalness of the image is restored.

[0083] Example 4

[0084] In this embodiment, an image analysis method using plt.cm.jet in PyTorch is applied to verify the model's ability to recover depth distance under low-light and complex lighting conditions. Traditional image enhancement techniques typically focus on restoring brightness, contrast, and details, but rarely effectively recover depth information, especially when low-light environmental factors severely affect image quality, often resulting in the loss or blurring of object depth information. Figure 4 As shown, the depth map generated from a blurred image is affected by insufficient lighting and environmental factors, resulting in a severe loss of image details and depth information. The spatial relationship between foreground and background objects is blurred, and depth differences are difficult to discern. The depth map generated from the original clear image provides accurate depth information, where closer objects are displayed in bright colors, while distant objects appear in dark colors, making depth relationships immediately apparent. The depth map generated from the enhanced image significantly improves the sense of space and hierarchy, making the relationship between foreground and background clearer, and effectively restoring details of distant objects. Especially in low-light environments, the enhanced image not only appears more natural visually but also recovers more depth information, making previously blurred objects clearer, and revealing obvious details in distant billboards, buildings, etc. Through multi-scale convolution and a hierarchical radiative transfer ODE model, the spatial layout and depth hierarchy of the image are effectively restored, significantly improving image recognition and visibility. Although environmental factors still affect the image, with the help of the enhancement algorithm, the depth information and spatial sense of the image are effectively restored, making objects, details, and distance relationships in the image clearer. Especially in fields such as autonomous driving, robot vision, and 3D reconstruction, this depth restoration technology will significantly improve the usability and accuracy of images.

[0085] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE, characterized in that, Includes the following steps: S1. Obtain a public dataset and construct an image augmentation dataset; S2. Preprocess the images in the image enhancement dataset to obtain blurred images. and original clear image ; S3. Construct an image enhancement model based on multi-scale convolution and hierarchical radiative transfer ODE, including a frequency sensing layer, a first-scale convolutional layer, a first radiative transfer modeling module, a second-scale convolutional layer, a second radiative transfer modeling module, a first ODE solving module, a second ODE solving module, and a dynamic illumination adjustment module; the blurred image The image enhancement model, based on multi-scale convolution and hierarchical radiative transfer ODE, is used to obtain the enhanced image O; specifically, it includes: The input image is fed into a frequency sensing layer, and the output of the frequency sensing layer is fed into a first-scale convolutional layer and a second-scale convolutional layer, respectively. The output of the first-scale convolutional layer is fed into a first radiative transfer modeling module; the output of the second-scale convolutional layer is fed into a second radiative transfer modeling module; the outputs of the first-scale convolutional layer and the first radiative transfer modeling module are summed point-by-point and then fed into a first ODE solving module and a second ODE solving module, respectively. The outputs of the second-scale convolutional layer and the second radiative transfer modeling module are summed point-by-point and then fed into a second ODE solving module; the outputs of the first ODE solving module and the second ODE solving module are fed into a dynamic illumination adjustment module; the dynamic illumination adjustment module outputs an enhanced image. The calculation formula for the first radiative transfer modeling module is expressed as follows: ， Where γ and δ represent two different adjustment coefficients; Represents the first convolutional feature exist The value of the position; Indicated in the first convolution feature In three-dimensional space Gradient of position; Indicates the first convolution feature Integrate across all points in the three-dimensional space Ω; Indicates point-by-point multiplication; integration variable Represents the first convolutional feature Coordinates in the integration space This indicates the position of the feature map in the horizontal dimension. This indicates the position of the feature map in the vertical dimension. This indicates the position of the feature map in the channel dimension; the denominator is processed by weighted summation of the convolutional features at different positions. Represents the coordinates within the weighted sum; Indicates the first radiation characteristic exist The results of location calculations, and the results obtained from the calculations at each location, constitute the first radiation feature. ; The second radiative transfer modeling module performs the same calculation process as the first radiative transfer modeling module; The calculation process of the first ODE solution formula is as follows: ， in, Indicates the first combination of features exist The result of the location calculation; This represents the first parameter controlling the diffusion term; Indicated in the first modeling feature In three-dimensional space Gradient of position; This represents the first adjustment coefficient; Represents the Euclidean distance between two points in space; Indicates the first spatial attenuation factor; This indicates the first dimension of the adjusted features; the calculated value for each position... according to Position combination yields the first combination feature. ; the first combination of features The input is fed into the normalization layer to obtain the first solution feature. ; The calculation process of the second ODE solution formula is as follows: ， in, Indicates the second combination feature exist The result of the location calculation; This represents the second parameter controlling the diffusion term; Indicated in the second modeling feature In three-dimensional space Gradient of position; This represents the second adjustment coefficient; Indicates the second spatial attenuation factor; This indicates the second dimension of the adjusted features; the calculated value for each position... according to Positional combinations yield the second set of features. The second combination of features The input is fed into the normalization layer to obtain the second solution feature. ; S4. Construct a metric-enhanced image O and the original sharp image. The mean squared error loss function of the difference between them is used to optimize the model parameters using the Adam optimization algorithm to obtain the trained model; S5. Input the blurred image to be processed into the trained model to obtain the final enhanced image.

2. The image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE according to claim 1, characterized in that, The SURF-based image registration algorithm is used to ensure a one-to-one correspondence between the blurred image and the original sharp image.

3. The image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE according to claim 2, characterized in that, Blurry image The input is fed into the frequency sensing layer to obtain frequency sensing features. : Blurred images are transformed using two-dimensional discrete Fourier transform. Transforming from the spatial domain to the frequency domain yields the frequency domain representation. ; for frequency domain representation High-pass filtering is performed to obtain the frequency domain image. For frequency domain images Standardization processing is performed to obtain a standardized image. Standardized images and blurry images The frequency-sensing features are obtained by adding them point by point in the frequency domain. .

4. The image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE according to claim 3, characterized in that, The first-scale convolutional layer includes a first convolutional layer, a normalization layer, a second convolutional layer, a normalization layer, a third convolutional layer, and a normalization layer; the frequency-aware feature The input is fed into the first-scale convolutional layer to obtain the first convolutional feature. .

5. The image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE according to claim 4, characterized in that, The first convolutional features The first radiation feature is obtained by inputting it into the first radiative transfer modeling module. The first radiation feature and the first convolutional features The first modeling feature is obtained by adding the features point by point in the frequency domain. .

6. The image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE according to claim 5, characterized in that, The second-scale convolutional layer includes a fourth convolutional layer, a normalization layer, a fifth convolutional layer, a normalization layer, a sixth convolutional layer, and a normalization layer; it incorporates the frequency-aware features. The input is fed into the second-scale convolutional layer to obtain the second convolutional features. .

7. The image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE according to claim 6, characterized in that, Second convolution feature The input is fed into the second radiative transfer modeling module to obtain the second convolutional features. exist Location calculation results ; to each position according to The combination of positions yields the second radiation feature. The second radiation feature Second convolution features The second modeling feature is obtained by adding the features point by point in the frequency domain. .

8. The image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE according to claim 7, characterized in that, The first ODE solving module includes a first dimension adjustment layer, a first ODE solving formula, and a normalization layer; the first dimension adjustment layer includes a seventh convolutional layer and a normalization layer; the second modeling feature The input is fed into the first-dimensional adjustment layer to obtain the first-dimensional adjusted features. .

9. The image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE according to claim 8, characterized in that, The second ODE solving module consists of a second dimension adjustment layer, a second ODE solving formula, and a normalization layer group; the second dimension adjustment layer includes an eighth convolutional layer and a normalization layer; the first modeling features are... The input is fed into the dimension adjustment layer to obtain the second dimension adjusted features. .

10. The image enhancement method based on multi-scale convolution and hierarchical radiative transfer ODE according to claim 9, characterized in that, In the dynamic illumination adjustment module, the first solved feature Second solution characteristics The splicing is performed along the channel dimension to obtain the splicing features. splicing features After adjusting the brightness function, the illumination characteristics are obtained. The illumination features The enhanced image O is obtained by dimensionality reduction through an average pooling layer.

Images