Adaptive image enhancement method and system based on physical modeling and dynamic prediction

Abstract

This invention provides an adaptive image enhancement method, system, device, and medium based on physical modeling and dynamic prediction. The method includes: constructing an image degradation mathematical model based on received multimodal input data; analyzing continuous frame sequences and predicting the changing trends of scene state and degradation parameters at the next moment using a dynamic scene prediction model employing a temporal convolutional network and a long short-term memory network; performing image reconstruction and enhancement based on the image degradation mathematical model and prediction results; evaluating the quality of the enhanced image and feeding the evaluation results back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization; and deploying the model in a lightweight manner and accelerating it with hardware to adapt to real-time operation on edge computing devices, thereby solving the problems of poor adaptability of traditional empirical image enhancement algorithms and insufficient generalization and dynamic prediction of basic deep learning algorithms in image enhancement technology.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and in particular to an adaptive image enhancement method, system, device, and medium based on physical modeling and dynamic prediction. Background Technology

[0002] In today's era where digital technology permeates all sectors, images, as the core carrier of information transmission and scene perception, directly determine the efficiency and accuracy of downstream applications. Whether it's the precise capture of human activity and object details in security monitoring, the reliable identification of road markings, pedestrians, and obstacles in autonomous driving, the clear presentation of lesions in medical imaging, or the accurate extraction of ground feature information in satellite remote sensing, all require high-quality image data. With the rapid development of intelligent transportation, smart security, precision medicine, and spatial information, the requirements for image quality have upgraded from "recognizable" to "high definition, high fidelity, and real-time." Especially in complex scenarios such as rain, fog, low light, and motion blur, overcoming environmental interference to improve image quality has become a key requirement for various industries to promote the implementation of these technologies.

[0003] Current mainstream image enhancement technologies are mainly divided into two categories: traditional empirical algorithms and basic deep learning algorithms, both of which have significant limitations. Traditional algorithms, such as histogram equalization and Retinex, adopt a "one-size-fits-all" fixed parameter approach, which can only optimize for a single type of degradation (such as low contrast or simple noise). They cannot adapt to complex situations such as sudden changes in lighting, superposition of multiple types of noise, and dynamic scene switching, and are prone to problems such as color distortion, loss of detail, or over-enhancement in the enhanced image. Although basic deep learning algorithms can learn enhancement rules through data-driven learning, they rely heavily on a large amount of labeled data and do not combine the physical nature of image degradation. They can only fit known scenes and have poor generalization ability when facing untrained degradation types (such as atmospheric scattering under special weather conditions or irregular blurring caused by equipment shake). At the same time, most algorithms only focus on static image processing and lack the ability to predict scene changes. In scenarios such as video streaming and real-time perception, they are prone to inter-frame jumps and cannot meet the requirements of real-time performance and continuity. The limitations of existing technologies have become a bottleneck restricting high-quality development in many fields: In autonomous driving scenarios, traditional algorithms cannot cope with sudden changes in lighting when entering and exiting tunnels, easily leading to temporary visual recognition failures; in the field of medical imaging, basic deep learning algorithms, because they do not incorporate the physical processes of imaging, may mistakenly enhance or mask lesion details, affecting diagnostic accuracy; in the field of satellite remote sensing, algorithms lacking dynamic scene prediction capabilities struggle to handle image degradation caused by cloud movement and atmospheric scattering changes in real time. With the advancement of edge computing and AI chip technologies, the computing power of edge devices, such as vehicle terminals, surveillance cameras, and medical imaging terminals, is gradually increasing, providing a hardware foundation for the real-time deployment of complex algorithms. The industry urgently needs an image enhancement technology that integrates physical modeling and dynamic prediction capabilities and possesses adaptive adjustment characteristics to overcome existing limitations, achieve a closed loop of "accurate modeling - dynamic prediction - adaptive enhancement - real-time optimization," meet the demand for high-quality images in complex scenarios across multiple fields, and drive the upgrade of image enhancement technology from "passive processing" to "active perception and optimization."

[0004] Therefore, there is an urgent need for an adaptive image enhancement method based on physical modeling and dynamic prediction to solve the technical problems of poor adaptability of traditional empirical image enhancement algorithms and insufficient generalization and dynamic prediction of basic deep learning algorithms. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this disclosure provides an adaptive image enhancement method, system, device and medium based on physical modeling and dynamic prediction, so as to solve the technical problems of poor adaptability of traditional empirical image enhancement algorithms and insufficient generalization and dynamic prediction of basic deep learning algorithms in image enhancement technology.

[0006] This specification provides one or more embodiments of an adaptive image enhancement method based on physical modeling and dynamic prediction, comprising the following steps: A mathematical model of image degradation is constructed based on the received multimodal input data; A dynamic scene prediction model using temporal convolutional networks and long short-term memory networks is used to analyze continuous frame sequences and predict the changing trends of scene state and degradation parameters at the next moment. Image reconstruction and enhancement are performed based on the mathematical model of image degradation and the prediction results. The enhanced image is quality assessed, and the assessment results are fed back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization. The model is deployed in a lightweight manner and hardware accelerated to adapt to real-time operation on edge computing devices.

[0007] Preferably, the multimodal input data includes at least two of the following: RGB image, infrared image, depth image, and polarization image; The image degradation mathematical model includes an atmospheric optical model, a motion blur model, and a noise model; An atmospheric optical model is established based on atmospheric optics principles to estimate transmittance maps and global atmospheric light in real time. The expression for the atmospheric optical model is as follows: I(x) = J(x) * t(x) + A * (1 - t(x)); Where I(x) is the observed image, J(x) is the clear image to be recovered, t(x) is the transmittance map, and A is the global atmospheric light; A motion blur model is established based on the motion characteristics of the camera and the object, and the blur kernel is estimated by image feature matching. The noise model is a signal-dependent noise model. A Poisson-Gaussian noise model is established based on the statistical characteristics of image noise to analyze the noise distribution pattern.

[0008] Preferably, the image reconstruction and enhancement based on the image degradation mathematical model and prediction results specifically includes the following steps: By embedding a mathematical model of image degradation into a deep learning network through differentiable programming, a differentiable rendering process is constructed, and a clear image is obtained from the degraded image through an inverse physical process. Based on a reinforcement learning policy network, a target enhancement algorithm or a combination of algorithms is selected from a preset enhancement algorithm library according to the current environmental parameters and the prediction parameters of the dynamic scene prediction model. The preset enhancement algorithm library includes histogram equalization algorithm, Retinex algorithm and homomorphic filtering algorithm. The parameters of the target enhancement algorithm are adjusted, including gain intensity, gamma value and filter size, and then the image enhancement operation is performed.

[0009] Preferably, the step of performing quality assessment on the enhanced image and feeding the assessment results back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization specifically includes the following steps: A pre-trained no-reference image quality assessment model was used to calculate the sharpness score, naturalness score, and information entropy score of the enhanced image. The clarity score, naturalness score, and information entropy score are compared with a preset quality threshold. If the score is lower than the threshold, a feedback signal is generated. Feedback signals are sent to the dynamic scene prediction model to adjust weights and to the image reconstruction and enhancement process to optimize enhancement algorithm parameters.

[0010] Preferably, the lightweight deployment and hardware acceleration of the model specifically includes the following steps: Pruning deep learning models removes redundant neurons and connections, reducing model complexity. The pruned model is quantized to convert 32-bit floating-point precision to 16-bit or 8-bit fixed-point precision, reducing the amount of computation. The underlying hardware chips for edge computing devices are adapted, including NVIDIA Jetson chips, Huawei Atlas chips, and Bitmain AI chips, to optimize instruction execution efficiency for real-time operation.

[0011] This specification provides one or more embodiments of an adaptive image enhancement system based on physical modeling and dynamic prediction, including a model building module, a dynamic scene prediction module, an adaptive enhancement module, an optimization module, and a deployment module; The model building module is used to build an image degradation mathematical model based on the received multimodal input data; The dynamic scene prediction module is used to analyze continuous frame sequences and predict the changing trends of scene state and degradation parameters at the next moment using a dynamic scene prediction model of temporal convolutional network and long short-term memory network. The adaptive enhancement module is used to perform image reconstruction and enhancement based on the image degradation mathematical model and prediction results; The optimization module is used to evaluate the quality of the enhanced image and feed the evaluation results back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization. The deployment module is used to deploy the model in a lightweight manner and accelerate it with hardware to adapt it for real-time operation on edge computing devices.

[0012] Preferably, the multimodal input data includes at least two of the following: RGB image, infrared image, depth image, and polarization image; The image degradation mathematical model includes an atmospheric optical model, a motion blur model, and a noise model; An atmospheric optical model is established based on atmospheric optics principles to estimate transmittance maps and global atmospheric light in real time. The expression for the atmospheric optical model is as follows: I(x) = J(x) * t(x) + A * (1 - t(x)); Where I(x) is the observed image, J(x) is the clear image to be recovered, t(x) is the transmittance map, and A is the global atmospheric light; A motion blur model is established based on the motion characteristics of the camera and the object, and the blur kernel is estimated by image feature matching. The noise model is a signal-dependent noise model. A Poisson-Gaussian noise model is established based on the statistical characteristics of image noise to analyze the noise distribution pattern.

[0013] Preferably, the adaptive enhancement module includes a differentiable programmable unit and a reinforcement learning policy network unit; The differentiable programming unit is used to embed the image degradation mathematical model into a deep learning network through differentiable programming, construct a differentiable rendering process, and solve for a clear image from the degraded image through a reverse physical process; The reinforcement learning policy network unit is used to select a target enhancement algorithm or a combination of algorithms from a preset enhancement algorithm library based on the reinforcement learning policy network and the prediction parameters of the current environment parameters and the dynamic scene prediction model. The preset enhancement algorithm library includes histogram equalization algorithm, Retinex algorithm and homomorphic filtering algorithm, and adjusts the parameters of the target enhancement algorithm, including gain intensity, gamma value and filter size, and performs image enhancement operation.

[0014] This specification provides one or more embodiments of a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the adaptive image enhancement method based on physical modeling and dynamic prediction as described above.

[0015] This specification provides one or more embodiments of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the adaptive image enhancement method based on physical modeling and dynamic prediction described above.

[0016] This disclosure provides an adaptive image enhancement method, system, device, and medium based on physical modeling and dynamic prediction. Its advantages lie in constructing an image degradation mathematical model based on received multimodal input data, establishing an accurate degradation model through multimodal data fusion, quantifying the physical mechanisms of image degradation in different scenarios, and providing precise physical constraints for subsequent enhancement. It employs a dynamic scene prediction model using temporal convolutional networks and long short-term memory networks to analyze continuous frame sequences and predict the changing trends of scene states and degradation parameters at the next moment. By utilizing a combined model of temporal convolutional networks and long short-term memory networks, it achieves forward-looking prediction of changes in scene states and degradation parameters, overcoming the limitations of static processing, gaining lead time for enhancement strategy adjustments, and avoiding enhancement lag caused by sudden scene changes. Based on the image degradation mathematical model and prediction results… If image reconstruction and enhancement are performed, the enhancement process is combined with the degradation model and prediction results. This ensures that the enhancement process conforms to physical laws to avoid distortion, while also achieving accurate and adaptive image quality improvement based on the prediction and dynamic adaptation strategy. The enhanced image is then evaluated for quality, and the evaluation results are fed back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization. This quality evaluation forms a closed-loop feedback, which can correct the prediction model and enhancement strategy in real time, continuously optimize the enhancement effect, and solve the problem of insufficient adaptability caused by the lack of self-iteration capability of the algorithm. The model is deployed in a lightweight manner and accelerated by hardware to adapt to real-time operation on edge computing devices, enabling the model to run efficiently on edge devices, meeting real-time requirements, solving the problem that complex algorithms are difficult to implement in edge scenarios due to large computational loads, and expanding the application scope of the technology. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating an adaptive image enhancement method based on physical modeling and dynamic prediction, provided for one or more embodiments of this specification; Figure 2 A schematic diagram of the structure of an adaptive image enhancement system based on physical modeling and dynamic prediction provided for one or more embodiments of this specification; Figure 3 This is a schematic diagram of the structure of a computer device provided for one or more embodiments of this specification. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this invention.

[0020] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings.

[0021] Method Implementation Examples According to embodiments of the present invention, an adaptive image enhancement method based on physical modeling and dynamic prediction is provided, such as... Figure 1 The diagram shown is a flowchart of the adaptive image enhancement method based on physical modeling and dynamic prediction provided in this embodiment. Based on the Linux Ubuntu 20.04 operating system, it adopts a Docker containerized deployment method, encapsulating the execution components of each step as independent services. The adaptive image enhancement method based on physical modeling and dynamic prediction according to this embodiment includes the following steps: S110. An image degradation mathematical model is constructed based on the received multimodal input data and deployed in Docker container 1. It receives real-time data from multimodal cameras and a light sensor, with a runtime of 10ms. The multimodal cameras include one RGB camera (1920×1080 resolution, 30fps), one infrared camera (detection distance 0-50m), and one depth camera (using TOF technology, accuracy ±3%). The multimodal input data includes at least two of RGB images, infrared images, depth images, and polarized images. The RGB camera acquires images of the road ahead, the infrared camera simultaneously acquires thermal images for identifying object contours in rainy or foggy weather, and the depth camera obtains target distance information. The acquired multimodal data is timestamped, distortion corrected, and converted into a unified image coordinate system to provide more constraints for physical modeling. The light sensor collects ambient light intensity in real time to provide lighting parameters for dynamic scene prediction, and millimeter-wave radar assists in acquiring object motion vector information to improve dynamic prediction accuracy.

[0022] S120, deployed in Docker container 2, receives degradation parameters and motion data from millimeter-wave radar. With a runtime of 20ms, it employs a dynamic scene prediction model combining temporal convolutional networks and long short-term memory networks to analyze parameters such as transmittance maps, atmospheric light values, and object motion vectors in continuous frame sequences, forming a time-series dataset. The temporal convolutional network layer (3×3 kernel size, dilation coefficients 1, 2, 4) captures short-term trends in transmittance maps and atmospheric light, while the long short-term memory network layer (64 hidden units) learns long-term dependencies in object motion vectors and predicts the scene state and degradation parameter trends for the next moment. For example, in autonomous driving scenarios, when the vehicle... As the vehicle approaches the tunnel, the millimeter-wave radar detects a fixed obstacle 50 meters ahead, and the light sensor detects that the light intensity drops from 10,000 lux to 5,000 lux. The dynamic scene prediction model outputs a warning through the following steps: inputting light intensity data from the past 10 frames (10,000 lux → 8,000 lux → 6,000 lux → 5,000 lux), the temporal convolutional network layer captures the decreasing trend of light intensity, and the long short-term memory network layer predicts the next frame based on training data of historical tunnel scenes. For example, it predicts that the light intensity will drop to 1,000 lux in 0.03 seconds, and the atmospheric light intensity (A) will drop from 180 to 80, generating a "sudden dimming warning signal". This signal is sent to the image reconstruction and enhancement step via the Robot Operating System (ROS) topic, activating the low-light enhancement mode 0.03 seconds in advance to achieve a seamless and smooth transition and avoid abrupt changes in the image.

[0023] S130: Based on the image degradation mathematical model and prediction results, image reconstruction and enhancement are performed. A physical model is embedded into a deep learning network to construct a differentiable rendering pipeline, deployed in a Docker container. The image enhancement algorithm is executed using the GPU computing power of NVIDIA Jetson, with a runtime of 33ms. The network learns how to reverse the physical process to extract a clear image from the degraded image, ensuring the result conforms to physical reality and preventing distortion. Simultaneously, a policy network is trained for different scenarios such as fog, night scenes, backlighting, and motion blur. This network dynamically selects or combines the most suitable enhancement algorithm and its parameters, such as gain intensity, gamma value, and filter size, based on the current and predicted environmental state.

[0024] S140. Perform real-time, online quality assessment on the enhanced image, and feed the assessment results back to the dynamic scene prediction model and image reconstruction and enhancement process for optimization, forming a closed-loop feedback. Deployed in Docker container 4, synchronously receive the enhanced image, with a running cycle of 50ms.

[0025] The S150 uses lightweight deployment and hardware acceleration to adapt to real-time operation on edge computing devices such as drones, vehicle systems, surveillance cameras, and mobile phones. It selects NVIDIA Jetson AGX Orin as the core edge computing device. This chip has 200 TOPS of AI computing power, supports multimodal data parallel processing, is integrated into various containers, and optimizes deep learning models through TensorRT.

[0026] The method provided in this embodiment constructs an image degradation mathematical model based on received multimodal input data, establishes an accurate degradation model through multimodal data fusion, quantifies the physical mechanisms of image degradation under different scenarios, and provides precise physical constraints for subsequent enhancement. A dynamic scene prediction model using temporal convolutional networks and long short-term memory networks analyzes continuous frame sequences and predicts the changing trends of scene state and degradation parameters at the next moment. By combining the temporal convolutional network and long short-term memory network model, it achieves forward-looking prediction of changes in scene state and degradation parameters, overcoming the limitations of static processing, gaining lead time for enhancement strategy adjustments, and avoiding enhancement lag caused by sudden scene changes. Image reconstruction and enhancement are performed based on the image degradation mathematical model and prediction results, combined with the degradation model. Enhancement is performed on the predicted results, ensuring that the enhancement process conforms to physical laws to avoid distortion, while also achieving accurate and adaptive image quality improvement based on dynamic adaptation strategies according to predictions. The enhanced images are then evaluated for quality, and the evaluation results are fed back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization. This quality evaluation forms a closed-loop feedback loop, allowing for real-time correction of the prediction model and enhancement strategy, continuously optimizing the enhancement effect and addressing the lack of adaptability caused by the algorithm's lack of self-iteration capabilities. The model is deployed in a lightweight manner and accelerated by hardware to adapt to real-time operation on edge computing devices, achieving efficient operation on edge devices, meeting real-time requirements, and solving the problem that complex algorithms are difficult to implement in edge scenarios due to their large computational load, thus expanding the scope of technology applications.

[0027] In one embodiment, the multimodal input data includes at least two of the following: RGB image, infrared image, depth image, and polarization image; The image degradation mathematical model includes an atmospheric optical model, a motion blur model, and a noise model; An atmospheric optical model is established based on atmospheric optics principles to estimate transmittance maps and global atmospheric light in real time. The expression for the atmospheric optical model is as follows: I(x) = J(x) * t(x) + A * (1 - t(x)); Where I(x) is the observed image, J(x) is the sharp image to be recovered, t(x) is the transmittance map, and A is global atmospheric light. The parameters are estimated through the following steps: A dark channel prior algorithm is used to extract the dark channel of the image and initially screen the candidate regions of atmospheric light A. Combined with the distance information obtained by the depth camera, the dark channel is corrected to eliminate interference from near objects and determine the final atmospheric light A, with a value range of 0-255. In this case, A is about 220 in rainy and foggy weather. Based on the contour information of the infrared image, the estimation of the transmittance map t(x) is optimized to avoid transmittance distortion at the edges of objects in traditional algorithms. The final output pixel value of t(x) ranges from 0.1 to 0.8, that is, the higher the rain and fog concentration, the smaller t(x) is.

[0028] A motion blur model is established based on the motion characteristics of the camera and the object. The blur kernel is estimated by image feature matching. Specifically, the relative motion speed between the camera and the scene is calculated based on the longitudinal speed of the vehicle obtained by millimeter-wave radar. The blur direction and the size of the blur kernel are determined by matching feature points of adjacent frames of the image.

[0029] The noise model is a signal-dependent noise model. A Poisson-Gaussian noise model is established based on the statistical characteristics of image noise to analyze the noise distribution pattern. Specifically, dark frame images under different illumination intensities are collected, the noise distribution is statistically analyzed, and a mapping relationship between noise intensity and illumination intensity is established. When the illumination intensity is below 500 lux, the Poisson-Gaussian noise model is activated, where the Poisson noise coefficient is inversely proportional to the illumination intensity, and the standard deviation of Gaussian noise is set to 5.

[0030] The method provided in this embodiment offers rich and reliable raw data for image processing through multimodal data input. Three mathematical models for image degradation address three common and impactful image degradation problems: atmospheric interference, motion blur, and noise interference, providing precise and effective processing methods. The overall solution effectively addresses image degradation problems in various complex scenarios, significantly improving the quality of degraded images, restoring key details and information, and providing high-quality image data for subsequent applications such as image analysis, target recognition, and scene understanding. It has broad application prospects and high practical value, meeting the high-quality image data needs of multiple fields such as security monitoring, autonomous driving, remote sensing, and medical imaging.

[0031] In one embodiment, image reconstruction and enhancement are performed based on an image degradation mathematical model and prediction results, specifically including the following steps: By embedding a mathematical model of image degradation into a deep learning network through differentiable programming, a differentiable rendering process is constructed. The model's backpropagation is achieved by solving for the clear image from the degraded image through a reverse physical process: the network outputs the clear image J(x) to be restored, with the degraded image I(x) as input. The loss function adopts "perceptual loss + L1 loss", where the perceptual loss is calculated based on the feature map of the VGG-16 network to ensure the visual consistency of J(x), and the L1 loss is used to reduce the pixel error between J(x) and the same scene image collected on a sunny day as the real clear image. The network is trained with the Adam optimizer. After 10,000 iterations, the model achieves a PSNR (Peak Signal-to-Noise Ratio) of 32dB in rainy and foggy scenes.

[0032] The reinforcement learning policy network employs the DQN algorithm. Based on the current environmental parameters and the prediction parameters of the dynamic scene prediction model, it selects a target enhancement algorithm or a combination of algorithms from a pre-set enhancement algorithm library. This library includes histogram equalization, Retinex, and homomorphic filtering algorithms. The parameters of the target enhancement algorithm, including gain intensity, gamma value, and filter size, are adjusted before performing image enhancement operations. Specific embodiments further illustrate this: In the rainy / foggy scene, the current mean t(x) is 0.3, and the predicted mean t(x) is 0.2 (due to increased fog concentration). The policy network selects the combination of "inverse solution of atmospheric optical model + Retinex algorithm", where the scale parameters of the Retinex algorithm are set to 3, 5, and 7 to enhance edge details. In the tunnel entrance scene, the current illumination intensity is 5000 lux and the predicted illumination intensity is 1000 lux. The policy network is selected as a combination of "homomorphic filtering + contrast-limited adaptive histogram equalization (CLAHE)". The cutoff frequency of the homomorphic filter is set to 0.1 and the cliplimit of CLAHE is set to 2.0 to avoid amplification of noise in dark areas. In high-speed motion scenes, the current blur kernel is 9×9, the predicted blur kernel is 11×11 (increased vehicle speed), the policy network is selected as "motion blur model reverse deblurring + gamma correction", the gamma value is set to 0.8 to improve image brightness.

[0033] The method provided in this embodiment, from the perspective of the overall image processing quality, solves for a clear image from the degraded image through a reverse physical process in the image reconstruction stage, providing a high-quality "base image" for subsequent enhancement operations, avoiding the "degradation and amplification" problem that may be caused by directly enhancing the degraded image; the image enhancement stage intelligently selects algorithms and adjusts parameters based on the dynamic needs of the scene, further optimizing the visual effect and practical value of the image, so that the final output image not only conforms to physical reality, but also meets the application needs in different scenarios.

[0034] In one embodiment, the enhanced image is quality-assessed, and the assessment results are fed back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization. This includes the following steps: A pre-trained no-reference image quality assessment model is adopted. Specifically, a pre-trained MobileNet-V2 model can be used as the NR-IQA model. The enhanced image is input, and the sharpness score, naturalness score, and information entropy score of the enhanced image are calculated. The sharpness score is calculated based on the image gradient magnitude, the naturalness score is calculated based on the similarity between the color histogram and the sunny scene, and the information entropy score is calculated based on the entropy value of the image grayscale distribution.

[0035] The clarity score, naturalness score, and information entropy score are compared with preset quality thresholds. If the score is lower than the threshold, a feedback signal is generated. For example, after enhancement, the naturalness score is 68, which is lower than the threshold of 70. The feedback signal is sent to the dynamic scene prediction engine to adjust the prediction weight of atmospheric light A. At the same time, it is sent to the adaptive enhancement rendering core to increase the scale parameter of the Retinex algorithm to 9 and optimize color reproduction.

[0036] Feedback signals are sent to the dynamic scene prediction model to adjust weights and to the image reconstruction and enhancement process to optimize enhancement algorithm parameters, thereby achieving system self-optimization.

[0037] The method provided in this embodiment uses a no-reference evaluation model that does not require the original image to quickly output the sharpness, naturalness, and information entropy scores of the enhanced image to avoid subjective bias. The scores are compared with preset thresholds, and when the scores are low, a precise feedback signal is generated to build a quality "gatekeeping mechanism" and provide a basis for optimization. The feedback signal can also adjust the weights of the scene prediction model and optimize the parameters of the enhancement algorithm, forming an "evaluation-feedback-optimization" closed loop, improving the self-improvement and adaptive capabilities of the solution, and achieving stable output of high-quality images in dynamic scenes.

[0038] In one embodiment, lightweight deployment and hardware acceleration of the model are implemented, specifically including the following steps: The deep learning model undergoes network pruning to remove redundant neurons and connections, reducing model complexity. The pruned model is then quantized using TensorRT, converting 32-bit floating-point precision to 16-bit or 8-bit fixed-point precision, reducing computational load and increasing inference speed from 20fps to 35fps, meeting the 30fps real-time requirement of automotive cameras. Low-level adaptation is performed for the hardware chips of edge computing devices, with shared memory optimization for image convolution operations reducing global memory accesses. Stream parallelism is employed for multimodal data processing, executing data reading and model inference in parallel. The hardware chips include NVIDIA Jetson, Huawei Atlas, and Bitmain AI chips, optimizing instruction execution efficiency for real-time operation.

[0039] The method provided in this embodiment significantly improves the practicality and running efficiency of the model through lightweight deployment and hardware acceleration: First, the deep learning model is pruned to remove redundant neurons and connections, effectively reducing model complexity and resource consumption; then, the pruned model is quantized, converting 32-bit floating-point precision to 16-bit or 8-bit fixed-point precision, greatly reducing the amount of computation and further reducing the hardware running pressure; finally, the underlying adaptation is performed for edge computing device hardware chips such as NVIDIA Jetson, Huawei Atlas, and Bitmain AI chips to optimize instruction execution efficiency, ultimately enabling the model to run in real time on edge devices, meeting the needs for low latency and high response speed in multiple scenarios.

[0040] System Implementation Examples According to embodiments of the present invention, an adaptive image enhancement system based on physical modeling and dynamic prediction is provided, such as... Figure 2 The diagram shown is a structural schematic of the adaptive image enhancement system based on physical modeling and dynamic prediction provided in this embodiment. The adaptive image enhancement system based on physical modeling and dynamic prediction according to this embodiment includes a model building module 21, a dynamic scene prediction module 22, an adaptive enhancement module 23, an optimization module 24, and a deployment module 25.

[0041] Model building module 21 is used to build a mathematical model of image degradation based on the received multimodal input data.

[0042] The dynamic scene prediction module 22 is used to analyze continuous frame sequences and predict the changing trends of scene state and degradation parameters at the next moment using a dynamic scene prediction model that employs a temporal convolutional network and a long short-term memory network.

[0043] The adaptive enhancement module 23 is used to perform image reconstruction and enhancement based on the image degradation mathematical model and prediction results.

[0044] The optimization module 24 is used to evaluate the quality of the enhanced image and feed the evaluation results back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization.

[0045] Deployment module 25 is used for lightweight deployment and hardware acceleration of the model to adapt to real-time operation on edge computing devices.

[0046] The system provided in this embodiment constructs an image degradation mathematical model based on received multimodal input data through a model building module 21. It establishes an accurate degradation model through multimodal data fusion, quantifying the physical mechanisms of image degradation in different scenarios and providing precise physical constraints for subsequent enhancement. The dynamic scene prediction module 22 uses a dynamic scene prediction model combining a temporal convolutional network and a long short-term memory network to analyze continuous frame sequences and predict the changing trends of scene states and degradation parameters at the next moment. By utilizing a combined model of temporal convolutional networks and long short-term memory networks, it achieves forward-looking prediction of changes in scene states and degradation parameters, overcoming the limitations of static processing and allowing for advance adjustments to enhancement strategies, avoiding enhancement lag caused by sudden scene changes. The adaptive enhancement module 23 performs image reconstruction and enhancement based on the image degradation mathematical model and prediction results. The enhancement module combines the degradation model and prediction results to ensure that the enhancement process conforms to physical laws to avoid distortion, while also achieving accurate and adaptive image quality improvement based on the prediction and dynamic adaptation strategy. The optimization module 24 performs quality evaluation on the enhanced image and feeds the evaluation results back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization. Through quality evaluation, a closed-loop feedback is formed, which can correct the prediction model and enhancement strategy in real time, continuously optimize the enhancement effect, and solve the problem of insufficient adaptability caused by the lack of self-iteration capability of the algorithm. The deployment module 25 performs lightweight deployment and hardware acceleration of the model to adapt to real-time operation on edge computing devices, realizes the efficient operation of the model on edge devices, meets the real-time requirements, solves the problem that complex algorithms are difficult to implement in edge scenarios due to large computational load, and expands the application scope of the technology.

[0047] In one embodiment, the multimodal input data includes at least two of the following: RGB image, infrared image, depth image, and polarization image.

[0048] The image degradation mathematical model includes an atmospheric optical model, a motion blur model, and a noise model.

[0049] An atmospheric optical model is established based on atmospheric optics principles to estimate transmittance maps and global atmospheric light in real time. The expression for the atmospheric optical model is as follows: I(x) = J(x) * t(x) + A * (1 - t(x)); Where I(x) is the observed image, J(x) is the clear image to be recovered, t(x) is the transmittance map, and A is the global atmospheric light.

[0050] A motion blur model is established based on the motion characteristics of the camera and the object, and the blur kernel is estimated by image feature matching.

[0051] The noise model is a signal-dependent noise model. A Poisson-Gaussian noise model is established based on the statistical characteristics of image noise to analyze the noise distribution pattern.

[0052] The system provided in this embodiment offers rich and reliable raw data for image processing through multimodal data input. Three mathematical models for image degradation address three common and impactful image degradation problems: atmospheric interference, motion blur, and noise interference, providing precise and effective processing methods. The overall solution effectively addresses image degradation issues in various complex scenarios, significantly improving the quality of degraded images, restoring key details and information, and providing high-quality image data for subsequent applications such as image analysis, target recognition, and scene understanding. It has broad application prospects and high practical value, meeting the high-quality image data needs of multiple fields such as security monitoring, autonomous driving, remote sensing, and medical imaging.

[0053] In one embodiment, the adaptive enhancement module 23 includes a differentiable programmable unit and a reinforcement learning policy network unit.

[0054] Differentiable programming units are used to embed image degradation mathematical models into deep learning networks through differentiable programming, construct differentiable rendering processes, and solve for clear images from degraded images through inverse physical processes.

[0055] The reinforcement learning policy network unit is used to select a target enhancement algorithm or a combination of algorithms from a preset enhancement algorithm library based on the reinforcement learning policy network, according to the current environmental parameters and the prediction parameters of the dynamic scene prediction model. The preset enhancement algorithm library includes histogram equalization algorithm, Retinex algorithm and homomorphic filtering algorithm, and adjusts the parameters of the target enhancement algorithm, including gain intensity, gamma value and filter size, and performs image enhancement operation.

[0056] The system provided in this embodiment, from the perspective of the quality of the entire image processing process, solves for a clear image from the degraded image through a reverse physical process in the image reconstruction stage, providing a high-quality "base image" for subsequent enhancement operations, avoiding the "degradation and amplification" problem that may be caused by directly enhancing the degraded image; the image enhancement stage intelligently selects algorithms and adjusts parameters based on the dynamic needs of the scene, further optimizing the visual effect and practical value of the image, so that the final output image not only conforms to physical reality, but also meets the application needs in different scenarios.

[0057] The embodiments of the present invention are device embodiments corresponding to the above method embodiments. The specific operations of each module processing step can be understood with reference to the description of the method embodiments, and will not be repeated here.

[0058] like Figure 3 As shown, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the adaptive image enhancement method based on physical modeling and dynamic prediction in the above embodiments, or when the computer program is executed by a processor, it implements the adaptive image enhancement method based on physical modeling and dynamic prediction in the above embodiments.

[0059] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0060] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and the contents not described in detail in the specification of the present invention are known to those skilled in the art.

Claims

1. An adaptive image enhancement method based on physical modeling and dynamic prediction, characterized in that, Includes the following steps: A mathematical model of image degradation is constructed based on the received multimodal input data; A dynamic scene prediction model using temporal convolutional networks and long short-term memory networks is used to analyze continuous frame sequences and predict the changing trends of scene state and degradation parameters at the next moment. Image reconstruction and enhancement are performed based on the mathematical model of image degradation and the prediction results. The enhanced image is quality assessed, and the assessment results are fed back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization. The model is deployed in a lightweight manner and hardware accelerated to adapt to real-time operation on edge computing devices.

2. The adaptive image enhancement method based on physical modeling and dynamic prediction as described in claim 1, characterized in that, The multimodal input data includes at least two of the following: RGB image, infrared image, depth image, and polarization image. The image degradation mathematical model includes an atmospheric optical model, a motion blur model, and a noise model; An atmospheric optical model is established based on atmospheric optics principles to estimate transmittance maps and global atmospheric light in real time. The expression for the atmospheric optical model is as follows: I(x) = J(x) * t(x) + A * (1 - t(x)); Where I(x) is the observed image, J(x) is the clear image to be recovered, t(x) is the transmittance map, and A is the global atmospheric light; A motion blur model is established based on the motion characteristics of the camera and the object, and the blur kernel is estimated by image feature matching. The noise model is a signal-dependent noise model. A Poisson-Gaussian noise model is established based on the statistical characteristics of image noise to analyze the noise distribution pattern.

3. The adaptive image enhancement method based on physical modeling and dynamic prediction as described in claim 1, characterized in that, The image reconstruction and enhancement based on the image degradation mathematical model and prediction results specifically includes the following steps: By embedding a mathematical model of image degradation into a deep learning network through differentiable programming, a differentiable rendering process is constructed, and a clear image is obtained from the degraded image through an inverse physical process. Based on a reinforcement learning policy network, a target enhancement algorithm or a combination of algorithms is selected from a preset enhancement algorithm library according to the current environmental parameters and the prediction parameters of the dynamic scene prediction model. The preset enhancement algorithm library includes histogram equalization algorithm, Retinex algorithm and homomorphic filtering algorithm. The parameters of the target enhancement algorithm are adjusted, including gain intensity, gamma value and filter size, and then the image enhancement operation is performed.

4. The adaptive image enhancement method based on physical modeling and dynamic prediction as described in claim 1, characterized in that, The process of evaluating the quality of the enhanced image and feeding the evaluation results back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization includes the following steps: A pre-trained no-reference image quality assessment model was used to calculate the sharpness score, naturalness score, and information entropy score of the enhanced image. The clarity score, naturalness score, and information entropy score are compared with a preset quality threshold. If the score is lower than the threshold, a feedback signal is generated. Feedback signals are sent to the dynamic scene prediction model to adjust weights and to the image reconstruction and enhancement process to optimize enhancement algorithm parameters.

5. The adaptive image enhancement method based on physical modeling and dynamic prediction as described in claim 1, characterized in that, The lightweight deployment and hardware acceleration of the model specifically includes the following steps: Pruning deep learning models removes redundant neurons and connections, reducing model complexity. The pruned model is quantized to convert 32-bit floating-point precision to 16-bit or 8-bit fixed-point precision, reducing the amount of computation. The underlying hardware chips for edge computing devices are adapted, including NVIDIA Jetson chips, Huawei Atlas chips, and Bitmain AI chips, to optimize instruction execution efficiency for real-time operation.

6. An adaptive image enhancement system based on physical modeling and dynamic prediction, characterized in that, It includes a model building module, a dynamic scene prediction module, an adaptive enhancement module, an optimization module, and a deployment module; The model building module is used to build an image degradation mathematical model based on the received multimodal input data; The dynamic scene prediction module is used to analyze continuous frame sequences and predict the changing trends of scene state and degradation parameters at the next moment using a dynamic scene prediction model of temporal convolutional network and long short-term memory network. The adaptive enhancement module is used to perform image reconstruction and enhancement based on the image degradation mathematical model and prediction results; The optimization module is used to evaluate the quality of the enhanced image and feed the evaluation results back to the dynamic scene prediction model and the image reconstruction and enhancement process for optimization. The deployment module is used to deploy the model in a lightweight manner and accelerate it with hardware to adapt it for real-time operation on edge computing devices.

7. The adaptive image enhancement system based on physical modeling and dynamic prediction as described in claim 6, characterized in that, The multimodal input data includes at least two of the following: RGB image, infrared image, depth image, and polarization image. The image degradation mathematical model includes an atmospheric optical model, a motion blur model, and a noise model; An atmospheric optical model is established based on atmospheric optics principles to estimate transmittance maps and global atmospheric light in real time. The expression for the atmospheric optical model is as follows: I(x) = J(x) * t(x) + A * (1 - t(x)); Where I(x) is the observed image, J(x) is the clear image to be recovered, t(x) is the transmittance map, and A is the global atmospheric light; A motion blur model is established based on the motion characteristics of the camera and the object, and the blur kernel is estimated by image feature matching. The noise model is a signal-dependent noise model. A Poisson-Gaussian noise model is established based on the statistical characteristics of image noise to analyze the noise distribution pattern.

8. The adaptive image enhancement system based on physical modeling and dynamic prediction as described in claim 6, characterized in that, The adaptive enhancement module includes a differentiable programmable unit and a reinforcement learning policy network unit; The differentiable programming unit is used to embed the image degradation mathematical model into a deep learning network through differentiable programming, construct a differentiable rendering process, and solve for a clear image from the degraded image through a reverse physical process; The reinforcement learning policy network unit is used to select a target enhancement algorithm or a combination of algorithms from a preset enhancement algorithm library based on the reinforcement learning policy network and the prediction parameters of the current environment parameters and the dynamic scene prediction model. The preset enhancement algorithm library includes histogram equalization algorithm, Retinex algorithm and homomorphic filtering algorithm, and adjusts the parameters of the target enhancement algorithm, including gain intensity, gamma value and filter size, and performs image enhancement operation.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the adaptive image enhancement method based on physical modeling and dynamic prediction as described in any one of claims 1 to 5.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the adaptive image enhancement method based on physical modeling and dynamic prediction as described in any one of claims 1 to 5.

Images