An infrared image enhancement method and system based on deep reinforcement learning and a storage medium

Abstract

The application discloses an infrared image enhancement method and system based on deep reinforcement learning, a storage medium, and the like. An original infrared image to be enhanced is taken as an initial state, and an image signal processing model based on an Actor-Critic architecture and pre-trained is input. The model outputs a hybrid action at each step in a step-by-step enhancement process through forward propagation. The hybrid action includes an operator index for selecting a corresponding operator from an enhancement operator library and a normalized intensity parameter for adjusting processing intensity. A corresponding processing function is called according to the operator index, and a current image is processed based on the intensity parameter to obtain an enhanced intermediate image as a next state. The above steps are repeated until a preset step number is reached, and a final enhanced image is output. Through a step-by-step decision mechanism, the application adaptively plans an optimal enhancement path without high-quality paired data, and realizes the collaborative improvement of contrast, texture details and visual naturalness.

Description

Technical Field

[0001] This application relates to the field of image signal processing technology, and in particular to an infrared image enhancement method, system, and storage medium based on deep reinforcement learning. Background Technology

[0002] Infrared thermal imaging technology, with its ability to withstand various lighting conditions and penetrate smoke, has demonstrated irreplaceable application value in fields such as all-weather military reconnaissance and search and rescue in extreme environments. This technology can capture the thermal radiation information of target objects and convert it into visual images, thus providing crucial visual support at night or in adverse weather conditions.

[0003] To improve the visual quality of infrared images, various image enhancement methods have been developed, which can be broadly categorized into traditional algorithms based on mathematical transformations. For example, histogram equalization methods improve overall contrast by adjusting the grayscale distribution of the image; these methods are computationally simple and widely used. Methods based on filtering theory attempt to separate noise from signals in an image, thus suppressing noise while retaining useful information. Methods based on Retinex models simulate the characteristics of human vision, enhancing details by estimating illumination components. However, these traditional methods generally have significant limitations in practical applications: histogram processing essentially adjusts global statistical characteristics, easily altering local texture details and amplifying background noise while enhancing contrast; filtering methods, during noise separation, often inevitably smooth out important subtle textures as noise; and Retinex-based methods, when processing low-contrast infrared images, are prone to over-enhancement or under-enhancement due to inaccurate illumination component estimation. It is evident that existing traditional algorithms have always struggled to achieve a balance between contrast enhancement, detail preservation, and noise suppression, often falling into a dilemma where enhancing contrast amplifies noise, while suppressing noise results in texture loss, thus failing to achieve a comprehensive improvement in image quality.

[0004] In summary, how to effectively preserve subtle texture details and suppress background noise amplification while improving the contrast of infrared images, and achieve synergistic optimization of multi-dimensional quality indicators, has become a technical challenge that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the aforementioned issues, this application provides an infrared image enhancement method, system, and storage medium based on deep reinforcement learning.

[0006] The embodiments of this application disclose the following technical solutions: The first aspect of this application provides an infrared image enhancement method based on deep reinforcement learning, comprising: The original infrared image to be enhanced is taken as the current state; the current state is input into a pre-trained image signal processing model to generate a hybrid action. The pre-trained image signal processing model is trained by a deep reinforcement learning algorithm based on the Actor-Critic architecture. It takes the original infrared image as input and outputs the hybrid action corresponding to each step in the step-by-step enhancement process through forward propagation. The hybrid action includes an operator index for selecting the corresponding image processing operator from a preset enhancement operator library and a normalized intensity parameter for adjusting the processing intensity of the selected image processing operator. The corresponding processing function in the enhancement operator library is called according to the operator index, and the current image is processed based on the normalized intensity parameter to obtain the enhanced intermediate image as the next state; The next state is taken as the new current state, and the process of inputting the current state into the policy network and subsequent steps is repeated until a preset number of steps are reached. The final intermediate image is then output as the enhanced target image.

[0007] In one possible implementation, the training process of the image signal processing model includes: The current state is set as a preliminary image and input into a preliminary model to generate a hybrid action. The target model includes a policy network and a value network. The hybrid action includes an operator index determined by sampling the discrete probability distribution output by the policy network and a normalized intensity parameter determined by sampling the continuous Beta distribution output by the policy network. The corresponding processing function in the enhancement operator library is called according to the operator index, and the current image is processed based on the normalized intensity parameter to obtain an enhanced intermediate image as the next state. The incremental comprehensive quality score of the image before and after processing is calculated as an immediate reward. The current state, hybrid action, immediate reward, and next state are stored as experience data. The next state is used as the new current state, and the process of inputting the current state into the policy network and subsequent steps is repeated until a preset number of steps is reached. Based on stored empirical data, a deep reinforcement learning algorithm is used to iteratively train the policy network and value network to obtain an image signal processing model with optimized network parameters.

[0008] In one possible implementation, before inputting the current state into the preparatory model to generate the hybrid action, the method further includes: An enhancement operator library is constructed, which integrates multiple image processing operators, each configured to receive continuous intensity parameters to process the image.

[0009] In one possible implementation, the image processing operators integrated in the enhancement operator library include at least one or more of the following: contrast-limited adaptive histogram equalization, dynamic detail enhancement, logarithmic transformation, gamma correction, single-scale Retinex, nonlocal mean denoising, adaptive median filtering, unsharpening mask, and wavelet thresholding denoising; each operator is encapsulated as a callable function interface, and the input normalized intensity parameter is mapped to the specific execution parameters inside the operator.

[0010] In one possible implementation, before inputting the current state into the preparatory model to generate the hybrid action, the method further includes: A backbone feature extraction network is constructed, which employs a pre-trained deep convolutional neural network to extract deep semantic features from the input infrared image. A policy network and a value network are built on top of the backbone feature extraction network. The output layer of the policy network adopts a two-branch decoupled structure including discrete and continuous branches. The discrete branch outputs the selection probability distribution of each enhancement operator through the Softmax activation function, and the continuous branch characterizes the intensity adjustment range of the selected operator by outputting the probability density function parameter of the Beta distribution. The output layer of the value network is used to estimate the expected cumulative reward of the current image state and outputs a scalar value.

[0011] In one possible implementation, storing the current state, mixed actions, immediate rewards, and the next state as experience data includes: Store the current state, mixed actions, immediate rewards, and next state as experience data in the experience replay buffer. The stored empirical data is used to iteratively train the policy network and value network using a deep reinforcement learning algorithm to obtain an image signal processing model with optimized network parameters, including: A preset number of empirical data are randomly sampled from the empirical replay buffer. A deep reinforcement learning algorithm is used to iteratively train the policy network and the value network based on the preset number of empirical data to optimize the network parameters and obtain the trained image signal processing model.

[0012] In one possible implementation, the deep reinforcement learning algorithm includes a proximal policy optimization algorithm, which combines generalized advantage estimation to calculate the advantage function during training to balance bias and variance; the policy network update adopts a truncation mechanism, which limits the update magnitude of the probability ratio between the new and old policies and introduces policy entropy as a regularization term in the loss function; the value network update is achieved by minimizing the error between the state value estimation and the multi-step reward objective.

[0013] In one possible implementation, the incremental composite quality score of the computed image before and after processing is used as an immediate reward, including: The first comprehensive quality score of the image in the current state is calculated based on a linear weighted combination of fused information entropy, average gradient, enhancement measure and no-reference natural image quality evaluation index. After performing the blending action and processing the current image, the same linear weighted combination is used to calculate the second comprehensive quality score of the image in the next state; Calculate the difference between the second comprehensive quality score and the first comprehensive quality score, and use the difference as the immediate reward for the current step.

[0014] A second aspect of this application provides an infrared image enhancement system based on deep reinforcement learning, comprising: A generation unit is used to take the original infrared image to be enhanced as the current state; input the current state into a pre-trained image signal processing model to generate hybrid actions. The pre-trained image signal processing model is trained by a deep reinforcement learning algorithm based on the Actor-Critic architecture. It takes the original infrared image as input and outputs the hybrid actions corresponding to each step in the step-by-step enhancement process through forward propagation. The hybrid actions include operator indices for selecting corresponding image processing operators from a preset enhancement operator library and normalized intensity parameters for adjusting the processing intensity of the selected image processing operators. The processing unit is used to call the corresponding processing function in the enhancement operator library according to the operator index, and process the current image based on the normalized intensity parameter to obtain the enhanced intermediate image as the next state; An iterative unit is used to take the next state as the new current state, repeatedly execute the steps of inputting the current state into the policy network and subsequent steps until a preset number of steps are reached, and output the final intermediate image as the enhanced target image.

[0015] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the infrared image enhancement method based on deep reinforcement learning as described in the first aspect above.

[0016] A fourth aspect of this application provides a computer program product that, when run on a computer, executes the infrared image enhancement method based on deep reinforcement learning as described in the first aspect above.

[0017] A fifth aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the infrared image enhancement method based on deep reinforcement learning as described in the first aspect above.

[0018] Compared with the prior art, this application has the following beneficial effects: The original infrared image to be enhanced is used as the initial state and input into a pre-trained image signal processing model. The model outputs a hybrid action corresponding to each step of the step-by-step enhancement process through forward propagation. This hybrid action is not a single global operation, but a refined instruction containing operator indexes and normalized intensity parameters. It can dynamically select the operator type most suitable for the current image state from the enhancement operator library and precisely control its processing intensity. By calling the corresponding processing function according to the operator index and processing the image based on the intensity parameters, the intermediate image is obtained as the next state in a loop execution. This achieves progressive optimization by observing the current state, deciding the optimal action, and executing step-by-step enhancement, completely abandoning the one-size-fits-all global adjustment mode of traditional algorithms. This step-by-step mechanism enables the model to precisely intervene in the specific degradation features of the image at each step: when improving contrast, the boundary of noise amplification is precisely controlled by selecting appropriate operators and intensity parameters; when enhancing texture details, targeted enhancement is performed on weak edges rather than overall smoothing. After a preset number of iterations, the final output target image achieves a dynamic balance between contrast, detail richness, and noise suppression level that is impossible to achieve with traditional methods. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating an infrared image enhancement method based on deep reinforcement learning, provided for an embodiment of this application; Figure 2 This is a schematic diagram of the overall framework of an infrared image enhancement method based on deep reinforcement learning. Figure 3 A flowchart illustrating an infrared image enhancement implementation provided in this application embodiment; Figure 4 This is a comparison diagram of the enhancement effects of the embodiments of the present invention and traditional methods; Figure 5 This is a comparison chart showing the enhancement effects of the embodiments of this application and deep learning methods; Figure 6 This is a comparison chart showing the effects of different variant strategies in the embodiments of this application; Figure 7 This is a structural diagram of an infrared image enhancement system based on deep reinforcement learning, provided in an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0022] To facilitate understanding of the technical solutions provided in the embodiments of this application, the technical terms involved in the embodiments of this application will be explained below.

[0023] Continuous Beta distribution sampling refers to the process of randomly sampling from the probability density function of the Beta distribution in a continuous action space to obtain specific action values. The Beta distribution is a continuous probability distribution defined on the closed interval [0,1], and its distribution shape is controlled by two shape parameters α and β, which can flexibly represent various probability characteristics such as symmetry, left skewness, right skewness, or uniformity.

[0024] Single-scale Retinex is an image enhancement algorithm based on the theory of color constancy. Its core idea is to simulate the human visual system's perception mechanism of illumination and reflection. The algorithm assumes that an image can be obtained by multiplying an illumination component and a reflection component. The illumination component represents the lighting conditions of a scene, changing slowly and concentrated in the low-frequency range; the reflection component represents the reflective properties of an object itself, containing rich texture details and changing dramatically. Single-scale Retinex transforms the multiplicative relationship into an additive relationship through a logarithmic domain transformation, then estimates the illumination component through Gaussian low-pass filtering, and finally subtracts the illumination component from the original image in the logarithmic domain to obtain the enhanced reflection component. In the embodiments of this application, single-scale Retinex, as a basic operator in the enhancement operator library, is mainly used to correct the contrast reduction problem in infrared images caused by uneven illumination or thermal radiation interference. By adjusting its scale parameter, the intensity and range of detail enhancement can be controlled.

[0025] The Softmax activation function is a mathematical function that maps a real-valued vector to a probability distribution. It is widely used in the output layer of multi-class classification problems. For a K-dimensional real-valued vector, the Softmax function exponentially normalizes each component, ensuring that all output values ​​lie in the (0,1) interval and sum to 1, thus forming an output that conforms to the characteristics of a probability distribution.

[0026] The Actor-Critic architecture is a deep reinforcement learning framework that combines policy-based and value-based methods, consisting of two core components: an actor and a critic. Common Chinese translations include "actor-critic architecture" or "actor-critic architecture." The actor, corresponding to the policy network, is responsible for outputting the probability distribution or specific action value based on the current state, directly participating in the decision-making process. The critic, corresponding to the value network, is responsible for evaluating the expected cumulative reward of the current state or action, providing the actor with optimization directions. The critic's evaluation results are fed back to the actor in the form of an advantage function, guiding it to adjust its policy to select actions that yield higher rewards.

[0027] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0028] As mentioned earlier, traditional methods struggle to balance contrast enhancement, detail preservation, and noise suppression; deep learning methods heavily rely on high-quality paired data and are prone to artifacts; existing methods lack adaptability and generalization ability; and model training is unstable and inference efficiency is low.

[0029] This application models the image enhancement process as a Markov decision process, employing a step-by-step observation, decision, and execution mechanism instead of the traditional end-to-end one-time processing. In each decision step, the policy network dynamically selects the most suitable operator type from the enhancement operator library based on the current image's specific state and finely adjusts its intensity parameters. This step-by-step optimization avoids the one-size-fits-all global adjustments of traditional algorithms, enabling precise control of noise amplification while improving contrast and selectively enhancing subtle textures, thus achieving a dynamic balance across multiple quality dimensions.

[0030] This application designs a hybrid reward function that does not require reference images, completely eliminating the dependence on paired "sharp-blurred" training data. This reward function integrates information entropy, average gradient, augmentation measure, and no-reference natural image quality evaluation metrics to comprehensively assess image quality from multiple dimensions, including information content, sharpness, contrast, and visual naturalness. By calculating the increase in quality score before and after image processing as an immediate reward, the model can be trained using only the original low-quality infrared image and spontaneously converges towards a direction consistent with human visual perception during optimization. This fundamentally solves the problems of scarce paired data and the tendency of generative models to produce false textures.

[0031] This application constructs an enhancement operator library containing various classic image processing operators and employs a reinforcement learning-based decision-making mechanism, enabling the model to adaptively plan enhancement paths based on the specific degradation features of the input image. Whether the image is low-contrast, high-noise, or texture-blurred, the model can dynamically adjust the operator combination and parameter configuration, achieving flexible responses to different scenarios. Furthermore, the modular design of the operator library supports the flexible addition or replacement of novel processing algorithms, significantly improving the method's versatility and engineering adaptability.

[0032] It should be noted that the infrared image enhancement method, system, and medium based on deep reinforcement learning provided in this application can be applied to the field of image signal processing technology. The above are merely examples and do not limit the application areas of the infrared image enhancement method, system, and medium based on deep reinforcement learning provided in this application. Furthermore, the embodiments of this application may not limit the executing entity of the infrared image enhancement based on deep reinforcement learning. For example, the infrared image enhancement method based on deep reinforcement learning in the embodiments of this application can be applied to data processing devices such as terminal devices or servers. The terminal device can be an electronic device such as a computer or a personal digital assistant (PDA). The server can be a standalone server, a cloud server, or a cluster server composed of multiple servers.

[0033] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0034] The following embodiment illustrates an infrared image enhancement method based on deep reinforcement learning provided in this application. See also... Figure 1 ,Should Figure 1 A flowchart of an infrared image enhancement method based on deep reinforcement learning provided in this application embodiment, the method comprising: S101. Construct an enhanced operator library.

[0035] The operator library, serving as the action space within the reinforcement learning framework, aims to provide models with flexible image processing capabilities. Specifically, it integrates various classic and complementary image processing algorithms, including Contrast-Limited Adaptive Histogram Equalization (CLAHE), dynamic detail enhancement, logarithmic transformation, gamma correction, single-scale Retinex, nonlocal mean denoising, adaptive median filtering, unsharpening masks, and wavelet thresholding denoising. These operators cover multiple dimensions, including contrast adjustment, detail enhancement, brightness correction, and denoising, effectively addressing common infrared image problems such as insufficient contrast, blurred edges, and noise interference. It is important to emphasize that the operators listed above are merely illustrative and can be flexibly extended or replaced according to specific needs in practical applications, reflecting the framework's modular design philosophy.

[0036] To support fine-tuning of infrared images in subsequent steps, each enhancement operator is encapsulated as a unified function interface, designed to receive continuous values ​​normalized to a standard closed interval [0,1] as intensity control parameters. The core advantage of this design lies in mapping the originally discrete or irregular operator parameter space to a continuous, standardized interval, thus simplifying the action output space of the reinforcement learning model. Within the operator, the received normalized parameters are dynamically mapped to specific execution parameters, such as the truncation threshold and grid size of CLAHE, the kernel size of Gaussian filtering, and the exponent value of gamma transform. This parameter mapping mechanism ensures that the model does not need to concern itself with the specific implementation details of the underlying operators; it only needs to output actions in a unified, continuous space to achieve precise control over image contrast, brightness, and texture details. In this way, the operator library lays a solid foundation for subsequent step-by-step decision-making, enabling the model to, like a human expert, progressively select the most suitable processing tool and adjust its intensity based on the current image state, ultimately achieving a gradual improvement in infrared image quality.

[0037] S102. Build a policy network and value network based on the Actor-Critic architecture.

[0038] A deep reinforcement learning model based on an Actor-Critic architecture is constructed to provide a decision-making and evaluation foundation for subsequent step-by-step image enhancement. The model consists of two key components: a policy network (Actor) responsible for generating enhancement actions based on the current image state, and a value network (Critic) used to evaluate the expected cumulative reward of the current state. The two work together to improve learning efficiency and stability. To efficiently process high-dimensional infrared image data, the policy network and value network share a pre-trained backbone feature extraction network. This network can employ deep convolutional neural networks (CNNs), visual transformers (ViTs), multilayer perceptrons (MLPs), or combinations thereof, aiming to extract rich deep semantic features from the raw input to provide strong support for subsequent decision-making.

[0039] To address the challenge of simultaneously selecting enhancement operator types (discrete actions) and their intensity parameters (continuous actions) in infrared image enhancement tasks, the policy network's output layer is designed as a decoupled two-branch structure. The discrete branch outputs the selection probability distribution of each enhancement operator through a Softmax activation function, enabling the model to flexibly determine the appropriate processing method based on current image features. The continuous branch outputs the probability density function parameters of the Beta distribution, representing the adjustment range of the selected operator's intensity parameters. The Beta distribution is chosen because its definition within the closed interval [0,1] perfectly matches the normalized intensity parameter space and allows for flexible representation of various probability distribution forms through shape parameter adjustments, facilitating model exploration and utilization of different parameter values. The value network's output layer is relatively simple, outputting only a scalar value to estimate the expected cumulative return in the current state. This estimate serves as the basis for subsequent advantage function calculations.

[0040] During training, the system incorporates immediate rewards from environmental feedback and utilizes the Generalized Advantage Estimation (GAE) method to calculate the advantage function. This involves obtaining an advantage estimate through a weighted combination of temporal difference errors, which is then used to update the weighted policy gradient. The introduction of GAE reduces gradient estimation variance while keeping bias within a controllable range, thus significantly improving model convergence stability and training efficiency.

[0041] That is, one branch is used to output the probability distribution in the discrete action space. One branch represents the possibilities of choosing different enhancement operators, and this branch uses the Softmax activation function; the other branch outputs the probability density function of the Beta distribution over the continuous action space. and The parameter represents the intensity adjustment range of the selected operator. The output layer of the value network is a scalar value. This is used to estimate the expected cumulative reward of the current state. Based on this state value estimate, the system further combines the immediate reward from environmental feedback to calculate the generalized advantage estimate, i.e. (1) In formula (1) For time-series difference error, this advantage function is used to update the magnitude of the weighted policy gradient, thereby reducing the gradient estimation variance while keeping the bias within a controllable range, significantly improving the stability and efficiency of model convergence.

[0042] Figure 2 This diagram illustrates the overall framework of a deep reinforcement learning-based infrared image enhancement method, showcasing the complete interactive logic from steps S102 to S106: Given a low-quality raw infrared image as input, the model aims to learn an optimal policy that maps the current image state to an optimal distribution of enhancement actions, where are the learnable parameters of the policy network. Establishing this mapping relationship relies on the interactive data collection and iterative parameter updates in subsequent steps S103 to S106, ultimately enabling the model to possess adaptive, step-by-step enhancement capabilities. That is, given a low-quality raw infrared image as input... The goal of the model is to learn an optimal policy. This strategy maps the current image state to the optimal augmentation action distribution, which can be expressed as: (2) In formula (2) These are the learnable parameters of the policy network. To successfully train such a model capable of adaptive step-by-step augmentation, steps S103 to S106 are executed sequentially.

[0043] The following are Figure 2 The English annotations in the text provide explanations. Figure 2 middle The image state at the current step. Score(I) t Score(I) represents the overall quality score of the current state image. t+1 ) represents the overall image quality score for the next state, r t This represents the immediate reward for the current step. The Actor-Critic network uses an actor-critic architecture, comprising two sub-networks: the policy network (Actor) outputs mixed actions (discrete operator indices + continuous intensity parameters), and the value network (Critic) evaluates the value of the current state (expected cumulative reward). An enhanced operator library integrates a set of image processing operators. Figure 2Examples include: CLAHE (Contrast-Limited Adaptive Histogram Equalization), Gamma (Gamma Correction), Log (Logarithmic Transformation), and SSR (Single Scale Retinex). In the field of image enhancement, Retinex theory simulates the human visual system's perception of illumination and reflection, decomposing the image into illumination and reflection components. Single-scale Retinex estimates the illumination component through logarithmic domain transformation and Gaussian low-pass filtering, thereby extracting the reflection component to enhance image details. In the enhancement operator library of this patent, SSR, as a basic operator, is mainly used to correct the contrast reduction problem in infrared images caused by uneven illumination or thermal radiation interference. Its scale parameter can adjust the intensity and range of detail enhancement.

[0044] S103. Model the image enhancement task as a Markov decision process and execute step-by-step enhancement interactions.

[0045] The model is based on the current strategy For the current image state Perform reasoning to generate hybrid actions ;where discrete action components The specific operator index to be executed is determined by sampling the class probability distribution output by the policy network, and the continuous action components are then processed. Normalized intensity parameters are determined by sampling from the predicted probability distribution. Subsequently, the environment executes a state transition function. That is, calling the index The corresponding image processing operator and the applied parameters For the current image Enhancement processing is performed to generate an enhanced intermediate image. This serves as the state input for the next moment. The aforementioned "observation-decision-execution" process repeats cyclically until the preset maximum number of steps T is reached, thus forming a complete augmentation trajectory. This is used for subsequent value assessment.

[0046] The probability distribution output by the policy network is transformed into specific image processing operations, and a complete augmentation trajectory is generated through multi-step interaction with the environment, providing foundational data for subsequent value assessment and model updates. Specifically, the model infers the current image state based on the current policy network, generating hybrid actions that include discrete operator selection and continuous parameter adjustment. During action generation, the discrete action component determines the specific operator index to be executed by sampling the class probability distribution output by the discrete branches of the policy network. This sampling mechanism, rather than simply taking the maximum value, helps the model explore more possible processing paths in the early stages of training, avoiding premature convergence to local optima. The continuous action component determines the normalized intensity parameter by sampling the Beta distribution predicted by the continuous branches of the policy network. The domain of the Beta distribution perfectly matches the range of the normalized parameter, which can flexibly represent the probability density of different parameter values, providing a probabilistic representation basis for fine-grained image adjustment.

[0047] After the action is generated, the system enters the state transition phase. The environment executes a predefined state transition function, which calls the corresponding image processing algorithm from the enhancement operator library based on the operator index determined by the discrete action components. It also maps the normalized intensity parameter determined by the continuous action components to the specific execution parameters within the operator, such as mapping 0.7 to the exponent value of the gamma transform, 1.5. The operator is then applied to enhance the current infrared image. The processed image serves as the state input for the next time step, allowing the model to perform a new round of observation and decision-making. This process rigorously simulates the step-by-step image editing logic of human experts using image processing tools: first, observing the current image features; then, deciding which tool to use and adjusting its intensity; next, observing the processing effect; and finally, deciding on the next operation.

[0048] The aforementioned cycle of observation, decision-making, and execution continues until a preset maximum number of interaction steps is reached, which can be set to 3-5 steps, ensuring both enhancement effectiveness and inference efficiency. When the maximum number of steps is reached, the system completes a full enhancement trajectory. This trajectory includes the state, action, reward, and next state information for each step from the initial state to the final state. This data can be stored in an experience replay buffer for subsequent value network evaluation and policy network updates. This step-by-step enhancement trajectory design has significant advantages over traditional end-to-end generative models. On the one hand, it makes the model's decision-making process interpretable, allowing for traceability and analysis of each step. On the other hand, the trajectory data accumulated through multiple interactions provides rich training samples for reinforcement learning algorithms, helping the model learn more robust and efficient enhancement strategies. Through repeated execution of step S103, the model gradually accumulates interaction experience covering various infrared image degradation scenarios, laying a solid data foundation for subsequent reward calculation and parameter optimization.

[0049] S104. Calculate the hybrid reward function based on the characteristics of human visual perception.

[0050] To guide the model towards optimization in a direction consistent with human visual perception when high-quality paired reference images are lacking, this invention designs a hybrid reward function that does not require reference images, addressing the key challenge of the lack of high-quality paired data in infrared image enhancement. In traditional supervised learning methods, model training heavily relies on pairs of low-quality and high-quality images, but obtaining such data is extremely difficult in the infrared domain. To address this limitation, this invention proposes a reward mechanism based on multi-dimensional, reference-free quality assessment, enabling the model to optimize towards a direction consistent with human visual perception solely based on the features of the original image itself, without any ground truth reference.

[0051] The design philosophy of this reward function is to comprehensively evaluate the visual quality of infrared images from multiple complementary dimensions, ensuring the comprehensiveness and balance of model optimization. Specifically, the evaluation metrics cover at least the following four dimensions: First, the information content dimension, using information entropy (IE) to measure the richness of information contained in the image; higher information entropy means more details the image carries. Second, the sharpness dimension, using average gradient (AG) to evaluate the texture detail and edge sharpness of the image; a larger average gradient indicates clearer image edges and richer texture. Third, the contrast dimension, using enhancement measure (EME) to measure the dynamic range and contrast performance of the image; a higher enhancement measure means more vivid contrast between light and dark areas. Fourth, the visual naturalness and noise dimension, using the no-reference natural image quality evaluation metric (BRISQUE) to evaluate the perceptual quality of the image; a lower BRISQUE value indicates that the image better conforms to the statistical characteristics of a natural image, with fewer artifacts and noise. It should be noted that the above metrics are only preferred embodiments; in practical applications, other similar metrics, such as spatial frequency (SF), Laplacian gradient sum, contrast per pixel (CPP), or NIQE, can be flexibly replaced or supplemented according to specific needs.

[0052] In this embodiment, to take into account the unique imaging characteristics of infrared images (such as low contrast, high noise, and weak texture), a linear weighted combination of information entropy (IE), average gradient (AG), enhancement measure (EME), and the no-reference natural image quality evaluation index (BRISQUE) is specifically adopted, namely: (3) The use of the -BRISQUE form for the BRISQUE transformation is intended to maintain consistency with other metrics, meaning a higher value indicates better quality. The weighting coefficients can be adjusted based on different application scenarios and preferences to balance the importance of each dimension. This linearly weighted combination is both simple and interpretable, facilitating both calculation and debugging.

[0053] Finally, the instant reward at step t. Defined as the incremental change in the overall quality score of an image before and after motion processing, i.e. This drives the model to continuously improve image quality. By using relative changes rather than absolute values, the model focuses more on the actual improvement brought by each step of the operation, rather than pursuing a fixed absolute score. Secondly, incremental rewards are naturally sparse and delayed, which is in line with the optimization goal of reinforcement learning for long-term cumulative rewards. Finally, positive increments encourage the model to continue to optimize in the current direction, while negative increments indicate that the model needs to adjust its strategy, thus forming a clear optimization direction.

[0054] Through this multi-dimensional, no-reference hybrid reward function design, the model can gradually learn enhancement strategies that conform to human visual perception, relying solely on the feature evolution of the original infrared image itself, even in the absence of paired data. The quality score increment brought by each operation step becomes the core signal driving the continuous optimization of the model, ensuring that the model does not only focus on single-dimensional improvement, such as simply increasing contrast which may lead to noise amplification, but also pursues a comprehensive balance of information content, sharpness, contrast, and naturalness. The final output enhanced image can present rich details and sharp contrast, while maintaining a natural visual experience and avoiding artifacts and noise interference.

[0055] S105. Update network parameters based on deep reinforcement learning algorithm.

[0056] This step utilizes the interaction experience data collected in the previous steps to iteratively update the parameters of the policy network and value network using a specific optimization algorithm. This allows the model to learn from experience and gradually improve decision-making quality. This process simulates the cognitive mechanism by which humans summarize experience from practice. By repeatedly trying different augmentation operations and observing the quality changes brought about by these operations, the model ultimately learns to make optimal decisions under various infrared image conditions.

[0057] The model collects enhanced trajectory data, including state, action, reward, and next state, through interaction with the environment, and stores it as tuples in an experience replay buffer. This experience replay mechanism breaks the strong correlation between time-series data, ensuring that randomly sampled samples from the buffer satisfy the independent and identically distributed assumption, which is beneficial for stable training of the neural network. Furthermore, it allows the model to repeatedly utilize historical experience for learning, improving the efficiency of data sample utilization, which is particularly important for high-cost data acquisition, such as infrared images.

[0058] Once a sufficient number of samples have accumulated in the buffer, the system enters the model update phase, randomly sampling small batches of data from the buffer for network parameter optimization. During the model update phase, the system randomly samples small batches of data from the buffer and updates the network parameters based on a deep reinforcement learning algorithm. The deep reinforcement learning algorithm is preferably a policy gradient algorithm or an actor-critic architecture algorithm.

[0059] In selecting the deep reinforcement learning algorithm, this invention prioritizes the Proximal Policy Optimization (PPO) algorithm, which is one of the most stable and efficient algorithms in the field of reinforcement learning. The PPO algorithm is an improved version of policy gradient methods, and its core advantage lies in its ability to strictly limit the magnitude of each update while improving policy performance, avoiding training crashes or performance drops caused by excessively large step sizes. Combined with Generalized Advantage Estimation (GAE), the algorithm can more accurately estimate the superiority or inferiority of each action relative to the average level. GAE finds the optimal trade-off between multi-step accumulation of temporal difference errors and single-step estimation by balancing bias and variance, providing a high-quality optimization direction for policy updates. This invention employs a clipped Surrogate Objective. The algorithm first calculates the probability ratio of the old and new policies on the same state-action pair. This ratio reflects the change in the likelihood of choosing that action after the policy update: (4) in, and Let represent the probability distributions of the current policy network and the old policy network, respectively. The total loss function of the policy network. It consists of the truncated policy loss and the entropy regularization term, and its specific calculation formula is as follows: (5) In the formula, Estimating the advantage function, To truncate hyperparameters, This is a cutoff function used to limit the ratio to a certain value. Within the range; For the entropy of the strategy, This is the entropy regularization coefficient, which encourages the model to explore the unknown action space and prevents premature convergence to local optima. For value networks, the update objective is to minimize the state value estimate. With true return target The difference between them is expressed using mean squared error as the loss function. : (6) The system utilizes gradient descent to minimize the policy loss and value loss, respectively, thereby updating the network. Therefore, the entire parameter update process is iteratively performed using gradient descent, with each update only fine-tuning the network parameters to ensure smooth policy evolution. Through multiple rounds of data collection, experience replay, and parameter update cycles, the policy network gradually learns to automatically select the optimal combination of enhancement operators and their parameter configurations based on the degradation characteristics of different infrared images, ultimately achieving the goal of adaptive step-by-step enhancement. This PPO-based training mechanism ensures learning stability while balancing exploration and utilization, providing a reliable and efficient optimization scheme for infrared image enhancement tasks.

[0060] S106. Use the trained image signal processing model to process infrared images.

[0061] The raw infrared image to be processed is input as the initial state into the trained policy network. Based on the visual features of the current image, the policy network adaptively plans a set of optimal enhancement operator sequences and their corresponding intensity parameters through forward propagation. The system then calls functions from the enhancement operator library to progressively enhance the image. After a preset number of decision interaction steps T, a high-quality infrared image with high contrast, rich texture details, and no significant noise is obtained. See [link to specific steps] for details. Figure 3 , Figure 3 A flowchart for implementing infrared image enhancement provided in this application includes: S201. Take the original infrared image to be enhanced as the current state; input the current state into the pre-trained image signal processing model to generate a hybrid action.

[0062] The pre-trained image signal processing model is a policy network trained using an Actor-Critic architecture and a near-end policy optimization algorithm. This network takes infrared images as input, extracts image features through a shared deep convolutional backbone network, and employs a dual-branch decoupled structure to output the probability distribution of discrete operator selection and the Beta distribution of continuous parameters, thereby enabling sampling decisions in a hybrid action space. Without requiring high-quality paired reference images, this model can adaptively plan the optimal enhancement operator sequence and its fine-tuned parameters through a multi-step interactive process, based on a hybrid reward function that integrates information entropy, average gradient, enhancement metric, and no-reference quality evaluation index. This achieves a synergistic improvement in infrared image contrast, texture detail, and visual naturalness.

[0063] S202. Call the corresponding processing function in the enhancement operator library according to the operator index, and process the current image based on the normalized intensity parameter to obtain the enhanced intermediate image as the next state.

[0064] S203. Take the next state as the new current state, and repeat the steps of inputting the current state into the policy network and subsequent steps until a preset number of steps are reached. Output the final intermediate image as the enhanced target image.

[0065] Step S106 marks the formal transition of the deep reinforcement learning-based infrared image enhancement method from the training phase to the application and verification phase. Its core task is to deploy the well-trained policy network as a practical image signal processing model to process infrared images to be enhanced in real-world scenes, and to comprehensively verify the effectiveness and superiority of the method through systematic experimental design. In practical application, the user only needs to input the original infrared image to be processed as the initial state into the trained policy network. The network will then, based on its knowledge learned from massive interactive experiences, rapidly infer from the visual features of the current image via forward propagation, adaptively planning a set of optimal enhancement operator sequences and their corresponding intensity parameters. This inference process only involves the forward computation of the network, requiring no iterative optimization or online learning, thus possessing extremely high computational efficiency. The system then sequentially calls the corresponding function interfaces in the enhancement operator library according to the planning results, gradually refining the image. After a preset number of decision interaction steps T, a high-quality infrared image with high contrast, rich texture details, and no obvious noise artifacts can be output. This step-by-step processing mechanism ensures both the stability of the enhancement effect and the real-time requirements of actual deployment.

[0066] To evaluate the infrared image enhancement method based on deep reinforcement learning provided in this application, extensive experimental verification was conducted. All experiments were implemented using the PyTorch deep learning framework, and model training was completed on a single NVIDIA Tesla P100 GPU computing platform. For parameter optimization of the policy network and value network, this embodiment uses the AdamW optimizer, where the initial learning rate of the policy network (Actor) is set to... The initial learning rate of the Critic network is set to... Furthermore, a cosine annealing strategy is employed to dynamically decay the learning rate during training to prevent oscillations and promote model convergence. In addition, to comprehensively and objectively evaluate the enhancement quality of infrared images, this invention utilizes five widely used no-reference evaluation metrics, specifically: contrast per pixel (CPP), enhancement measure (EME), information entropy (IE), average gradient (AG), and spatial frequency (SF).

[0067] The selection of experimental data fully considers the diversity and challenges of infrared image enhancement tasks, primarily using the FLIR and MSRS public datasets. The FLIR dataset contains a large number of 640×512 resolution infrared thermal images, covering typical monitoring environments such as urban streets and traffic scenes; the MSRS dataset contains 640×480 resolution multispectral infrared images, involving various lighting conditions and weather conditions. These two datasets together cover a variety of typical application scenarios, including complex daytime streets, extremely low nighttime illumination, and rich texture details, providing a solid data foundation for verifying the generalization ability of the method. To fully demonstrate the advancement of this invention, the research team systematically compared the method with three traditional infrared image enhancement algorithms (AGCWD, DUAL, and SKWGIF-DDE) and four high-performing deep learning enhancement algorithms in computer vision (CLIP-LIT, Restormer, OneRestore, and Zero-DCE) under the same hardware environment and parameter settings.

[0068] The experiments were conducted on two publicly available datasets, FLIR and MSRS. Tables 1, 2, and 3 show the objective performance test results of the proposed method and the aforementioned comparative methods on different datasets. Bold text indicates the best performance, and underlined text indicates the second best. Experimental data show that compared to traditional algorithms that easily lead to background overexposure or noise amplification, and existing deep learning methods that easily produce smoothing artifacts, the proposed method achieves the best or second best results on all key metrics. Compared to the current best-performing comparative method, especially on the FLIR publicly available dataset compared to the current best deep learning augmentation method, information entropy, spatial frequency, and average gradient are improved by 11.2%, 19.1%, and 76.6%, respectively. These data fully demonstrate that the proposed method, through a step-by-step decision-making mechanism, significantly enhances image contrast and edge sharpness while greatly enriching the information contained in the image and successfully preserving more subtle texture details that would otherwise be easily smoothed out.

[0069] Table 1. Comparison of objective evaluation indicators between the embodiments of the present invention and traditional methods

[0070] Table 2. Comparison of enhancement effects of different methods on the FLIR dataset

[0071] Table 3. Comparison of Augmentation Effects of Different Methods on the MSRS Dataset

[0072] Table 4. Comparison of objective evaluation indicators for different variants of ablation experiments

[0073] also, Figure 4 This is a comparison diagram of the enhancement effects of the embodiments of the present invention and traditional methods; Figure 5 This is a comparison chart showing the enhancement effects of the embodiments of this application and deep learning methods; Figure 4 and Figure 5 The visualization comparison of the enhancement effects of different algorithms in typical scenarios further confirms the conclusion of the objective indicators: the image processed by the method of this invention is significantly better than the comparison method in terms of visual perception. It avoids the overexposure and noise amplification problems common in traditional algorithms, and also overcomes the defects of false texture and smoothing artifacts that are easy to be generated by deep learning methods.

[0074] In the ablation experiment, the research team designed a meticulously crafted comparison scheme to verify the effectiveness of each key component in the reinforcement learning framework proposed in this invention, as well as the necessity of the hybrid action space design. Three sets of variant models were constructed for comparative testing: the first set was the "fixed action strategy" variant, which completely removed the action decision network of reinforcement learning and uniformly processed all images using a preset fixed image enhancement operation sequence, aiming to verify the necessity of the adaptive decision mechanism; the second set was the "fixed parameter strategy" variant, which retained the discrete action selection function but removed the continuous parameter adjustment branch, and all selected operators were processed using preset default parameters, aiming to verify the importance of fine-tuning continuous parameters; the third set was the "key operator removal strategy" variant, which removed the Limit Contrast Adaptive Histogram Equalization (CLAHE) operator from the action space, relying only on other operators for enhancement, aiming to verify the key role of the CLAHE operator in the framework of this invention.

[0075] Figure 6 Table 4 shows a comparison of the effects of different variant strategies in the embodiments of this application, along with the objective indicator comparison results. Figure 6 The intuitive comparisons revealed key conclusions: the complete algorithm significantly outperformed all variant models across all metrics, with the "fixed parameter strategy" showing the most significant decline, indicating that the lack of continuous parameter adjustment capability prevented the model from adapting to the diverse needs of different images. Conversely, the "CLAHE removal strategy" performed worst when processing low-contrast images, demonstrating the irreplaceable role of the CLAHE operator in enhancing infrared image contrast. These ablation experiment results fully demonstrate that the hybrid action space design proposed in this invention is a necessary technical feature for achieving high-quality infrared image enhancement. The combination of discrete operator selection and continuous parameter adjustment ensures that the algorithm achieves an optimal balance in enhancing contrast, suppressing noise, and maintaining visual naturalness.

[0076] The following are Figure 4 , Figure 5 and Figure 6The English annotations in the text provide explanations. Figure 4 The horizontal axis labels are Infraredimage, AGCWD, DUAL, SKWGIF-DDE, and Ours, respectively. Infrared image refers to the original infrared thermal image, i.e., the input image without enhancement processing. It serves as a baseline in the comparison chart, visually demonstrating the enhancement effect of different methods relative to the original image. AGCWD, or Adaptive Gamma Correction with Weighted Distribution, is an adaptive gamma correction method that enhances image contrast through gamma correction and probability distribution of luminance pixels. Its drawback is that it can easily lead to local overexposure or noise amplification. DUAL, or Dual Contrast Enhancement, is a strategy that combines global and local enhancement, aiming to balance overall contrast and local details. When compared with traditional algorithms, its results are prone to uneven contrast enhancement or artifacts. SKWGIF-DDE, short for Dynamic Detail Enhancement Based on Statistical Kernel Weighted Guided Filtering, is a detail enhancement algorithm based on guided filtering. It decomposes an image into basal and detail layers, enhances the detail layers, and then fuses the layers to enhance texture while preserving edges. As a traditional contrast method, its performance has limitations in suppressing noise and preserving details. Ours refers to the infrared image enhancement method proposed in this application, namely… Figure 4 The enhanced result of the last column.

[0077] Figure 5The horizontal axis labels are Infrared image, CLIP-LIT, Restormer, OneRestore, Zero-DCE, and Ours, respectively. Infrared image refers to the original infrared image, i.e., the input image without any enhancement processing, serving as the baseline reference in the contrast image. CLIP-LIT is a low-light image enhancement method based on a contrastive language-image pre-trained model, utilizing the semantic capabilities of CLIP to improve the quality of low-light images, but it is prone to texture smoothing on infrared images. Restormer is an efficient Transformer restoration network, adept at denoising and deblurring, but its contrast enhancement effect on infrared images is limited. OneRestore is a unified restoration model capable of handling various image degradation problems end-to-end, but it is inferior to this invention in terms of infrared detail preservation. Zero-DCE is a zero-reference depth curve estimation method that enhances low-light images by iteratively adjusting the brightness curve in an unsupervised manner, but it is prone to noise amplification or color distortion. Ours represents the infrared image enhancement method proposed in this invention based on deep reinforcement learning and step-by-step decision-making, achieving optimal performance in contrast, detail preservation, and noise suppression. These labels together constitute the... Figure 5 A horizontal comparison of different methods to enhance the effect.

[0078] Figure 6 This paper presents a comparison of the enhancement effects of different variant strategies in ablation experiments. The figure includes three typical scene sub-images: (a), (b), and (c). The horizontal columns of each sub-image are labeled as follows: Infrared image, 1), 2), 3), and proposed. The Infrared image represents the original infrared image, serving as the baseline reference for all enhancement results. 1) represents the "fixed action strategy" variant, which removes the action decision network from reinforcement learning and uses a pre-defined fixed enhancement operator sequence to uniformly process all images, used to verify the necessity of adaptive decision-making. 2) represents the "fixed parameter strategy" variant, which retains the discrete operator selection function but removes the continuous parameter adjustment branch; all selected operators use default fixed intensity parameters, used to verify the importance of fine parameter adjustment. 3) represents the "CLAHE key operator removal strategy" variant, which removes the Limiting Contrast Adaptive Histogram Equalization (CLAHE) operator from the enhancement operator library and relies solely on other operators for enhancement, used to verify the key role of CLAHE in improving infrared contrast. "Proposed" refers to the complete algorithm proposed in this invention, which includes a step-by-step enhancement framework involving discrete operator selection and continuous intensity parameter adjustment. By comparing the processing results of the original infrared image with those of various variants and the complete method, it can be intuitively seen that the complete method significantly outperforms any variant in terms of contrast, detail preservation, and noise suppression, thus verifying the effectiveness of the hybrid action space design.

[0079] The following explains the English annotations in Tables 1-4: CPP (Contrast Per Pixel) measures the contrast strength of a local area of ​​an image. A higher value indicates higher image contrast. EME (Measure of Enhancement) evaluates the degree of contrast improvement after image enhancement by calculating the contrast of each block and averaging the values. A higher value indicates a more significant contrast improvement. Entropy reflects the amount of information contained in an image. A higher entropy value indicates richer image details and more texture information. AG (Average Gradient) evaluates image sharpness and edge sharpness. A higher value indicates clearer image edges and more prominent texture details. SF (Spatial Frequency) reflects the spatial frequency, comprehensively reflecting the spatial activity of image grayscale values. A higher value indicates richer image details and denser texture. It should be noted that the meanings of CPP, EME, Entropy, AG, and SF in Tables 1-4 are the same and will not be repeated hereafter.

[0080] AGCWD (Adaptive Gamma Correction with Weighted Distribution): This method enhances image contrast by adaptively adjusting the gamma curve and histogram weights. DUAL (Dual Contrast Enhancement): This method combines global and local contrast enhancement strategies to balance overall brightness with local detail. SKWGIF-DDE (Statistical Kernel Weighted Guided Image Filter based Dynamic Detail Enhancement): This method decomposes the image into a basal layer and a detail layer, dynamically enhances the detail layer, and then fuses them to preserve edges and improve texture. Ours is the method proposed in this invention.

[0081] Table 2 compares the enhancement effects of different methods on the FLIR dataset, a publicly available standard dataset in the field of infrared image enhancement. Table 3 compares the enhancement effects of different methods on the MSRS dataset, a publicly available multispectral infrared dataset containing paired visible and infrared images, and is a standard test dataset in the field of infrared enhancement. In Tables 2 and 3, "Dataset" refers to the name of the publicly available infrared image dataset used for the experimental comparison. "Method" refers to the name of the different image enhancement methods used in the comparison. "Dataset" indicates the source of the test data, and "Method" indicates the specific algorithm used in the test. Together, they form the row and column indexes of the table, facilitating a horizontal comparison of objective evaluation metrics of different methods on different datasets.

[0082] Tables 2 and 3 list CLIP-LIT (CLIP-based Low-light Image Enhancement): A low-light image enhancement method based on a contrastive language-image pre-trained model, utilizing CLIP's semantic capabilities to guide enhancement. Restormer (Restoration Transformer): An efficient Transformer restoration network, adept at handling image restoration tasks such as denoising and deblurring. OneRestore: A unified restoration model capable of end-to-end processing of various image degradation problems. Zero-DCE (Zero-Reference Deep Curve Estimation): Zero-reference depth curve estimation, requiring no paired data, achieving low-light enhancement through iterative adjustment of the brightness curve. Ours: The method of this invention.

[0083] In Table 4, 1) is a fixed action strategy variant. The action decision network of reinforcement learning is removed, and a preset fixed sequence of enhancement operators is used for all images. 2) is a fixed parameter strategy variant. The discrete operator selection function is retained, but the continuous intensity parameter adjustment is removed, and all operators use default parameters. 3) is a variant of the strategy to remove the CLAHE key operator. The Limiting Contrast Adaptive Histogram Equalization (CLAHE) operator is removed from the enhancement operator library. The proposed method is the complete infrared image enhancement method proposed in this invention, which is the same technical solution as Ours mentioned above, and is the complete benchmark solution for ablation experiments.

[0084] The above are some specific implementations of the infrared image enhancement method based on deep reinforcement learning provided in the embodiments of this application. Based on this, this application also provides a corresponding system for infrared image enhancement based on deep reinforcement learning. The system provided in the embodiments of this application will be described below from the perspective of functional modularity. Figure 7 This is a structural diagram of an infrared image enhancement system based on deep reinforcement learning, provided in an embodiment of this application.

[0085] The system includes: The generation unit 110 is used to take the original infrared image to be enhanced as the current state; input the current state into a pre-trained image signal processing model to generate hybrid actions. The pre-trained image signal processing model is trained by a deep reinforcement learning algorithm based on the Actor-Critic architecture. It takes the original infrared image as input and outputs the hybrid actions corresponding to each step in the step-by-step enhancement process through forward propagation. The hybrid actions include operator indices for selecting corresponding image processing operators from a preset enhancement operator library and normalized intensity parameters for adjusting the processing intensity of the selected image processing operators. The processing unit 111 is used to call the corresponding processing function in the enhancement operator library according to the operator index, and process the current image based on the normalized intensity parameter to obtain the enhanced intermediate image as the next state; The iteration unit 112 is used to take the next state as the new current state, repeatedly execute the steps of inputting the current state into the policy network and subsequent steps until a preset number of steps are reached, and output the final intermediate image as the enhanced target image.

[0086] This application also provides corresponding devices and computer storage media for implementing the infrared image enhancement scheme based on deep reinforcement learning provided in this application.

[0087] The device includes a memory and a processor. The memory stores instructions or code, and the processor executes the instructions or code to enable the device to perform the infrared image enhancement method based on deep reinforcement learning as described in any embodiment of this application.

[0088] The computer storage medium stores code, and when the code is executed, the device running the code implements the infrared image enhancement method based on deep reinforcement learning as described in any embodiment of this application.

[0089] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0090] It should be understood that in this application, "at least one" refers to one or more items, and "more" refers to two or more items. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one" or similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, "at least one" of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0091] It should be understood that the terms center, longitudinal, transverse, up, down, front, back, left, right, vertical, horizontal, top, bottom, inside, outside, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0092] It should be noted that, unless otherwise explicitly specified and limited, the terms installation, connection, and linking should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0093] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the statement "including a…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0094] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0095] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An infrared image enhancement method based on deep reinforcement learning, characterized in that, include: Use the original infrared image to be enhanced as the current state; The current state is input into a pre-trained image signal processing model to generate hybrid actions. The pre-trained image signal processing model is trained using a deep reinforcement learning algorithm based on the Actor-Critic architecture. It takes the original infrared image as input and outputs the hybrid actions corresponding to each step in the step-by-step enhancement process through forward propagation. The hybrid actions include operator indices for selecting corresponding image processing operators from a preset enhancement operator library and normalized intensity parameters for adjusting the processing intensity of the selected image processing operators. The corresponding processing function in the enhancement operator library is called according to the operator index, and the current image is processed based on the normalized intensity parameter to obtain the enhanced intermediate image as the next state; The next state is taken as the new current state, and the process of inputting the current state into the policy network and subsequent steps is repeated until a preset number of steps are reached. The final intermediate image is then output as the enhanced target image.

2. The method according to claim 1, characterized in that, The training process of the image signal processing model includes: The current state is set as a preliminary image and input into a preliminary model to generate a hybrid action. The target model includes a policy network and a value network. The hybrid action includes an operator index determined by sampling the discrete probability distribution output by the policy network and a normalized intensity parameter determined by sampling the continuous Beta distribution output by the policy network. The corresponding processing function in the enhancement operator library is called according to the operator index, and the current image is processed based on the normalized intensity parameter to obtain an enhanced intermediate image as the next state. The incremental comprehensive quality score of the image before and after processing is calculated as an immediate reward. The current state, hybrid action, immediate reward, and next state are stored as experience data. The next state is used as the new current state, and the process of inputting the current state into the policy network and subsequent steps is repeated until a preset number of steps is reached. Based on stored empirical data, a deep reinforcement learning algorithm is used to iteratively train the policy network and value network to obtain an image signal processing model with optimized network parameters.

3. The method according to claim 2, characterized in that, Before inputting the current state into the preparatory model to generate the hybrid action, the process also includes: An enhancement operator library is constructed, which integrates multiple image processing operators, each configured to receive continuous intensity parameters to process the image.

4. The method according to claim 3, characterized in that, The image processing operators integrated in the enhancement operator library include at least one or more of the following: contrast-limited adaptive histogram equalization, dynamic detail enhancement, logarithmic transformation, gamma correction, single-scale Retinex, nonlocal mean denoising, adaptive median filtering, unsharpening mask, and wavelet threshold denoising; each operator is encapsulated as a callable function interface, and the input normalized intensity parameter is mapped to the specific execution parameters inside the operator.

5. The method according to claim 2, characterized in that, Before inputting the current state into the preparatory model to generate the hybrid action, the process also includes: A backbone feature extraction network is constructed, which employs a pre-trained deep convolutional neural network to extract deep semantic features from the input infrared image. A policy network and a value network are built on top of the backbone feature extraction network. The output layer of the policy network adopts a two-branch decoupled structure including discrete and continuous branches. The discrete branch outputs the selection probability distribution of each enhancement operator through the Softmax activation function, and the continuous branch characterizes the intensity adjustment range of the selected operator by outputting the probability density function parameter of the Beta distribution. The output layer of the value network is used to estimate the expected cumulative reward of the current image state and outputs a scalar value.

6. The method according to claim 2, characterized in that, The method of storing the current state, mixed actions, immediate rewards, and the next state as experience data includes: Store the current state, mixed actions, immediate rewards, and next state as experience data in the experience replay buffer. The stored empirical data is used to iteratively train the policy network and value network using a deep reinforcement learning algorithm to obtain an image signal processing model with optimized network parameters, including: A preset number of empirical data points are randomly sampled from the empirical replay buffer. A deep reinforcement learning algorithm is then used to iteratively train the policy network and the value network based on the preset number of empirical data points to optimize the network parameters and obtain the trained image signal processing model.

7. The method according to claim 2, characterized in that, The deep reinforcement learning algorithm includes a proximal policy optimization algorithm, which combines generalized advantage estimation to calculate the advantage function during training to balance bias and variance; the policy network update adopts a truncation mechanism, which limits the update magnitude of the probability ratio between the new and old policies and introduces policy entropy as a regularization term in the loss function; the value network update is achieved by minimizing the error between the state value estimation and the multi-step reward objective.

8. The method according to claim 2, characterized in that, The incremental increase in the overall quality score of the image before and after processing is used as an immediate reward, including: The first comprehensive quality score of the image in the current state is calculated based on a linear weighted combination of fused information entropy, average gradient, enhancement measure and no-reference natural image quality evaluation index. After performing the blending action and processing the current image, the same linear weighted combination is used to calculate the second comprehensive quality score of the image in the next state; Calculate the difference between the second comprehensive quality score and the first comprehensive quality score, and use the difference as the immediate reward for the current step.

9. An infrared image enhancement system based on deep reinforcement learning, characterized in that, include: The generation unit is used to take the original infrared image to be enhanced as the current state; The current state is input into a pre-trained image signal processing model to generate hybrid actions. The pre-trained image signal processing model is trained using a deep reinforcement learning algorithm based on the Actor-Critic architecture. It takes the original infrared image as input and outputs the hybrid actions corresponding to each step in the step-by-step enhancement process through forward propagation. The hybrid actions include operator indices for selecting corresponding image processing operators from a preset enhancement operator library and normalized intensity parameters for adjusting the processing intensity of the selected image processing operators. The processing unit is used to call the corresponding processing function in the enhancement operator library according to the operator index, and process the current image based on the normalized intensity parameter to obtain the enhanced intermediate image as the next state; An iterative unit is used to take the next state as the new current state, repeatedly execute the steps of inputting the current state into the policy network and subsequent steps until a preset number of steps are reached, and output the final intermediate image as the enhanced target image.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the infrared image enhancement method based on deep reinforcement learning as described in any one of claims 1-8.

Images