A camera exposure control method based on deep reinforcement learning and nelder-mead online expert

Abstract

The application discloses a camera exposure control method based on deep reinforcement learning and Nelder-Mead online experts, and relates to the technical field of computer vision and imaging control, which firstly collects continuous historical images to construct input states, outputs initial exposure control actions through a deep reinforcement learning strategy network, and realizes global adaptive adjustment; subsequently, a Nelder-Mead simplex algorithm is introduced for online optimization, the strategy network output is taken as an initial vertex, images under different exposure parameters are simulated and generated based on a camera response function, the optimal exposure value is searched through iteration, and local fine optimization is completed; finally, exposure time and sensor gain are adaptively distributed according to the static and dynamic characteristics of a scene, and the exposure time is limited in a dynamic scene to suppress motion blur. The application combines global decision-making of reinforcement learning and local optimization of simplex, effectively improves exposure control precision and stability, and reduces overexposure and underexposure.

Description

Technical Field

[0001] This invention relates to the field of computer vision and imaging control technology, and more specifically to a camera exposure control method based on deep reinforcement learning and Nelder-Mead online expert. Background Technology

[0002] With the rapid development of machine vision technology, vision-based intelligent systems are widely used in fields such as autonomous driving, industrial inspection, and robot navigation. In these applications, the quality of image acquisition directly affects the performance of subsequent vision tasks, and camera exposure control, as a key step in the image acquisition process, is of great significance for obtaining clear, stable, and information-rich images.

[0003] To obtain high-quality images, existing systems typically employ automatic exposure (AE) algorithms to adjust camera parameters. Traditional methods often rely on overall image brightness statistics (such as average brightness or histogram distribution) for control. Although these methods are simple to implement and have low computational overhead, they tend to overlook exposure differences in local areas under complex lighting conditions, leading to overexposure or underexposure in the image and affecting the accuracy of visual tasks.

[0004] With the development of deep learning technology, researchers have gradually introduced reinforcement learning methods into exposure control tasks. By constructing state, action, and reward functions, the model can adaptively adjust exposure parameters based on environmental feedback. These methods exhibit stronger environmental adaptability in complex scenarios compared to traditional methods. However, they still face a series of challenges in practical applications.

[0005] First, reinforcement learning methods primarily rely on policy networks to output exposure control actions. Their decision-making process typically exhibits some discreteness or instability, lacking fine-tuning capabilities when approaching the optimal solution, which can easily lead to insufficient exposure control precision. Second, in dynamic scenes, due to changes in lighting or drastic target movement, policy networks struggle to respond promptly to environmental changes, easily causing brightness fluctuations or exposure oscillations, thus affecting the stability of image sequences. Furthermore, most existing methods lack further optimization mechanisms for policy outputs, failing to fully utilize current image information for local correction of exposure parameters, leaving room for improvement in overall control performance.

[0006] To address these issues, some studies have attempted to introduce optimization algorithms to adjust exposure parameters, such as gradient-based or heuristic search methods for iterative optimization of exposure values. However, these methods typically rely on specific assumptions or require additional computational overhead, making it difficult to balance real-time performance with optimization accuracy.

[0007] Therefore, how to introduce an efficient and stable local optimization mechanism on the basis of global decision-making in reinforcement learning to improve exposure control accuracy and suppress brightness fluctuations has become a key problem that urgently needs to be solved in the field of camera exposure control. Summary of the Invention

[0008] In view of this, the present invention provides a camera exposure control method based on deep reinforcement learning and Nelder-Mead online expert. Under the reinforcement learning framework, image temporal information is used as the state input, and the initial exposure control action is output through the policy network to achieve global adaptive adjustment of exposure parameters. On this basis, the Nelder-Mead simplex optimization method is introduced to construct an online optimization module to perform local search and fine optimization on the policy output results, thereby obtaining better exposure control parameters.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A camera exposure control method based on deep reinforcement learning and Nelder-Mead online expert includes: S1: Acquire continuous historical images from the camera and perform size normalization processing to construct an input state with a uniform scale; S2, The input state is input into a pre-trained deep reinforcement learning policy network. The policy network outputs an exposure control action based on the input state and obtains the exposure value for the next time step by combining the current effective exposure value. S3, Select a preset reward function according to different task requirements, and use the negative of the preset reward function as the objective function for online optimization of the Nelder-Mead simplex algorithm; S4, calculate the average brightness value of the initial vertex using the exposure value of the next time step, determine the search step size based on the average brightness value, and construct an initial simplex; iterate the initial simplex based on the objective function, and in each subsequent round of simplex iteration evaluation, simulate the image corresponding to the exposure value of each vertex based on the camera response function, thereby replacing the repeated imaging of the real camera to quickly calculate the objective function value of each vertex, and then update the simplex based on the objective function value; iterate until the change in exposure value is less than a preset threshold or the maximum number of iterations is reached, and output the target effective exposure value; S5. After obtaining the target effective exposure value, select the allocation method according to the scene, obtain the target exposure time, limit the target exposure time using the maximum allowable exposure time, obtain the final exposure time, calculate the target sensor gain based on the final effective exposure value and the final exposure time, and compensate the exposure amount through the target sensor gain.

[0010] Preferably, S1 specifically includes: S1-1, for the current time step Given an input image, the system retrieves the previous four historical images to construct the state input. If the current image is the first frame in the sequence, it copies the frame four times to ensure the state tensor dimensions remain consistent. Otherwise, it sequentially reads the previous four images from the cache or storage. , , , As historical information; S1-2: Normalize the four frames of images acquired in S1-1 and scale them to a fixed size of 84. 84. Subsequently, the four normalized images are combined along the channel dimension to form a three-dimensional state tensor (4,84,84) as the input state.

[0011] Preferably, the deep reinforcement learning policy network in S2 specifically includes: S2-1, for the current time step Input status Input the state tensor into the deep reinforcement learning policy network Utilizing continuous motion space to output exposure control action ; S2-2, Based on the exposure control action Combined with the current effective exposure value Calculate the exposure value at the next time step. : .

[0012] Preferably, the preset reward function in S3 includes a regional brightness statistical reward. With AKAZE feature reward The specific definition is as follows: Regional brightness statistics reward Divide the image evenly into N=9 regions, calculate the brightness statistics of each region, and set the target normalized brightness. The area brightness statistical reward is defined as: ; ; ; in, Indicates the first Image regions, and These represent the width and height of the region, respectively. Used to measure the brightness of each area Deviation from target brightness; Used to measure the brightness of the current frame. brightness with adjacent frames The brightness variation between these parameters is used to suppress brightness flicker during the exposure adjustment process. To normalize the target brightness, and For the corresponding weights.

[0013] AKAZE Feature Rewards The reward takes into account both the number of feature points detected in the current frame and the number of feature points successfully matched between adjacent frames, and is defined as follows: ; in, This indicates the number of feature points detected in the current frame. This indicates the number of feature points that were successfully matched between adjacent frames. and For the corresponding weights; Based on the different image quality requirements of downstream tasks, rewards are derived from regional brightness statistics. and AKAZE feature rewards Select the corresponding reward function as the reward function for the current task. When downstream tasks focus on image exposure quality, local brightness uniformity, and exposure stability between adjacent frames, regional brightness statistics are selected as the reward. As a reward function, AKAZE features are selected as rewards when downstream tasks focus on the performance of feature point detection, feature matching, visual odometry, image registration, or other visual perception tasks. As a reward function.

[0014] Preferably, S4 specifically includes: S4-1, the exposure value of the next time step The input is fed into the optimizer constructed using the Nelder-Mead simplex algorithm as the initial vertex. ; S4-2, based on the exposure value of the next time step Corresponding image Calculate the average brightness ; S4-3, based on the average brightness value Determine the simplex initialization step size : ; S4-4, Determine the simplex initialization step size Post-construction extended vertices : ; From the initial vertex With extended vertices Together they form the initial simplex of the Nelder-Mead simplex algorithm; S4-5, iterate the initial simplex based on the objective function, and during the local optimization process, base the optimization on the camera response function. Simulated image generation is performed for different exposure parameters to replace the repetitive imaging process of a real camera under different exposure parameters, thereby reducing image acquisition overhead and improving optimization efficiency. ; in, It is the inverse function of the camera response function. , These represent the pixel positions of the image at the current and previous time steps, respectively. The brightness value at that location; and These represent the exposure values ​​at the current moment and the previous moment, respectively; S4-6 uses the negative of the reward function as the objective function, evaluates at each vertex of the simplex, and performs the following steps: (1) Calculate the centroids of all vertices except the worst point. ; (2) Based on the objective function value, perform one of the following operations in sequence: reflection, expansion, contraction, shrinkage; S4-7, repeat the simplex update process until the relative change in exposure value in adjacent iterations is less than a preset threshold or the preset maximum number of iterations is reached, then stop the iteration and obtain the target effective exposure value. .

[0015] Preferably, S5 specifically includes: S5-1, before allocating exposure time and gain, first determine whether the current scene is static or dynamic based on the amount of motion between adjacent frames. Then, perform feature point detection and matching on the current frame and the previous frame, and calculate the average pixel displacement of the matched feature points between adjacent frames as an indicator of scene motion. : ; in, and They represent the first The pixel coordinates of each matching feature point in the current frame and the previous frame. Indicates the number of successfully matched feature points; S5-2, when the scene motion index Greater than or equal to the preset motion threshold The current scene is determined to be a dynamic scene; when the scene's motion index... Less than the preset motion threshold The current scene is determined to be a static scene. S5-3: When facing a static scene, prioritize the allocation of exposure time, and then calculate the exposure gain after the exposure time reaches the upper limit. S5-4, When facing dynamic scenes, the final effective exposure value is based on S4. Calculate the corresponding target exposure time ;in, Indicates the preset base gain; S5-5, using the maximum permissible exposure time Target exposure time Limit the exposure time to obtain the final exposure time. : ; S5-6, Based on the target effective exposure value With the final exposure time The relationship between them is used to inversely calculate the corresponding target sensor gain. : .

[0016] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a camera exposure control method based on deep reinforcement learning and Nelder-Mead online expert, with the following beneficial effects: 1. By introducing a simplex optimization mechanism on top of the reinforcement learning policy network output, local fine-tuning of exposure parameters is achieved. Reinforcement learning enables global adaptive modeling of complex lighting environments, while the online expert module dynamically corrects the policy output using current image information, thereby effectively improving the accuracy and stability of exposure control and significantly reducing overexposure and underexposure.

[0017] 2. By constructing an online expert module based on simplex optimization, this invention achieves adaptive search and rapid convergence of exposure parameters. The method iteratively optimizes the exposure parameters through operations such as reflection, expansion, contraction, and shrinkage, achieving efficient local search without relying on gradient information. This compensates for the insufficient fine-tuning ability of reinforcement learning near the optimal solution, thereby significantly improving the smoothness and robustness of exposure adjustment.

[0018] 3. This invention requires no additional labeled data and can be directly embedded into existing vision acquisition systems. It has good versatility and engineering application value, and has broad application prospects in scenarios such as autonomous driving, robot vision and industrial inspection. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 A schematic diagram of the framework of the camera exposure control method provided by the present invention; Figure 2 A comparison chart of convergence step size between data provided by this invention and the camera's automatic exposure algorithm in indoor and outdoor scenes; Figure 3 This is a comparison chart of the experimental performance of left and right stereo cameras for pose estimation under different initial exposure conditions in the camera exposure dataset provided by the present invention. Figure 4 A comparison of feature tracking performance under different exposure control strategies in simulated image sequences provided by this invention; Figure 5 This is a comparison chart of the number of SIFT feature points obtained from images captured in indoor and outdoor scenes provided by the present invention, and of the automatic exposure algorithm and the pure deep reinforcement learning method. Figure 6 This is a performance comparison chart of the captured images and the automatic exposure algorithm provided by this invention in an image segmentation task; Figure 7 A flowchart of the method steps provided by the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] This invention discloses a camera exposure control method based on deep reinforcement learning and the Nelder-Mead online expert, such as... Figure 7 As shown, it includes: S1: Acquire continuous historical images from the camera and perform size normalization processing to construct an input state with a uniform scale; S2, the input state is fed into the pre-trained deep reinforcement learning policy network. The policy network outputs exposure control actions based on the input state and obtains the exposure value for the next time step by combining the current effective exposure value. S3. Select a preset reward function according to different task requirements, and use the negative of the preset reward function as the objective function for online optimization of the Nelder-Mead simplex algorithm. S4: Using the exposure value of the next time step as the initial vertex, calculate the average brightness value, determine the search step size based on the average brightness value, and construct the initial simplex. Iterate the initial simplex based on the objective function. In each subsequent round of simplex iteration evaluation, simulate the image corresponding to the exposure value of each vertex based on the camera response function. This replaces the repeated imaging of the real camera to quickly calculate the objective function value of each vertex. Then, update the simplex based on the objective function value. Iterate until the change in exposure value is less than the preset threshold or the maximum number of iterations is reached, and output the target effective exposure value. S5: After obtaining the target effective exposure value, select the allocation method according to the scene, obtain the target exposure time, limit the target exposure time using the maximum allowed exposure time, obtain the final exposure time, calculate the target sensor gain based on the final effective exposure value and the final exposure time, and compensate the exposure amount through the target sensor gain.

[0023] In specific embodiment 1: Step S1: Obtaining Historical Status like Figure 1 As shown, in order to obtain The time-state is used as input for deep reinforcement learning to capture continuous states. Constructing a uniform-scale input state from frame history images ; S1-1, For the current time step The input image is used to construct the state input by obtaining the previous four historical images. If the current image is the first frame in the sequence, it is copied four times as historical frames to ensure the consistency of the state tensor dimensions. Otherwise, the previous four images are read sequentially from the cache or storage. , , , As historical information; S1-2. Normalize the four frames of images obtained in step S1-1, and scale the images to a fixed size of 84. 84. Subsequently, the four normalized images are combined along the channel dimension to form a three-dimensional state tensor (4,84,84), which is used as input to the deep reinforcement learning policy network.

[0024] Step S2: Output of deep reinforcement learning strategy

[0025] In order to achieve initial adaptive adjustment of exposure parameters, Input status at any moment The input is fed into a pre-trained deep reinforcement learning policy network, Actor. The Actor network predicts exposure adjustment actions for the current scene based on historical image states. And use this action to adjust the current effective exposure value. Make initial corrections to obtain the exposure value for the next time step. This serves as the initial search point for the subsequent Nelder-Mead simplex online optimization process.

[0026] S2-1, targeting Input status at all times Input the state tensor into the deep reinforcement learning policy network The network, trained offline, can predict the optimal exposure adjustment direction for scene perception based on historical observations. Internally, the Actor network utilizes a continuous action space to output exposure control actions. This allows for adaptive adjustment of the exposure value.

[0027] The exposure value is defined by formula (1): (1) in, and These represent exposure time and sensor gain, respectively.

[0028] S2-2, Output actions through the Actor network. And combined with the current effective exposure value Actions are output by the Actor network. The exposure value for the next time step is obtained using formula (2): (2) Under the above definition, EV indicates double the effective exposure, while EV indicates that the effective exposure is halved. Therefore, the range of motion... Allows the target exposure to vary between one-quarter and four times the current exposure.

[0029] Step S3: Selection of reward function

[0030] Based on different task requirements, this invention designs two different reward functions. It is used to evaluate image quality and task adaptability under different exposure parameters, and to provide a target basis for training deep reinforcement learning policy networks and online optimization of the Nelder-Mead simplex algorithm; Reward function design: Two reward functions were designed based on image statistical characteristics and feature points, according to different downstream tasks: and Their specific definitions are as follows: (1) Regional brightness statistics reward To avoid neglecting local exposure issues in global brightness evaluation, this invention uniformly divides the image into N=9 regions, calculates the brightness statistics for each region, and sets the normalized target brightness as... The area brightness statistical reward is defined as: (3) (4) (5) in, Indicates the first Image regions, and These represent the width and height of the area, respectively, and are used to measure the brightness of each area. Deviation from target brightness; Used to measure brightness changes between adjacent frames to suppress brightness flicker during exposure adjustment. Weights are set to... , .

[0031] (2) AKAZE Feature Reward To improve the adaptability of the exposure strategy to visual perception tasks, this invention further introduces the AKAZE feature reward. This reward considers both the number of feature points detected in the current frame and the number of feature points successfully matched between adjacent frames. It is defined as follows: (6) in, This indicates the number of feature points detected in the current frame. This indicates the number of feature points that were successfully matched between adjacent frames. and These are the corresponding weights, and all are set to 0.05.

[0032] Reward function selection; based on the different image quality requirements of downstream tasks, a reward is selected from the defined region brightness statistics. and AKAZE feature rewards Select the corresponding reward function as the reward function for the current task. When downstream tasks primarily focus on image exposure quality, local brightness uniformity, and exposure stability between adjacent frames, regional brightness statistics are selected as the reward. When downstream tasks primarily focus on the performance of feature point detection, feature matching, visual odometry, image registration, or other visual perception tasks, AKAZE features are selected as the reward function. As a reward function.

[0033] The purpose of the reward function is determined; the selected reward function will be... The target evaluation criteria used for exposure control guide the training of deep reinforcement learning strategy networks, enabling the Actor network to learn from historical states. To the exposure action Mapping; In the online optimization of the Nelder-Mead simplex algorithm, As the objective function, the candidate exposure values ​​for the next time step Local optimization is performed to obtain the final exposure value.

[0034] Step S4: Simplex Algorithm Optimization

[0035] In order to further optimize the exposure strategy for the characteristics of the current scene, candidate exposure values ​​for the next moment are needed. Perform local optimization.

[0036] S4-1, Set the exposure value for the next time step. The input is fed into the optimizer constructed using the Nelder-Mead simplex algorithm as the initial vertex. ; S4-2, Exposure value at the next time step based on the current scene Corresponding image Calculate its average brightness. ; S4-3, Based on the average brightness Determine the simplex initialization step size using formula (7). Used to control the simplex search range: (7) S4-4. Determine the step size Then, construct another simplex vertex according to formula (8). : (8) From the initial vertex With extended vertices Together they form the initial simplex of the Nelder-Mead simplex algorithm, which is used for subsequent local optimization search of exposure values.

[0037] S4-5, Image Simulation Accelerated Optimization: Iterate the initial simplex based on the objective function, and during the local optimization process, base the optimization on the camera response function. Images corresponding to different exposure parameters are simulated and generated according to formula (9) to replace the repeated imaging process of a real camera under different exposure parameters, thereby reducing image acquisition overhead and improving optimization efficiency: (9) in, It is the inverse function of the camera response function. , These represent the pixel positions of the image at the current and previous time steps, respectively. The brightness value at that location; and These represent the exposure values ​​at the current moment and the previous moment, respectively.

[0038] S4-6, Simplex Iterative Optimization; Using the negative of the reward function as the objective function, evaluate at each vertex of the simplex and perform the following steps: (1) Calculate the centroids of the vertices other than the worst point. (2) Based on the objective function value, perform one of the following operations in sequence: reflection, expansion, contraction, shrink.

[0039] S4-7. Repeat the simplex update process until the relative change in exposure value in adjacent iterations is less than a preset threshold. The iteration stops when the preset maximum number of iterations (50) is reached, and the target effective exposure value is obtained. .

[0040] Step S5: Exposure parameter allocation yields the final exposure parameters.

[0041] In order to optimize the target effective exposure value Converting the exposure control parameters into camera-executable parameters, this invention further allocates exposure parameters to the final effective exposure value to obtain the final target exposure time. and final target sensor gain .

[0042] S5-1 Scene Motion Determination: Before allocating exposure time and gain, the current scene is first determined to be static or dynamic based on the amount of motion between adjacent frames. Specifically, feature point detection and matching are performed between the current frame and the previous frame, and the average pixel displacement of the matched feature points between adjacent frames is calculated as an indicator of scene motion. : (10) in, and They represent the first The pixel coordinates of each matching feature point in the current frame and the previous frame. This indicates the number of feature points that were successfully matched.

[0043] S5-2, When the scene's motion index Greater than or equal to the preset motion threshold The current scene is determined to be a dynamic scene; when the scene's motion index... Less than the preset motion threshold The current scene is determined to be a static scene. In one optional implementation, a preset motion threshold is used. Set to 5 pixels.

[0044] S5-3, Static Scene; When facing a static scene, prioritize the allocation of exposure time, and after the exposure time reaches the upper limit, calculate the exposure gain in reverse according to formula (1); S5-4, Dynamic Scenes; When facing dynamic scenes, the effective exposure value of the target obtained in step S4 is used as a basis. And the corresponding target exposure time is obtained according to formula (11). : (11) in, This indicates the preset base gain. In one optional implementation, It can be set to 0, at which point the target exposure time... Equal to the target's effective exposure value Furthermore, this invention constrains the exposure time by estimating image motion information between adjacent frames and calculates the maximum permissible exposure time, which is defined as: (12) in, For adjustment coefficients, This is the median vector of optical flow between two frames. Indicates the inter-frame time interval. This represents a very small positive number used to ensure computational stability.

[0045] S5-5, Utilize the maximum permissible exposure time Target exposure time Limit the exposure time to obtain the final exposure time. : (13) S5-6. Subsequently, based on the target effective exposure value... With the final exposure time The relationship between the two is used to inversely calculate the corresponding target sensor gain using formula (1). : (14) Therefore, in dynamic scenes, motion blur is suppressed by limiting the exposure time and the exposure is compensated by adjusting the sensor gain, thereby achieving a balance between image brightness and motion sharpness.

[0046] The following is a simulation experiment of the camera exposure control method based on deep reinforcement learning and Nelder-Mead online expert of the present invention.

[0047] Experimental setup

[0048] The method of this invention can be implemented on various computing platforms, including servers or embedded vision systems. In a specific embodiment, the method is based on a deep learning framework and the model training and inference process is completed on a computing device equipped with a graphics processing unit (GPU).

[0049] Furthermore, the method was validated on exposure control datasets, simulated image sequences, and real indoor and outdoor environments. The results show that the method of the present invention can achieve stable and effective exposure control in different scenarios.

[0050] Figure 2 The results show the exposure convergence experiments conducted on two MV-CE013-50GM industrial cameras in indoor and outdoor scenes, respectively. The experimental results demonstrate that, under different environmental conditions, the method proposed in this invention can achieve convergence within 2-3 frames, resulting in a stable exposure state and good image quality. In contrast, the camera's built-in automatic exposure algorithm requires significantly more convergence steps, approximately 15 frames in indoor scenes and approximately 10 frames in outdoor scenes to achieve a stable exposure state.

[0051] Figure 3 The results show the stability experiments of stereo camera pose estimation and feature tracking conducted on an exposure control dataset. To evaluate the robustness of the proposed method under different initial exposure conditions, image pairs with different exposure parameters were selected from 25 scenes to construct three types of input conditions: underexposure, normal exposure, and overexposure. ORB features were used for detection and matching, and RANSAC was combined to remove mismatched points. Pose estimation was completed based on the inlier set. Table 1 shows the average and maximum values ​​of each method in terms of rotation error Er (°) and translation error Et (cm). In the table, bold indicates the optimal result, and underline indicates the suboptimal result. Experimental results show that existing methods generally suffer from performance degradation under extreme exposure conditions, while the method of this invention exhibits stability under all three exposure conditions. Under normal exposure, the rotation error is 1.38° / 4.73° and the translation error is 9.45 / 26.22 cm. Under underexposure and overexposure conditions, the rotation errors are 1.75° / 4.73° and 1.35° / 4.61°, respectively, with minimal performance fluctuations. This indicates that the method of this invention can effectively suppress the influence of exposure changes on feature matching and geometric estimation, achieve stable pose estimation under different initial exposure conditions, and has good robustness.

[0052] Table 1 Comparison of methods

[0053] Figure 4 The results show the feature tracking performance evaluation of the method of this invention on a simulated image sequence (0-1000 frames). Experiments show that D2S-akaze and D2S-ras consistently track significantly more feature points than the baseline method throughout the sequence, with more stable fluctuations. Their average number of tracked features is approximately 180-200, while the baseline method only has 120-150, and in some frames (around frames 400 and 900), it drops below 50, exhibiting significant tracking failure. Visualization results further demonstrate that the baseline method is prone to overexposure due to unstable exposure control, leading to feature point loss and tracking drift. In contrast, the method of this invention can maintain a stable exposure level, ensuring feature region consistency, thereby achieving stable feature tracking over long time periods and demonstrating excellent robustness.

[0054] Figure 5 The results show the performance evaluation of the method of this invention in real indoor and outdoor scenes. The algorithm is comprehensively analyzed from two aspects: image information content and module ablation. Experimental results show that compared with the traditional automatic exposure (AE) algorithm, the method of this invention has significant advantages in SIFT feature extraction tasks. The average number of feature points increases from 2082.79 to 3017.07, an improvement of about 44.8%, indicating that it can acquire images with higher dynamic range and richer detail information under complex lighting conditions, thereby enhancing the ability to express image structural information. Furthermore, the average number of feature points is 3007.18 without the simplex optimizer, and increases to 3060.40 after its introduction, an increase of about 1.77%. This shows that on the basis of the better exposure strategy obtained by deep reinforcement learning, the simplex optimizer can further refine the exposure search space, improve the exposure control accuracy, and make the result closer to the optimal exposure state.

[0055] Figure 6 The diagram illustrates the performance improvement of the method described in this invention in downstream vision tasks. Experimental results show that, compared to images acquired using traditional automatic exposure (AE) algorithms, images obtained using the method described in this invention perform better in segmentation tasks, specifically by approximately 10% more target detections, approximately 4.33% more average detection confidence, and approximately 10.46% more average mask coverage. These results demonstrate that the method described in this invention can generate images with richer detail and clearer target contours, thereby more accurately preserving the shape, structure, and edge information of targets. This provides a more stable and information-rich input for vision tasks such as target detection and instance segmentation, ultimately improving the overall performance of downstream tasks.

[0056] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The methods disclosed in the embodiments are described simply because they correspond to the methods disclosed in the embodiments; relevant parts can be found in the method section.

[0057] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A camera exposure control method based on deep reinforcement learning and Nelder-Mead online expert, characterized in that, include: S1: Acquire continuous historical images from the camera and perform size normalization processing to construct an input state with a uniform scale; S2, The input state is input into a pre-trained deep reinforcement learning policy network. The policy network outputs an exposure control action based on the input state and obtains the exposure value for the next time step by combining the current effective exposure value. S3, Select a preset reward function according to different task requirements, and use the negative of the preset reward function as the objective function for online optimization of the Nelder-Mead simplex algorithm; S4, calculate the average brightness value of the initial vertex using the exposure value of the next time step, determine the search step size based on the average brightness value, and construct the initial simplex; The initial simplex is iterated based on the objective function. In each subsequent round of simplex iteration evaluation, the image corresponding to the exposure value of each vertex is simulated based on the camera response function. This replaces the repeated imaging of the real camera to quickly calculate the objective function value of each vertex. The simplex is then updated based on the objective function value. The iteration continues until the change in exposure value is less than a preset threshold or the maximum number of iterations is reached, and the target effective exposure value is output. S5. After obtaining the target effective exposure value, select the allocation method according to the scene, obtain the target exposure time, limit the target exposure time using the maximum allowable exposure time, obtain the final exposure time, calculate the target sensor gain based on the final effective exposure value and the final exposure time, and compensate the exposure amount through the target sensor gain.

2. The camera exposure control method based on deep reinforcement learning and Nelder-Mead online expert as described in claim 1, characterized in that, S1 specifically includes: S1-1, for the current time step Given an input image, the system retrieves the previous four historical images to construct the state input. If the current image is the first frame in the sequence, it copies the frame four times to ensure the state tensor dimensions remain consistent. Otherwise, it sequentially reads the previous four images from the cache or storage. , , , As historical information; S1-2: Normalize the four frames of images acquired in S1-1 and scale them to a fixed size of 84.

84. Subsequently, the four normalized images are combined along the channel dimension to form a three-dimensional state tensor (4,84,84) as the input state.

3. The camera exposure control method based on deep reinforcement learning and Nelder-Mead online expert as described in claim 2, characterized in that, The deep reinforcement learning policy network in S2 specifically includes: S2-1, for the current time step Input status Input the state tensor into the deep reinforcement learning policy network Utilizing continuous motion space to output exposure control action ; S2-2, Based on the exposure control action Combined with the current effective exposure value Calculate the exposure value at the next time step. : .

4. The camera exposure control method based on deep reinforcement learning and Nelder-Mead online expert as described in claim 1, characterized in that, The preset reward function in S3 includes a regional brightness statistical reward. With AKAZE feature reward The specific definition is as follows: Regional brightness statistics reward Divide the image evenly into N=9 regions, calculate the brightness statistics of each region, and set the target normalized brightness. The area brightness statistical reward is defined as: ； ； ； in, Indicates the first Image regions, and These represent the width and height of the region, respectively. Used to measure the brightness of each area Deviation from target brightness; Used to measure the brightness of the current frame. brightness with adjacent frames The brightness variation between these parameters is used to suppress brightness flicker during the exposure adjustment process. and For the corresponding weights, To normalize the target brightness; AKAZE Feature Rewards The reward takes into account both the number of feature points detected in the current frame and the number of feature points successfully matched between adjacent frames, and is defined as follows: ； in, This indicates the number of feature points detected in the current frame. This indicates the number of feature points that were successfully matched between adjacent frames. and For the corresponding weights; Based on the different image quality requirements of downstream tasks, rewards are derived from regional brightness statistics. and AKAZE feature rewards Select the corresponding reward function as the reward function for the current task. When downstream tasks focus on image exposure quality, local brightness uniformity, and exposure stability between adjacent frames, regional brightness statistics are selected as the reward. As a reward function, AKAZE features are selected as rewards when downstream tasks focus on the performance of feature point detection, feature matching, visual odometry, image registration, or other visual perception tasks. As a reward function.

5. The camera exposure control method based on deep reinforcement learning and Nelder-Mead online expert as described in claim 1, characterized in that, S4 specifically includes: S4-1, the exposure value of the next time step The input is fed into the optimizer constructed using the Nelder-Mead simplex algorithm as the initial vertex. ; S4-2, based on the exposure value of the next time step Corresponding image Calculate the average brightness ; S4-3, based on the average brightness value Determine the simplex initialization step size : ； S4-4, Determine the simplex initialization step size Post-construction extended vertices : ； From the initial vertex With extended vertices Together they form the initial simplex of the Nelder-Mead simplex algorithm; S4-5, iterate the initial simplex based on the objective function, and during the local optimization process, base the optimization on the camera response function. Simulated image generation is performed for different exposure parameters to replace the repetitive imaging process of a real camera under different exposure parameters, thereby reducing image acquisition overhead and improving optimization efficiency. ； in, It is the inverse function of the camera response function. , These represent the pixel positions of the image at the current and previous time steps, respectively. The brightness value at that location; and These represent the exposure values ​​at the current moment and the previous moment, respectively; S4-6 uses the negative of the reward function as the objective function, evaluates at each vertex of the simplex, and performs the following steps: (1) Calculate the centroids of all vertices except the worst point. ; (2) Based on the objective function value, perform one of the following operations in sequence: reflection, expansion, contraction, shrinkage; S4-7, repeat the simplex update process until the relative change in exposure value in adjacent iterations is less than a preset threshold or the preset maximum number of iterations is reached, then stop the iteration and obtain the target effective exposure value. .

6. The camera exposure control method based on deep reinforcement learning and Nelder-Mead online expert as described in claim 1, characterized in that, S5 specifically includes: S5-1, before allocating exposure time and gain, first determine whether the current scene is static or dynamic based on the amount of motion between adjacent frames. Then, perform feature point detection and matching on the current frame and the previous frame, and calculate the average pixel displacement of the matched feature points between adjacent frames as an indicator of scene motion. : ； in, and They represent the first The pixel coordinates of each matching feature point in the current frame and the previous frame. Indicates the number of successfully matched feature points; S5-2, when the scene motion index Greater than or equal to the preset motion threshold The current scene is determined to be a dynamic scene; when the scene's motion index... Less than the preset motion threshold The current scene is determined to be a static scene. S5-3: When facing a static scene, prioritize the allocation of exposure time, and then calculate the exposure gain after the exposure time reaches the upper limit. S5-4, When facing dynamic scenes, the final effective exposure value is based on S4. Calculate the corresponding target exposure time ;in, Indicates the preset base gain; S5-5, using the maximum permissible exposure time Target exposure time Limit the exposure time to obtain the final exposure time. : ; S5-6, Based on the target effective exposure value With the final exposure time The relationship between them is used to inversely calculate the corresponding target sensor gain. : .

Images