Camera automatic exposure method and device and electronic equipment

Abstract

The embodiment of the invention provides a camera automatic exposure method and device and electronic equipment, and relates to the technical field of imaging. According to the method, a key sample region in a current sampling image is identified by using a pre-trained segmentation model, and a foreground image is generated, so that an exposure decision can focus on a target concerned by a user. And jointly constructing a target image sequence according to the sampling images of the first N frames, the current sampling image and the foreground image corresponding to the current sampling image, and realizing joint modeling of the time sequence visual information and the semantic features. The exposure time is predicted based on the target image sequence by using the pre-trained intelligent agent, the exposure adjustment amount with environmental adaptability is obtained, and the control precision and stability under the dynamic illumination change are improved; the latest exposure time is determined by combining the current exposure time corresponding to the image of the current frame and the predicted exposure adjustment amount, and closed-loop exposure parameter updating is completed, so that the robustness and the adaptive capacity of camera imaging are improved.

Description

Technical Field

[0001] This invention relates to the field of imaging technology, and more specifically, to a method, apparatus, and electronic device for automatic camera exposure. Background Technology

[0002] Current automatic exposure methods for cameras generally adjust parameters based on the overall or partial brightness statistics of the image, lacking the ability to semantically recognize the image content. Whether using classic algorithms such as averaging, center-weighted, or matrix metering, the goal is to make the global brightness approach the preset gray level, which is easily affected by strong background light or local dark areas, resulting in underexposure or overexposure.

[0003] Even with the introduction of heuristic optimization or deep learning regression models, there is still a lack of understanding of image content. Under high contrast, low illumination, or special imaging modes (such as fluorescence or dark field), exposure decisions often deviate from the observed subject, resulting in the loss of key target details. In addition, most methods lag in response to dynamic scenes and are highly dependent on manual parameter tuning or fixed scene assumptions, exhibiting poor adaptability to new environments. This further exacerbates the problem of inaccurate exposure in the core observation area, affecting imaging consistency and analytical reliability. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a camera automatic exposure method, apparatus and electronic device that can improve the robustness and adaptability of imaging.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, the present invention provides a method for automatic exposure of a camera, the method comprising: Preprocess the image of the current frame to obtain the current sampled image; The key sample regions in the current sampled image are identified using a pre-trained segmentation model to obtain the foreground image corresponding to the current sampled image; A target image sequence is generated based on the sampled images of the previous N frames, the current sampled image, the foreground images corresponding to the sampled images of the previous N frames, and the foreground images corresponding to the current sampled image. The exposure adjustment amount is obtained by using a pre-trained agent to predict the exposure time based on the target image sequence; The latest exposure time is determined based on the current exposure time corresponding to the current frame image and the exposure adjustment amount, so that the camera performs exposure processing based on the latest exposure time.

[0006] In an optional implementation, the step of using a pre-trained segmentation model to identify key sample regions in the current sampled image to obtain the foreground image corresponding to the current sampled image includes: The current sampled image is input into a pre-trained segmentation model to mark key sample regions in the current sampled image, thereby obtaining a binary mask image corresponding to the current sampled image. Generate a foreground image corresponding to the current sampled image based on the current sampled image and the corresponding binary mask image.

[0007] In an optional implementation, the preprocessing of the image of the current frame to obtain the current sampled image includes: The received image of the current frame is downsampled to obtain the current grayscale image; The current grayscale image is normalized to obtain the current sampled image.

[0008] In an optional implementation, the segmentation model is obtained through the following training method: Multiple first original images and foreground labels corresponding to each first original image are obtained; the foreground labels are used to mark key sample regions in the first original images. Enhancement processing is performed on each first original image to obtain multiple first training images and foreground labels corresponding to each first training image; Each of the first training images is input into the target detection neural network to identify key sample regions in the first training images, thereby obtaining a binary mask image corresponding to each of the first training images. The first loss value is determined based on the foreground label and binary mask image corresponding to the first training image; The weights of the target detection neural network are updated by backpropagation based on the first loss value to obtain the initial segmentation model; The initial segmentation model is quantized, compressed, and calibrated to obtain the segmentation model.

[0009] In an optional implementation, the agent is obtained through the following training method: Multiple sets of second original image sequences acquired based on different optical parameters and a deep reinforcement learning network to be trained are obtained; each set of second original image sequences contains multiple second original images with different exposure times; the deep reinforcement learning network includes a policy network and a first value network; Multiple sets of second training image sequences are generated based on each set of the second original image sequences; the second training image sequence includes multiple second training images and a foreground image corresponding to each second training image; Each group of the second training image sequence is input into the policy network to obtain the exposure time adjustment amount; Generate a predicted training image sequence and a predicted adjustment amount corresponding to the second training image sequence based on each group of the second training image sequence and the corresponding exposure time adjustment amount; The second training image sequence and the exposure time adjustment are input into the first value network to obtain the first value, and the predicted training image sequence corresponding to the second training image sequence and the prediction adjustment are input into the second value network to obtain the second value. The weights of the policy network, the first value network, and the second value network are updated by reverse propagation based on the first value and the second value, thus obtaining the updated deep reinforcement learning network. The updated deep reinforcement learning network is quantized, pruned, and corrected to obtain the agent.

[0010] In an optional implementation, generating multiple sets of second training image sequences based on each set of the second original image sequences includes: Multiple image combinations are generated based on the second original image sequence; each image combination includes at least one second original image. An image generator is used to generate multiple second training images corresponding to each image combination based on each image combination; The segmentation model is used to identify the foreground image of each second training image corresponding to each image combination; the multiple second training images and the corresponding foreground images corresponding to each image combination constitute a set of second training image sequences.

[0011] In an optional implementation, generating a predicted training image sequence and a predicted adjustment amount corresponding to the second training image sequence based on each group of the second training image sequences and the corresponding exposure time adjustment amount includes: The second exposure time is determined based on the exposure time of the last second training image in each group of the second training image sequence and the exposure time adjustment amount; The image generator is used to generate a second generated image corresponding to the second exposure time based on the last second training image in the second training image sequence, the exposure time of the last second training image in the second training image sequence, and the second exposure time. A foreground image corresponding to the second generated image is generated based on the second generated image using a pre-trained segmentation model; Generate a corresponding prediction training image sequence based on the second to last second training images in the second training image sequence, the second generated image, and the corresponding foreground image; The predicted training image sequence is input into the policy network to obtain the prediction adjustment amount.

[0012] In an optional implementation, the step of generating a second generated image corresponding to the second exposure time using an image generator based on the last second training image in the second training image sequence, the exposure time of the last second training image in the second training image sequence, and the second exposure time includes: The last second training image in the second training image sequence, the exposure time of the last second training image in the second training image sequence, and the second exposure time are input into the image generator; The corrected image is obtained by subtracting the preset black level value from the last second training image in the second training image sequence; The grayscale values ​​in the corrected image are converted into light intensity values, and a target light intensity map is generated based on the light intensity values, the exposure time of the last second training image in the second training image sequence, and the second exposure time. The illumination intensity values ​​in the target illumination intensity map are converted into grayscale values, and a preset black level value is added for normalization processing to obtain the second generated image corresponding to the second exposure time.

[0013] In a second aspect, the present invention provides an automatic exposure device for a camera, the device comprising: The processing module is used to preprocess the image of the current frame to obtain the current sampled image; identify key sample regions in the current sampled image using a pre-trained segmentation model to obtain the foreground image corresponding to the current sampled image; and generate a target image sequence based on the sampled images of the previous N frames, the current sampled image, the foreground images corresponding to the sampled images of the previous N frames, and the foreground images corresponding to the current sampled image. An adjustment module is used to predict the exposure time based on the target image sequence using a pre-trained agent to obtain an exposure adjustment amount; and to determine the latest exposure time based on the current exposure time corresponding to the current frame image and the exposure adjustment amount, so that the camera can perform exposure processing based on the latest exposure time.

[0014] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing a computer program executable by the processor, the processor executing the computer program to implement the automatic exposure method for a camera as described in any of the foregoing embodiments.

[0015] Compared to existing technologies, the automatic exposure method, apparatus, and electronic device for cameras provided in this invention utilize a pre-trained segmentation model to identify key sample regions in the currently sampled image and generate corresponding foreground images. This allows exposure decisions to focus on the user's target rather than global brightness affected by background interference. A target image sequence is then constructed based on the sampled images of the previous N frames, the current sampled image, and its corresponding foreground image, achieving joint modeling of temporal visual information and semantic features. Next, a pre-trained agent predicts the exposure time based on the target image sequence, obtaining an environmentally adaptive exposure adjustment amount, improving control accuracy and stability under dynamic lighting changes. Finally, the latest exposure time is determined by combining the current exposure time corresponding to the current frame image and the predicted exposure adjustment amount, completing a closed-loop exposure parameter update.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic flowchart of an automatic camera exposure method provided by an embodiment of the present invention is shown.

[0019] Figure 2 This diagram illustrates another flowchart of the automatic exposure method for cameras provided in an embodiment of the present invention.

[0020] Figure 3 This diagram illustrates another flowchart of the automatic exposure method for cameras provided in an embodiment of the present invention.

[0021] Figure 4 A block diagram illustrating an agent training method provided by an embodiment of the present invention is shown.

[0022] Figure 5 A block diagram of an automatic exposure device for a camera provided in an embodiment of the present invention is shown.

[0023] Figure 6 A block diagram of an electronic device provided in an embodiment of the present invention is shown. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0025] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0026] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0027] The inventors discovered that automatic exposure systems aim to ensure image quality by adjusting optical parameters in real time. Currently, camera exposure control methods are mainly divided into three categories: the classic numerical method, which adjusts the combination of aperture, shutter speed, and ISO through statistical analysis of image brightness to make the image brightness approach the target value (such as 18% gray). The core algorithms include the average method (calculating the average brightness of all pixels in the image, which has a fast response but is easily affected by local extreme brightness / darkness, resulting in overall overexposure or underexposure), the weighted average method (typically center-weighted metering, which gives higher weight to the center of the image to highlight the subject, but fails in backlight or when the subject is off-center), area metering / matrix metering (dividing the image into multiple areas and comprehensively evaluating brightness according to preset rules, which can handle high-contrast scenes, but is difficult to adapt to unconventional compositions due to the fixed rules), and spot metering (precise exposure for a very small area of ​​1%-5%, which requires manual intervention and completely ignores other parts of the image).

[0028] While such methods offer advantages in speed and stability, their reliance on pure luminance statistics and lack of semantic understanding leads to issues such as loss of detail in high dynamic range scenes, exposure deviations in special environments, and interference from localized strong light or dark areas, severely weakening their robustness and adaptability under complex lighting conditions.

[0029] The second category comprises heuristic optimization methods that transform exposure control into an objective function optimization problem. These methods dynamically adjust parameter combinations through iterative search or feedback mechanisms. Key methods include PID control algorithms (which calculate parameter increments in real-time based on brightness errors to achieve smooth video exposure transitions), genetic algorithm optimization (which generates a random parameter population and iteratively searches for the global optimum using a fitness function), and fuzzy logic control (which makes decisions based on a pre-defined rule base). While these methods incorporate dynamic optimization concepts, they generally suffer from bottlenecks such as insufficient dynamic performance, lack of semantic understanding (inability to recognize main content), and parameter sensitivity (requiring repeated tuning for different device models).

[0030] The third category comprises deep learning-based methods that utilize neural networks to learn the mapping relationship from images to exposure parameters end-to-end, breaking through the limitations of traditional rules. The main technical approaches include CNN regression networks (using lightweight architectures such as MobileNet to process preview frames and jointly optimize exposure error and perceptual loss, which can improve parameter prediction accuracy, but accuracy decays severely during model transfer) and Transformer multimodal fusion (integrating image, ambient light, and motion pose data through cross-modal attention mechanisms to output a local exposure mask to improve dynamic range). This type of method faces core challenges such as strong data dependence, high computational cost, and failure in edge scenes (performance collapses in low-light conditions <1 lux).

[0031] In summary, existing technologies have two major shortcomings: First, insufficient scene understanding and semantic awareness. Traditional AE algorithms rely excessively on image brightness statistics and lack the ability to semantically recognize image content (such as cell structures and areas of user interest), leading to inaccurate exposure of key targets (underexposure or overexposure). Furthermore, they cannot distinguish special optical scenes (such as dark scenes and fluorescence), which can easily lead to incorrect exposure strategies. Second, although deep learning-based exposure algorithms are more adaptable in general scenes, they have problems such as weak adaptability to new scenes and insufficient adaptation to user habits. Moreover, their high model complexity limits their real-time performance.

[0032] Based on this, embodiments of the present invention provide an automatic exposure method for a camera. The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0033] Please refer to Figure 1 , Figure 1 A schematic flowchart of a camera automatic exposure method provided by an embodiment of the present invention is shown. The method includes the following steps: Step S100: Preprocess the image of the current frame to obtain the current sampled image.

[0034] Step S110: Use a pre-trained segmentation model to identify key sample regions in the current sampled image to obtain the foreground image corresponding to the current sampled image.

[0035] In this embodiment of the invention, the segmentation model is a trained and quantized compressed image segmentation model (such as the FastSAM model), loaded using the TensorRT or ONNX Runtime inference engine, with the model file path being . / models / fastsam_quantized.onnx, ensuring efficient execution. If the detection results indicate the existence of key sample regions, a foreground image containing only the key sample regions is generated. If the detection results indicate no valid key sample regions, the current sampled image is used as the foreground image to generate a target image sequence, ensuring process continuity.

[0036] As one possible approach, taking the acquisition of microscope images via a microscope as an example, the key sample region to be identified is the key biological sample region (such as target structures like cell nuclei and cytoplasm). A pre-trained segmentation model is used to identify the key biological sample region in the current sampled image corresponding to the microscope image, thus obtaining the foreground image corresponding to the current sampled image.

[0037] As another possible implementation, taking the acquisition of portrait images by a consumer-grade camera as an example, the key sample region to be identified is the region containing the key person. A pre-trained segmentation model is used to identify the key person region in the current sampled image corresponding to the portrait image, thus obtaining the foreground image corresponding to the current sampled image.

[0038] As another possible implementation, taking the acquisition of workpiece images by an industrial camera as an example, the key sample region to be identified is the region containing the workpiece to be tested. A pre-trained segmentation model is used to identify the region of the workpiece to be tested in the current sampled image corresponding to the workpiece image, thus obtaining the foreground image corresponding to the current sampled image.

[0039] It should be understood that automatic exposure cameras include, but are not limited to, cameras for stereo microscopes, fluorescence microscopes, industrial cameras, consumer cameras, and professional cameras. This invention does not limit this scope; the following description uses a microscope camera as an example.

[0040] Step S120: Generate a target image sequence based on the sampled images of the previous N frames, the current sampled image, the foreground images corresponding to the sampled images of the previous N frames, and the foreground images corresponding to the current sampled image.

[0041] In this embodiment of the invention, a sliding window buffer is maintained (removing the oldest frame and adding a new frame) to store historical data of the most recent N frames, including historical sampled images of N consecutive frames and their corresponding foreground images. When a new current sampled image and its corresponding foreground image are generated, they are combined with the existing historical frames in the buffer to form a set of shared data. The target image sequence of the frame.

[0042] As one possible implementation, taking N=3 as an example, assuming the size of each frame is... The first three sampled images and the current sampled image are used as the first four frames of the target image sequence. The foreground images corresponding to the first three sampled images and the foreground images corresponding to the current sampled image are used as the last four frames of the target image sequence. These are stacked along the time dimension into a state tensor of shape 8×128×128, resulting in the target image sequence. This target image sequence reflects recent image change trends and the spatial distribution characteristics of the target region, providing the agent with sufficient temporal context information.

[0043] Step S130: The exposure time is predicted based on the target image sequence using a pre-trained agent to obtain the exposure adjustment amount.

[0044] In this embodiment of the invention, the pre-trained agent is constructed based on the Soft Actor-Critic (SAC) algorithm. The target image sequence is input into the pre-trained agent, and the agent uses deterministic measurements to perform forward inference to obtain action decisions (i.e., exposure adjustment amounts) through the policy network within the agent. The exposure adjustment amount ranges from [-1, 1], representing the adjustment direction and magnitude relative to the current exposure level.

[0045] Step S140: Determine the latest exposure time based on the current exposure time and exposure adjustment amount corresponding to the current frame image, so that the camera can perform exposure processing based on the latest exposure time.

[0046] In this embodiment of the invention, the formula for calculating the latest exposure time is:

[0047] in, This is the latest exposure time; This is the current exposure time; It's the exposure adjustment amount; It is an adjustable parameter with a default value of 2. It is used to control the non-linear response of exposure changes, ensuring smooth transitions and avoiding inter-frame flickering.

[0048] The exponential mapping of the latest exposure time ensures that exposure adjustments exhibit a symmetrical growth or decay trend regardless of whether the action is positive or negative, avoiding the asymmetric response problem caused by linear increases or decreases. Furthermore, to prevent exposure parameters from exceeding the hardware's supported range, a hard constraint is imposed on the calculation results, forcibly limiting the final exposure time to the camera's effective exposure area (e.g., [10ms, 500ms]). Once the latest exposure time is determined, the underlying API interface is called via the camera SDK (such as Micro-Manager or OpenCV) to send this parameter to the camera in real time, completing this round of exposure configuration update.

[0049] It should be noted that the electronic device executing the automatic camera exposure method requires the deployment of a pre-trained segmentation model and agent, and the configuration of the initial environment upon startup. First, the parameters of the policy network in the pre-trained agent are loaded, i.e., the pre-trained weight file is loaded from a specified path (e.g., . / models / sac_actor.pth). Then, the initial exposure parameters are set, including but not limited to the initial exposure value (e.g., expo_init = 100 ms) and camera gain (e.g., gain = 1.0). Next, the hardware interface is initialized, connecting the camera and the frame rate (e.g., 20fps) via the camera SDK (e.g., Micro-Manager or OpenCV).

[0050] In summary, the automatic exposure method for cameras provided in this invention utilizes a pre-trained segmentation model to identify key sample regions in the currently sampled image and generate corresponding foreground images. This allows exposure decisions to focus on the target of interest to the user rather than global brightness affected by background interference. Furthermore, a target image sequence is constructed based on the sampled images of the previous N frames, the current sampled image, and its corresponding foreground image, achieving joint modeling of temporal visual information and semantic features. Next, a pre-trained agent predicts the exposure time based on the target image sequence, obtaining an environmentally adaptive exposure adjustment amount, improving control accuracy and stability under dynamic lighting changes. Finally, the latest exposure time is determined by combining the current exposure time corresponding to the current frame image and the predicted exposure adjustment amount, completing a closed-loop exposure parameter update.

[0051] As can be seen, by integrating semantic segmentation results with deep reinforcement learning decision-making mechanisms, this method achieves precise exposure control of key sample regions without human intervention, effectively alleviating the problem of inaccurate exposure of key targets caused by ignoring content semantics in traditional methods, and improving the robustness and adaptability of the camera under complex optical conditions.

[0052] Alternatively, one possible implementation for generating the foreground image is provided below. Figure 1 The sub-steps of step S110 may include: Step S111: Input the current sampled image into the pre-trained segmentation model, mark the key sample regions in the current sampled image, and obtain the binary mask image corresponding to the current sampled image.

[0053] In this embodiment of the invention, the segmentation model receives the current sampled image (e.g., a normalized grayscale image of size 1×128×128) and performs forward inference, outputting a binary mask of the same resolution, where each pixel takes a value of 0 or 1. A pixel with a value of 1 corresponds to an identified key generated sample region (i.e., foreground); a pixel with a value of 0 represents a background region. The binary mask essentially models the spatial distribution of semantic importance in the current sampled image, forming the basis for subsequent image reconstruction.

[0054] Step S112: Generate the foreground image corresponding to the current sampled image based on the current sampled image and the corresponding binary mask image.

[0055] In this embodiment of the invention, an element-wise multiplication operation is performed, that is, each pixel value of the current sampled image is multiplied by the pixel value in the binary mask image at its corresponding position to construct a biologically meaningful foreground image. This not only improves the target targeting of exposure control, but also provides high-quality feature input for subsequent time-series-based state modeling, thereby significantly improving the quality stability and detail visibility of microscopic images under complex lighting conditions.

[0056] Foreground Image The calculation formula is:

[0057] in, This is the currently sampled image; It is a binary mask image.

[0058] Alternatively, one possible implementation for generating the current sampled image is provided below. Figure 1 The sub-steps of step S100 may include: The received image of the current frame is downsampled to obtain the current grayscale image; the current grayscale image is then normalized to obtain the current sampled image.

[0059] In this embodiment of the invention, the preprocessing process begins after each frame of image is acquired. First, the image of the current frame is captured via a camera interface (e.g., based on the USB 3.0 communication protocol). To reduce processor processing latency, hardware acceleration is used to read the image of the current frame into a memory buffer.

[0060] If the image is in RAW format, the resolution is typically 1920×1080 pixels. A spatial downsampling operation is performed on the image, and a bilinear interpolation algorithm is used to compress the image from 1920×1080 resolution to 128×128 pixels, resulting in the current grayscale image.

[0061] If the image is an RGB (red, green, blue) three-channel image, it is converted to a grayscale image (i.e., the current grayscale image), and a single-channel current grayscale image (shape such as 1×128×128) is output. The grayscale value conversion formula for pixel coordinates (x, y) is:

[0062] in, It's a grayscale value. This is the weight of the red channel, for example, set to 0.299; This is the weight of the green channel, for example, set to 0.587; This is the weight of the blue channel, for example, set to 0.114; .

[0063] Next, the current grayscale image is normalized, that is, all pixel values ​​are linearly mapped from the original integer dynamic range of [0, 255] to the floating-point range of [0, 1].

[0064] Alternatively, regarding how to train the segmentation model, the following is one possible implementation. Please refer to... Figure 2 This includes the following steps: Step S200: Obtain multiple first original images and foreground labels corresponding to each first original image; the foreground labels are used to mark key sample regions in the first original images.

[0065] In this embodiment of the invention, the training process of the segmentation model begins with the construction of a large-scale, high-quality dataset. During this process, a large number of initial raw images (e.g., 10,000 images) are first acquired. These images originate from actual microscopic imaging environments, covering approximately 200 typical biological sample types, including cell sections and tissue sections. They also cover different lighting conditions ranging from 2500K to 7500K in color temperature and include three typical exposure states: underexposure, normal exposure, and overexposure, ensuring that the training data fully reflects the diversity and complexity of real-world application scenarios.

[0066] Next, pixel-level annotation is performed on each of the first original images. Key sample regions, including but not limited to biologically significant structures such as cell nuclei and cytoplasm, are manually delineated using professional annotation tools (such as LabelMe or CVAT), generating corresponding foreground labels. These foreground labels can be stored in PNG format. Background areas in the foreground labels are marked as 0 to distinguish them from key sample regions.

[0067] Step S210: Enhance each first original image to obtain multiple first training images and foreground labels corresponding to each first training image.

[0068] In this embodiment of the invention, in order to further improve the anti-interference ability and generalization performance of the model, a variety of data augmentation processing methods are performed simultaneously on the first original image and its corresponding foreground label, including but not limited to random rotation (rotation angle range controlled within ±15 degrees), brightness perturbation (brightness adjustment range of ±20%), and adding Gaussian noise (noise standard deviation σ is set to 0.01).

[0069] Through the aforementioned data augmentation techniques, the training set of the segmentation model was expanded from 10,000 images to 300,000 first training images. Each first training image is equipped with a corresponding augmented foreground label, forming a complete set of first training images.

[0070] The first training image set can be divided into 80% training set, 10% validation set, and 10% test set, and organized and managed using a standard image database structure to facilitate subsequent segmentation model training and performance evaluation.

[0071] Step S220: Input each first training image into the target detection neural network to identify key sample regions in the first training images and obtain a binary mask image corresponding to each first training image.

[0072] In this embodiment of the invention, each first training image is input into a target detection neural network (e.g., a lightweight YOLO11n-seg model, which is optimized for embedded devices and has fast inference capabilities) to perform a semantic segmentation task, and outputs a binary mask image with a resolution of 128×128, which can meet the real-time requirements of the embedded platform while ensuring sufficient spatial details.

[0073] Step S230: Determine the first loss value based on the foreground label and binary mask image corresponding to the first training image.

[0074] Step S240: Update the weights of the target detection neural network by backpropagation based on the first loss value to obtain the initial segmentation model.

[0075] In this embodiment of the invention, during the initialization phase, the pre-trained weights of the object detection neural network are loaded as initial parameters, which helps to accelerate the model convergence process. During training, the Adam optimizer is used for parameter updates; for example, the initial learning rate is set to 1e-4, the weight decay coefficient is set to 1e-5, the batch size is 32, and the total training cycles are 200. The loss function adopts a weighted combination of cross-entropy loss and Dice loss, aiming to balance the class imbalance problem with boundary segmentation accuracy.

[0076] The entire training process is based on the PyTorch deep learning framework and is completed on a workstation equipped with an NVIDIA GPU. During training, a first loss value is calculated based on the training set using a loss function for the foreground label and binary mask image corresponding to the first training image, and the weights of the object detection neural network are updated using the first loss value.

[0077] An early stopping mechanism (patience=50) is triggered when no significant improvement is observed on the validation set for 50 consecutive rounds to prevent overfitting. The final trained initial segmentation model achieved 92.5% mIoU accuracy on the test set, with a single-frame inference time of less than 5 milliseconds, indicating a good balance between accuracy and efficiency.

[0078] Step S250: Quantize, compress, and calibrate the initial segmentation model to obtain the segmentation model.

[0079] In this embodiment of the invention, to adapt to embedded deployment, the trained initial segmentation model is quantized and compressed to reduce reliance on high-performance hardware. Specifically, Post-Training Quantization (PTQ) can be used to convert the FP32 (single-precision floating-point number) format used in the initial segmentation model into INT8 (8-bit integer) representation, thereby significantly reducing model size and memory bandwidth consumption.

[0080] During the calibration phase, 1000 labeled images not used in training are selected as the calibration set. The min-max calibration algorithm is applied to determine the optimal quantization parameters, ensuring that accuracy loss is minimized. The quantization process can be implemented using mainstream inference engine toolchains such as TensorRT or ONNX Runtime, ensuring cross-platform compatibility.

[0081] After quantization, the storage size of the resulting segmentation model is reduced to approximately 35% of the initial segmentation model (e.g., compressed from 10MB to 3.5MB). While maintaining functional integrity, the mIoU accuracy decreases by less than 0.5 percentage points (still remaining above 92%), and the single-frame inference time is further shortened to less than 3 milliseconds. The final output segmentation model is packaged in a standardized format (such as .engine or .onnx) for easy application in camera automatic exposure methods, and can be directly called as a runtime module.

[0082] Alternatively, regarding how to train the agent, the following is one possible implementation. Please refer to... Figure 3 This includes the following steps: Step S300: Obtain multiple sets of second original image sequences acquired based on different optical parameters and a deep reinforcement learning network to be trained; each set of second original image sequences contains multiple second original images with different exposure times; the deep reinforcement learning network includes a policy network and a first value network.

[0083] In this embodiment of the invention, a programmable control hardware system, including an adjustable light source and a motorized stage, is interfaced with a camera software development kit (SDK) to achieve a high degree of automation and precise synchronization in the image acquisition process.

[0084] The adjustable light source has a color temperature range of 2500K to 7500K, and is adjusted in 500K increments, covering 11 independent color temperature nodes. The light source intensity is adjusted at 12.5% ​​intervals within the 0% to 100% range, forming 9 intensity levels. The motorized stage is precisely positioned in 0.1 mm increments, or the sample position can be finely adjusted manually when necessary, ensuring that each type of sample can be repeatedly sampled from different areas with spatial texture differences.

[0085] Building upon this foundation, image acquisition was performed on over 400 sample classes (including 300 biological tissue slices and 100 multicolor synthetic samples) across five independent imaging regions for each sample (the center point and the edges of the four quadrants). Each set of second raw image sequences consisted of eight images with exposure times set to 0.1 ms, 1 ms, 10 ms, 50 ms, 100 ms, 300 ms, 600 ms, and 1000 ms to comprehensively cover the full dynamic range from underexposed to overexposed. This resulted in a second raw image sequence containing 176,000 images (i.e., 400 sample classes × 5 imaging regions per class × 11 color temperature nodes × 8 images per set), thus avoiding training bias.

[0086] As one possible implementation, such as Figure 4 As shown, the agent includes a policy network (i.e., the Actor network) and a first value network (i.e., the first Critic network). The policy network and the first value network are implemented based on the PyTorch architecture and use the ReLU activation function.

[0087] The policy network comprises an input layer, an image encoding layer, a fully connected layer, and an output layer. The input layer receives a sequence of images (e.g., a second training image sequence). Features are extracted from the image sequence by an image editing layer (a four-layer convolutional structure and a flattening layer: the first three convolutional layers contain activation functions, the fourth convolutional layer does not, and the flattening layer converts the output into a one-dimensional vector). These features are then processed by a fully connected layer containing three 512-dimensional hidden layers (including LayerNorm normalization). Finally, the output layer uses a reparameterization technique to output an exposure time adjustment based on the generated action mean and logarithmic standard deviation. The exposure time adjustment is constrained to the range [-1, 1] in the output layer using the Tanh function.

[0088] The first value network employs a four-Q network structure to reduce overestimation bias. Each Q network structure includes an input layer, an image encoding layer, a fully connected layer, and an output layer. The input layer receives each set of second training image sequences (i.e., the received state) and the corresponding exposure time adjustment (i.e., the action). Each Q network shares an image editing layer with the policy network (parameters are frozen to avoid redundant computation), and concatenates the flattened features with the action vector (i.e., the exposure time adjustment), feeding it into a fully connected layer containing four 512 hidden layers for processing. Finally, it outputs a Q value (i.e., the first value). The network weights are initialized using the Xavier method to ensure training stability.

[0089] The agent training process includes, but is not limited to, the initialization phase, parameter configuration phase, interactive training phase, and model optimization phase. In the initialization phase, a random seed (default value 0) is set to ensure the determinism of all random number generation processes, thereby achieving reproducibility of experiments across training cycles. Simultaneously, a log file is created, recording training metrics such as reward and loss in TensorBoard or CSV format, and training parameters such as environment configuration and network architecture are also written to the log to support subsequent analysis and traceability.

[0090] Based on this, an environment simulator based on a physical imaging model and an experience replay buffer using a Prioritized Experience Replay (PER) mechanism are instantiated, and the instantiation of the SAC agent is completed. The weight parameters of its policy network and first value network are randomly initialized from a normal distribution (e.g., mean 0, standard deviation 0.01).

[0091] During the parameter configuration phase, convergence speed, policy stability, and exploration efficiency are balanced by appropriately configuring hyperparameters. For example, a discount factor of 0.99 is set to assign a higher weight to future rewards, enhancing the agent's focus on long-term exposure control effects. Both the policy network and the first value network use a learning rate of 1e-4 and employ the Adam optimizer for gradient updates, balancing training stability and convergence efficiency. Each batch samples 256 empirical data points from the experience replay buffer for network parameter updates. This buffer has a total capacity of 5000 experiences, each containing a state (e.g., the second training image sequence), action, reward value, and the next state (e.g., predicting the training image sequence).

[0092] The total number of training cycles is set to 100,000, with each cycle including 100 interactions (i.e., 100 sets of second training image sequences). Network parameters are updated every 120 steps to ensure a smooth learning process. Model checkpoints are saved every 1000 episodes, and the output format is a .pt file. The initial value of the temperature parameter specific to the SAC algorithm is set to 0.2, and an automatic adjustment mechanism is supported to dynamically optimize the trade-off between exploration and utilization. Parameter configuration can be set according to the actual application scenario, and this invention does not limit this.

[0093] Step S310: Generate multiple sets of second training image sequences based on each set of second original image sequences; the second training image sequence includes multiple second training images and a foreground image corresponding to each second training image.

[0094] In this embodiment of the invention, to reduce reliance on real hardware and improve training efficiency, an environment simulator based on Python and OpenCV libraries is implemented. This simulator can accurately simulate image effects under different exposure parameters using the camera's response function (CRF). The environment simulator generates multiple second training images from multiple sets of first training image sequences corresponding to each set of first original image sequences. A segmentation model is then used to generate a foreground image corresponding to each second training image, ultimately resulting in multiple sets of second training image sequences corresponding to each set of second original image sequences.

[0095] To enhance the agent's adaptability to various perturbations in real imaging environments, diverse mutations need to be introduced during the interactive training phase to improve robustness. To this end, a dynamic data augmentation mechanism was integrated into the environment simulator to ensure the agent maintains high accuracy in diverse scenarios. Test results show that the data augmentation mechanism can reduce the agent's generalization error by more than 15%. Data augmentation operations include, but are not limited to, image blurring, image rotation, temporal reversal, spatial flip, and temporal acceleration.

[0096] Image blurring simulates defocusing blur caused by camera mechanical vibration. It employs either Gaussian blur (fixed 5×5 kernel, random sampling with standard deviation σ ∈ [0.5, 2.5]) or motion blur (random direction 0°-180°, random length ∈ [5, 20] pixels), applied independently to a single frame with a 20% probability to force the model to adapt to defocusing scenarios. Image rotation overcomes sample slide placement deviations by performing random rotation (uniform sampling at ±10° angle), applying it to a single frame with a 15% probability after bicubic interpolation and filling the black boundaries (value = 0), thus improving the model's robustness to geometric deformations.

[0097] Time reversal randomly reverses the image sequence along the timeline (e.g., changing the frame order from [1,2,3,4] to [4,3,2,1]) to simulate a sudden change in lighting, with a trigger probability of 20%. Spatial flip flips the image horizontally or vertically (15% probability each) to enhance the model's robustness to changes in sample orientation. Time acceleration traverses the image sequence at different speeds (e.g., 2x or 0.5x speed) and generates intermediate frames using interpolation algorithms (such as bilinear interpolation) to simulate dynamic lighting changes.

[0098] Step S320: Input each group of second training image sequences into the policy network to obtain the exposure time adjustment amount.

[0099] Step S330: Generate a predicted training image sequence and a predicted adjustment amount corresponding to the second training image sequence based on each group of second training image sequences and the corresponding exposure time adjustment amount.

[0100] In this embodiment of the invention, a policy network is used to analyze the second training image sequence and output the exposure time adjustment amount. Then, a predicted training image sequence is generated based on the second training image sequence and the corresponding exposure time adjustment amount, and the policy network is used to generate the predicted adjustment amount for the predicted training image sequence.

[0101] It should be noted that the agent's training is completed in a customized, forced learning environment. This learning environment simulates a camera's exposure control scenario, and its core components include, but are not limited to, a state space, an action space, action mapping rules, and a reward function.

[0102] The state space is defined as a sequence of eight images, each a 1×128×128 pixel grayscale image, used to represent historical observation information. The first four frames are downsampled and normalized versions of the original images, while the last four frames are foreground images extracted from the corresponding foreground regions using a segmentation model. Together, they construct a complete context input, ensuring the consistency of the input data.

[0103] The action space is set to a continuous value within the range of [-1, 1], representing the relative adjustment amount of exposure time, with the unit being Exposure Value (EV). It covers typical exposure adjustment needs, with negative values ​​indicating reduced exposure and positive values ​​indicating increased exposure.

[0104] The actual exposure time needs to be generated using motion mapping rules based on the relative adjustment between the current exposure time and the exposure time. The expression for the motion mapping rule is:

[0105] To guide the agent in learning the optimal policy, a reward function with a weighted combination of multiple objectives is used, the expression of which is:

[0106] in, It is a reward value; It is the average brightness of the foreground region in the current frame, normalized to the range of 0-1. The target brightness is set to 0.5, which corresponds to the ideal exposure level. This item encourages the agent to maintain the image brightness in a moderate range. The degree of flicker between frames is measured by calculating the luminance variance of consecutive frames. The smaller the value, the more stable the image and the less visual flicker. It is based on the output of deep learning models (such as MANIQA, HyperIQA, etc.) and reflects the overall image quality; a higher score indicates better image quality. Weight parameters (default) =1, It is 0.5. (0.3) Through grid search optimization, the contribution between different indicators is balanced, so that the agent can flicker consistently and maintain high-quality imaging while approaching the target brightness.

[0107] During training, the agent interacts with the learning environment. In each training cycle, the action output by the policy network (i.e., exposure adjustment), the reward value output by the policy network, the current state (i.e., the current image), and the next state (i.e., the image generated according to the adjusted exposure time) are stored in the replay buffer as the interaction data for this training.

[0108] Step S340: Input the second training image sequence and exposure time adjustment into the first value network to obtain the first value, and input the predicted training image sequence and prediction adjustment corresponding to the second training image sequence into the second value network to obtain the second value.

[0109] In this embodiment of the invention, the first value network sets its output to a fixed Q value, while the second value network outputs four Q values ​​by default. The first value network evaluates the second training image sequence and the exposure time adjustment to obtain a first value. The second value network evaluates the predicted training image sequence corresponding to the second training image sequence and the prediction adjustment to obtain four Q values, and determines the largest Q value as the second value.

[0110] Step S350: Based on the first value and the second value, the weights of the updated policy network, the weights of the first value network and the weights of the second value network are backpropagated to obtain the updated deep reinforcement learning network.

[0111] In this embodiment of the invention, the loss value (including policy loss and Q-value loss) is calculated periodically (e.g., every 120 training cycles) based on the first value and the second value. The weights of the policy network, the first value network, and the second value network are then updated through backpropagation to obtain the updated deep reinforcement learning network.

[0112] The entire training process was executed on an NVIDIA GPU platform, and visualization analysis was performed using Matplotlib or TensorBoard tools. When training converged, the average reward stabilized above 0.8, and the exposure control error was less than 5%, indicating that the updated deep reinforcement learning network has accurate and stable exposure adjustment capabilities.

[0113] Step S360: Quantize, prune, and correct the updated deep reinforcement learning network to obtain the agent.

[0114] In this embodiment of the invention, to adapt to embedded deployment, the updated deep reinforcement learning network is optimized for lightweighting to reduce reliance on high-performance hardware. One approach is to perform weight pruning, using PyTorch's pruning module to remove redundant connections (e.g., weights with absolute values ​​< 1e-3). Assuming a pruning rate of 30%, accuracy is restored through iterative pruning and fine-tuning (e.g., fine-tuning over 10 training epochs with a learning rate of 1e-5).

[0115] Secondly, INT8 quantization is performed, using post-trained static quantization (PTQ) to convert the FP32 model to 8-bit integer format. The calibration set uses 1000 environment state samples not involved in training, implemented through the TensorRT toolchain. The optimized model reduces computation by 70% (from 1 GFLOPs to 0.3 GFLOPs), with inference latency controlled to <10 ms. Accuracy loss is <2% (average reward decrease <0.1), ensuring real-time performance. The optimized model output is in a standard format (e.g., .engine or .tflite).

[0116] It should be noted that during the training and lightweight optimization of the agent, the agent parameters (including the weights of the policy network and the first value network) are periodically saved to disk (e.g., files in .pt format). The storage location is specified as a dedicated target, along with storage metadata (such as training configuration and random seeds) to facilitate subsequent loading and inference.

[0117] Alternatively, one possible implementation for generating the second training image sequence is provided below. Figure 3 The sub-steps of step S310 may include: Step S311: Generate multiple image combinations based on the second original image sequence; each image combination includes at least one second original image.

[0118] In this embodiment of the invention, an image combination is randomly obtained from at least one second original image in the second original image sequence, and multiple image combinations are generated through multiple random samplings. For example, two or three second original images are obtained from the second original image sequence.

[0119] Step S312: Use an image generator to generate multiple second training images corresponding to each image combination based on each image combination.

[0120] In this embodiment of the invention, the target exposure time corresponding to the image to be generated is determined according to the exposure time of each second original image in each image combination according to a preset generation rule. The second original image as a reference, the exposure time corresponding to the second original image as a reference, and the target exposure time are input into the image generator, and the generated image corresponding to the target exposure time is output. The second original image and the generated image corresponding to the target exposure time in a set of image combinations are determined as multiple second training images corresponding to the image combination.

[0121] Step S313: Use the segmentation model to identify the foreground image of each second training image corresponding to each image combination; the multiple second training images and the corresponding foreground images corresponding to each image combination constitute a set of second training image sequences.

[0122] In this embodiment of the invention, each second training image corresponding to the image combination is input into the segmentation model to obtain a binary mask image corresponding to the second training image, and a foreground image corresponding to the second training image is generated based on the second training image and the corresponding binary mask image, thereby forming a second training image sequence corresponding to each image combination.

[0123] Alternatively, one possible implementation for generating the prediction training image sequence is provided below. Figure 3 The sub-steps of step S330 may include: Step S331: Determine the second exposure time based on the exposure time and exposure time adjustment of the last second training image in each group of second training image sequences.

[0124] In this embodiment of the invention, the second exposure time is calculated based on the exposure time and exposure time adjustment of the last second training image, referring to the aforementioned calculation formula for the latest exposure time.

[0125] Step S332: Using an image generator, a second generated image corresponding to the second exposure time is generated based on the last second training image in the second training image sequence, the exposure time of the last second training image in the second training image sequence, and the second exposure time.

[0126] In this embodiment of the invention, the last second training image in the second training image sequence, the exposure time of the last second training image in the second training image sequence, and the second exposure time are input into the image generator, and the second generated image corresponding to the second exposure time is output.

[0127] Step S333: Generate a foreground image corresponding to the second generated image based on the second generated image using a pre-trained segmentation model.

[0128] In this embodiment of the invention, the second generated image is input into the segmentation model to identify key sample regions, and a binary mask image corresponding to the second generated image is output. The second generated image and the corresponding binary mask image are multiplied element by element to obtain the foreground image corresponding to the second generated image.

[0129] Step S334: Generate a corresponding prediction training image sequence based on the second to last second training images, the second generated image, and the corresponding foreground image in the second training image sequence.

[0130] Step S335: Input the predicted training image sequence into the policy network to obtain the prediction adjustment amount.

[0131] In this embodiment of the invention, second training images from the second image to the last image in the second training image sequence are obtained, and foreground images corresponding to the second training images from the second image to the last image are obtained simultaneously. The second generated image is stored after the last second training image and placed at the beginning of the prediction training image sequence. The foreground images corresponding to the second generated image are stored after the foreground images corresponding to the last second training image and placed at the end of the prediction training image sequence, resulting in a complete prediction training image sequence. Finally, a policy network is used to generate the action (i.e., the prediction adjustment amount) corresponding to the prediction training image sequence.

[0132] Optionally, regarding how to generate the second generated image based on the adjusted second exposure time, a possible implementation is provided below. The sub-steps of step S332 may include: Step S332-1: Input the last second training image in the second training image sequence, the exposure time of the last second training image in the second training image sequence, and the second exposure time into the image generator.

[0133] In this embodiment of the invention, the image generator processes the image data step by step according to a preset imaging conversion rule based on the received second training image and the corresponding exposure time, ensuring that the generated second image conforms to the response characteristics of a real camera at the second exposure time.

[0134] Step S332-2: Subtract the preset black level value from the last second training image in the second training image sequence to obtain the corrected image.

[0135] In this embodiment of the invention, black level correction is first performed to eliminate noise inherent in the image sensor. The expression for black level correction is:

[0136] in, It corrects the grayscale values ​​of pixels in an image; It is the grayscale value of the pixel in the second training image; It is a preset black level value, usually provided by the camera manufacturer or determined through dark field calibration experiments.

[0137] Step S332-3: Convert the grayscale values ​​in the corrected image into light intensity values, and generate a target light intensity map based on the light intensity values, the exposure time of the last second training image in the second training image sequence, and the second exposure time.

[0138] Next, in the pixel conversion stage, a predefined look-up table (LUT) is used to convert the grayscale value of each pixel in the calibrated image into an illumination intensity value. The LUT is generated through camera calibration experiments, recording the nonlinear mapping relationship between the sensor output grayscale and the actual incident light intensity under different illumination conditions, and is stored in CSV file format for later retrieval.

[0139] Entering the exposure ratio transformation stage, based on the linear imaging assumption—that image brightness is proportional to exposure time—exposure changes can be simulated through scaling under fixed light source conditions. The conversion formula for the target illumination intensity map is:

[0140] in, It is a target illumination intensity map; It is an image obtained by converting the grayscale value of each pixel in the corrected image into the illumination intensity value; This is the second exposure time; It is the exposure time of the last second training image in the second training image sequence.

[0141] Step S332-4: Convert the light intensity value in the target light intensity map into a grayscale value, and add a preset black level value for normalization processing to obtain the second generated image corresponding to the second exposure time.

[0142] During the illuminance conversion stage, an inverse LUT is applied to convert the illuminance values ​​in the target illuminance map back to grayscale values. The inverse LUT is the inverse function of the aforementioned LUT, ensuring that the physical light intensity is remapped to the digital range that the sensor can represent, thus avoiding the introduction of interpolation distortion or dynamic range compression.

[0143] Finally, the preset black level is added back to the converted grayscale value, and the entire image is normalized to obtain the second generated image corresponding to the second exposure time. Using the second generated image as the input to the last frame of the prediction training image sequence, it closely approximates the actual shooting result in both visual performance and numerical characteristics.

[0144] It should be noted that the image generator runs on a standard workstation (such as one equipped with an Intel i7 CPU and 32GB RAM), supports real-time simulation, and has a frame rate greater than 30 FPS, providing an efficient and controllable environment for agent training.

[0145] Based on the same inventive concept, the basic principle and technical effects of the automatic exposure device for cameras provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments.

[0146] Please refer to Figure 5 , Figure 5 This is a block diagram of a camera automatic exposure device 400 provided in an embodiment of the present invention. The camera automatic exposure device 400 includes a processing module 410 and an adjustment module 420.

[0147] The processing module 410 is used to preprocess the image of the current frame to obtain the current sampled image; use a pre-trained segmentation model to identify key sample regions in the current sampled image to obtain the foreground image corresponding to the current sampled image; and generate a target image sequence based on the sampled images of the previous N frames, the current sampled image, the foreground images corresponding to the sampled images of the previous N frames, and the foreground images corresponding to the current sampled image. The adjustment module 420 is used to predict the exposure time based on the target image sequence using a pre-trained agent to obtain the exposure adjustment amount; and to determine the latest exposure time based on the current exposure time and exposure adjustment amount corresponding to the current frame image so that the camera can perform exposure processing based on the latest exposure time.

[0148] In summary, the automatic exposure device for cameras provided in this embodiment of the invention utilizes a pre-trained segmentation model to identify key sample regions in the currently sampled image and generate corresponding foreground images. This allows exposure decisions to focus on the target of interest to the user rather than global brightness affected by background interference. Furthermore, a target image sequence is constructed based on the sampled images of the previous N frames, the current sampled image, and its corresponding foreground image, achieving joint modeling of temporal visual information and semantic features. Next, a pre-trained agent predicts the exposure time based on the target image sequence, obtaining an environmentally adaptive exposure adjustment amount, improving control accuracy and stability under dynamic lighting changes. Finally, the latest exposure time is determined by combining the current exposure time corresponding to the current frame image and the predicted exposure adjustment amount, completing a closed-loop exposure parameter update.

[0149] Optionally, the processing module 410 is specifically used to input the current sampled image into a pre-trained segmentation model, mark the key sample regions in the current sampled image, and obtain a binary mask image corresponding to the current sampled image; and generate a foreground image corresponding to the current sampled image based on the current sampled image and the corresponding binary mask image.

[0150] Optionally, the processing module 410 is specifically used to downsample the image of the received current frame to obtain the current grayscale image; and to normalize the current grayscale image to obtain the current sampled image.

[0151] Optionally, the processing module 410 is further configured to train a segmentation model, including: acquiring multiple first original images and foreground labels corresponding to each first original image; the foreground labels are used to mark key sample regions in the first original images; performing enhancement processing on each first original image to obtain multiple first training images and foreground labels corresponding to each first training image; inputting each first training image into a target detection neural network to identify key sample regions in the first training images and obtain a binary mask image corresponding to each first training image; determining a first loss value based on the foreground labels and binary mask images corresponding to the first training images; updating the weights of the target detection neural network through backpropagation based on the first loss value to obtain an initial segmentation model; and performing quantization compression and calibration on the initial segmentation model to obtain a segmentation model.

[0152] Optionally, the processing module 410 is further configured to train an agent, including: acquiring multiple sets of second original image sequences acquired based on different optical parameters and a deep reinforcement learning network to be trained; each set of second original image sequences contains multiple second original images with different exposure times; the deep reinforcement learning network includes a policy network and a first value network; generating multiple sets of second training image sequences based on each set of second original image sequences; the second training image sequences include multiple second training images and a foreground image corresponding to each second training image; inputting each set of second training image sequences into the policy network to obtain an exposure time adjustment amount; generating a predicted training image sequence corresponding to the second training image sequence based on each set of second training image sequences and the corresponding exposure time adjustment amount; inputting the second training image sequence into the first value network to obtain a first value amount, and inputting the predicted training image sequence corresponding to the second training image sequence into the second value network to obtain a second value amount; updating the weights of the policy network, the first value network, and the second value network by backpropagation based on the first value amount and the second value amount to obtain an updated deep reinforcement learning network; and quantizing, pruning, and correcting the updated deep reinforcement learning network to obtain an agent.

[0153] Optionally, the processing module 410 is specifically configured to generate multiple image combinations based on the second original image sequence; each image combination includes at least one second original image; generate multiple second training images corresponding to each image combination using an image generator; identify the foreground image of each second training image corresponding to each image combination using a segmentation model; and the multiple second training images and the corresponding foreground images corresponding to each image combination constitute a set of second training image sequences.

[0154] Optionally, the processing module 410 is specifically configured to: determine the second exposure time based on the exposure time and exposure time adjustment amount of the last second training image in each group of second training image sequences; generate a second generated image corresponding to the second exposure time using an image generator based on the last second training image in the second training image sequence, the exposure time of the last second training image in the second training image sequence, and the second exposure time; generate a foreground image corresponding to the second generated image based on the second generated image using a pre-trained segmentation model; generate a corresponding prediction training image sequence based on the second to last second training images in the second training image sequence, the second generated image, and the corresponding foreground image; and input the prediction training image sequence into a policy network to obtain the prediction adjustment amount.

[0155] Optionally, the processing module 410 is specifically used to input the last second training image in the second training image sequence, the exposure time of the last second training image in the second training image sequence, and the second exposure time into the image generator; subtract a preset black level value from the last second training image in the second training image sequence to obtain a corrected image; convert the grayscale values ​​in the corrected image into light intensity values, and generate a target light intensity map based on the light intensity values, the exposure time of the last second training image in the second training image sequence, and the second exposure time; convert the light intensity values ​​in the target light intensity map into grayscale values, add a preset black level value for normalization processing, and obtain the second generated image corresponding to the second exposure time.

[0156] Please refer to Figure 6 This is a block diagram illustrating an electronic device 500 provided in an embodiment of the present invention. The electronic device 500 includes, but is not limited to, a personal computer (PC), a personal digital assistant (PDA), a laptop computer, a tablet computer, and a server. The electronic device 500 includes a memory 510, a processor 520, and a communication module 530. The memory 510, processor 520, and communication module 530 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0157] The memory 510 is used to store programs or data. The memory 510 may be, but is not limited to, random access memory, read-only memory, programmable read-only memory, erasable read-only memory, electrically erasable read-only memory, etc.

[0158] The processor 520 is used to read / write data or programs stored in the memory 510 and perform corresponding functions. For example, when a computer program stored in the memory 510 is executed by the processor 520, the automatic exposure method for a camera disclosed in the above embodiments can be implemented.

[0159] The communication module 530 is used to establish a communication connection between the electronic device 500 and other communication terminals via a network, and to send and receive data via the network.

[0160] It should be understood that, Figure 6 The structure shown is only a schematic diagram of the electronic device 500. The electronic device 500 may also include components that are larger than those shown. Figure 6 The more or fewer components shown, or having the same Figure 6 The different configurations shown. Figure 6 The components shown can be implemented using hardware, software, or a combination thereof.

[0161] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor 520, implements the automatic camera exposure method disclosed in the above embodiments.

[0162] This invention also provides a program product that, when executed by processor 520, implements the automatic camera exposure method disclosed in the above embodiments.

[0163] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0164] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0165] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of camera auto exposure, characterized by, The method comprises: preprocessing the image of the current frame to obtain a current sampling image; using a pre-trained segmentation model to identify a key sample area in the current sampling image to obtain a foreground image corresponding to the current sampling image; generating a target image sequence according to the sampling images of the previous N frames, the current sampling image, the foreground images corresponding to the sampling images of the previous N frames, and the foreground image corresponding to the current sampling image; using a pre-trained agent to predict an exposure adjustment amount based on the target image sequence; determining a latest exposure time according to the current exposure time corresponding to the image of the current frame and the exposure adjustment amount, so that the camera performs exposure processing based on the latest exposure time.

2. The camera auto exposure method of claim 1, wherein, The use of a pre-trained segmentation model to identify a key sample area in the current sampling image to obtain a foreground image corresponding to the current sampling image comprises: inputting the current sampling image into a pre-trained segmentation model to label the key sample area in the current sampling image to obtain a binary mask image corresponding to the current sampling image; generating a foreground image corresponding to the current sampling image according to the current sampling image and the corresponding binary mask image.

3. The camera auto exposure method of claim 1, wherein, The preprocessing of the image of the current frame to obtain a current sampling image comprises: down-sampling the received image of the current frame to obtain a current grayscale image; normalizing the current grayscale image to obtain a current sampling image.

4. The camera auto exposure method of claim 1, wherein, The segmentation model is obtained by the following training method: obtaining a plurality of first original images and foreground labels corresponding to each of the first original images; the foreground label is used to label a key sample area in the first original image; enhancing each first original image to obtain a plurality of first training images and foreground labels corresponding to each of the first training images; inputting each of the first training images into a target detection neural network to identify a key sample area in the first training image to obtain a binary mask image corresponding to each of the first training images; determining a first loss value according to the foreground label and the binary mask image corresponding to the first training image; updating the weight of the target detection neural network in a reverse propagation manner according to the first loss value to obtain a segmentation initial model; quantizing, compressing and calibrating the segmentation initial model to obtain the segmentation model.

5. The camera auto exposure method of claim 1, wherein, The agent is obtained by the following training method: obtaining a plurality of second original image sequences collected based on different optical parameters and a deep reinforcement learning network to be trained; each of the second original image sequences comprises a plurality of second original images with different exposure times; the deep reinforcement learning network comprises a policy network and a first value network; generating a plurality of second training image sequences according to each of the second original image sequences; each of the second training image sequences comprises a plurality of second training images and a foreground image corresponding to each of the second training images; inputting each of the second training image sequences into the policy network to obtain an exposure time adjustment amount; generating a predicted training image sequence and a predicted adjustment amount corresponding to each of the second training image sequences according to each of the second training image sequences and the corresponding exposure time adjustment amount; inputting the second training image sequence and the exposure time adjustment amount into the first value network to obtain a first value amount, and inputting a predicted training image sequence corresponding to the second training image sequence and the predicted adjustment amount into a second value network to obtain a second value amount; updating weights of the policy network, weights of the first value network, and weights of the second value network in a reverse direction according to the first value amount and the second value amount, to obtain an updated deep reinforcement learning network; quantizing, pruning, and correcting the updated deep reinforcement learning network to obtain the agent.

6. The camera auto exposure method of claim 5, wherein, The method further includes: generating a plurality of image combinations according to the second original image sequence; each of the image combinations includes at least one second original image; generating a plurality of second training images corresponding to each of the image combinations by using an image generator based on each of the image combinations; identifying a foreground image of each of the second training images corresponding to each of the image combinations by using the segmentation model; the plurality of second training images corresponding to each of the image combinations and the corresponding foreground image constitute a second training image sequence.

7. The camera auto exposure method of claim 5, wherein, The method further includes: determining a second exposure time according to an exposure time of a last second training image in each of the second training image sequences and the exposure time adjustment amount; generating a second generated image corresponding to the second exposure time by using an image generator based on the last second training image in the second training image sequence, the exposure time of the last second training image in the second training image sequence, and the second exposure time; generating a foreground image corresponding to the second generated image by using a pre-trained segmentation model based on the second generated image; generating a corresponding predicted training image sequence according to the second training image sequence, the second generated image, and the corresponding foreground image; inputting the predicted training image sequence into the policy network to obtain a predicted adjustment amount.

8. The camera auto exposure method of claim 7, wherein, The method further includes: inputting the last second training image in the second training image sequence, the exposure time of the last second training image in the second training image sequence, and the second exposure time into the image generator; subtracting a preset black level value from the last second training image in the second training image sequence to obtain a corrected image; converting a gray value in the corrected image into an illumination intensity value, and generating a target illumination intensity map according to the illumination intensity value, the exposure time of the last second training image in the second training image sequence, and the second exposure time; and inputting the target illumination intensity map into the policy network to obtain the predicted adjustment amount. The illumination intensity values in the target light intensity map are converted into gray scale values, and a preset black level value is added for normalization to obtain a second generated image corresponding to the second exposure time.

9. A camera auto exposure apparatus, characterized by, The device comprises: The processing module is configured to pre-process an image of a current frame to obtain a current sampling image, identify a key sample region in the current sampling image by using a pre-trained segmentation model to obtain a foreground image corresponding to the current sampling image, and generate a target image sequence according to sampling images of previous N frames, the current sampling image, foreground images corresponding to the sampling images of the previous N frames, and a foreground image corresponding to the current sampling image. The adjusting module is configured to predict an exposure time based on the target image sequence by using a pre-trained intelligent agent to obtain an exposure adjustment amount, and determine a latest exposure time according to a current exposure time corresponding to the image of the current frame and the exposure adjustment amount, so that the camera performs exposure processing based on the latest exposure time.

10. An electronic device, comprising: The device comprises a processor and a memory, the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the camera automatic exposure method of any one of claims 1-8.

Images