Real-time exposure control and image synthesis method of multi-lens array

Abstract

The invention relates to the technical field of image processing, in particular to a real-time exposure control and image synthesis method for a multi-lens array, which comprises the following steps: an acquisition step: acquiring images shot by a plurality of lenses and corresponding sensor read-out rate information; a processing step comprising: determining exposure parameters through a multi-level hardware level exposure control unit based on the image and the sensor readout rate information; adjusting exposure settings of the plurality of lenses according to the exposure parameters; generating a mask based on the adjusted images shot by the plurality of lenses; performing image synthesis according to the mask and the images shot by the plurality of lenses; the output step comprises the step of outputting the synthesized image, the method can complete exposure parameter adjustment of a plurality of lenses within the millisecond level, and the real-time performance of the system is greatly improved. Meanwhile, an exposure prediction mechanism based on reinforcement learning enables the system to quickly adapt to scene changes, and the problem of response lag of a traditional method is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a real-time exposure control and image synthesis method for a multi-lens array. Background Art

[0002] With the rapid development of image processing technology, multi-lens array camera systems are increasingly used in various fields. This system captures scenes simultaneously with multiple lenses, which can theoretically obtain richer image information. However, in practical applications, multi-lens array systems face many challenges, the most prominent of which are real-time exposure control and image synthesis.

[0003] Traditional multi-lens array systems usually use independent exposure control mechanisms, that is, each lens adjusts the exposure parameters separately. Although this method is simple to implement, it often leads to inconsistent exposure between different lenses, which in turn affects the quality of subsequent image synthesis. In addition, due to the lack of a global coordination mechanism, this method is difficult to cope with complex and changing lighting environments, especially in high dynamic range scenes.

[0004] In terms of image synthesis, existing technologies mostly use simple pixel-level fusion or weight-based fusion methods. Although these methods have fast calculation speed, they often cannot handle the detail differences between images with different exposures well, resulting in artifacts or loss of details in the synthesized image. In particular, the limitations of existing methods are more obvious when dealing with motion scenes or high-contrast scenes.

[0005] On the other hand, with the rise of deep learning technology, some researchers have tried to apply neural networks to image synthesis tasks. However, these methods usually have high computational complexity and are difficult to meet the needs of real-time processing. In addition, due to the lack of sufficient consideration of hardware characteristics and scene features, the effects of these methods in practical applications are often not as ideal.

[0006] There is also a common problem with existing technologies, which is the lack of ability to respond quickly to dynamic changes in scenes. In fast-changing scenes, such as moving objects or sudden changes in lighting conditions, existing methods often lag in response, resulting in reduced image quality.

[0007] In summary, the existing multi-lens array real-time exposure control and image synthesis methods are difficult to simultaneously meet multiple requirements such as real-time performance, image quality, and adaptability. New technical solutions are urgently needed to solve these problems. Summary of the invention

[0008] The present invention aims to solve the above technical problems and provide a method capable of realizing multi-lens collaborative exposure control, rapid scene adaptation and high-quality image synthesis. The method of the present invention effectively overcomes the limitations of the prior art through the combination of innovative hardware design and intelligent algorithm.

[0009] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0010] A real-time exposure control and image synthesis method for a multi-lens array, comprising:

[0011] The acquisition steps include:

[0012] Obtain images captured by multiple lenses and corresponding sensor readout rate information;

[0013] Processing steps include:

[0014] determining exposure parameters by a multi-level hardware-level exposure control unit based on the image and the sensor readout rate information;

[0015] adjusting exposure settings of the plurality of lenses according to the exposure parameters;

[0016] Generate a mask based on the adjusted images captured by the multiple lenses;

[0017] Performing image synthesis according to the mask and the images captured by the multiple lenses;

[0018] Output steps include:

[0019] Output the synthesized image.

[0020] Preferably, determining the exposure parameters by means of a multi-level hardware-level exposure control unit specifically includes:

[0021] Determining, by a first-level hardware-level exposure control unit, an exposure time of each frame of the image based on the image and the sensor readout rate information;

[0022] Determine the gain of the current frame based on the exposure time of the current frame image and the exposure time of the previous frame image adjacent to the image by means of the next level hardware level exposure control unit;

[0023] The exposure gain of the current frame image is determined through the last layer of hardware-level exposure control unit.

[0024] As a preference, it also includes:

[0025] Determining, by the exposure control buffer unit, exposure correction parameters based on the exposure time of the current captured image;

[0026] The exposure correction parameter includes the exposure gain k, which is calculated according to the following formula:

[0027] k=1+ΔG

[0028] Wherein, ΔG is the exposure gain output by the exposure control buffer unit.

[0029] Preferably, generating a mask specifically includes:

[0030] Extracting local features of the images captured by the multiple lenses through a feature atlas;

[0031] The local features are fused through a feature fusion module to obtain global features;

[0032] Further extract high-level features through convolution blocks;

[0033] The high-level features are fused through a residual fusion module to generate the mask.

[0034] Preferably, performing image synthesis specifically includes:

[0035] Calculate the geometric mean of the image signal gains in multiple frames of images as the image signal gain for image synthesis;

[0036] Calculate the total exposure time of multiple frames of images based on the exposure time;

[0037] Align multiple frames of images;

[0038] Accumulate the signals of each frame image and normalize them to obtain a normalized result;

[0039] The normalized result is convolved to obtain an image synthesis processing result.

[0040] As a preference, it also includes:

[0041] Use machine learning methods to estimate motion in local areas of the scene;

[0042] Performing scene classification on the image sequences output by the multiple lenses;

[0043] According to the scene classification result, the parameters of the multiple lenses are dynamically adjusted.

[0044] As a preference, it also includes:

[0045] Use a deep convolutional neural network model to predict the maximum exposure quantity and exposure mode corresponding to different areas;

[0046] According to the maximum exposure quantity, the focal lengths, aperture sizes and shutter speeds of the multiple lenses are adjusted to balance the dynamic range and detail loss.

[0047] Preferably, the multi-level hardware-level exposure control unit comprises:

[0048] Exposure control unit, used for coordinated exposure control of multiple lenses;

[0049] The exposure prediction unit uses the exposure parameters of the previous frames of multiple cameras and the current frame image to predict the exposure parameters of the next frame based on the principle of reinforcement learning;

[0050] An exposure adjustment unit is used to coordinate the outputs of the exposure control unit and the exposure prediction unit.

[0051] As a preference, it also includes:

[0052] Using the global camera as a reference, the pixel brightness variance is calculated;

[0053] The other camera parameters are adjusted through the adaptive exposure control algorithm to achieve consistent exposure among multiple cameras.

[0054] Preferably, performing image synthesis further comprises:

[0055] Design visually adaptive weighting functions to weight visual effects, details, and dynamic range based on the difference between the original image and the mask image;

[0056] The final grayscale value of each pixel is calculated using the following formula:

[0057] Final grayscale value = k·o+(1-k)·g

[0058] Among them, o is the pixel gray value of the original image, g is the pixel gray value of the mask image, and k is the weight;

[0059] The weight k is defined by the following formula: k = max(|go|).

[0060] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0061] Specifically, the beneficial effects of the present invention are mainly reflected in the following aspects:

[0062] First, the multi-level hardware-level exposure control unit proposed in the present invention realizes precise coordinated control between multiple lenses. Through the high-speed processing capability achieved by FPGA, the method can complete the exposure parameter adjustment of multiple lenses within milliseconds, greatly improving the real-time performance of the system. At the same time, the exposure prediction mechanism based on reinforcement learning enables the system to quickly adapt to scene changes, effectively solving the problem of delayed response of traditional methods.

[0063] Secondly, the deep learning-based mask generation technology introduced in the present invention significantly improves the quality of image synthesis. Compared with the traditional pixel-level fusion method, this method can better preserve image details and reduce the generation of artifacts. Especially when processing high dynamic range scenes, this method can preserve local details to the maximum extent while maintaining the overall dynamic range, thereby obtaining a more natural and realistic image.

[0064] Furthermore, the content-based adaptive weighting mechanism proposed in the present invention further optimizes the image synthesis effect. This mechanism can dynamically adjust the synthesis weight according to the importance of the image content, effectively balancing the visual effects, detail preservation and dynamic range of different regions. This innovation greatly improves the overall quality of the synthesized image, especially when processing complex scenes.

[0065] In addition, the method of the present invention not only realizes high-quality image processing, but also ensures the real-time performance of the entire processing process through hardware acceleration and algorithm optimization. This enables the method to be widely used in scenarios requiring real-time high-quality images, such as intelligent monitoring, autonomous driving and other fields.

[0066] Finally, the method of the present invention has good adaptability and scalability. By introducing scene recognition and parameter adaptive adjustment mechanism, the method can maintain stable performance in various complex environments. At the same time, the modular design makes it easy to integrate the method into the existing multi-lens array system to improve the overall performance of the system.

[0067] In summary, the multi-lens array real-time exposure control and image synthesis method provided by the present invention not only solves the key problems in the prior art, but also achieves significant improvements in image quality, processing speed and system adaptability, thus opening up a new path for the application and development of multi-lens array camera systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 The figure is an overall flow chart of the method of the present invention.

[0069] Figure 2 It is a logic block diagram of the multi-level hardware-level exposure control unit of the present invention.

[0070] Figure 3 The logical block diagram of the mask generation of the present invention.

[0071] Figure 4 It is a logic block diagram of the image synthesis of the present invention. DETAILED DESCRIPTION

[0072] like Figure 1-4 As shown, the present invention provides a real-time exposure control and image synthesis method for a multi-lens array. The method uses a multi-level hardware-level exposure control unit to achieve real-time exposure adjustment, and generates a high-quality mask through deep learning technology, and finally achieves the synthesis of high dynamic range images. The technical solution of the present invention will be described in detail below.

[0073] First, the method of the present invention includes an acquisition step, a processing step and an output step. In the acquisition step, the method acquires images taken by multiple lenses and corresponding sensor readout rate information. This information is the basic data for subsequent processing.

[0074] In the processing step, the method first determines exposure parameters through a multi-level hardware-level exposure control unit based on the acquired image and sensor readout rate information. Preferably, these exposure parameters include exposure time, gain, etc. After determining the exposure parameters, the method adjusts the exposure settings of multiple lenses according to these parameters. This step ensures that multiple lenses can work together to capture the best image information.

[0075] Next, the method generates a mask based on the adjusted images captured by the multiple lenses. The generation of the mask is a key innovation of the present invention, which helps to balance the dynamic range and details in the subsequent image synthesis process. Finally, the method performs image synthesis based on the generated mask and the images captured by the multiple lenses.

[0076] In the output step, the method outputs a synthesized image. The final output image has a high dynamic range and rich details, which can meet the imaging requirements in various complex lighting environments.

[0077] Furthermore, in the method of the present invention, the process of determining the exposure parameters by the multi-level hardware-level exposure control unit specifically includes the following steps:

[0078] First, the first-level hardware-level exposure control unit determines the exposure time of each frame based on the acquired image and sensor readout rate information. The purpose of this step is to set an appropriate exposure time for each lens to ensure that sufficient light information is captured. Preferably, the exposure time can be determined based on factors such as scene brightness and dynamic range. For example, in a low-light environment, the exposure time may be set to a longer time, such as 1 / 30 second; while in a bright environment, the exposure time may be shortened to 1 / 1000 second or shorter.

[0079] Then, the next level hardware-level exposure control unit determines the gain of the current frame based on the exposure time of the current frame image and the exposure time of the previous frame image adjacent to the image. The gain adjustment can further optimize the brightness and contrast of the image. Generally, the gain value can be adjusted between 1 and 10, depending on the scene requirements and the characteristics of the camera sensor.

[0080] Finally, the last layer of hardware-level exposure control unit determines the exposure gain of the current frame image. This final exposure gain is a comprehensive consideration of the results of the previous steps and is used to fine-tune the final exposure effect of the image.

[0081] The method of the present invention further includes determining an exposure correction parameter based on the exposure time of the current captured image through an exposure control buffer unit. The exposure correction parameter includes an exposure gain k, which is calculated according to the following formula:

[0082] k=1+ΔG

[0083] Wherein, k is the exposure gain, and ΔG is the exposure gain output by the exposure control buffer unit.

[0084] The introduction of this exposure correction parameter enables the method to control exposure more accurately, thereby obtaining higher quality images. For example, when ΔG is 0.2, the final exposure gain k is 1.2, which means that the brightness of the image will increase by 20%. This fine-tuning mechanism enables the method to adapt to various complex lighting conditions.

[0085] The method of the present invention realizes accurate real-time exposure control of a multi-lens array through the collaborative work of a multi-level hardware-level exposure control unit and an exposure control buffer unit. This control method can not only adapt to complex and changeable lighting environments, but also ensure the exposure consistency between multiple lenses, laying a solid foundation for subsequent image synthesis. At the same time, the hardware-level implementation ensures the real-time nature of the entire process, which is particularly important for application scenarios that require rapid response (such as high-speed photography or real-time monitoring).

[0086] From the above description, it can be seen that the method of the present invention has significant innovation and practicality in real-time exposure control of multi-lens arrays. It not only solves the exposure control problem of traditional methods in complex lighting environments, but also improves the processing speed through hardware implementation, providing strong support for high-quality image synthesis.

[0087] In one embodiment of the present invention, the process of generating a mask is a key step in image synthesis. Specifically, generating a mask includes the following steps:

[0088] First, local features of images taken by multiple lenses are extracted through feature atlases. The purpose of this step is to capture detailed information in the image. Preferably, the feature atlas can contain multiple convolutional layers, each of which is responsible for extracting features at different scales. For example, shallow convolutions may focus on low-level features such as edges and textures, while deep convolutions may capture more abstract high-level features.

[0089] Next, the local features are fused through the feature fusion module to obtain global features. The purpose of this step is to integrate feature information of different scales and different positions to form a comprehensive understanding of the entire image. In a preferred embodiment of the present invention, the feature fusion module can use attention mechanism or pyramid pooling technology to better process multi-scale information.

[0090] Then, high-level features are further extracted through convolution blocks. The purpose of this step is to further refine more representative features based on global features. Preferably, the convolution block can contain multiple residual units to avoid the gradient vanishing problem while deepening the network.

[0091] Finally, the residual fusion module is used to fuse the above high-level features to generate the final mask. The introduction of the residual fusion module can help retain the detail information of the original image while fusing high-level semantic information. The generated mask contains rich details and has good semantic consistency.

[0092] The image synthesis process in the method of the present invention is one of the core steps of the entire technical solution. Specifically, performing image synthesis includes the following key steps:

[0093] First, the geometric mean of the image signal gain in multiple frames is calculated as the image signal gain for image synthesis. The reason for using the geometric mean instead of the arithmetic mean is that the geometric mean is not sensitive to outliers and can better reflect the overall trend. For example, if the signal gains of three frames are 1.2, 1.5, and 1.8 respectively, the final image signal gain is (1.2×1.5×1.8) 1 / 3 ≈1.48.

[0094] Secondly, the total exposure time of multiple frames is calculated. The purpose of this step is to ensure that the synthesized image has sufficient exposure. Preferably, a weighted summation method can be used to give a higher weight to high-quality frames.

[0095] Next, the multiple frames are aligned. This is a crucial step because even a small displacement may cause blurring of the composite image. In a preferred embodiment of the present invention, a phase correlation method can be used to perform sub-pixel image alignment to ensure high accuracy.

[0096] Then, the signals of each frame are accumulated and normalized to obtain the normalized result. The purpose of this step is to fuse the information of multiple frames while keeping the overall brightness of the image within a reasonable range. Preferably, the normalization process can use min-max normalization or z-score normalization, and the specific choice depends on the characteristics of the image and the application scenario.

[0097] Finally, the normalized result is convolved to obtain the image synthesis processing result. This step can further enhance the details and sharpness of the image. Preferably, an adaptive convolution kernel can be used to dynamically adjust the convolution parameters according to the local characteristics of the image.

[0098] The method of the present invention also includes some intelligent scene recognition and parameter adjustment steps, which further improve the adaptability and robustness of the method. Specifically:

[0099] First, this method uses machine learning methods to estimate the motion of local areas in the scene. The purpose of this step is to identify moving objects in the image and provide a basis for subsequent parameter adjustment. Preferably, an optical flow method or a motion estimation algorithm based on deep learning can be used. For example, end-to-end motion estimation can be achieved using a neural network model such as FlowNet.

[0100] Secondly, the image sequences output by multiple lenses are classified into scenes. This step can help the system understand the type of scene currently being shot, so as to adopt a more targeted processing strategy. In one embodiment of the present invention, a pre-trained convolutional neural network (such as ResNet or DenseNet) can be used to classify scenes, and the scenes can be divided into multiple categories such as indoor, outdoor, night scene, sports scene, etc.

[0101] Finally, the parameters of multiple lenses are dynamically adjusted according to the scene classification results. This adaptive adjustment mechanism enables the method to obtain the best imaging effect in different scenes. For example, in night scene mode, the exposure time and ISO sensitivity may be increased; while in motion scenes, the exposure time may be shortened to reduce motion blur.

[0102] The method of the present invention also introduces deep learning technology to optimize exposure control. Specifically:

[0103] First, a deep convolutional neural network model is used to predict the maximum exposure number and exposure mode corresponding to different areas. The purpose of this step is to adaptively determine the best exposure strategy based on the image content. In a preferred embodiment of the present invention, a variant of a segmentation network such as U-Net or DeepLab can be used to implement this function and output the best exposure parameters for each pixel position.

[0104] Then, based on the predicted maximum number of exposures, the focal lengths, apertures, and shutter speeds of multiple lenses are adjusted to balance dynamic range and detail loss. This fine tuning of parameters enables the method to preserve image details to the maximum extent while maintaining a high dynamic range. For example, for high-contrast scenes, a multiple exposure strategy may be adopted, and the aperture and shutter speed may be adjusted accordingly.

[0105] From the above description, it can be seen that the method of the present invention has significant innovations in image processing, scene understanding, parameter optimization, etc. The comprehensive application of these technologies enables the method to obtain high-quality images in various complex shooting environments, greatly improving the performance and applicability of the multi-lens array camera system.

[0106] A key innovation of the present invention is the design of a multi-level hardware-level exposure control unit, which includes three main components: an exposure control unit, an exposure prediction unit, and an exposure adjustment unit.

[0107] The exposure control unit is used for collaborative exposure control of multiple lenses. In a preferred embodiment of the present invention, the unit is implemented using FPGA to ensure high-speed processing capability and low latency. The exposure control unit can process the exposure parameters of multiple lenses simultaneously to ensure that they can work together to capture the best image information. For example, when shooting a high dynamic range scene, different lenses may be set to different exposure parameters to capture the highlights and shadow details in the scene.

[0108] The exposure prediction unit is designed based on the principle of reinforcement learning. It uses the exposure parameters of the past frames of multiple cameras and the current frame image to predict the exposure parameters of the next frame. This prediction mechanism enables the method to respond to scene changes more quickly and improves the real-time performance of the system. In the specific implementation, reinforcement learning algorithms such as deep Q network (DQN) or policy gradient can be used. Preferably, the adjustment of exposure parameters can be modeled as a Markov decision process, where the state is the current image features and exposure parameters, the action is the adjustment of the exposure parameters, and the reward is the improvement of the image quality after adjustment.

[0109] The exposure adjustment unit is responsible for coordinating the outputs of the exposure control unit and the exposure prediction unit. The role of this unit is to find the best balance between hardware control and intelligent prediction. In one embodiment of the present invention, a weighted fusion method can be used to dynamically adjust the weights of the outputs of the two units based on historical performance. For example, if it is found that the prediction unit performs well in a certain type of scenario, then when encountering a similar scenario, the weight of the prediction unit output will be increased.

[0110] The method of the present invention also introduces a novel global exposure consistency control mechanism. Specifically, the method uses the global camera as a reference to calculate the pixel brightness variance. The purpose of this step is to evaluate the exposure consistency between multiple lenses. In practical applications, a key area in the image (such as the central area) can be selected as the range for calculating the brightness variance to improve the calculation efficiency.

[0111] Next, the method adjusts other camera parameters through an adaptive exposure control algorithm to achieve consistent exposure among multiple cameras. The core of this adaptive algorithm is to minimize the brightness difference between multiple lenses while maintaining the optimal exposure of each lens. Preferably, a gradient descent method can be used to optimize this multi-objective problem. For example, a loss function can be defined that takes into account both brightness difference and exposure quality, and then the best parameter settings can be found through iterative optimization.

[0112] The method of the present invention introduces a content-based adaptive weighting mechanism in the image synthesis stage, further improving the quality of the synthesized image. Specifically, the method designs a visual adaptive weighting function that weights the visual effect, details and dynamic range based on the difference between the original image and the mask image.

[0113] The core idea of ​​this weighting function is that the greater the difference between the original image and the mask image, the more likely the area contains important visual information or high-frequency details, and should be given a higher weight. Conversely, areas with smaller differences may be smooth areas or low-frequency information, and can be given a lower weight. This mechanism can effectively retain important details in the image while smoothing unimportant areas.

[0114] In a preferred embodiment of the present invention, the final grayscale value of each pixel is calculated using the following formula:

[0115] Final grayscale value = k·o+(1-k)·g

[0116] Among them, o is the pixel grayscale value of the original image, g is the pixel grayscale value of the mask image, and k is the weight. The weight k is defined by the following formula:

[0117] k=max(|go|)

[0118] The design of this formula takes into account the difference between the original image and the mask image. When the difference is large, the k value is close to 1, and the final grayscale value is more likely to retain the information of the original image; when the difference is small, the k value is close to 0, and the final grayscale value is more likely to adopt the information of the mask image.

[0119] This adaptive weighting mechanism enables the method to preserve image details to the maximum extent while maintaining a high dynamic range. For example, when capturing scenes with strong contrast between light and dark, this mechanism can effectively balance highlight and shadow areas, producing a more natural and detailed image.

[0120] Through the above detailed description, it can be seen that the method of the present invention has significant innovation and practicality in real-time exposure control and image synthesis of multi-lens arrays. From hardware-level exposure control to scene understanding based on deep learning, to content-based adaptive image synthesis, the present method covers multiple key links in image processing. This comprehensive and systematic method design enables the present invention to obtain high-quality images in various complex shooting environments, greatly improving the performance and applicability of multi-lens array camera systems.

[0121] The above description is only a preferred specific implementation manner of the present invention; however, the protection scope of the present invention is not limited thereto; any person familiar with the art who, within the scope disclosed by the present invention, makes equivalent replacements or changes based on the scheme and improved concepts of the present invention shall be covered within the protection scope of the present invention.

Claims

1. A method for real-time exposure control and image synthesis of a multi-lens array, characterized in that: include: The acquisition steps include: Obtain images captured by multiple lenses and corresponding sensor readout rate information; Processing steps include: determining exposure parameters by a multi-level hardware-level exposure control unit based on the image and the sensor readout rate information; adjusting exposure settings of the plurality of lenses according to the exposure parameters; Generate a mask based on the adjusted images captured by the multiple lenses; Performing image synthesis according to the mask and the images captured by the multiple lenses; Output steps include: Output the synthesized image.

2. The method according to claim 1, characterized in that Determining the exposure parameters by means of a multi-level hardware-level exposure control unit specifically includes: Determining, by a first-level hardware-level exposure control unit, an exposure time of each frame of the image based on the image and the sensor readout rate information; Determine the gain of the current frame based on the exposure time of the current frame image and the exposure time of the previous frame image adjacent to the image by means of the next level hardware level exposure control unit; The exposure gain of the current frame image is determined through the last layer of hardware-level exposure control unit.

3. The method according to claim 1, characterized in that Also includes: Determining, by the exposure control buffer unit, exposure correction parameters based on the exposure time of the current captured image; The exposure correction parameter includes the exposure gain k, which is calculated according to the following formula: k=1+ΔG Wherein, ΔG is the exposure gain output by the exposure control buffer unit.

4. The method according to claim 1, characterized in that: The generating of the mask specifically comprises: Extracting local features of the images captured by the multiple lenses through a feature atlas; The local features are fused through a feature fusion module to obtain global features; Further extract high-level features through convolution blocks; The high-level features are fused through a residual fusion module to generate the mask.

5. The method according to claim 1, characterized in that The performing of image synthesis specifically includes: Calculate the geometric mean of the image signal gains in multiple frames of images as the image signal gain for image synthesis; Calculate the total exposure time of multiple frames of images based on the exposure time; Align multiple frames of images; Accumulate the signals of each frame image and normalize them to obtain a normalized result; The normalized result is convolved to obtain an image synthesis processing result.

6. The method according to claim 1, characterized in that Also includes: Use machine learning methods to estimate motion in local areas of the scene; Performing scene classification on the image sequences output by the multiple lenses; According to the scene classification result, the parameters of the multiple lenses are dynamically adjusted.

7. The method according to claim 1, characterized in that Also includes: Use a deep convolutional neural network model to predict the maximum exposure quantity and exposure mode corresponding to different areas; According to the maximum exposure quantity, the focal lengths, aperture sizes and shutter speeds of the multiple lenses are adjusted to balance the dynamic range and detail loss.

8. The method according to claim 1, characterized in that The multi-level hardware-level exposure control unit comprises: Exposure control unit, used for coordinated exposure control of multiple lenses; The exposure prediction unit uses the exposure parameters of the previous frames of multiple cameras and the current frame image to predict the exposure parameters of the next frame based on the principle of reinforcement learning; An exposure adjustment unit is used to coordinate the outputs of the exposure control unit and the exposure prediction unit.

9. The method according to claim 1, characterized in that: Also includes: Using the global camera as a reference, the pixel brightness variance is calculated; The other camera parameters are adjusted through the adaptive exposure control algorithm to achieve consistent exposure among multiple cameras.

10. The method according to claim 1, characterized in that The performing of image synthesis further comprises: Design visually adaptive weighting functions to weight visual effects, details, and dynamic range based on the difference between the original image and the mask image; The final grayscale value of each pixel is calculated using the following formula: Final grayscale value = k·o+(1-k)·g Among them, o is the pixel gray value of the original image, g is the pixel gray value of the mask image, and k is the weight; The weight k is defined by the following formula: k = max(|go|).

Images