On-orbit multi-stage real-time automatic light adjusting device and method for space array camera

Abstract

The application relates to the technical field of image processing, in particular to a multi-stage real-time automatic light adjusting device and method of a space array camera in orbit. The device comprises the following steps: an imaging module obtains a scene original image according to a set integration time, and transmits image data to an image coarse adjustment module; the image coarse adjustment module calculates a first updated integration time according to the real-time statistics of the gray scale distribution of the original image, and sends image data without overexposure or underexposure to the image fine adjustment module; the image fine adjustment module receives the image, carries out scene recognition and gray scale evaluation through a deep learning model, dynamically adjusts exposure parameters according to the scene type, and realizes image exposure control; the image coarse adjustment module and the image fine adjustment module constitute a two-stage closed loop, so that the imaging module can obtain the integration time matched with the scene in real time. The device has the advantages that the integration time of the detector can be automatically adjusted according to the details of the target scene, and the target image with the best image perception and details can be captured.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an on-orbit multi-stage real-time automatic dimming device and method for a space area array camera. Background Art

[0002] When capturing images, aerospace cameras are often affected by numerous factors, including the reflectivity of the target object, optical lens design, solar altitude, and atmospheric and weather influences, resulting in over- or underexposure. Remote sensing image data volumes are often large, and over- or underexposed images have low usability. Downlinking poorly exposed images wastes channel resources and can also affect accuracy in specialized tasks like target recognition, making mission success more difficult.

[0003] Existing on-orbit dimming methods for space remote sensing cameras fall into two main categories: post-processing dimming, and integration time adjustment strategies based on fixed thresholds or lookup tables. Post-processing dimming methods often over-increase the brightness of dark areas, causing bright areas to overexpose; or suppress highlight areas, resulting in loss of local detail. Post-processing methods struggle to recover the detailed information contained in the target scene when over- or under-exposed. Furthermore, they can amplify noise, blur edge details, and cause hue shifts when processing color images. Currently, there are relatively few on-orbit, real-time automatic dimming methods, consisting of strategies that adjust the integration time based on the median or mean grayscale value of the image. These strategies suffer from poor real-time performance and robustness, making it difficult to adjust to the optimal integration time that maximizes the preservation of the target scene's image quality and detail.

[0004] Existing automatic dimming methods lack real-time image post-processing dimming strategies and cannot guide the capture of the next frame in real time. Furthermore, post-processed images are prone to introducing artifacts, making it difficult to accurately recover image details from over- or under-exposure. These on-orbit automatic dimming methods primarily use the image's grayscale mean or median as a priori information to calculate and update the detector integration time. These methods can only adjust the integration time by presetting a desired threshold or injecting a desired threshold from the ground. They cannot dynamically adjust the area array detector integration time to capture optimal contrast and detail based on the target scene. These methods lack dimming precision, fail to consider the image's inherent content, and lack control over image classification and detail. This is especially true when capturing ground objects, as cloud and fog obstructions can significantly interfere with dimming accuracy. Furthermore, different integration times should be used for different scenes. For example, if a space camera captures the ocean, adjusting the integration time based on the ocean scene can easily lead to underexposure in other scenes. Some researchers have also adopted dimming methods based on image histogram features or image area division dimming methods. These methods cannot capture the detailed distribution of the image and cannot achieve automatic dimming based on the characteristic distribution of the target scene.

[0005] In summary, existing research and inventions do not have an on-orbit real-time automatic dimming method for image details. Therefore, it is urgent to develop a real-time automatic dimming device and method to meet the needs of real-time precise adjustment of the integration time of aerospace remote sensing cameras on-orbit. Summary of the Invention

[0006] In order to solve the above problems, the present invention provides an on-orbit multi-level real-time automatic dimming device and method for a space area array camera.

[0007] The first object of the present invention is to provide an on-orbit multi-level real-time automatic light adjustment device for a space array camera, comprising an imaging module, an image coarse adjustment module, and an image fine adjustment module;

[0008] The imaging module acquires the original image of the scene according to the set integration time and transmits the image data to the image coarse adjustment module;

[0009] The image coarse adjustment module counts the grayscale distribution of the original image in real time and calculates the first update integration time, and packages the image data without overexposure or underexposure and sends it to the image fine adjustment module;

[0010] After receiving the image, the image fine-tuning module uses a deep learning model to perform scene recognition and grayscale evaluation, and dynamically adjusts exposure parameters according to the scene type to achieve image exposure control;

[0011] The image data output end of the imaging module is electrically connected to the image data input end of the image coarse adjustment module; the image coarse adjustment module and the image fine adjustment module form a two-stage closed loop, so that the imaging module obtains the integration time of scene matching in real time.

[0012] Preferably, the imaging module includes an area array CMOS detector; the image coarse adjustment module includes an imaging control FPGA; the image fine adjustment module includes an image processing CPU; the integration time control signal output end of the image coarse adjustment module is electrically connected to the integration time setting end of the imaging module.

[0013] Preferably, the image coarse adjustment module includes an imaging control FPGA; the image coarse adjustment module statistically analyzes the original image data, obtains grayscale distribution information, and determines whether the image is overexposed or underexposed based on the grayscale distribution information;

[0014] If the image is not overexposed or underexposed, the image data is packaged and sent to the image fine-tuning module;

[0015] If overexposure or underexposure is detected, a first update integration time is calculated according to a preset algorithm; and image data including the first update integration time is packaged and sent to the image fine-tuning module.

[0016] Preferably, the image fine-tuning module includes an image processing CPU; the deep learning model includes a feature extraction unit, a scene recognition unit, a grayscale evaluation unit and an exposure adjustment unit;

[0017] The feature extraction unit extracts features in the Conv1, Conv3 or Conv5 layer of the MobileNetV3-Large network with depthwise separable convolution using a multi-scale feature pyramid;

[0018] The scene recognition unit is used to identify the scene type; the scene type includes at least one of a city, an ocean, a mountain, a desert, a forest, a grassland, a farmland, a lake or a river, a cloud, and a starry sky;

[0019] The grayscale evaluation unit is used to determine the corresponding grayscale threshold interval according to the identified scene type;

[0020] The exposure adjustment unit is used to calculate the second update integration time and verify whether it is within a preset safety range.

[0021] A second object of the present invention is to provide an on-orbit multi-level real-time automatic dimming method for a spacecraft area array camera, wherein dimming is performed using an on-orbit multi-level real-time automatic dimming device for a spacecraft area array camera, and specifically comprises the following steps:

[0022] S1 start device; imaging module according to the set integration time to obtain the original image of the scene, and transmits the image data to the image coarse adjustment module;

[0023] S2. The image coarse adjustment module receives image data, calculates the image's pixel grayscale distribution in real time, and determines whether the image is overexposed or underexposed. If not, the image data is packaged and sent to the image fine adjustment module. If so, the image integration time is calculated and updated.

[0024] S3. By presetting the grayscale integration time range, it is determined whether the updated integration time is within the expected normal range; if the updated integration time is within the expected normal range, the imaging module works with the updated integration time; if the updated integration time exceeds the normal range, the imaging module works with the original set integration time;

[0025] S4. The image coarse adjustment module packages the exposure-corrected image and sends it to the image fine adjustment module. The image fine adjustment module receives the image data and triggers the calculation of the dimming mode, and performs scene recognition through the deep learning model;

[0026] S5. Check whether the identified scene type is a foggy scene and calculate the foggy ratio. If the foggy ratio exceeds the preset threshold, the current image is determined to be a fog-dominated scene and the exposure parameters are not adjusted. If the foggy ratio does not exceed the preset threshold, the image fine-tuning module is used to perform grayscale evaluation and exposure adjustment to complete automatic light adjustment.

[0027] Preferably, in step S2, the specific method for determining whether the image is overexposed is as follows: using the 90% grayscale maximum value as a standard to determine the proportion of overexposed pixels; if the number of pixels with a grayscale maximum value greater than 90% exceeds 20% of the total number of pixels, the image is considered to be overexposed; an overexposure coefficient is determined to guide the capture of the next frame; the updated integration time in the case of overexposure is:

[0028] ;

[0029] in: is the updated integration time, is the original integration time, is the number of pixels with grayscale maximum value greater than 90%, is the number of pixels in the entire frame image.

[0030] Preferably, in step S2, the specific method for determining whether the image is underexposed is as follows: using the 90% grayscale maximum value as a standard to determine the proportion of underexposed pixels; if the number of pixels with a grayscale maximum value less than 10% exceeds 20% of the total number of pixels, the image is considered to be underexposed, and the underexposure coefficient is determined to guide the capture of the next frame; the updated integration time when underexposed is:

[0031] ;

[0032] in: is the updated integration time, is the original integration time, is the number of pixels with grayscale maximum values ​​less than 10%, is the number of pixels in the entire frame image.

[0033] Preferably, the deep learning model in step S4 includes a MobileNetV3-Large network with deep separable convolution and a multi-scale feature pyramid; the MobileNetV3-Large network with deep separable convolution is used as the backbone network, and the SSD multi-scale feature pyramid is used to extract features in the Conv1, Conv3 or Conv5 layer to output the scene type;

[0034] The scene type is at least one of city, ocean, mountain, desert, forest, grassland, farmland, lake and river, cloud and fog, and starry sky. When the scene type is a complex scene, the number and distribution of various targets are counted through the deep learning model to determine the dominant scene type.

[0035] Preferably, in step S5, the method for calculating the cloud and fog ratio includes: counting the number of pixels belonging to the cloud and fog scene in the image through a deep learning model; calculating the ratio of the number of cloud and fog pixels to the total number of pixels in the entire frame image to obtain the cloud and fog ratio; the preset threshold is 50%.

[0036] Preferably, in step S5, if the cloud and fog ratio in the image does not exceed 50% and the image is a non-cloud and fog-dominated scene type, it is determined whether the grayscale mean is within the expected grayscale threshold range;

[0037] If the image grayscale mean is within the expected grayscale threshold range, the automatic dimming is terminated;

[0038] If the image grayscale mean is not within the expected grayscale threshold range, the grayscale mean adjustment coefficient is calculated to guide the next frame shooting integration time; the updated integration time is:

[0039] ;

[0040] in, is the updated integration time, is the original integration time, is the grayscale mean of the original image, is the target grayscale mean in the discrimination scene.

[0041] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0042] (1) Multi-level real-time on-orbit automatic light adjustment strategy, which realizes initial adjustment of overexposure and underexposure through fast real-time feedback of imaging control FPGA, and realizes precise integration time adjustment based on target scene perception in real time through image processing CPU to avoid cloud and fog interference;

[0043] (2) Improve data quality: By automatically adjusting the integration time, the detector can obtain high-quality data under different working states and environmental conditions. The adjustment of the detector integration time is a dynamic adjustment based on the target scene and details, rather than a mechanical adjustment of the grayscale threshold, avoiding problems such as data blurring and saturation caused by improper integration time.

[0044] (3) Enhanced adaptability: It can quickly adapt to changes in the target scene environment without the need for ground personnel intervention, eliminates cloud and fog interference, improves the detector's response ability to complex target scenes, and expands the detector's application range;

[0045] (4) Reduced costs and risks: This reduces the operational burden on ground personnel, reduces the detection risk caused by delays or errors in ground command transmission, and also reduces mission costs;

[0046] (5) Fault tolerance mechanism: Design a multi-level dimming strategy and protection mechanism to ensure that the system can still output usable images under abnormal conditions.

[0047] In summary, the dimming method proposed in the present invention can be run on the dimming device proposed in the present invention to identify the type of target scene and automatically adjust the integration time of the detector according to the detailed content of the target scene to capture the target image with the best image appearance and details; it is mainly aimed at the need for real-time and precise adjustment of the integration time of aerospace remote sensing cameras on orbit, and adaptively adjusts according to the detailed content of the target scene to calculate the integration time for shooting the best contrast visual appearance and optimal details. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The figure is a schematic structural diagram of an on-orbit multi-stage real-time automatic dimming device for a space area array camera provided according to an embodiment of the present invention.

[0049] Figure 2 The present invention provides a flowchart of a method for on-orbit multi-stage real-time automatic dimming of a space array camera.

[0050] Reference numerals:

[0051] 1. Area array CMOS detector;

[0052] 2. Imaging control FPGA;

[0053] 3. Image processing CPU. DETAILED DESCRIPTION

[0054] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, identical modules are denoted by identical reference numerals. In the case of identical reference numerals, their names and functions are also identical. Therefore, their detailed description will not be repeated.

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0056] The present invention provides an on-orbit multi-stage real-time automatic light adjustment device for a space area array camera, comprising an imaging module, an image coarse adjustment module and an image fine adjustment module;

[0057] The imaging module acquires the original image of the scene according to the set integration time and transmits the image data to the image coarse adjustment module;

[0058] The image coarse adjustment module counts the grayscale distribution of the original image in real time and calculates the first update integration time, and packages the image data without overexposure or underexposure and sends it to the image fine adjustment module;

[0059] After receiving the image, the image fine-tuning module performs scene recognition and grayscale evaluation through a deep learning model, and dynamically adjusts exposure parameters according to the scene type to achieve image exposure control, improving the adaptability of the device and image quality;

[0060] The image data output end of the imaging module is electrically connected to the image data input end of the image coarse adjustment module; the image coarse adjustment module and the image fine adjustment module form a two-stage closed loop, so that the imaging module obtains the integration time of scene matching in real time.

[0061] Specifically, the imaging module includes an area array CMOS detector;

[0062] The image coarse adjustment module statistically analyzes the original image data to obtain grayscale distribution information, and determines whether the image is overexposed or underexposed based on the grayscale distribution information. If the image is not overexposed or underexposed, the image data is packaged and sent to the image fine adjustment module. If overexposed or underexposed is detected, the first update integration time is calculated according to a preset algorithm. The image data including the first update integration time is packaged and sent to the image fine adjustment module via the RapidIO interface.

[0063] The image coarse adjustment module includes an imaging control FPGA (Field-Programmable Gate Array); the high-speed serial interface (RapidIO port) of the image coarse adjustment module is bidirectionally electrically connected to the high-speed serial interface (RapidIO port) of the image fine adjustment module; the integration time control signal output terminal of the image coarse adjustment module is electrically connected to the integration time setting terminal of the imaging module;

[0064] Preferably, the image fine-tuning module includes an image processing CPU; the deep learning model includes a feature extraction unit, a scene recognition unit, a grayscale evaluation unit and an exposure adjustment unit; the feature extraction unit uses a multi-scale feature pyramid to extract features in the Conv1, Conv3 or Conv5 layer of the MobileNetV3-Large network with deep separable convolution; the scene recognition unit is used to identify at least one scene type such as city, ocean, mountain, desert, forest, grassland, farmland, lake and river, cloud, starry sky, etc.; the grayscale evaluation unit is used to determine the corresponding grayscale threshold interval according to the identified scene type; the exposure adjustment unit is used to calculate the second update integration time and verify whether it is within a preset safety range; specifically, the MobileNetV3-Large network is used as the backbone network to extract image features, and deep separable convolution is used in the MobileNetV3-Large network; the multi-scale feature pyramid (Single Shot The module uses a MobileNetV3-Large network based on depthwise separable convolution to extract features at the Conv1, Conv3, or Conv5 layers using the SSD multi-scale feature pyramid. It then outputs at least one scene type: city, ocean, mountain, desert, forest, grassland, farmland, lake or river, fog, or starry sky. When fog is less than 50%, the module calculates the second update integration time based on the scene grayscale threshold range and transmits it back after safety verification.

[0065] In a specific embodiment, if the proportion of fog in an image exceeds 50%, meaning fog is the dominant scene type, the image is considered invalid and cannot guide the dimming strategy. The image processing CPU instructs the imaging control FPGA to use the original integration time as the exposure time for the next frame. In other words, when an image is determined to be invalid (dominated by fog), the automatic dimming process skips this adjustment and continues to use the previous exposure setting.

[0066] If the image is not dominated by fog or cloud, determine whether the grayscale mean is within the expected grayscale threshold range. Different threshold ranges are set according to the type of target dominant scene. The specific threshold range of the scene can be set based on previous images. If the image grayscale mean is within this grayscale threshold range, the automatic light adjustment is terminated. If it is not within this grayscale threshold range, the grayscale mean adjustment coefficient is calculated to guide the integration time of the next frame. The updated integration time is:

[0067] ;

[0068] in, is the updated integration time, is the original integral time, is the original image gray mean value, is the target gray mean value in the identified scene;

[0069] Similarly, the image processing CPU identifies whether the updated integral time is in the expected normal range, if the updated integral time is in the expected normal range, the imaging control FPGA is guided to control the CMOS detector to work according to the updated integral time; if the updated integral time is abnormal, the imaging control FPGA is guided to control the CMOS detector to work according to the original integral time.

[0070] The image data output end of the imaging module is electrically connected with the image data input end of the image coarse adjustment module; the image coarse adjustment module and the image fine adjustment module constitute a two-stage closed loop, so that the imaging module can obtain the integral time matched with the scene in real time; the device further comprises a power management module for supplying power to the imaging control FPGA, the image processing CPU and the area array CMOS detector.

[0071] Specifically, the working process of the device is as follows:

[0072] Starting automatic dimming: the system is started, and the automatic dimming process is started;

[0073] Image acquisition and gray scale statistics: the imaging module acquires an image, and the imaging control FPGA collects the image gray scale distribution;

[0074] Overexposure / underexposure judgment and integral time updating: the imaging control FPGA judges whether the image is overexposed or underexposed; if the image is overexposed or underexposed, a first updated integral time is calculated; otherwise, the original integral time is maintained;

[0075] Data sending: the image and parameter data are sent to the image processing CPU;

[0076] Scene recognition: the image processing CPU runs the MobileNetV3-Large+SSD network to identify the scene type;

[0077] Cloud and fog detection and gray scale evaluation: it is judged whether the image is a cloud and fog scene; if it is not a cloud and fog scene, it is evaluated whether the gray mean value is in the target gray threshold interval;

[0078] Exposure adjustment: if the gray mean value is not in the target interval, a second updated integral time is calculated;

[0079] Integral time verification: it is verified whether the second updated integral time is safe, if it is safe, the integral time is updated; if it is not safe, the original integral time is maintained.

[0080] End of dimming: the automatic dimming process is completed.

[0081] Through the above process, the device can intelligently adapt to different light conditions and scene types, realize accurate image exposure control, and optimize image quality.

[0082] Referring to Figure 1 , the on-orbit multi-stage real-time automatic light adjustment device of the space plane array camera specifically comprises a plane array CMOS detector 1, an imaging control FPGA 2, and an image processing CPU 3; the plane array CMOS detector 1 and the imaging control FPGA 2 are connected bidirectionally through image data and control signals, to realize transmission of image data and sending of exposure control signals; the imaging control FPGA 2 and the image processing CPU 3 are connected unidirectionally through graphic and parameter data, to realize transmission of parameter data and updating of integration time; the image processing CPU 3 updates the integration time according to a processing result, and sends the updated integration time back to the imaging control FPGA 1, to realize closed-loop control; the whole device realizes the on-orbit multi-stage real-time automatic light adjustment function of the space plane array camera through cooperative work of the three modules, can intelligently adapt to different light conditions and scene types, and optimizes exposure and quality of images.

[0083] Based on the above on-orbit multi-stage real-time automatic light adjustment device of the space plane array camera, the present application provides an on-orbit multi-stage real-time automatic light adjustment method of the space plane array camera, and a flow chart is shown in Figure 2 , and specifically comprises the following steps:

[0084] S1. Image acquisition: starting the device; the imaging module acquires a scene original image according to a set integration time, and transmits image data to the image coarse adjustment module.

[0085] S2. Gray scale statistics and exposure time updating: the image coarse adjustment module receives the image data, statistically analyzes pixel gray scale distribution of the image in real time, judges whether the image has overexposure or underexposure, sends the image data to the image fine adjustment module if there is no overexposure or underexposure, and calculates updated integration time if there is overexposure or underexposure.

[0086] The specific method for judging whether the image has overexposure is as follows: taking 90% gray scale maximum value as a standard to judge the proportion of overexposed pixel points; if the pixel points greater than 90% gray scale maximum value account for more than 20% of the total pixel points, the image is considered to have overexposure; an overexposure coefficient is determined to guide shooting of the next frame of image; the updated integration time when overexposure occurs is as follows:

[0087]

[0088] wherein: is the updated integration time, is the original integration time, is the number of pixel points greater than 90% gray scale maximum value, is the number of pixel points of the whole frame of image.​

[0089] The specific method for determining whether an image is underexposed is as follows: the proportion of underexposed pixels is determined using the 90% grayscale maximum value as the standard. If the number of pixels with a grayscale value less than 10% exceeds 20% of the total number of pixels, the image is considered underexposed, and the underexposure coefficient is determined to guide the next frame capture. The updated integration time for underexposure is:

[0090] ;

[0091] in: is the updated integration time, is the original integration time, is the number of pixels with grayscale maximum values ​​less than 10%, is the number of pixels in the entire frame.

[0092] S3. Integration time verification and control: By presetting the grayscale integration time range, determine whether the updated integration time is within the expected normal range; if the updated integration time is within the expected normal range, control the imaging module to operate with the updated integration time; if the updated integration time is abnormal, control the imaging module to operate with the originally set integration time.

[0093] S4. Image Processing and Precision Dimming: The coarse image adjustment module packages the exposure-corrected image and sends it to the fine image adjustment module for fine dimming. The fine image adjustment module triggers the calculation dimming mode and performs scene recognition using a deep learning model.

[0094] Specifically, the deep learning model includes a MobileNetV3-Large network with deep separable convolution and a multi-scale feature pyramid. The MobileNetV3-Large network with deep separable convolution serves as the backbone network, and uses the SSD multi-scale feature pyramid to extract features in the Conv1, Conv3, or Conv5 layers and output the scene type.

[0095] The scene type is at least one of city, ocean, mountain, desert, forest, grassland, farmland, lake and river, cloud, starry sky, etc.; when the scene type is a complex scene, the number and distribution of various targets are counted through the deep learning model to determine the dominant scene type.

[0096] S5. Cloud Detection and Grayscale Assessment: Among the identified scene types, check whether they are cloud-dominated and calculate the cloud-fog ratio. If the cloud-fog ratio exceeds a preset threshold, the image is determined to be cloud-dominated and the exposure parameters are not adjusted. If the cloud-fog ratio does not exceed the preset threshold, grayscale assessment and exposure adjustment are performed using the image fine-tuning module.

[0097] Specifically, different threshold intervals are set according to the type of target-dominated scene; the scene-specific threshold interval can be set based on past prior images;

[0098] Among the identified scene types, special checks are made to determine whether they are cloud and fog scenes. A deep learning model is used to count the number of pixels in the image that belong to cloud and fog scenes. The ratio of the number of cloud and fog pixels to the total number of pixels in the entire frame is calculated to obtain the cloud and fog ratio.

[0099] If the cloud and fog ratio in the image exceeds 50%, the image is considered invalid and cannot guide the dimming strategy. The image processing CPU instructs the imaging control FPGA to use the original integration time as the exposure time for the next frame of the image.

[0100] If the cloud and fog ratio in the image does not exceed 50%, and the image is a non-cloud-dominated scene type, then determine whether the grayscale mean is within the expected grayscale threshold range. If the image grayscale mean is within this grayscale threshold range, then end the automatic light adjustment. If it is not within this grayscale threshold range, calculate the grayscale mean adjustment coefficient to guide the next frame shooting integration time. The updated integration time is:

[0101] ;

[0102] in, is the updated integration time, is the original integration time, is the grayscale mean of the original image, is the target grayscale mean in the discrimination scene;

[0103] The image processing CPU determines whether the updated integration time is within the expected normal range: if the updated integration time is within the expected normal range, it instructs the imaging control FPGA to control the operation of the CMOS detector with the calculated updated integration time; if the updated integration time is abnormal, it instructs the imaging control FPGA to control the operation of the CMOS detector with the original integration time.

[0104] The key technical points of the present invention are as follows:

[0105] (1) A multi-level real-time on-orbit automatic light adjustment strategy adjusts the integration time through multi-level processing of the imaging control FPGA and the image processing CPU, effectively identifies the exposure status of the image, prevents overexposure and underexposure of the image, and finds the optimal integration time for the best image appearance and details;

[0106] (2) A precise adjustment strategy of the integration time based on scene perception is proposed, so that the on-track automatic dimming is no longer a simple adjustment strategy based on the grayscale mean or grayscale median, but a dynamic integration time adjustment mode based on the target scene and target details;

[0107] (3) Can adapt to different environmental changes, and eliminate cloud interference, discriminant calculation exception, so that the integral time adjustment is more accurate, strong robustness.

[0108] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of the present disclosure can be achieved, which is not limited herein.

[0109] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An on-orbit multi-level real-time automatic dimming device for aerospace area array cameras, characterized by: It includes an imaging module, an image coarse adjustment module and an image fine adjustment module; The imaging module acquires the original image of the scene according to the set integration time and transmits the image data to the image coarse adjustment module; The image coarse adjustment module counts the grayscale distribution of the original image in real time and calculates the first update integration time, and packages the image data without overexposure or underexposure and sends it to the image fine adjustment module; After receiving the image, the image fine-tuning module uses a deep learning model to perform scene recognition and grayscale evaluation, and dynamically adjusts exposure parameters according to the scene type to achieve image exposure control; The image data output end of the imaging module is electrically connected to the image data input end of the image coarse adjustment module; the image coarse adjustment module and the image fine adjustment module form a two-stage closed loop, so that the imaging module obtains the integration time of scene matching in real time.

2. The on-orbit multi-level real-time automatic dimming device for aerospace area array cameras according to claim 1, characterized in that: The imaging module includes an array CMOS detector; the image coarse adjustment module includes an imaging control FPGA; the image fine adjustment module includes an image processing CPU; the integration time control signal output end of the image coarse adjustment module is electrically connected to the integration time setting end of the imaging module.

3. The on-orbit multi-level real-time automatic dimming device for aerospace area array cameras according to claim 1, characterized in that: The image coarse adjustment module includes an imaging control FPGA; the image coarse adjustment module statistically analyzes the original image data, obtains grayscale distribution information, and determines whether the image is overexposed or underexposed based on the grayscale distribution information; If the image is not overexposed or underexposed, the image data is packaged and sent to the image fine-tuning module; If overexposure or underexposure is detected, a first update integration time is calculated according to a preset algorithm; and image data including the first update integration time is packaged and sent to the image fine-tuning module.

4. The on-orbit multi-level real-time automatic dimming device for aerospace area array cameras according to claim 1, characterized in that: The image fine-tuning module includes an image processing CPU; the deep learning model includes a feature extraction unit, a scene recognition unit, a grayscale evaluation unit and an exposure adjustment unit; The feature extraction unit extracts features in the Conv1, Conv3 or Conv5 layer of the MobileNetV3-Large network with depthwise separable convolution using a multi-scale feature pyramid; The scene recognition unit is used to identify the scene type; the scene type includes at least one of a city, an ocean, a mountain, a desert, a forest, a grassland, a farmland, a lake or a river, a cloud, and a starry sky; The grayscale evaluation unit is used to determine the corresponding grayscale threshold interval according to the identified scene type; The exposure adjustment unit is used to calculate the second update integration time and verify whether it is within a preset safety range.

5. A method for on-orbit multi-stage real-time automatic dimming of a spacecraft area array camera, using the on-orbit multi-stage real-time automatic dimming device for a spacecraft area array camera according to claim 1 for dimming, characterized in that: The specific steps include: S1 start device; imaging module according to the set integration time to obtain the original image of the scene, and transmits the image data to the image coarse adjustment module; S2. The image coarse adjustment module receives image data, calculates the image's pixel grayscale distribution in real time, and determines whether the image is overexposed or underexposed. If not, the image data is packaged and sent to the image fine adjustment module. If so, the image integration time is calculated and updated. S3. By presetting the grayscale integration time range, it is determined whether the updated integration time is within the expected normal range; if the updated integration time is within the expected normal range, the imaging module works with the updated integration time; if the updated integration time exceeds the normal range, the imaging module works with the original set integration time; S4. The image coarse adjustment module packages the exposure-corrected image and sends it to the image fine adjustment module. The image fine adjustment module receives the image data and triggers the calculation of the dimming mode, and performs scene recognition through the deep learning model; S5. In the identified scene type, check whether it is a foggy scene and calculate the foggy ratio; If the cloud and fog ratio exceeds the preset threshold, the current image is determined to be a cloud-dominated scene and the exposure parameters are not adjusted; If the cloud and fog ratio does not exceed the preset threshold, the image fine-tuning module is used to perform grayscale evaluation and exposure adjustment to complete automatic light adjustment.

6. The on-orbit multi-level real-time automatic dimming method for aerospace area array cameras according to claim 5, characterized in that: In step S2, the specific method for determining whether the image is overexposed is as follows: the proportion of overexposed pixels is determined based on the 90% grayscale maximum value; if the number of pixels with a grayscale maximum value greater than 90% exceeds 20% of the total number of pixels, the image is considered overexposed; an overexposure coefficient is determined to guide the capture of the next frame; the updated integration time for overexposure is: ； in: is the updated integration time, is the original integration time, is the number of pixels with grayscale maximum value greater than 90%, is the number of pixels in the entire frame image.

7. The on-orbit multi-level real-time automatic dimming method for aerospace area array cameras according to claim 5, characterized in that: In step S2, the specific method for determining whether the image is underexposed is as follows: the proportion of underexposed pixels is determined based on the 90% grayscale maximum value; if the number of pixels with a grayscale maximum value less than 10% exceeds 20% of the total number of pixels, the image is considered underexposed, and the underexposure coefficient is determined to guide the capture of the next frame; the updated integration time for underexposure is: ； in: is the updated integration time, is the original integration time, is the number of pixels with grayscale maximum values ​​less than 10%, is the number of pixels in the entire frame image.

8. The on-orbit multi-level real-time automatic dimming method for aerospace area array cameras according to claim 5, characterized in that: The deep learning model in step S4 includes a MobileNetV3-Large network with deep separable convolution and a multi-scale feature pyramid; the MobileNetV3-Large network with deep separable convolution is used as the backbone network, and the SSD multi-scale feature pyramid is used to extract features in the Conv1, Conv3 or Conv5 layer to output the scene type; The scene type is at least one of city, ocean, mountain, desert, forest, grassland, farmland, lake and river, cloud and fog, and starry sky. When the scene type is a complex scene, the number and distribution of various targets are counted through the deep learning model to determine the dominant scene type.

9. The on-orbit multi-level real-time automatic dimming method for aerospace area array cameras according to claim 5, characterized in that: In step S5, the method for calculating the cloud and fog ratio includes: counting the number of pixels belonging to the cloud and fog scene in the image through a deep learning model; calculating the ratio of the number of cloud and fog pixels to the total number of pixels in the entire frame image to obtain the cloud and fog ratio; the preset threshold is 50%.

10. The on-orbit multi-level real-time automatic dimming method for aerospace area array cameras according to claim 9, characterized in that: In step S5, if the cloud and fog ratio in the image does not exceed 50%, and the image is a non-cloud and fog-dominated scene type, it is determined whether the grayscale mean is within the expected grayscale threshold range; If the image grayscale mean is within the expected grayscale threshold range, the automatic dimming is terminated; If the image grayscale mean is not within the expected grayscale threshold range, the grayscale mean adjustment coefficient is calculated to guide the next frame shooting integration time; the updated integration time is: ； in, is the updated integration time, is the original integration time, is the grayscale mean of the original image, is the target grayscale mean in the discrimination scene.

Images