Satellite-borne camera real-time adaptive imaging method based on target detection

Abstract

The invention discloses a satellite-borne camera real-time adaptive imaging method based on target detection, and belongs to the technical field of space-based information on-satellite processing. From the perspective of satellite detection imaging engineering, imaging parameters of corresponding detection modules are adjusted based on the change condition of the exposure information amount of a frame anchor image where an on-satellite target detection result is located according to the characteristic of large change of the radiation intensity of a detection dynamic target; wherein in consideration of the fact that the gradient value of a target edge pixel area is large and the gradient value of a pixel area in a target contour is small, in order to balance the importance of a weak gradient and a strong gradient, a logarithmic function is used for mapping the relation between the gradient amplitude and the gradient information amount, and therefore the accuracy amount of the exposure information amount of the frame anchor image is achieved; based on this, the method can adapt to the change of the radiation intensity of the target in real time, thereby not only guaranteeing the effective detection of the dark and weak target, but also avoiding the detection saturation, and achieving the imaging of complete target characteristics.

Description

Technical Field

[0001] This invention belongs to the field of space-based information on-board processing technology, and more specifically, relates to a real-time adaptive imaging method for spaceborne cameras based on target detection. Background Technology

[0002] Regarding target detection capabilities, the increasing dynamic range of the radiant energy of detected objects poses a challenge to the target energy dynamic range of satellite systems. For some targets, the radiant energy can change by tens of thousands of times within minutes, and the current dynamic range of spaceborne cameras under a single integration time is often insufficient to cope with such rapid energy changes over a wide range. Therefore, researching a real-time adaptive imaging method for spaceborne cameras is of great significance.

[0003] In existing real-time adaptive imaging methods for spaceborne cameras, adaptive adjustment of camera parameters is a key step in achieving high-quality imaging. Patent CN117454080A proposes a method for adjusting satellite payload gain by dividing the image into grids based on Earth's latitude and longitude. This method requires pre-classifying different grids according to land cover types (water, vegetation, snow, etc.) and fixing the payload gain to these types. However, this method cannot adjust the gain for targets that change in real time. Patent CN110689505A proposes a scene-based adaptive correction method for spaceborne remote sensing instruments, but this method is only applicable to radiometric calibration correction and cannot be used for adaptive imaging. Patent CN106791508A proposes a method for adjusting the imaging quality of a digital domain TDI camera. This method sets imaging parameters by adjusting the average grayscale value of the pre-captured area, but it cannot set imaging parameters based on the target being detected. Patent CN115727856A proposes an adaptive correction method for satellite remote sensing images. This method performs coordinate transformation on the image using known landmark locations, but it cannot be used for adaptive adjustment of imaging parameters. Patent CN111770283A proposes a method for adjusting the gain by comparing the image brightness with a preset brightness, but this method cannot adjust the gain based on the target in the image. Summary of the Invention

[0004] To address the above-mentioned deficiencies or improvement needs of existing technologies, this invention provides a real-time adaptive imaging method for spaceborne cameras based on target detection. The purpose is to adaptively adjust the imaging parameters of the spaceborne camera based on the on-board target detection results, so as to adapt to changes in target radiation intensity in real time, thereby ensuring effective detection of faint targets and avoiding detection saturation, and achieving imaging of complete target characteristics.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a real-time adaptive imaging method for a spaceborne camera based on target detection, comprising: Target detection is performed on the current frame image captured by each detection module in the spaceborne camera. The area containing the background is then cropped with the center of the detected target as the center. A sub-image of a certain size is used as the current frame anchor image for the corresponding detection module, and the corresponding exposure information is calculated; where, It is a positive integer; the exposure information is the sum of the gradient information at all pixels in the corresponding anchor image; for any pixel i in the anchor image, when the gradient magnitude at that pixel is... Less than the preset threshold At that time, the amount of gradient information at that location The gradient magnitude at that pixel is 0. Greater than or equal to the preset threshold At that time, the amount of gradient information at that location ; These are preset coefficients; ; The imaging parameters of each detection module in each onboard camera are adaptively adjusted based on the exposure information of the corresponding current frame anchor image to achieve imaging: Specifically, when the exposure information of the current frame anchor image of any detection module is greater than the exposure information of the previous frame anchor image of that detection module, the integration time of that detection module is increased. and / or circuit gain ; When the exposure information of the current frame anchor image of any detection module is less than the exposure information of the previous frame anchor image of that detection module, the integration time of that detection module is reduced. and / or circuit gain ; When the exposure information of the current frame anchor image of any detection module is equal to the exposure information of the previous frame anchor image of that detection module, the integration time of that detection module is maintained. and circuit gain constant.

[0006] More preferably, the above-mentioned real-time adaptive imaging method for spaceborne cameras based on target detection further includes: When the target radiation intensity in the current frame anchor image of any detection module is less than the first preset intensity, during the integration time of that detection module... After reaching the maximum integration time, increase the circuit gain of the detection module. Among them, the maximum integration time ; and These are the pixel size and focal length of the detection module, respectively; This represents the distance from the target in the current frame anchor image of the detection module to the detection module. This represents the velocity of the target in the current frame anchor image.

[0007] More preferably, the above-mentioned real-time adaptive imaging method for spaceborne cameras based on target detection further includes: When the target radiation intensity in the current frame anchor image of any detection module is greater than the second preset intensity, the integration time of that detection module is adjusted. and circuit gain The following relationship must be satisfied:

[0008] Wherein, the second preset strength is greater than or equal to the first preset strength; The photocurrent value is obtained by converting the target radiation intensity in the current frame anchor image of the detection module; This refers to the dark current of the detection module; This refers to the Poisson noise of the optical signal in the detection module. This is the capacitance value of the pixel-level integrating capacitor of the detection module; At the end of the exposure process of the detection module, the capacitance value of the pixel-integrating capacitor of the detection module is the voltage value accumulated on the pixel-integrating capacitor when the full well capacity is reached.

[0009] More preferably, the aforementioned spaceborne camera is a satellite array camera.

[0010] More preferably, the method for target detection of the current frame image captured by the detection module in the spaceborne camera includes: Background suppression, threshold segmentation, and spatial connectivity processing are sequentially performed on the current frame image captured by the detection module in the spaceborne camera and the consecutive frames preceding the current frame image to obtain the suspected targets in each frame image. Then, a multi-target correlation tracking algorithm is used to generate the target trajectory of the detected target to eliminate the interference of false targets, thereby obtaining the target region in the current frame image captured by the detection module and the target velocity.

[0011] In a second aspect, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the real-time adaptive imaging method for a spaceborne camera provided in the first aspect of the present invention.

[0012] Thirdly, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein the computer program, when executed by a processor, controls the device containing the storage medium to perform the real-time adaptive imaging method for a spaceborne camera provided in the first aspect of the present invention.

[0013] Fourthly, the invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the real-time adaptive imaging method for a spaceborne camera provided in the first aspect of the invention.

[0014] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects: This invention, from the perspective of satellite detection and imaging engineering, addresses the challenge of detecting dynamic targets with large variations in radiation intensity. It provides a real-time adaptive imaging method for spaceborne cameras based on target detection. The method adjusts the imaging parameters of the corresponding detection module based on the exposure information of the anchor image containing the onboard target detection results. Considering that gradient values ​​are larger in the target edge pixel region and smaller in the pixel region within the target outline, a logarithmic function is used to map the relationship between gradient magnitude and gradient information content to balance the importance of weak and strong gradients. This achieves accurate measurement of the exposure information content of the anchor image. Based on this, the invention can adapt to changes in target radiation intensity in real time, ensuring effective detection of faint targets while avoiding detection saturation, thus achieving imaging of complete target characteristics. Attached Figure Description

[0015] Figure 1 A hardware block diagram of a satellite real-time adaptive imaging method based on target detection provided in an embodiment of the present invention; Figure 2 This is the on-board processing, detection, and target locking process provided in this embodiment of the invention; Figure 3 This is a flowchart of a real-time adaptive imaging process for a spaceborne camera based on target detection, provided in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0017] To achieve the above objectives, in a first aspect, the present invention provides a real-time adaptive imaging method for a spaceborne camera based on target detection, comprising: Target detection is performed on the current frame image captured by each detection module in the spaceborne camera. The area containing the background is then cropped with the center of the detected target as the center. A sub-image of a certain size is used as the current frame anchor image for the corresponding detection module, and the corresponding exposure information is calculated; where, It is a positive integer; the exposure information is the sum of the gradient information at all pixels in the corresponding anchor image; for any pixel i in the anchor image, when the gradient magnitude at that pixel is... Less than the preset threshold At that time, the amount of gradient information at that location The gradient magnitude at that pixel is 0. Greater than or equal to the preset threshold At that time, the amount of gradient information at that location ; These are preset coefficients; ; The imaging parameters of each detection module in each onboard camera are adaptively adjusted based on the exposure information of the corresponding current frame anchor image to achieve imaging: Specifically, when the exposure information of the current frame anchor image of any detection module is greater than the exposure information of the previous frame anchor image of that detection module, the integration time of that detection module is increased. and / or circuit gain ; When the exposure information of the current frame anchor image of any detection module is less than the exposure information of the previous frame anchor image of that detection module, the integration time of that detection module is reduced. and / or circuit gain ; When the exposure information of the current frame anchor image of any detection module is equal to the exposure information of the previous frame anchor image of that detection module, the integration time of that detection module is maintained. and circuit gain constant.

[0018] In one alternative implementation, the above-mentioned real-time adaptive imaging method for spaceborne cameras based on target detection further includes: When the target radiation intensity in the current frame anchor image of any detection module is less than the first preset intensity, during the integration time of that detection module... After reaching the maximum integration time, increase the circuit gain of the detection module. Among them, the maximum integration time ; and These are the pixel size and focal length of the detection module, respectively; This represents the distance from the target in the current frame anchor image of the detection module to the detection module. This represents the velocity of the target in the current frame anchor image.

[0019] In one alternative implementation, the above-mentioned real-time adaptive imaging method for spaceborne cameras based on target detection further includes: When the target radiation intensity in the current frame anchor image of any detection module is greater than the second preset intensity, the integration time of that detection module is adjusted. and circuit gain The following relationship must be satisfied:

[0020] Wherein, the second preset strength is greater than or equal to the first preset strength; The photocurrent value is obtained by converting the target radiation intensity in the current frame anchor image of the detection module; This refers to the dark current of the detection module; This refers to the Poisson noise of the optical signal in the detection module. This is the capacitance value of the pixel-level integrating capacitor of the detection module; At the end of the exposure process of the detection module, the capacitance value of the pixel-integrating capacitor of the detection module is the voltage value accumulated on the pixel-integrating capacitor when the full well capacity is reached.

[0021] In one alternative implementation, the aforementioned spaceborne camera is a satellite area array camera.

[0022] In one alternative implementation, the method for target detection of the current frame image captured by the detection module in the spaceborne camera includes: Background suppression, threshold segmentation, and spatial connectivity processing are sequentially performed on the current frame image captured by the detection module in the spaceborne camera and the consecutive frames preceding the current frame image to obtain the suspected targets in each frame image. Then, a multi-target correlation tracking algorithm is used to generate the target trajectory of the detected target to eliminate the interference of false targets, thereby obtaining the target region in the current frame image captured by the detection module and the target velocity.

[0023] To further illustrate the real-time adaptive imaging method for a spaceborne camera provided in the first aspect of the present invention, a detailed description is provided below with reference to a specific embodiment: This embodiment employs a method that adaptively adjusts the imaging parameters (gain / integration time) of the onboard camera based on the onboard target detection processing results, thereby adapting to changes in target radiation intensity in real time. For example... Figure 1 As shown, the detected images undergo real-time target detection, and high-probability targets are extracted through multi-frame correlation. The characteristics of the suspected targets and the detection system are analyzed, the gradient values ​​of the target region and the adjacent background region are calculated, and the optimal adaptive parameters are searched for. Adjustment commands based on the imaging parameters are sent to the camera. The adaptive imaging mechanism exits after the target disappears. For satellite array cameras, each detector module can perform multi-target adaptive imaging. The specific implementation includes the following steps: 1) Target whole-frame sorting selection strategy based on confidence The satellite performs real-time multi-frame correlation target detection and extracts highly probable targets from the camera-detected images. The onboard processing then sorts the detected targets in the current frame according to their confidence level, selecting the target with the highest confidence level exceeding a set threshold as the basis for adaptive imaging adjustments. For example... Figure 2 As shown, the detected image is used to find suspected targets through background suppression, threshold segmentation, and spatial connectivity. The target motion trajectory is obtained by multi-frame correlation and the interference of false alarm targets is eliminated to obtain highly suspected targets.

[0024] Eliminating false alarm interference: Confidence scores are used to indicate the probability that a detected target is a real target; a higher score indicates a greater probability, and vice versa. Confidence scores are obtained based on characteristics such as the speed, energy, and size of detected potential targets. A frame-by-frame update and whole-frame sorting strategy is employed for confidence scores. The confidence scores of detected potential targets within a frame are sorted, and the highest-scoring potential target exceeding a threshold is selected and used as the basis for adaptive parameter tuning.

[0025] 2) Imaging parameter adjustment strategy based on target and detection system characteristics The quantization relationship between target radiation intensity and integration time / circuit gain is as follows:

[0026] Where Q is the number of electrons and gain is the circuit gain. The photocurrent converted from the target radiation intensity. For optical signal Poisson noise, C is the dark current of the detector itself, C is the integrating capacitance within the pixel, and U is the integrated voltage read out by quantization.

[0027] When the integrated voltage U is quantized into a digital signal N, the ADC also introduces noise. Assuming the detector does not introduce noise, the limiting signal-to-noise ratio of the acquired image is:

[0028] Readout noise from the detector This can be calculated from a stationary background and radiometric calibration. When the signal strength equals the square of the root mean square of the readout noise, the Poisson noise of the optical signal and the readout noise contribute equally to the image noise. When adjusting the imaging parameters of the detection system, the quantized signal... (i.e., target radiation intensity) should ideally satisfy the following relationship to ensure image quality:

[0029] Based on the speed of detecting suspected targets v Energy e, size s, distance d Features, and the focal length of the detection system. f Pixel sizep Using parameters such as these, the maximum integration time that prevents motion blur in the imaging of the suspected target is calculated. t .

[0030]

[0031] For fast-moving targets with low radiation intensity, i.e., when the target radiation intensity in the current frame of the anchor image of any detection module is less than a first preset intensity, when adjusting imaging parameters, the circuit gain is preferentially increased after the integration time reaches the maximum integration time. In this embodiment, the aforementioned first preset intensity is... At the target radiation intensity Less than At that time, after the integration time reaches the maximum integration time, the circuit gain is increased to increase the target radiation intensity. Greater than or equal to .

[0032] For targets with high radiation intensity, that is, when the radiation intensity of the target in the current frame anchor image of any detection module is greater than the second preset intensity (in this embodiment, the value is taken as...), ... When adjusting imaging parameters, it is required that the detector pixels not be saturated, which is generally limited to 80% of the full-well capacity, i.e.:

[0033] 3) Gradient-based adaptive imaging parameter adjustment Setting a fixed average value as a reference for whether the image exposure is appropriate or not, the algorithm struggles to achieve appropriate and rapid exposure for the target area under special lighting conditions such as high radiation intensity and rapid movement of space-based targets.

[0034] Using gradient as a standard for exposure evaluation, an image is considered to have a small or no gradient in overexposed / underexposed areas, while a well-exposed image also has a significant gradient. The image gradient exhibits a monotonically increasing and then monotonically decreasing trend with respect to parameters. For example... Figure 3 As shown, the imaging parameter search can start with a low exposure time and gain, and then be adjusted towards the peak in a certain step size until the gradient value of the target region in the next step is less than that in the current step, that is, the optimal imaging parameters have been found.

[0035] For each frame of the anchor image, its gradient value at each pixel i is:

[0036] in, and These are the gradients in the horizontal and vertical directions, respectively.

[0037] The gradient values ​​in the pixel regions at the edge of the target will be larger, while the gradient values ​​in the pixel regions within the target contour will be smaller. To balance the importance of weak and strong gradients, this embodiment uses a logarithmic function to map the relationship between gradient magnitude and gradient information content.

[0038]

[0039] in, , It is a pixel The gradient magnitude at that point shows that the gradient is less than the threshold. When the gradient magnitude is zero, the gradient magnitude is mapped to zero.

[0040] parameter Determine the activation threshold: the mapping function will be less than The gradient value is considered noise and ignored. Parameters The mapping trend is determined by... Set to a smaller value to emphasize the contrast between the target edge and the background, or by... A larger value is set to emphasize the variation in radiation intensity within the target contour. In this embodiment, The value is 10.

[0041] Based on this, the amount of information exposed is:

[0042] The larger the value of M, the better the exposure effect and the more information contained in the target area.

[0043] 4) Adaptive parameter adjustment by region according to detector module Satellite array cameras typically employ multi-detector module stitching to achieve a large field of view. When the detection system detects multiple suspected targets in the same scene, N detector modules are selected centered on each suspected target. For each detector module, an independent adaptive parameter adjustment is performed on the pixel region N containing the background. Image acquisition and output are performed in cycles of low frame rate outputting full-frame images and high frame rate outputting target region images, which improves the target data acquisition frame rate and solves the impact of highly reflective cloud areas, strong landmarks, and celestial bodies on the weighting of imaging parameters.

[0044] 5) Set up a target locking and exit mechanism. Object detection requires the accumulation of background images over a certain period of time for background suppression. Therefore, a time threshold is set to exit the adaptive parameter tuning mechanism after the target disappears for a certain period of time, thus avoiding interruption of background accumulation and object detection.

[0045] In summary, this embodiment, from the perspective of satellite detection and imaging engineering, proposes a real-time adaptive imaging method based on target detection, addressing the characteristic of large variations in the radiation intensity of detected dynamic targets. First, the onboard processing receives images from the satellite's detection camera and performs multi-frame associated target detection and extraction of highly suspected targets in real time. The onboard processing then sorts the detected targets in the current frame image according to their confidence level, selecting the target with the highest confidence level exceeding a set threshold as the basis for adaptive imaging adjustment. Based on the characteristics of the suspected target and the detection system, different imaging parameter adjustment strategies are selected. The onboard processing then judges the gradient value of the target and its neighboring region. When the gradient value is higher than the previous frame, a request to increase the imaging parameters (gain / integration time) by one level is issued; when the gradient value is lower than the previous frame, a request to decrease the imaging parameters (gain / integration time) by one level is issued. No adjustment is made when the gradient value is at its peak. Simultaneously, the onboard processing generates imaging parameter adjustment instructions based on the adjustment requests and sends them to the satellite maintenance team via the 1553B bus, which in turn sends them to the camera. The camera executes the instructions and completes the imaging parameter adjustment.

[0046] In a second aspect, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the real-time adaptive imaging method for a spaceborne camera provided in the first aspect of the present invention.

[0047] The related technical solutions are the same as the real-time adaptive imaging method for spaceborne cameras provided in the first aspect of this invention, and will not be described in detail here.

[0048] Thirdly, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein the computer program, when executed by a processor, controls the device containing the storage medium to perform the real-time adaptive imaging method for a spaceborne camera provided in the first aspect of the present invention.

[0049] The related technical solutions are the same as the real-time adaptive imaging method for spaceborne cameras provided in the first aspect of this invention, and will not be described in detail here.

[0050] Fourthly, the invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the real-time adaptive imaging method for a spaceborne camera provided in the first aspect of the invention.

[0051] The related technical solutions are the same as the real-time adaptive imaging method for spaceborne cameras provided in the first aspect of this invention, and will not be described in detail here.

[0052] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A real-time adaptive imaging method for a spaceborne camera based on target detection, characterized in that, include: Target detection is performed on the current frame image captured by each detection module in the spaceborne camera. The area containing the background is then cropped with the center of the detected target as the center. A sub-image of a certain size is used as the current frame anchor image for the corresponding detection module, and the corresponding exposure information is calculated; where, The value is a positive integer; the exposure information is the sum of the gradient information at all pixels in the corresponding anchor image; for any pixel i in the anchor image, when the gradient magnitude at that pixel is... Less than the preset threshold At that time, the amount of gradient information at that location The gradient magnitude at that pixel is 0. Greater than or equal to the preset threshold At that time, the amount of gradient information at that location ; These are preset coefficients; ; The imaging parameters of each detection module in each onboard camera are adaptively adjusted based on the exposure information of the corresponding current frame anchor image to achieve imaging: Specifically, when the exposure information of the current frame anchor image of any detection module is greater than the exposure information of the previous frame anchor image of that detection module, the integration time of that detection module is increased. and / or circuit gain ; When the exposure information of the current frame anchor image of any detection module is less than the exposure information of the previous frame anchor image of that detection module, the integration time of that detection module is reduced. and / or circuit gain ; When the exposure information of the current frame anchor image of any detection module is equal to the exposure information of the previous frame anchor image of that detection module, the integration time of that detection module is maintained. and circuit gain constant.

2. The real-time adaptive imaging method for a spaceborne camera according to claim 1, characterized in that, Also includes: When the target radiation intensity in the current frame anchor image of any detection module is less than the first preset intensity, during the integration time of that detection module... After reaching the maximum integration time, increase the circuit gain of the detection module. Among them, the maximum integration time ; and These are the pixel size and focal length of the detection module, respectively; This represents the distance from the target in the current frame anchor image of the detection module to the detection module. This represents the velocity of the target in the current frame anchor image.

3. The real-time adaptive imaging method for a spaceborne camera according to claim 1, characterized in that, Also includes: When the target radiation intensity in the current frame anchor image of any detection module is greater than the second preset intensity, the integration time of that detection module is adjusted. and circuit gain The following relationship must be satisfied: Wherein, the second preset strength is greater than or equal to the first preset strength; The photocurrent value is obtained by converting the target radiation intensity in the current frame anchor image of the detection module; This refers to the dark current of the detection module; This refers to the Poisson noise of the optical signal in the detection module. This is the capacitance value of the pixel-level integrating capacitor of the detection module; At the end of the exposure process of the detection module, the capacitance value of the pixel-integrating capacitor of the detection module is the voltage value accumulated on the pixel-integrating capacitor when the full well capacity is reached.

4. The real-time adaptive imaging method for a spaceborne camera according to any one of claims 1-3, characterized in that, The onboard camera is a satellite array camera.

5. The real-time adaptive imaging method for a spaceborne camera according to any one of claims 1-3, characterized in that, Methods for target detection in the current frame image captured by the detection module in the spaceborne camera include: Background suppression, threshold segmentation, and spatial connectivity processing are sequentially performed on the current frame image captured by the detection module in the spaceborne camera and the consecutive frames preceding the current frame image to obtain the suspected targets in each frame image. Then, a multi-target correlation tracking algorithm is used to generate the target trajectory of the detected target to eliminate the interference of false targets, thereby obtaining the target region in the current frame image captured by the detection module and the target velocity.

6. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the real-time adaptive imaging method for a spaceborne camera according to any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein when the computer program is run by a processor, it controls the device containing the storage medium to execute the real-time adaptive imaging method of the spaceborne camera according to any one of claims 1-5.

8. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by a processor, implements the real-time adaptive imaging method for a spaceborne camera as described in any one of claims 1-5.

Images