Self-adaptive imaging system based on CMOS detector

Abstract

The invention provides a self-adaptive imaging system based on a CMOS detector. The self-adaptive imaging system comprises a CMOS focal plane circuit, a data processing circuit, an interface circuit and a camera integrated electron. The interface circuit receives a power supply provided by the power supply module, and provides the power supply to the CMOS focal plane circuit and the data processing circuit after filtering conversion; each control signal of the camera integrated electronics is received and sent to the data processing circuit, and telemetering information of the data processing circuit is forwarded to the camera integrated electronics; the CMOS focal plane circuit comprises a CMOS detector; the CMOS detector receives a time sequence driving signal generated by the data processing circuit and drives the CMOS detector to work normally, and the CMOS detector outputs multichannel original image data; the data processing circuit carries out image data automatic alignment on original image data output by the CMOS detector, carries out real-time monitoring in the imaging process, carries out arrangement and preprocessing on the data, carries out target detection and image interception in real time, recognizes a target area image, carries out automatic exposure adjustment on the recognized target area image, and carries out automatic exposure adjustment on the recognized target area image. And outputting the target valid data.

Description

Technical Field

[0001] This invention can be applied to space target information processing and application technology to perform real-time automatic target detection and image capture, separate the target from the background, and perform rapid automatic exposure imaging for effective targets, achieving minimum and stable data transmission bandwidth and optimal image quality. Background Technology

[0002] Currently, lightweight, low-power area array CMOS image detector cameras are widely used in target recognition, deep space exploration, and process monitoring. Due to the complexity of the deep space environment, the irreversibility of the space environment, and the lag in ground processing, as well as the lack of data transmission outside the national land control area, it is necessary to solve the problem of real-time data transmission from overseas, while requiring minimal and stable data transmission bandwidth and clear target images. Therefore, the camera needs to achieve fully adaptive imaging on-board for the target, retaining the original target data without losing any details, and being able to monitor and adjust the multi-channel image data output by the CMOS detector in real time. It should be able to quickly adjust imaging parameters, identify valid targets as soon as possible, and only window out and output the target area to obtain the best imaging effect. At the same time, the size, weight, power consumption, and reliability of the camera need to be carefully considered. The adaptive imaging system needs to be stable, reliable, and simple to implement in both software and hardware.

[0003] Existing on-board adaptive imaging systems have limited adaptive capabilities, relying on automatic exposure of the entire image to achieve adaptive imaging. This type of system cannot completely separate the target from the background, and the image transmission data is large, often involving targets of no interest. Furthermore, target detection and image cropping cannot meet the demands for rapid target identification and real-time imaging. Compared to image compression, automatic target detection and image cropping functions can significantly compress image data while preserving the original target data without loss of information. Summary of the Invention

[0004] The technical problem solved by this invention is to provide an adaptive imaging system based on a CMOS detector that monitors the target in real time during the imaging process.

[0005] The solution of the present invention is: an adaptive imaging system based on a CMOS detector, comprising a CMOS focal plane circuit, a data processing circuit, an interface circuit, and integrated camera electronics; The interface circuit receives power from the power supply module, filters and converts it, and then supplies it to the CMOS focal plane circuit and the data processing circuit; it receives control signals from various channels of the camera integrated electronics and sends them to the data processing circuit, and forwards the telemetry information from the data processing circuit to the camera integrated electronics. The CMOS focal plane circuit includes a CMOS detector; the CMOS detector receives timing drive signals generated by the data processing circuit, drives the CMOS detector to work normally, and the CMOS detector outputs multi-channel raw image data; The data processing circuit automatically aligns the multi-channel raw image data output by the CMOS detector and monitors it in real time during the imaging process. It also arranges the data, performs real-time target detection and image cropping, identifies the target area image, automatically adjusts the exposure of the identified target area image, and then outputs the effective target data.

[0006] Preferably, the data processing circuit includes an uploading refresh module, a DDR3 module, a FLASH program storage module, a data transmission interface module, and an FPGA control processing module; The control signals include CAN bus communication data, uploading refresh data, and second pulse signals. The CAN bus communication data includes GPS broadcast time, exposure time command settings, effective pixel count threshold on / off command settings, automatic exposure grayscale threshold and convergence interval command settings, and target detection grayscale threshold and size threshold command settings. The GPS broadcast time and second pulse signals are used to complete the time calibration and timing functions of the imaging system. The FPGA control processing module parses and executes each control signal. The uploading and refresh module, DDR3 module, and data transmission interface module are connected to the FPGA control and processing module; the FLASH program storage module is connected to the uploading and refresh module; the uploading and refresh module realizes on-rail uploading of the FPGA program and refreshes the FPGA program in real time through RS422 uploading; the DDR3 module realizes ping-pong buffering of raw image data; the FPGA control and processing module generates corresponding timing drive signals according to the initial exposure time of the system default setting, automatically aligns the multi-channel raw image data output by the CMOS detector, monitors it in real time during the imaging process, arranges the data, performs real-time target detection and image cropping, identifies the target area image, automatically adjusts the exposure of the identified target area image, and then outputs the effective target data to the data transmission interface for transmission to the next level system after preprocessing.

[0007] Preferably, the FLASH program storage module stores multiple copies of the FPGA program for backup.

[0008] Preferably, the FPGA control and processing module includes a CMOS detector driving unit, a target detection unit, an automatic exposure unit, and a data output unit; The CMOS detector driving unit generates corresponding timing driving signals to the CMOS focal plane circuit according to the initial exposure time set by the system default, and automatically aligns the received multi-channel raw image data, and sends it to the target detection unit and the automatic exposure unit after arranging it; at the same time, it monitors the image data in real time for inter-frame synchronization codes, and if an anomaly is detected, it performs automatic alignment operation between frames and adjusts the data output in the next frame; The target detection unit processes the received data line by line in real time and inputs the target detection results into the automatic exposure unit and the CMOS detector driving unit. The automatic exposure unit calculates the exposure time of the target area image data based on the received aligned and arranged data and the target detection results, and sends the calculated exposure time to the CMOS detector driving unit. The CMOS detector driving unit generates a corresponding timing driving signal to drive the CMOS detector to image based on the new exposure time calculated by the automatic exposure unit. It automatically aligns and arranges the generated multi-channel raw image data, and performs windowing and cropping of the image data after expanding the target area based on the target detection results. The cropped image data is then preprocessed and output through the data output unit.

[0009] Preferably, automatic alignment includes bit alignment, word alignment, and channel alignment; bit alignment aligns the sampling clock with the data center; word alignment aligns the most significant bit of the image pixels in all channels of the CMOS detector; and channel alignment aligns the pixel data of all channels.

[0010] Preferably, the automatic alignment includes detecting the original image data, finding the left and right edges of the data eye diagram, determining the effective window position, adjusting the delay to half the length of the effective window, so that the sampling point is located at the optimal sampling point of the effective window of the data eye diagram, and completing bit alignment; after bit alignment, comparing whether the data is the same as the synchronization code, if they are different, performing a shift operation until the output is the synchronization code, and completing word alignment; after bit alignment and word alignment, using the slowest transmission channel as a reference, recording the advance time of each channel, and delaying the advance data of each channel in the next frame image, until the last channel completes transmission, the data of multiple channels are read out simultaneously; that is, the aligned image data is obtained; the synchronization code is fixed data set by the CMOS detector output.

[0011] Preferably, the automatic exposure unit processes the following: First, stretch the image data to adjust the image contrast; The target coordinate range output by the target detection unit is read, and the average pixel value of the image data within this range that meets the automatic exposure grayscale threshold parameter is calculated to obtain the effective target image grayscale average value. When calculating the average value, it is simultaneously determined whether the number of effective pixels is greater than or equal to the effective pixel number threshold. If it is less than, the exposure time is adjusted to the maximum exposure time of the system. If the threshold requirement is met, the effective target image grayscale average value is obtained. If the mean grayscale value of the effective target image is within the automatic exposure convergence range, then no exposure time adjustment is performed, and the current exposure time remains unchanged. The CMOS detector driving unit drives the CMOS detector to image according to the current exposure time. If not, then it is further determined whether it is within the transition range. If it is within the transition range, then the current exposure time remains unchanged. If the mean grayscale value of the effective target image exceeds the transition range, then the exposure time is iteratively adjusted until the calculated mean grayscale value of the effective target image converges to the automatic exposure convergence range.

[0012] Preferably, the stretching transformation involves first normalizing the image data, then performing a nonlinear stretching transformation on it, and finally performing mean statistics.

[0013] Preferably, the target detection unit processes the data as follows: The image data after alignment and arrangement of CMOS detector driving units is subjected to downsampling, binarization, dilation, and connected component analysis. Image segmentation is performed based on the set target detection grayscale threshold parameter. Pixels with a DN value greater than or equal to the grayscale threshold are marked as 1, and the rest are marked as 0. Perform dilation processing row by row. For a point marked as 0, if there is a point marked as 1 in its neighborhood (top, bottom, left, right), update the point's mark to 1. Connectivity analysis is performed to form a surface target by grouping pixels belonging to the same connected component in the image. All surface targets are detected. Then, based on the set size threshold parameter, targets that occupy fewer than the size threshold are removed, and the outer envelope of the remaining targets is taken as the target detection result.

[0014] Preferably, the target detection result of the target detection unit is processed as follows: The CMOS detector driving unit drives the detector to image according to the exposure time calculated by the automatic exposure unit. Based on the results of the target detection unit, the target is windowed and cropped at the coordinates of the target above, below, left and right sides after being expanded by 16 pixels. The effective target image data of the windowed target is then transmitted to the data output unit.

[0015] The beneficial effects of this invention compared to the prior art are: Existing onboard adaptive imaging systems have limited adaptive functions, cannot completely separate targets from the background, and transmit large amounts of image data, mostly of targets of no interest. The real-time performance of target detection and image capture cannot meet the requirements for rapid target identification and imaging. Furthermore, the multi-channel raw image data output by the CMOS detector is not monitored in real time during the imaging process. In response to the problem of not being able to quickly adapt to the environment in complex scenes, this invention provides fully autonomous onboard imaging, which is easy to implement in both software and hardware, stable and reliable, and can improve imaging quality and extract effective information.

[0016] This invention enables fully autonomous and intelligent adaptive imaging on satellite, real-time target detection and image capture, rapid automatic exposure imaging of effective targets, removal of areas of no interest, and is used for space exploration. It achieves minimum and stable data transmission bandwidth and optimal image quality, solves the lag of manual intervention, and meets the requirements of real-time performance.

[0017] This invention proposes an adaptive imaging system that adaptively identifies targets, performs adaptive windowing imaging based on the target, and completes local automatic exposure to directly remove background components, making the output image exposure more suitable for visual observation. Simultaneously, it automatically aligns the multi-channel raw image data output from the CMOS detector and monitors it in real time during the imaging process. Through these methods, it avoids reception errors in key spatial detection actions, achieves minimal and stable data transmission bandwidth and optimal image quality, overcomes the lag of manual intervention, avoids the influence of background and noise on exposure calculations, and obtains the target area in real time to calculate the optimal exposure value, ensuring image quality. Attached Figure Description

[0018] Figure 1 This is a block diagram of an adaptive imaging system based on a CMOS detector according to an embodiment of the present invention; Figure 2 This is a block diagram of the automatic alignment strategy for CMOS detector data in an embodiment of the present invention; Figure 3 This is a block diagram of another adaptive imaging system based on a CMOS detector in an embodiment of the present invention; Figure 4 This is a flowchart illustrating the target detection implementation steps in an embodiment of the present invention; Figure 5 This is a flowchart illustrating the automatic exposure implementation steps in an embodiment of the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the embodiments.

[0020] Existing onboard adaptive imaging systems have limited adaptive functions, cannot completely separate targets from the background, and transmit large amounts of image data, often involving targets of little interest. The real-time performance of target detection and image capture cannot meet the demands for rapid target identification and imaging. Furthermore, the multi-channel raw image data output by the CMOS detector is not monitored in real-time during the imaging process. These systems also struggle to adapt quickly to complex environments and address the challenges of real-time data transmission outside national land control areas, requiring solutions for minimal and stable data transmission bandwidth while maintaining clear target images. This paper proposes an adaptive imaging system and method based on a CMOS detector. This system enables fully autonomous onboard imaging, is easy to implement in both software and hardware, and is stable and reliable. It improves imaging quality and extracts effective information, achieving minimal and stable data transmission bandwidth and optimal image quality. It overcomes the lag caused by manual intervention and meets real-time requirements.

[0021] like Figure 1 As shown, an adaptive imaging system based on a CMOS detector includes a CMOS focal plane circuit, a data processing circuit, an interface circuit, and camera integrated electronics. The interface circuit receives power from the power supply module, and after power filtering and conversion by the interface circuit, obtains clean power to supply the CMOS focal plane circuit and the data processing circuit. It receives control signals from various channels of the integrated electronics and sends them to the data processing circuit, and forwards telemetry information from the data processing circuit to the camera integrated electronics. The control signals include CAN bus communication data, second pulse signals, and uploading refresh data. The CAN bus communication data includes GPS broadcast time, exposure time command settings, effective pixel count threshold on / off command settings, automatic exposure grayscale threshold and convergence interval command settings, and target detection grayscale threshold and size threshold command settings. The GPS broadcast time and second pulse signals are used to complete the time calibration and timing functions of the imaging system. The FPGA control processing module parses and executes the various control signals. The CMOS focal plane circuit receives timing drive signals generated by the data processing circuit, driving the CMOS detector to operate normally. The CMOS detector configures itself for imaging according to the timing drive signals and outputs multi-channel raw image data. The data processing circuit automatically aligns the multi-channel raw image data output by the CMOS detector and monitors it in real time during the imaging process. It arranges and preprocesses the raw image data, performs target detection and image cropping in real time, detects target images that meet the target detection grayscale threshold and size threshold in the row direction (H) and column direction (V), removes background noise below the automatic exposure grayscale threshold, automatically adjusts the exposure of the identified target area image, and then outputs the valid target data to the next level system.

[0022] The DC / DC and LDO modules in the interface circuit convert the input power. The RS422 bus communication module and CAN bus communication module receive control signals from the camera's integrated electronics and send them to the data processing circuit, and forward telemetry data from the data processing circuit to the camera's integrated electronics. The upload / refresh module, DDR3 module, and data transmission interface module in the data processing circuit are connected to the FPGA control processing module, and the FLASH program storage module is connected to the upload / refresh module. The upload / refresh module uses RS422 upload to achieve on-orbit upload of the FPGA program and real-time refresh of the FPGA program. Two FLASH modules store four copies of the program. These two modules further ensure the stability and reliability of the adaptive imaging system. The DDR3 module includes two sets of DDR3 chips to achieve continuous, uninterrupted ping-pong buffering and retrieval of high-speed data.

[0023] The data processing circuit parses and executes the control signals of each channel, sends timing drive signals to drive the CMOS detector, and the CMOS detector drive unit receives the raw data from multiple channels of the CMOS focal plane circuit, completes channel alignment, and performs arrangement and preprocessing. Due to differences in the internal channels of the CMOS detector, environmental conditions, and incomplete equal length of circuit board wiring, the data from different channels arrives at the receiving end at inconsistent times. Therefore, it is necessary to first align the data output from the CMOS detector. Data alignment consists of three steps: bit alignment, word alignment, and channel alignment. Bit alignment aligns the sampling clock with the data center to ensure sampling stability. Word alignment aligns the most significant bit of all channel pixels. Channel alignment aligns the pixel data of all channels. Figure 2 As shown, the CMOS detector output is set to a fixed data, which is the synchronization code. The FPGA performs image data detection, finds the left and right edges of the data eye diagram, determines the effective window position, and adjusts the delay to half the length of the effective window so that the sampling point is located at the optimal sampling point within the effective window of the data eye diagram, thus completing bit alignment. After bit alignment, the data is compared with the synchronization code; if they are different, a shift operation is performed until the output is the synchronization code, thus completing word alignment. After bit and word alignment, the advance time of each channel is recorded based on the slowest transmission channel. In the next frame, the advanced data of each channel is delayed until the last channel completes transmission, and then the data from multiple channels is read out simultaneously.

[0024] To avoid the impact of complex conditions such as the space environment on CMOS detectors and FPGAs, a method is proposed to send synchronization codes and monitor them in real time during the imaging process of multi-channel raw image data output by CMOS detectors. The synchronization code words are judged between frames. Based on the detector selected in this implementation scheme and the actual application, if 10 non-synchronization codes are accumulated, the image data is considered abnormal. Automatic data alignment is performed in three steps between frames, and the data output is adjusted in the next frame to avoid imaging abnormalities during the imaging process.

[0025] like Figure 3 As shown, the data processing circuit sends timing drive signals to the CMOS focal plane circuit to drive the CMOS detector. The CMOS detector exposes according to the initial exposure time value (default binding) to obtain raw image data. The data processing circuit first receives the raw image data output by the CMOS detector and inputs it into the CMOS detector driving unit. The CMOS detector driving unit then... Figure 2 After the multi-channel image data is aligned and arranged, it is sent to the target detection unit and the automatic exposure unit.

[0026] Compared to image compression, automatic target detection and image cropping not only significantly compresses image data but also preserves the original target data without losing any detail. This function is crucial for reducing data transmission pressure and improving data timeliness. The target detection unit can remove most of the spatial background influence and eliminate targets that do not meet the criteria based on the set grayscale and size thresholds. This embodiment includes a set of default grayscale and size thresholds for spatial applications. For example, for 8-bit quantized data, considering the characteristics of spatial scene applications, the default grayscale threshold can be set to 15DN, and the default size threshold can be set to 64 pixels (the number of pixels occupied by the target after downsampling). The default grayscale and size thresholds can be modified by command, and the size threshold can be changed according to actual observations.

[0027] The specific operation steps of the target detection unit are as follows: Figure 4As shown, after entering the target detection unit, the first step is downsampling. In this embodiment, 4*4 downsampling is used based on the selected detector to reduce image resolution and improve processing speed. The second step is binarization. Image segmentation is performed according to the set target detection grayscale threshold parameter. Pixels with a DN value greater than or equal to the grayscale threshold are marked as 1, and the rest are marked as 0. To reduce the probability of splitting a target image point into multiple targets, the third step is dilation, which completes the merging of neighboring connected components. For a point marked as 0, if there is a point marked as 1 in its neighborhood (top, bottom, left, right), the point is updated to be marked as 1. The fourth step is connected component analysis. The image data is scanned line by line, and the entire process is scanned only once to improve the real-time performance of the system. Using the run-length extraction method, run information recording, connectivity information judgment, connectivity information merging, and connectivity information extraction are performed simultaneously with the input data. Pixels belonging to the same connected component in the image are grouped into a surface target. In this embodiment, 4-neighborhood analysis is used to detect all surface targets. Then, based on the set size threshold parameter, targets with fewer pixels than the size threshold are removed. The outer envelope of the remaining targets is taken as the target detection result, i.e., the target coordinate range, and input into the automatic exposure unit and CMOS detector driving unit. Dilation processing combined with 4-neighborhood connected component detection analysis enhances target features, reduces image complexity and the impact of noise, and reduces the amount of data that needs to be processed in subsequent analysis.

[0028] The CMOS detector driving unit drives the detector to image according to the exposure time calculated by the automatic exposure unit. Based on the target detection unit's results, it performs image windowing by expanding the target's coordinates by 16 pixels above, below, left, and right. The effective target image data from the windowed area is preprocessed (auxiliary data information, including exposure time and target detection results, is added) and transmitted to the data output unit. When one side of the target's range is less than 16 pixels from the edge of the field of view, the windowing on that side reaches the edge of the field of view. When the image window size is smaller than the original image's 160×160 pixels, a 160×160 pixel window is taken centered on the target. If the target is also located at the edge of the field of view and cannot be centered, 160 pixels are added on the other side. Since the target detection result affects subsequent image frames, this inter-frame motion compensation mechanism is designed to prevent the target from exceeding the effective image data window range due to inter-frame target movement and to confirm the 16-pixel expansion based on the application scenario. When the target detection result is no target, the CMOS detector driving unit outputs the full image data to the data output unit without performing image windowing.

[0029] To avoid overexposure or underexposure of the target area and to ensure simplicity and ease of implementation, the statistical characteristic mean method is used when calculating the image mean. This involves first stretching the entire image to adjust the image contrast, and then calculating the effective image mean. This prevents saturation in high-brightness areas of the image and overexposure of the target area when the contrast between the target and the background is too large or a certain area of ​​the target is too bright. This design achieves a better balance between the background and the target. Afterward, the new exposure time is calculated and output based on the mean and the exposure time of the previous frame.

[0030] The specific steps for automatic exposure are as follows: Figure 5As shown, the automatic exposure unit receives the image data aligned and arranged by the CMOS detector driving unit, reads the target coordinate range output by the target detection unit, and first performs normalization and stretching transformation on the image data within the target coordinate range in the mean calculation module. In this embodiment, a 0.7 power nonlinear stretching transformation is performed on the image data to adjust the image contrast before mean calculation. The mean calculation module calculates the average value of pixels in the image within the target coordinate range that are greater than the grayscale threshold parameter to obtain the effective target image grayscale average value. For example, for 8-bit quantized data, considering the characteristics of spatial scene applications, the grayscale threshold is set to 25DN by default. Considering that an image does not necessarily have to reach the absolutely ideal image brightness, the convergence interval is set to 90DN~128DN by default. The grayscale threshold, convergence interval, and the transition interval mentioned below are all subjected to nonlinear stretching transformation to ensure that all data are compared in the same perceptual space. In this embodiment, a set is bound by default. The exposure time adjustment module iteratively adjusts the exposure time according to the target image grayscale average value of the previous frame and the exposure time of the previous frame until the calculated target image grayscale average value converges to the convergence interval range. Furthermore, if the calculated grayscale mean of the target image is within the convergence interval, the exposure time remains unchanged. If it is not within the convergence interval, the system continues to determine if it is within the transition interval. If it is within the transition interval, the exposure time remains unchanged. If it is not within the transition interval, the exposure time is iteratively adjusted based on the grayscale mean of the target image in the previous frame and the exposure time of the previous frame to obtain a new exposure time, which is then sent to the CMOS detector driving unit. The CMOS detector driving unit generates a corresponding timing drive signal to the CMOS detector, causing the calculated grayscale mean of the target image to converge to the convergence interval. In this embodiment, the transition interval is set to be 16 DN beyond the upper and lower limits of the convergence interval, i.e., 74 DN to 144 DN. The transition interval is set according to the fluctuation range of the image mean and the influence of noise. If this transition interval is not set, when the system increases or decreases the exposure time, affected by light or noise, the system will oscillate repeatedly near the upper and lower limits of the convergence interval, resulting in continuous adjustment of the exposure time and flickering image brightness. That is, when the average gray level of the effective target image exceeds the automatic exposure convergence range, no new adjustment is made within the transition range, and the current exposure time remains unchanged. When the average gray level of the effective target image exceeds the transition range, the next round of adjustment is performed to bring the average gray level of the effective target image back to the convergence range.

[0031] When environmental conditions change, the default value of the automatic exposure grayscale threshold setting may exceed the background average. When the grayscale threshold is set too high, pixels in the target that are below the grayscale threshold will be treated as background. When the image is generally dark, only some overexposed points may be used as effective pixels to calculate the average value. This results in the average effective pixel value still being within the convergence range even if the entire image is dark, manifesting as the entire image being dark but the exposure time not being adjusted. To address this issue, an effective pixel count threshold is designed. When the grayscale threshold is set too high and an effective pixel count threshold is also configured, if the entire image is dark but some pixels are overexposed, and the number of overexposed points is lower than the effective pixel count threshold, the automatic exposure unit will proactively adjust the exposure time to the system's maximum exposure time all at once. Then, it will calculate the automatic exposure time based on the average grayscale of the effective pixels in the adjusted image, avoiding the abnormal problem of failing to adjust the exposure in dark scenes due to overexposed points. When the grayscale threshold is less than or equal to the average value of the entire image under completely dark conditions, the effective pixel average value can normally reflect the average grayscale of the target, and the automatic exposure unit can adjust the exposure time based on the effective pixel average value and the convergence interval. The effective pixel count threshold setting is only applicable when the grayscale threshold is set too high. Set this value according to the observed target. This function can be turned off if selected.

[0032] The CMOS detector driving unit receives the exposure time sent by the automatic exposure unit, controls the acquisition of a new frame image, receives the target detection result from the target detection unit, expands and captures the image data, adds auxiliary data information to the effective target image data after windowing and sends it to the data output unit, and outputs the image data to the next level system through the data transmission interface to complete adaptive imaging.

[0033] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

Claims

1. An adaptive imaging system based on a CMOS detector, characterized in that... This includes CMOS focal plane circuitry, data processing circuitry, interface circuitry, and integrated camera electronics. The interface circuit receives power from the power supply module, filters and converts it, and then supplies it to the CMOS focal plane circuit and the data processing circuit; it receives control signals from various channels of the camera integrated electronics and sends them to the data processing circuit, and forwards the telemetry information from the data processing circuit to the camera integrated electronics. The CMOS focal plane circuit includes a CMOS detector; the CMOS detector receives timing drive signals generated by the data processing circuit, drives the CMOS detector to work normally, and the CMOS detector outputs multi-channel raw image data; The data processing circuit automatically aligns the multi-channel raw image data output by the CMOS detector and monitors it in real time during the imaging process. It also arranges the data, performs real-time target detection and image cropping, identifies the target area image, automatically adjusts the exposure of the identified target area image, and then outputs the effective target data.

2. The system according to claim 1, characterized in that: The data processing circuit includes an uploading refresh module, a DDR3 module, a FLASH program storage module, a data transmission interface module, and an FPGA control and processing module. The control signals include CAN bus communication data, uploading refresh data, and second pulse signals. The CAN bus communication data includes GPS broadcast time, exposure time command settings, effective pixel count threshold on / off command settings, automatic exposure grayscale threshold and convergence interval command settings, and target detection grayscale threshold and size threshold command settings. The GPS broadcast time and second pulse signals are used to complete the time calibration and timing functions of the imaging system. The FPGA control processing module parses and executes each control signal. The uploading refresh module, DDR3 module, and data transmission interface module are connected to the FPGA control and processing module; the FLASH program storage module is connected to the uploading refresh module. The upload and refresh module uses RS422 upload to upload the FPGA program on the rail and refresh the FPGA program in real time; the DDR3 module implements ping-pong buffering of raw image data; the FPGA control and processing module generates corresponding timing drive signals according to the initial exposure time of the system default setup, automatically aligns the multi-channel raw image data output by the CMOS detector, monitors it in real time during the imaging process, arranges the data, performs real-time target detection and image cropping, identifies the target area image, automatically adjusts the exposure of the identified target area image, and then outputs the preprocessed target data to the data transmission interface for transmission to the next level system.

3. The system according to claim 2, characterized in that: The FLASH program storage module stores multiple copies of the FPGA program for backup.

4. The system according to claim 2, characterized in that: The FPGA control and processing module includes a CMOS detector driving unit, a target detection unit, an automatic exposure unit, and a data output unit; The CMOS detector driving unit generates corresponding timing driving signals to the CMOS focal plane circuit according to the initial exposure time set by the system default, and automatically aligns the received multi-channel raw image data, and sends it to the target detection unit and the automatic exposure unit after arranging it; at the same time, it monitors the image data in real time for inter-frame synchronization codes, and if an anomaly is detected, it performs automatic alignment operation between frames and adjusts the data output in the next frame; The target detection unit processes the received data line by line in real time and inputs the target detection results into the automatic exposure unit and the CMOS detector driving unit. The automatic exposure unit calculates the exposure time of the target area image data based on the received aligned and arranged data and the target detection results, and sends the calculated exposure time to the CMOS detector driving unit. The CMOS detector driving unit generates a corresponding timing driving signal to drive the CMOS detector to image based on the new exposure time calculated by the automatic exposure unit. It automatically aligns and arranges the generated multi-channel raw image data, and performs windowing and cropping of the image data after expanding the target area based on the target detection results. The cropped image data is then preprocessed and output through the data output unit.

5. The system according to claim 4, characterized in that: Automatic alignment includes bit alignment, word alignment, and channel alignment; bit alignment aligns the sampling clock with the data center; word alignment aligns the most significant bits of the image pixels across all channels of the CMOS detector. Channel alignment aligns the pixel data across all channels.

6. The system according to claim 5, characterized in that: The automatic alignment includes detecting the original image data, finding the left and right edges of the data eye diagram, determining the effective window position, adjusting the delay to half the length of the effective window, so that the sampling point is located at the optimal sampling point of the effective window of the data eye diagram, and completing bit alignment; after bit alignment, comparing whether the data is the same as the synchronization code, if they are different, performing a shift operation until the output is the synchronization code, and completing word alignment; After bit alignment and word alignment, the advance time of each channel is recorded based on the slowest transmission channel. In the next frame of the image, the advance data of each channel is delayed. After the last channel completes transmission, the data of multiple channels are read out simultaneously; that is, the aligned image data is obtained. The synchronization code is a fixed data set by the CMOS detector output.

7. The system according to claim 4, characterized in that: The automatic exposure unit processes the data as follows: First, stretch the image data to adjust the image contrast; Read the target coordinate range output by the target detection unit, calculate the average pixel value of the image data that meets the automatic exposure grayscale threshold parameter within the range, and obtain the effective target image grayscale average value; When calculating the mean, it is simultaneously determined whether the number of effective pixels is greater than or equal to the threshold of the number of effective pixels. If it is less than, the exposure time is adjusted to the maximum exposure time of the system. If the threshold requirement is met, the mean gray value of the effective target image is obtained. If the mean grayscale value of the effective target image is within the automatic exposure convergence range, then no exposure time adjustment is performed, and the current exposure time remains unchanged. The CMOS detector driving unit drives the CMOS detector to image according to the current exposure time. If not, then it is further determined whether it is within the transition range. If it is within the transition range, then the current exposure time remains unchanged. If the mean grayscale value of the effective target image exceeds the transition range, then the exposure time is iteratively adjusted until the calculated mean grayscale value of the effective target image converges to the automatic exposure convergence range.

8. The system according to claim 7, characterized in that: The stretching transformation involves first normalizing the image data, then performing a nonlinear stretching transformation on it, and finally calculating the mean.

9. The system according to claim 4, characterized in that: The target detection unit processes the data as follows: The image data after alignment and arrangement of CMOS detector driving units is subjected to downsampling, binarization, dilation, and connected component analysis. Image segmentation is performed based on the set target detection grayscale threshold parameter. Pixels with DN values ​​greater than or equal to the grayscale threshold are marked as 1, and the rest are marked as 0. Perform dilation processing row by row. For a point marked as 0, if there is a point marked as 1 in its neighborhood, update the point's mark to 1. Connectivity analysis is performed to form a surface target by grouping pixels belonging to the same connected component in the image. All surface targets are detected. Then, based on the set size threshold parameter, targets that occupy fewer than the size threshold are removed, and the outer envelope of the remaining targets is taken as the target detection result.

10. The system according to claim 9, characterized in that: The target detection results of the target detection unit are processed as follows: The CMOS detector driving unit drives the detector to image according to the exposure time calculated by the automatic exposure unit. Based on the results of the target detection unit, the target is windowed and cropped at the coordinates of the same number of pixels above, below, left and right. The effective target image data of the windowed target is then transmitted to the data output unit.

Images