An intelligent information processing device for a spaceborne micro-light remote sensor

Abstract

The application discloses a kind of intelligent information processing devices of spaceborne micro-light remote sensor, for the imaging control and optimal focal plane searching of visible micro-light remote sensing camera. For the information flow processing of visible micro-light remote sensing camera, combined with orbit and satellite attitude information, timing imaging with sub-second control accuracy is realized, the focusing mechanism is controlled to reach the optimal focal plane position before imaging, and the imaging parameters, windowing and on-orbit image correction algorithm of the micro-light detector are autonomously regulated and controlled, which improves the efficiency of bus data usage and the proportion of effective image data on orbit.

Description

TECHNICAL FIELD

[0001] The application relates to a kind of intelligent information processing devices of spaceborne micro-light remote sensor, and belongs to the technical field of space remote sensor. BACKGROUND

[0002] Modern space remote sensing camera functions are increasingly complex, and algorithms are increasingly diverse. For the current micro-light remote sensor, the instruction is much, and the image data volume is large. The traditional information processing mechanism needs a large number of ground instructions to interfere, and a large amount of redundant data is generated, which occupies the valuable transmission bandwidth between the star and the ground, and reduces the use efficiency of the remote sensor.

[0003] With the complication of functions, the camera parameter instruction set tends to be large and complex, and the constraint relationship between the instructions becomes more complex. Simply relying on the artificial judgment, screening and avoiding of the constraint to send instructions to the camera is prone to cause too many constraint conditions and lead to misoperation and misexecution. Higher requirements are put forward for the management control, automation and intelligentization of the remote sensing camera. SUMMARY

[0004] The technical problem to be solved by the application is to overcome the shortcomings of the prior art, improve the automation and intelligentization capability, and realize the self-management of the remote sensor.

[0005] The purpose of the application is achieved by the following technical solutions:

[0006] A kind of intelligent information processing device of spaceborne micro-light remote sensor, comprising:

[0007] A time system module for unifying the time reference of the entire remote sensor;

[0008] A mode analysis module for analyzing mode control instructions in the imaging process; the mode includes: window imaging mode, HDR imaging mode, conventional imaging mode and automatic focusing mode; the window mode is used for target ROI identification; the HDR imaging mode is used for HDR synthesis output of area array image, and higher dynamic range image is output; the conventional imaging mode is used for outputting high gain or low gain image; the automatic focusing mode is used for automatically adjusting the focusing mechanism to find the best focal plane mode according to the image of the area array;

[0009] A program control module for imaging control during on-orbit operation;

[0010] A timing standby / start module for camera standby and restart;

[0011] A focusing module for adjusting the best position of the camera focal plane

[0012] An image processing module as the terminal of the intelligent information processing system, receives execution instructions and returns digital telemetry.

[0013] Preferably, the focusing module is provided with an automatic focusing function for early determination of the optimal focal plane position, a service focusing function for daily imaging tasks and an object distance focusing function for special task applications.

[0014] Preferably, the time system module realizes time accuracy unification by taking a second pulse as a reference.

[0015] Preferably, the image processing module stores computer program instructions of an automatic exposure algorithm, and the computer program instructions, when loaded and run by the processor, enable the processor to execute the following method:

[0016] The integral time is automatically adjusted by the automatic exposure to keep the image brightness within a proper range.

[0017] Preferably, the automatic exposure algorithm determines the integral time of the current frame of image data by calculating the mean value of the previous frame of image data DN, and iterates continuously until the work is completed.

[0018] Preferably, in the automatic exposure process, the image histogram characteristic value needs to be calculated first; the histogram method has two exposure modes: high mode and low mode.

[0019] Preferably, the constraint priority of the saturation information in the high mode is the highest, and the constraint priority of the low radiance information in the low mode is the highest.

[0020] Preferably, the image processing module stores computer program instructions of a sharpness evaluation function, and the computer program instructions, when loaded and run by the processor, enable the processor to execute the following method:

[0021] In the automatic focusing mode, the sharpness evaluation function value of each frame of image received by the electronic receiver is integrated by using the sharpness evaluation function, and the position of the optimal focal plane is recorded and compared.

[0022] Preferably, the local gradient is calculated by using the difference of adjacent pixels on the image, and then the image sharpness evaluation function is established.

[0023] Preferably, in the on-orbit application, the automatic focusing mode is applied in the initial stage, and in the gaze imaging state, the best focal plane position of the remote sensing camera is automatically stopped by comparing the sharpness evaluation function of the image processing module; when the camera enters the stable running period, the imaging circuit is controlled by the program control module to image / stop imaging at regular time intervals, and it is ensured that the focal plane is at the optimal position when the first frame of image is formed.

[0024] Compared with the prior art, the present application has the following beneficial effects:

[0025] (1) The time system of the present application takes a second pulse as a reference, realizes higher time unification accuracy of the whole satellite, and in combination with the local mu s accuracy clock and standby start function, can realize imaging control with sub-second level accuracy;

[0026] (2) The imaging control is executed in a programmed form, the imaging parameters are autonomously adjusted and adapted, high-precision non-delay imaging is realized, the effective data rate is improved, the utilization rate of the star-ground channel is improved, the camera autonomous management of high-precision moving imaging can be adapted, and single-rail multi-target parameter rapid setting is realized.

[0027] (3) The automatic focusing method can be applied to early focusing of an optical remote sensing camera, the construction and adjustment precision requirement of the camera is reduced, for a long focal length system, the time length of early focusing is greatly reduced, and the application efficiency of the remote sensing camera is improved.

[0028] (4) The rich image processing algorithms provide more choices for the application of the remote sensing camera: the automatic exposure algorithm can be applied to intelligently adjust the exposure time of imaging, so that the image brightness is kept in a proper range, the user intervention is reduced under the premise of ensuring the image quality; the HDR, Binning and surround exposure algorithms can realize imaging of a richer target dynamic range under the existing conditions, and the applicable scene of the remote sensor is increased; the ROI windowing combined with other on-board image processing algorithms can improve the frame frequency by reducing the width, and realize target tracking and special applications. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The information flow conversion path of the satellite-borne micro-light remote sensor satellite system is shown in the figure.

[0030] Figure 2 The processing logic of the time system module of the present application is shown in the figure.

[0031] Figure 3 The automatic focusing mode control flow chart is shown in the figure.

[0032] Figure 4 The processing flow chart of the mode analysis module of the present application is shown in the figure.

[0033] Figure 5 The timing diagram of the programmed module is shown in the figure.

[0034] Figure 6 The automatic exposure algorithm control flow chart is shown in the figure. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0036] A satellite-borne micro-light remote sensor intelligent information processing device comprises:

[0037] Time system module, used to unify the time reference of the whole remote sensor. The module consists of second pulse and second star time broadcast. The management controller maintains the second accuracy based on the star time broadcast, the imaging module is based on the second pulse and the star time broadcast as reference, and the local crystal oscillator is used as the us accuracy to maintain the time reference of the remote sensor, the platform and other subsystems.

[0038] Mode analysis module, used to analyze the mode control instructions in the imaging process. The modes include: windowed imaging mode, HDR imaging mode, regular imaging mode and automatic focusing mode. The windowed mode is used for target ROI identification; the HDR imaging mode is used for HDR synthesis output of area array image, which can output higher dynamic range image; the regular imaging mode is used for outputting high gain or low gain image; the automatic focusing mode is used for automatically adjusting the focusing mechanism to find the best focal plane according to the image of the area array.

[0039] Program control module, used for imaging control during on-orbit operation. The program control data block includes information such as start / stop time accurate to ms level, start / stop action, imaging mode, exposure time, algorithm switch, etc. The management controller analyzes the program control block, combines the broadcast containing satellite orbit and attitude information, decomposes the corresponding focusing action, imaging action and imaging parameters under the program control data block, and respectively sends them to the corresponding modules for execution.

[0040] Timing standby / start module, used for standby and restart of the camera. The module is located in the imaging circuit, combined with the time system of the management controller and the focal plane circuit, to realize sub-second level timing standby / start. It effectively reduces the on-orbit power consumption while realizing accurate start / stop combined with the program control module.

[0041] Focusing module, used for the best position adjustment of the camera focal plane. The module has automatic focusing function for early determination of the best focal plane position, business focusing function applied to daily imaging tasks, and object distance focusing function applied to special task applications. The three kinds of focusing and regular focusing functions are coordinated with the main control software to complete intelligent information processing.

[0042] Image processing module, as the terminal of the intelligent information processing system, receives execution instructions and returns digital telemetry. The module is on the imaging circuit, including binning algorithm, HDR fusion, automatic exposure, high and low gain, surround exposure, ROI windowing, non-uniformity correction, blind cell replacement, digital filtering algorithm, dark cell correction, sharpness evaluation function, etc. Algorithm, as well as imaging frame rate and exposure time setting function, and HDR imaging mode function of the detector.

[0043] Embodiment:

[0044] The embodiment of the present application provides a kind of intelligent information processing device of spaceborne micro-light remote sensor (hereinafter referred to as processing device).The processing device is an important component of the information processing system of spaceborne micro-light remote sensing satellite system. Figure 1 The information flow path of the processing device is shown in the satellite system.The camera integrated electronic receives the second pulse and mode, program control, focusing instruction from the platform, uses the second pulse and directly forwards to the focal plane assembly as the time system of the entire processing device;Resolve mode, program control and focusing instruction, control secondary power supply and focal plane assembly to realize timing imaging, distribute imaging parameters and image processing algorithm to focal plane assembly, receive the image clarity condition feedback by focal plane assembly and control the focusing mechanism to manually or automatically adjust the focal plane position.Focal plane assembly downloads the processed image to the data transmission subsystem.

[0045] The time system module unifies the time reference of the entire remote sensor, and the function implementation process is as shown in Figure 2 After the camera integrated electronic receives the satellite service time reference second pulse forwarded by the satellite service, it is directly sent to the focal plane assembly.The camera focal plane electronic assembly starts local timer after power-on initialization, and starts counting in 1us unit, and the counting value is continuously not cleared in a imaging process.The counter width is 32 bits to ensure that it does not overflow in a imaging cycle.The accumulation of microsecond counter forms its own local imaging time, and the counting value is added to the agreed position in the image auxiliary data as the local imaging time of each row.After detecting the falling edge of the second pulse signal, the camera focal plane electronic assembly latches the value of the local counter (as the second pulse time) at each second pulse falling edge and punches into the agreed position of the image auxiliary data.At the same time, the satellite service time reference broadcast data sent by the camera integrated electronic through the camera secondary CAN bus is latched at each second pulse falling edge.The latched whole second time is added by 1 (the second part of the whole second time data is added by 1 as 32-bit binary number), as the on-board working time, and added to the agreed position in the image auxiliary data.

[0046] The mode analysis module is used to analyze the mode control instruction in the imaging process, and the mode includes: window imaging mode, HDR imaging mode, conventional imaging mode and automatic focusing mode.

[0047] The window mode is used for target ROI identification, and local image can be output in blocks, which can effectively improve the imaging frame frequency in this mode, realize tracking imaging of target, and also realize measurement of internal micro-vibration / disturbance of remote sensor.In window mode, the integrated electronic controls the imaging circuit to perform window operation, and when image processing function is provided, target identification algorithm is executed at the same time for tracking, scanning and subsequent actions.

[0048] The HDR imaging mode is used for the HDR synthesis output of the area array image, and a higher dynamic range image can be output. In the HDR imaging mode, the integrated electronic control imaging circuit outputs the HDR image, the imaging circuit reads the high and low gain images of the area array and performs the HDR processing algorithm, and a higher dynamic range HDR image is output.

[0049] The conventional imaging mode is a normal imaging mode distinguished from the HDR imaging mode, in which the high gain or low gain image is output according to the program module or indirect instruction. This mode is mainly used for daily imaging, imaging of experimental nature, and maintenance and upgrade of the camera software.

[0050] The automatic focusing mode is a mode in which the focusing mechanism is automatically adjusted according to the image of the area array to find the best focal plane. For a spaceborne remote sensor, the focusing time can be greatly shortened. Figure 3 The automatic focusing mode is a mode in which the focusing mechanism is automatically adjusted according to the image of the area array to find the best focal plane. For a spaceborne remote sensor, the focusing time can be greatly shortened.

[0051] The program module is a key module for controlling the camera imaging process and is also a core part of the processing device. Figure 4 The control flow of the mode analysis module is shown in the following table. The camera determines the imaging mode according to the received solar elevation angle and records the determination. The satellite sends the program data block according to the pre-agreed data block format, and the integrated electronics determines the imaging action, imaging time, end action, and end time. When the start action is "power on", the integrated electronics gives the secondary power supply power in advance according to the preset power-on time to ensure that the image is output at the imaging time; when the start action is "image output (stop standby)", the integrated electronics sends the image output time and the stop standby action to the imaging circuit in advance, and the imaging circuit executes the image output action at the image output time according to the local time system; when the end action is "power off", the integrated electronics powers off the secondary power supply at the end time; when the end action is "standby", the integrated electronics also sends the standby time and the standby action to the imaging circuit in advance, and the imaging circuit executes the action. After the imaging circuit is started and before the camera outputs the image, the integrated electronics sends the exposure time, working mode, and preset algorithm switch to the imaging circuit to ensure that the image at the image output time is the image after the parameters are adjusted. In addition to completing the settings related to the camera imaging, the program module also performs the focusing action according to the satellite downward-looking angle to compensate for the change in the object distance. According to the camera focal length, when the defocus causes the transmission function to drop by more than 10%, a focusing action corresponding to the defocus change is performed. When the camera focal length is determined, the value is a fixed value. The focusing motor is powered on and the focusing action is completed before the camera outputs the image. The timing sequence of the entire program module is shown in the following table.Figure 5 As shown.

[0052] The focusing module is used to determine the camera's focal plane. The autofocus function is used in automatic focusing mode to determine the optimal focal plane in the early on-orbit and later maintenance stages. The operational focusing is used by the programmable module to compensate for defocusing when the satellite is looking at the oblique angle during long-term on-orbit operation. The object distance focusing can automatically complete the focusing of the specified target based on the target's object distance information and can restore the original focal plane position after the current imaging is completed. The conventional command focusing is the traditional manual focal plane position adjustment, which is reserved as a means for maintenance and functional assurance.

[0053] The image processing module is integrated into the software of the imaging circuit, including binning algorithm, HDR fusion, automatic exposure, high and low gain, bracketing exposure, ROI windowing, non-uniformity correction, blind pixel replacement, digital filtering algorithm, dark pixel correction, sharpness evaluation function and other algorithms, as well as imaging frame rate and exposure time setting functions, standby and timed standby functions, and the detector's HDR imaging mode function.

[0054] Figure 6 This describes the implementation method of automatic exposure. The automatic exposure design can autonomously adjust the integration time to maintain image brightness within a suitable range. The automatic exposure algorithm determines the integration time of the current frame image by calculating the mean DN value of the previous frame image data, iterating continuously until the process ends. In the automatic exposure process, the image histogram feature values ​​need to be calculated first. The histogram method has two exposure modes: high-brightness mode and low-brightness mode. The principle of the "high-brightness mode" is to ensure normal imaging of high-brightness ground objects during imaging, while not guaranteeing or fully guaranteeing low-brightness objects. The advantage of this principle is that it avoids too many saturated areas in the image, but it will cause a decrease in the resolution of low-brightness object information. The principle of the "low-brightness mode" is to prioritize normal imaging of low-brightness ground objects during imaging, while not guaranteeing or fully guaranteeing high-brightness objects. This principle can ensure the maximum acquisition of low-brightness object information, but it is prone to local scene saturation during imaging. The constraint priority of saturation information is highest in the high-brightness mode, while the constraint priority of low-brightness information is highest in the low-brightness mode. To ensure normal operation in orbit, the threshold can be modified through data injection.

[0055] The sharpness evaluation function is an algorithm used to calculate the camera's out-of-focus distance in autofocus mode. In autofocus mode, the sharpness evaluation function values ​​of each frame received electronically are combined, recorded, and compared to determine the optimal focal plane position. The function calculates the local gradient using the difference between adjacent pixels in the image, and then establishes the image sharpness evaluation function based on this. The adjacent pixel gray-level gradient method is defined as follows:

[0056]

[0057] In the formula: I(x, y) is the gray value of the pixel at position (x, y), MxN is the total number of pixels.

[0058] In view of the changes in the scene during on-board focusing, the back light phenomenon that may exist during gaze imaging, and noise, etc., a corresponding threshold is added in the calculation process to improve the robustness of the algorithm. The non-edge region has a larger gray gradient value; the proportion in the entire image is smaller. In order to improve the accuracy and sensitivity of the algorithm, so as to quickly and accurately extract the edge information in the image, an edge threshold is introduced when calculating the image sharpness, as follows.

[0059]

[0060] Threshold judgment is performed pixel by pixel, and only when F(x, y) is greater than Th, the pixel is retained as an edge pixel, and other pixels are discarded.

[0061] In the processing device, in addition to the automatic focusing mode, the star service broadcast and the program control data block being real-time sent, the other mode settings (window opening, HDR, regular imaging) and the image processing algorithm switch are all state setting quantities. The integrated electronics autonomously sends these state setting quantities according to the preset module start time. In addition to the imaging parameters calculated in real time, the remaining parameters are set through indirect instructions.

[0062] In on-orbit application, the automatic focusing mode is used initially, and in the gaze imaging state, the best focal plane position of the remote sensing camera is automatically stopped by comparing the sharpness evaluation function of the image processing module. When the camera enters the stable running period, the program control function is mainly used to control the imaging circuit to image / stop imaging at regular intervals, and to ensure that the focal plane is at the best position when the first frame is imaged, and the imaging parameters use the best parameters and modes of the current task.

[0063] The processing device has been applied to a remote sensing camera system, greatly reducing the operation amount of the on-orbit instruction sending operator, and greatly improving the use efficiency of the remote sensor.

[0064] The contents not described in detail in the specification of the present application are the known technology of those skilled in the art.

[0065] Although the present application has been disclosed with reference to the preferred embodiments above, it is not intended to limit the present application, and any person skilled in the art can make possible changes and modifications to the technical solutions of the present application using the disclosed methods and technical contents without departing from the spirit and scope of the present application. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application, which does not deviate from the technical solutions of the present application, belongs to the protection scope of the technical solutions of the present application.

Claims

1. A kind of intelligent information processing device of spaceborne micro-light remote sensor, it is characterized in that, The application relates to a remote sensor system, which comprises the following modules: a time system module for unifying the time reference of the whole remote sensor; a mode analysis module for analyzing the mode control instructions in the imaging process; the modes include a window imaging mode, an HDR imaging mode, a regular imaging mode and an automatic focusing mode; the window imaging mode is used for target ROI identification; the HDR imaging mode is used for HDR synthesis output of the area array image, and an image with a higher dynamic range is output; the regular imaging mode is used for outputting an image with high or low gain; and the automatic focusing mode is used for automatically adjusting the focusing mechanism to find the best focal plane according to the image of the area array; a program control module for imaging control during on-orbit operation; a timing standby / starting module for standby and restart of the camera; a focusing module for best position adjustment of the camera focal plane; an image processing module as a terminal of an intelligent information processing system, which receives execution instructions and returns digital remote measurement.

2. The intelligent information processing apparatus according to claim 1, characterized by, The focusing module is provided with an automatic focusing function for early determination of the best focal plane position, a business focusing function applied to daily imaging tasks and an object distance focusing function applied to special task applications.

3. The intelligent information processing device according to claim 1, characterized by, The time system module realizes time precision unification by taking a second pulse as a reference.

4. The intelligent information processing device according to claim 1, wherein The image processing module is provided with computer program instructions of an automatic exposure algorithm, and the computer program instructions enable the processor to execute the following method when loaded and run by the processor: the integral time is automatically adjusted by the automatic exposure to keep the image brightness in a proper range.

5. The intelligent information processing device according to claim 4, wherein The automatic exposure algorithm determines the integral time of the current image by calculating the DN mean value of the previous image data, and the process is continuously iterated until the work is completed.

6. The intelligent information processing device according to claim 5, wherein In the automatic exposure process, the image histogram characteristic value needs to be calculated first; the histogram method has two exposure modes: a high mode and a low mode.

7. The intelligent information processing device according to claim 6, wherein The constraint priority of the saturation information is the highest in the high mode, and the constraint priority of the low radiance information is the highest in the low mode.

8. The intelligent information processing device of claim 1, wherein, The image processing module is provided with computer program instructions of a definition evaluation function, and the computer program instructions enable the processor to execute the following method when loaded and run by the processor: the definition evaluation function is used to comprehensively evaluate the definition evaluation function value of each image received by the electronic receiver in the automatic focusing mode, and the position of the best focal plane is recorded and compared.

9. The intelligent information processing device according to claim 8, wherein The local gradient is calculated by using the difference of adjacent pixels on the image, and then the image definition evaluation function is established. 10.The intelligent information processing apparatus according to any one of claims 1 to 9, characterized in that, In on-orbit application, the automatic focusing mode is applied in the initial stage, the remote sensor camera is automatically stopped at the best focal plane position by comparing the definition evaluation function of the image processing module in the gaze imaging state; when the camera enters the stable operation period, the imaging circuit is controlled by the program control module to realize timed imaging / stop imaging, and the focal plane is ensured to be in the best position when the first frame is imaged.

Images