Satellite-borne super-high-speed image intelligent processing method and device
By employing an architecture combining a high-speed data interface unit and an intelligent image processing unit in remote sensing satellites, and utilizing a combination of FPGA and GPU, the problem of real-time processing of high-code-rate payload data was solved, achieving efficient, lightweight, and low-power image processing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI SATELLITE ENG INST
- Filing Date
- 2026-02-26
- Publication Date
- 2026-06-23
AI Technical Summary
Existing technologies in remote sensing satellites struggle to meet the real-time throughput and processing requirements of high-data-rate payloads. Traditional solutions result in excessive weight and power consumption and cannot effectively address the pressure of transmitting and processing massive amounts of data.
The architecture combines a high-speed data interface unit with an intelligent image processing unit. It leverages the data preprocessing capabilities of a high-performance FPGA and the intelligent computing capabilities of a high-performance GPU to achieve intelligent image processing. By combining the FPGA interface module and the GPU module with data preprocessing and real-time detection of the target of interest, weight and power consumption are reduced.
It achieves efficient real-time image processing, reduces weight and power consumption, meets the real-time processing requirements under high throughput, avoids the repeated transmission of massive internal data, and improves processing efficiency.
Smart Images
Figure CN122269014A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of spaceborne image processing technology, specifically relating to a spaceborne ultra-high-speed image intelligent processing method and apparatus. The ultra-high speed refers to a transmission rate exceeding 40Gbps. Background Technology
[0002] With the continuous development of remote sensing technology, the bit rate of most remote sensing satellite payload cameras has increased from the original Mbit level to the current Gbit level, and even tens or hundreds of Gbit. The massive amount of remote sensing data brings severe delays to the satellite-to-ground transmission link, and at the same time, it brings great costs and pressure to the storage and processing of ground processing systems. Traditional on-board image compression cannot fundamentally solve the current situation of massive data transmission and processing pressure.
[0003] Furthermore, in recent years, the field of on-board image processing has developed rapidly. However, in order to adapt to the current real-time throughput and processing requirements of high bit rate payload data, it is usually achieved by using external large-capacity solid-state disks for storage and then post-processing, or by using a solution of stacking a large number of FPGAs and CPUs.
[0004] Such solutions come at the cost of significant weight, power consumption, and size, and struggle to guarantee real-time data access, processing, and output under ultra-high-speed load data input conditions.
[0005] To meet the requirements for efficient real-time access and processing of payload data, while reducing weight and power consumption, the spaceborne image intelligent processor designed in this invention adopts an architecture combining a high-speed data interface unit and an intelligent image data processing unit. It effectively utilizes the high-speed data interface capabilities and data preprocessing capabilities of a high-performance FPGA, while leveraging the intelligent computing capabilities of a high-performance GPU to meet both the intelligent processing requirements and traditional image processing requirements of spaceborne images.
[0006] Patent document CN111127294A discloses a spaceborne image AI processing device, including a hardware structure and system circuit; the system circuit consists of FPGA, DDR, SD card, SSD, cameralink, LVDS, gigabit Ethernet, CAN, IIC, rs422, rs485, spi, and io; this solution cannot avoid the repeated transmission of massive amounts of internal data, thus causing pressure on the transmission bandwidth.
[0007] This problem urgently needs to be solved. Summary of the Invention
[0008] To address the shortcomings of existing technologies, the purpose of this invention is to provide a spaceborne ultra-high-speed image intelligent processing method and apparatus.
[0009] According to the present invention, a spaceborne ultra-high-speed image intelligent processing device includes: a high-speed data interface unit, an intelligent image processing unit, and a main control unit; The high-speed data interface unit is connected to the external data input unit, the intelligent image processing unit, and the main control unit, respectively.
[0010] Preferably, it further includes: a power distribution unit; the power distribution unit converts the primary power input from the external power system into secondary power, and supplies power to the high-speed data interface unit, the intelligent image processing unit and the main control unit.
[0011] Preferably, the data processing circuit of the high-speed data interface unit includes: a data interface chip, a data preprocessing chip, a data reconstruction and multiplexing chip, and an image slice extraction chip; The data interface chip connects the data preprocessing chip and the image slice extraction chip; the data preprocessing chip is connected to the data reconstruction and multiplexing chip.
[0012] Preferably, the data interface chip has a single-channel data rate of 10Gbps or greater; The image slice extraction chip has slice adaptation sizes including: 512*512, 1024*1024, 3072*3072, 5120*5120 or 10240*10240; where * represents product.
[0013] Preferably, the intelligent image processing unit includes: a second data interface chip and a second data cache chip; The second data interface chip receives data input from the high-speed data interface unit via the UDP module; The second data cache chip caches and processes the data input of the high-speed data interface unit.
[0014] Preferably, the second data interface chip uses a 10 Gigabit Ethernet network port communication method.
[0015] Preferably, the main control unit includes: an RS422 interface chip, a CAN bus chip, an antifuse FPGA chip, and a relay; The RS422 interface chip inputs the program from the external measurement and control system into the antifuse FPGA chip; the CAN bus chip can receive control signals and input them into the antifuse FPGA chip; the antifuse FPGA chip outputs OC commands, and the relay switches the power supply based on the OC commands.
[0016] Preferably, the high-speed data interface unit outputs data to an external storage object via optical fiber, i.e., optical signal output.
[0017] According to the present invention, a spaceborne ultra-high-speed image intelligent processing method is implemented based on a spaceborne ultra-high-speed image intelligent processing device. The high-speed data interface unit receives raw data and outputs image data to the intelligent image processing unit; the intelligent image processing unit outputs the image processing result.
[0018] Preferably, the main control unit provides power distribution control for the high-speed data interface unit and the intelligent image processing unit.
[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention provides a universal, ultra-high-speed spaceborne image intelligent processing solution, abandoning the traditional FPGA+CPU stacked spaceborne processing solution. It adopts a combination of a high-throughput FPGA interface module and a high-computing-power GPU intelligent module, improving overall processing efficiency and reducing weight and power consumption. Furthermore, the dual-module combination can be configured according to actual application requirements, offering high flexibility.
[0020] 2. This invention provides a data preprocessing method for FPGA image quantization and downsampling to reduce the amount of data, and utilizes the high intelligent computing power of the GPU to perform real-time detection and processing of the target of interest in the image, thus meeting the real-time image processing requirements under high throughput.
[0021] 3. This invention provides a processing scheme for separating slice calculation and slice extraction, which keeps the original data on the front-end FPGA, and the GPU module is only used for slice position calculation, avoiding the repeated transmission of massive internal data, thereby reducing the pressure on transmission bandwidth.
[0022] 4. The spaceborne ultra-high-speed image intelligent processing device provided by this invention is stable and reliable, and can be widely used in the field of satellite-borne intelligent processing. Attached Figure Description
[0023] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 The structural block diagram provided for this invention; Figure 2 The system block diagram provided for this invention; Figure 3 The main control unit principle block diagram provided by the present invention; Figure 4 This is a schematic diagram of the information processing flow provided by the present invention. Detailed Implementation
[0024] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.
[0025] According to the present invention, a spaceborne ultra-high-speed image intelligent processing device is provided, the system device comprising: a high-speed data interface unit, an intelligent image processing unit, a main control unit, and a power distribution unit; The high-speed data interface unit is connected to the external data input unit, the intelligent image processing unit, and the main control unit, respectively. The high-speed data interface unit receives raw payload data and outputs processed image data through the high-speed data interface. The high-speed data interface unit performs operations such as packetization, caching, preprocessing, and slice extraction on the image data. The intelligent image data processing unit receives preprocessed image data forwarded by the high-speed data interface unit via a high-speed internal bus communication method, and performs operations such as intelligent target detection, region extraction, and target localization. The main control unit receives OC commands to perform power distribution control on the above modules; it receives remote control commands from the external integrated electronic system to perform program loading and reload management, parameter management, and simultaneously collects telemetry data from each module and forwards it to the integrated electronic system; it receives on-orbit program data forwarded by the telemetry and control system and performs configuration operations on the corresponding SPI FLASH. The power distribution unit provides power to the above units.
[0026] This invention provides a universal, ultra-high-speed spaceborne image intelligent processing solution. It abandons the traditional FPGA+CPU stacked spaceborne processing scheme, employing a combination of a high-throughput FPGA interface module and a high-computing-power GPU intelligent module to improve overall processing efficiency and reduce weight and power consumption. Furthermore, the dual-module combination can be configured according to actual application requirements, offering high flexibility. High throughput refers to a transmission rate greater than 40Gbps; high computing power refers to a single-chip intelligent computing power exceeding 30TOPS.
[0027] Specifically, the intelligent image data processing unit is connected to the high-speed data interface unit and the main control unit in terms of information flow; in terms of interface function, it receives preprocessed image data forwarded by the high-speed data interface unit through the communication method of the high-speed internal bus; in terms of data processing function, it performs operations such as intelligent target detection, region extraction, and target localization. The main control unit is connected to the high-speed data interface unit and the intelligent image processing unit respectively; it receives OC commands to perform power distribution control on the above modules; it receives remote control commands from the external integrated electronic system to perform program loading and reload management, parameter management on the above units, and collects telemetry data from each module and forwards it to the integrated electronic system; it receives on-orbit program data forwarded by the telemetry and control system and performs configuration operations on the corresponding SPI FLASH.
[0028] The power distribution unit receives primary power input from an external power system, converts it into secondary power, and then supplies power to the aforementioned units.
[0029] Specifically, the high-speed data interface unit is composed of a high-performance FPGA and includes several data processing circuits. Each of the data processing circuits includes a data interface chip, a data preprocessing chip, a data cache chip, a data reconstruction and multiplexing chip, an image slice extraction chip, a configuration chip, etc. The data interface chip, including the Aurora module, receives several external image inputs through a fiber optic interface, and simultaneously outputs the processed image data of this system to the outside. The data preprocessing chip performs data parsing on the received data, separates image data and auxiliary data, and performs operations such as radiometric calibration, image quantization, and image downsampling on the image data. The data cache chip stores the raw image data in spectral band partitions for image slice extraction operations. The data reconstruction and multiplexing chip packages camera-specific data, auxiliary data, and image data according to protocol requirements, and reassembles and multiplexes multiple data streams before sending them to the intelligent image data processing unit. The image slice extraction chip performs original resolution slice extraction operations on the slice auxiliary data sent by the intelligent image data processing unit and the cached original image data. The configuration chip receives configuration parameters forwarded by the main control unit, configures the working mode, and frames the telemetry information of this module and sends it to the main control module within a specified time.
[0030] Specifically, the high-speed data interface unit and the data interface chip use an Aurora module to achieve real-time access to fiber optic data, with a single-channel data rate of up to 10Gbps. The entire module uses four parallel optical signal access channels with a rate of up to 40Gbps.
[0031] The data preprocessing chip performs data parsing on the received data, separating image data and auxiliary data, and performing operations such as radiometric calibration, image quantization, and image downsampling on the image data. Specifically, the high-speed data interface unit, the data and processing chip, the image quantization operation can perform 8-bit quantization processing on image data within 16 bits, and the image downsampling operation can perform downsampling processing on the original image data by a factor of 1 to N according to actual application requirements.
[0032] Specifically, the high-speed data interface unit, specifically the image slice extraction chip, can extract slices from target slices or regions according to remote control commands. The slices can be adapted to different sizes such as 512*512, 1024*1024, 3072*3072, 5120*5120, or 10240*10240.
[0033] Specifically, the intelligent image data processing unit is composed of a high-performance GPU and includes several data processing circuits. Each of the data processing circuits includes a second data interface chip, a second data cache chip, a target slice chip, a region slice chip, a positioning calculation chip, a configuration chip, etc.
[0034] The second data interface chip includes a UDP module, which receives data input from the high-speed data interface unit via a 10 Gigabit fiber optic connection, and simultaneously outputs the processed data of this unit to the high-speed data interface unit; in other words, it receives data input from the high-speed data interface unit and outputs the processed data of this unit to the high-speed data interface unit; it adopts a 10 Gigabit Ethernet port communication method and a UDP communication protocol, with a single-channel rate of up to 10Gbps; The second data cache chip performs block-based caching processing on the received image data and performs caching processing on the auxiliary data according to its type; The target slice chip uses deep learning intelligent algorithms and YOLO v5s models to detect and identify targets of interest, and calculates the position of the target slice based on the slice size. The regional slicing chip performs inverse positioning operations on the latitude and longitude points marked on the ground, and calculates the position of the regional slices based on the slice size; this is achieved using a controlless geometric positioning solution method. The positioning calculation chip adopts a control-point-free positioning calculation method, which uses satellite-assisted data and camera-assisted data to calculate the Earth-fixed system position of image points; it is implemented using a control-point-free geometric positioning calculation method. The configuration chip receives configuration parameters forwarded by the main control unit, configures the working mode and slice size; and frames the telemetry information of this module and sends it to the main control module within a specified time.
[0035] Specifically, the intelligent image data processing unit, the data interface chip, adopts a 10 Gigabit Ethernet port communication method and UDP communication protocol, with a single-channel rate of up to 10Gbps; Specifically, the intelligent image data processing unit, specifically the target slice chip, is implemented using a deep learning architecture and a YOLO v5s network model.
[0036] Specifically, the intelligent image data processing unit, the region slicing chip, and the positioning calculation chip are implemented using a control-point-free geometric positioning calculation method.
[0037] Specifically, the main control unit includes an RS422 interface chip, a CAN bus chip, an antifuse FPGA chip, and a relay; The RS422 interface chip is used to receive the input program from the external measurement and control system and input it into the antifuse FPGA chip; The CAN bus chip is used to receive control signals from an external integrated electronic system and input them to the antifuse FPGA chip. The antifuse FPGA chip receives and processes the input program and writes it into the SPI FLASH. The antifuse FPGA chip parses the control signal and sends it to the high-speed data interface unit and the intelligent image data processing unit. It is also used to receive telemetry data from the high-speed data interface unit and the intelligent image data processing unit and package it into a telemetry package to send to an external integrated electronic system. The relay is controlled by the antifuse FPGA chip outputting the OC command to switch the power output of the system power supply unit.
[0038] Specifically, based on the actual camera data input rate requirements and processing needs, the high-speed data interface unit and intelligent image data processing unit can be combined and multiplied at the unit level to meet the processing requirements under higher load rates and stronger intelligent computing power.
[0039] The power distribution unit receives primary power input from the outside and converts it into secondary power to power the high-speed data interface unit, intelligent image processing unit, and main control unit.
[0040] This invention proposes an architecture design based on a combination of a high-speed data interface unit and an intelligent image data processing unit. The entire system design can meet the requirements of high throughput, high computing power, and intelligence, and solve the technical problems of low image processing efficiency and high power consumption in existing technologies.
[0041] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a further detailed explanation of the spaceborne ultra-high-speed image intelligent processing device proposed in this invention. The advantages and features of this invention will become clearer from the following description and claims.
[0042] like Figures 1 to 3 As shown, this embodiment provides a spaceborne ultra-high-speed image intelligent processing device, from... Figure 1As can be seen, the overall structure of this implementation consists of a high-speed data interface unit, an intelligent image data processing unit, a main control unit, and a power distribution unit. Each unit is arranged on the base plate unit through a plug-in structure to form a complete system.
[0043] like Figure 1 As shown, based on the actual load input rate, the present invention is modified in a certain way, that is, the high-speed data interface unit and the intelligent image data processing unit are combined and configured to meet the requirements of higher load rate and stronger intelligent computing.
[0044] like Figure 2 As shown, the high-speed data interface unit receives input data from an external payload camera, internally performs data interaction operations with the intelligent image data processing unit, and simultaneously sends the processed data to the external unit.
[0045] like Figure 2 and Figure 3 As shown, the main control unit interacts with the high-speed data interface unit, the intelligent image data processing unit, and several external units for telemetry and remote control, and receives OC commands and completes program uploading and management functions.
[0046] like Figure 2 As shown, in terms of power distribution, the power distribution unit is electrically connected to the high-speed data interface unit, the intelligent image data processing unit, and the main control unit, respectively, and is used to receive the primary power supply of the external overall circuit, convert it into secondary power supply, and supply power to each unit.
[0047] like Figure 2 As shown, in this embodiment, the high-speed data interface unit is composed of a high-performance FPGA, which is configured with 8GB of DDR4 memory and 900 million logic gates.
[0048] like Figure 2 As shown, in this embodiment, the intelligent image data processing unit is composed of a high-performance AGX with a computing power of 32 TOPS and an EMMC storage resource of 32 GB.
[0049] The processing system of this embodiment will be further described below with reference to the accompanying drawings.
[0050] like Figure 4 As shown, in this embodiment, the high-speed data interface unit receives external payload camera data and outputs processed slice data. The interface adopts an optical fiber interface to meet the requirements of ultra-high-speed transmission rate. The interface data protocol adopts the Aurora64 / 66b protocol, and 4-channel parallel transmission and reception can achieve 40Gbps data transmission and reception.
[0051] The high-speed data interface unit performs data parsing on the received camera data, separating image data and auxiliary data, storing the original image data in different bands, and performing radiometric calibration, image quantization, and image downsampling on the image data. The image quantization operation can perform 8-bit quantization processing on image data up to 16 bits, and the image downsampling operation can perform downsampling processing on the original image data by multiples of 1 to N according to actual application requirements.
[0052] The high-speed data interface unit will package camera-specific data, auxiliary data, and image data according to the transmission processing board protocol requirements, and multiplex and reassemble multiple data streams before sending them to the intelligent image data processing unit. The data transmission protocol uses 10 Gigabit Ethernet UDP.
[0053] like Figure 4 As shown, after receiving the preprocessed image data sent by the high-speed data interface unit, the intelligent image data processing unit separates the image data and auxiliary data.
[0054] The intelligent image data processing unit is equipped with a deep learning environment and uses the YOLO v5s algorithm to perform intelligent image processing for target detection and recognition, obtaining the category and image plane location of the target of interest. It then combines auxiliary data to calculate the Earth-fixed system location of the target of interest.
[0055] The intelligent image data processing unit calculates the position of the center point of the region of interest on the image plane based on the latitude and longitude coordinates of the region of interest marked on the ground and in combination with auxiliary data.
[0056] The intelligent image data processing unit packages information such as slice location and transmits it to the high-speed data interface unit via the internal bus UDP method.
[0057] like Figure 4 As shown, the high-speed data interface unit receives data such as the slice position sent by the intelligent image data processing unit and performs original resolution slice extraction operation on the cached original image data; according to the remote control command requirements, it realizes the extraction of target slices or regions, and the slices can be adapted to different sizes such as 512*512 or 1024*1024 or 3072*3072 or 5120*5120 or 10240*10240.
[0058] The high-speed data interface unit sends the framed slice data information to the external storage module via optical fiber.
[0059] This invention provides ideas for the high speed and intelligence required for the design of spaceborne image processing systems.
[0060] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.
[0061] In the description of this application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.
[0062] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.
Claims
1. A spaceborne ultra-high-speed intelligent image processing device, characterized in that, include: High-speed data interface unit, intelligent image processing unit, main control unit; The high-speed data interface unit is connected to the external data input unit, the intelligent image processing unit, and the main control unit, respectively.
2. The spaceborne ultra-high-speed image intelligent processing device according to claim 1, characterized in that, Also includes: Power distribution unit; The power distribution unit converts the primary power input from the external power system into secondary power and supplies it to the high-speed data interface unit, the intelligent image processing unit, and the main control unit.
3. The spaceborne ultra-high-speed image intelligent processing device according to claim 1, characterized in that, The data processing circuit of the high-speed data interface unit includes: a data interface chip, a data preprocessing chip, a data reconstruction and multiplexing chip, and an image slice extraction chip; The data interface chip connects the data preprocessing chip and the image slice extraction chip; the data preprocessing chip is connected to the data reconstruction and multiplexing chip.
4. The spaceborne ultra-high-speed image intelligent processing device according to claim 3, characterized in that, The data interface chip has a single-channel data rate greater than or equal to 10Gbps; The image slice extraction chip has slice adaptation sizes including: 512*512, 1024*1024, 3072*3072, 5120*5120 or 10240*10240; where * represents product.
5. The spaceborne ultra-high-speed image intelligent processing device according to claim 1, characterized in that, The intelligent image processing unit includes: a second data interface chip and a second data cache chip; The second data interface chip receives data input from the high-speed data interface unit via the UDP module; The second data cache chip caches and processes the data input of the high-speed data interface unit.
6. The spaceborne ultra-high-speed image intelligent processing device according to claim 5, characterized in that, The second data interface chip uses 10 Gigabit Ethernet for network communication.
7. The spaceborne ultra-high-speed image intelligent processing device according to claim 1, characterized in that, The main control unit includes: an RS422 interface chip, a CAN bus chip, an antifuse FPGA chip, and a relay; The RS422 interface chip inputs the program from the external measurement and control system into the antifuse FPGA chip; the CAN bus chip can receive control signals and input them into the antifuse FPGA chip; the antifuse FPGA chip outputs OC commands, and the relay switches the power supply based on the OC commands.
8. The spaceborne ultra-high-speed image intelligent processing device according to claim 1, characterized in that, The high-speed data interface unit outputs data to an external storage object via optical fiber, i.e., optical signal.
9. A spaceborne ultra-high-speed image intelligent processing method, implemented based on the spaceborne ultra-high-speed image intelligent processing device according to any one of claims 1 to 8, characterized in that, The high-speed data interface unit receives raw data and outputs image data to the intelligent image processing unit; the intelligent image processing unit outputs the image processing result.
10. The spaceborne ultra-high-speed image intelligent processing method according to claim 9, characterized in that, The main control unit controls the power distribution of the high-speed data interface unit and the intelligent image processing unit.