High-performance spaceborne intelligent computing terminal and starting method thereof

Abstract

The invention relates to a high-performance spaceborne intelligent calculation terminal and a starting method thereof. The terminal comprises a routing unit, a storage unit and an intelligent calculation unit, each of the routing unit and the intelligent calculation unit comprises a hardware physical layer, a hardware driving layer, an embedded operating system, a resource scheduling management layer and an application service layer; the application service layer is used for configuring different application service APPs according to spaceflight application scenes and function requirements, and the application service APPs comprise service plug-ins corresponding to the resource scheduling management layer called by an embedded operating system and universal task distribution and resource management modules in the operating system according to the different application service APPs related to intelligent AI model intelligent calculation. In combination with corresponding hardware drive, computing power and hardware resources required by the intelligent AI model are flexibly configured for multi-user application software; the routing unit is used for providing interface communication with the outside, and realizing optical fiber data receiving and transmitting health management, Ethernet bus communication, RS422 bus communication, data routing management, task scheduling management, remote control and telemetering, centralized management of a system program or an application program or an FPGA configuration program, and a power supply control function; the intelligent calculation unit dynamically loads the on-satellite intelligent AI model and parameters thereof and is used for completing operation of the on-satellite intelligent AI model; and the storage unit is used for storing on-satellite load original data, intermediate data processed by the on-satellite intelligent AI model and processed data.

Description

TECHNICAL FIELD

[0001] The present application relates to a high-performance on-board intelligent computing terminal and a starting method thereof, and belongs to the field of on-board high-performance intelligent computing. BACKGROUND

[0002] With the rapid development of modern aerospace remote sensing technology, the satellite earth observation capability has been greatly improved, and the resolution is getting higher and higher, resulting in a large amount of image data. The data volume has grown from the previous hundreds of megabytes and gigabytes to the current hundreds of gigabytes. Due to the existence of a large amount of invalid data in the image data acquired by the satellite in orbit, if all the image data is transmitted back to the ground for processing, it will inevitably occupy a large amount of valuable resources of the satellite-ground link, and the timeliness will also be greatly discounted. The traditional service mode from satellite data acquisition -> ground processing -> actual application has long processing delay and cannot respond to emergencies in time, which has been unable to meet the demand of satellite rapid response. The new application mode from satellite data acquisition -> in-orbit intelligent processing -> actual application directly generates effective information on the satellite, which has short processing delay and can greatly improve the timeliness from data to information. In-orbit intelligent processing technology has gradually become a hot spot and future development trend in the field of aerospace remote sensing.

[0003] With the rapid development of artificial intelligence technology, the algorithm is updated and iterated quickly, and the demand for hardware resources is getting higher and higher. The traditional on-board processing system with FPGA+DSP architecture has problems such as low computing power, high development difficulty, and poor algorithm portability, which cannot meet the urgent needs of satellite for fast updating of deep learning algorithm, high processing performance, and high timeliness. SUMMARY

[0004] The technical problem solved by the present application is to overcome the shortcomings of the prior art and provide a high-performance on-board intelligent computing terminal and a starting method thereof. Under the premise of meeting the high-reliability on-board intelligent computing demand, the algorithm transplantation and application deployment process is greatly simplified, the mainstream AI framework and APP in-orbit evolution upgrade are supported, and the satellite is transformed from a functional solidified hardware platform to an intelligent platform with sustainable growth capability.

[0005] The technical solution of the present application is: a high-performance on-board intelligent computing terminal, which comprises a routing unit, a storage unit and an intelligent computing unit. The routing unit and the intelligent computing unit each comprise a hardware physical layer, a hardware driver layer, an embedded operating system, a resource scheduling management layer, and an application service layer; the application service layer is configured with different application services APP according to space application scenarios and functional requirements, the application services APP including an APP that uses an intelligent AI model for operation, and the embedded operating system calls corresponding service plug-ins of the resource scheduling management layer and a universal task distribution and resource management module in the operating system, in combination with corresponding hardware drivers, to flexibly configure computing power and hardware resources required by the intelligent AI model for multi-user application software; The routing unit is configured to provide interface communication with the outside, implement fiber data transceiving, health management, Ethernet bus communication, bus communication, data routing management, task scheduling management, remote control and telemetry, and centralized management of embedded operating system programs or application programs or hardware configuration programs of the routing unit and the intelligent computing unit, and power control functions. The intelligent computing unit is configured to dynamically load an intelligent AI model on a satellite and parameters thereof, and complete operation of the intelligent AI model on the satellite. The storage unit is configured to store original data of a satellite-borne payload, intermediate data processed by the intelligent AI model on the satellite, and processed data.

[0006] Preferably, the routing unit, the storage unit, and the intelligent computing unit are connected through three buses, which are independent of each other and are respectively responsible for control flow, service flow, and energy flow interaction. The control flow is implemented by using RS422, the routing unit distributes task instructions, upload data, and remote control instructions to the intelligent computing unit through the control flow, and collects telemetry state information of each unit to realize monitoring and scheduling of the terminal. The service flow is implemented by using Ethernet, the intelligent computing unit reads data to be processed from the storage unit through the service flow when processing data, and writes the processed result data back to the storage unit through the service flow. The energy flow provides primary power for the routing unit, the intelligent computing unit, and the storage unit, and the routing unit, the intelligent computing unit, and the storage unit convert the primary power into secondary power or tertiary power and provide the internal circuit with the secondary power or the tertiary power.

[0007] Preferably, upload data and remote control and telemetry data flow are logically separated by different communication protocols and are subjected to priority management.

[0008] Preferably, the routing unit and the intelligent computing unit each comprise an FPGA, a CPU, an SSD solid state disk, a GPU, or an NPU. FPGA module, connected with CPU, GPU or NPU, SSD solid state disk and external interface, used for protocol conversion of external data interface, responsible for external data transceiver, controlling data access of SSD solid state disk, completing front-end processing of load data, including image hardware acceleration and data format conversion processing, forming algorithmic complementation with GPU or NPU; CPU module, used for running embedded operating system and resource scheduling management layer program, loading AI model from SSD solid state disk and calling GPU or NPU to execute intelligent calculation; SSD solid state disk, used for storing embedded operating system kernel, service plug-in, application service APP program, AI model and temporary data; GPU or NPU, interconnected with CPU through high-speed PCIE bus, used as intelligent calculation coprocessor, responsible for unloading intensive intelligent calculation tasks from CPU, including AI inference and parallel image processing intelligent calculation.

[0009] Preferably, the GPU or NPU in the intelligent calculation unit is at least two, and the GPU or NPU is installed on the board card where the intelligent calculation unit is located in the form of a module.

[0010] Preferably, the embedded operating system is a general Linux operating system, which includes general mathematical intelligent calculation tools, image processing public library, neural network model library, running scheduling module, database and load balancing module. The general mathematical intelligent calculation tool includes library functions of general mathematical intelligent calculation. The image processing tool includes library functions of general image processing. The neural network model library includes YOLO, SSD and ResNet. The running scheduling module dynamically allocates intelligent calculation tasks to available intelligent calculation resources according to the monitoring result of the load balancing module, and manages the execution life cycle of the tasks; the intelligent calculation resources include CPU, GPU or NPU and FPGA. The database is used for storing system configuration parameters, task metadata, state information of processing result and system health management data. The load balancing module monitors the load state, memory and display memory usage of each GPU or NPU in real time, and provides decision basis for the running scheduling module.

[0011] Preferably, the embedded operating system further includes an information security module, which performs identity authentication and integrity check on uplink instructions, uplink programs or model parameters in the process of starting the intelligent calculation machine system, executing application programs, and encrypts and decrypts the data stored and transmitted to prevent malicious code execution.

[0012] Preferably, the hardware driving layer comprises CPU driving, NPU driving and GPU driving, and the embedded operating system identifies and accesses different physical chips through the CPU driving, NPU driving and GPU driving, and directly interacts with the physical chips of the corresponding hardware resource layer.

[0013] Preferably, the storage unit adopts a solid-state storage unit single-board master-slave structure, and the entire solid-state storage unit comprises two module boards, all of which are installed on the solid-state storage unit structural member and then inserted into the BUS mother board in the single machine.

[0014] Another technical scheme of the application is a starting method of the high-performance satellite-borne intelligent terminal. Step one: first, send a starting instruction to the terminal, and the configuration chip of the FPGA first writes the BIT file in the FLASH to the FPGA configuration area; if the FPGA configuration is successful, jump to step four, otherwise, enter step two; Step two: the configuration chip of the FPGA starts the second loading, rewrites the BIT file in the FLASH to the configuration area of the FPGA, and if the FPGA configuration is successful, jump to step four, otherwise, enter step three; Step three: the configuration chip of the FPGA starts the third loading, rewrites the BIT file in the FLASH to the configuration area of the FPGA, and if the FPGA configuration is successful, jump to step four, otherwise, enter step one; Step four: start to determine whether to start the CPU, if the CPU is started, enter step five, otherwise, still in step four; Step five: start the UBOOT of the CPU, the CPU reads the UBOOT configuration data, completes the UBOOT starting, and after the CPU initialization is completed, enter step six, otherwise, enter step four; Step six: start the operating system, the CPU reads the operating system kernel from the SSD, completes the operating system starting, and after the control right is transferred to the operating system, enter step seven, otherwise, enter step four; Step seven: start the service plug-in of the CPU, the CPU reads the service plug-in from the SSD, completes the starting of the service plug-in, and enter step eight, otherwise, enter step four; Step eight: the CPU completes the starting and performs the normal working state, receives the working mode instruction, if the GPU or NPU needs to be started, enter step nine, otherwise, enter step ten; Step nine: start the GPU or NPU, power on the GPU or NPU, and after the power on, the GPU or NPU reads the configuration file to complete the starting, and enter step ten; Step ten: the CPU completes the corresponding application service APP starting according to the received application service APP starting instruction, and after the starting is completed, the terminal enters the working state.

[0015] The beneficial effects of the present application compared with the prior art are: (1) The present application realizes true software and hardware decoupling and application modular management by constructing a five-layer protocol architecture based on the operating system "hardware physical layer-hardware driver-embedded operating system-resource scheduling management layer-application service layer", greatly simplifies the algorithm transplantation and application deployment process under the premise of meeting the high-reliability satellite intelligent computing demand, supports the on-orbit evolution and upgrade of mainstream AI framework and APP, and changes the satellite from a functional solidified hardware platform to an intelligent platform with sustainable growth capability.

[0016] (2) The routing, intelligent computing and storage units of the present application use all-optical Ethernet for standard protocol interconnection, build a high-speed and unified network communication environment, and guarantee the efficient transmission of high-throughput service streams such as payload data and processing data, while having excellent electromagnetic compatibility and system expandability compared with the non-standard (such as 2711, GTH) scheme in the prior art.

[0017] (3) The present application uses a minimum unit integration technology based on "energy flow-service flow-control flow" three buses to refine the system into three core units of routing, intelligent computing and storage, and realizes interconnection through an optical / electrical backplane, which simplifies the internal structure of the system and supports on-demand splicing and integration of high, medium and low configurations, and flexibly adapts to various satellite application scenarios compared with the prior art architecture with dispersed functional modules and low integration.

[0018] (4) The present application realizes the synergy and shared access capability of management, intelligent computing and storage capability by using a general-purpose computing and storage integrated architecture of "high-performance CPU+FPGA+high-computing GPU or NPU (GPU or NPU)+storage", which effectively improves the system resource utilization efficiency.

[0019] (5) The intelligent computing unit of the present application realizes flexible configuration and on-demand calling of intelligent computing power by using GPU or NPU modular design and starting process controlled by CPU according to instructions, which realizes the optimization of system energy efficiency and the unity of high computing power and high energy efficiency compared with the prior art scheme of fixed power-on of computing unit and high power consumption under the premise of meeting the complex AI algorithm processing demand. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a hardware architecture diagram of the satellite intelligent computing terminal of the embodiment of the present application; Figure 2 is a satellite intelligent computing terminal platform architecture diagram of the embodiment of the present application; Figure 3 The software architecture of the satellite-borne intelligent terminal of the embodiment of the application; Figure 4 The starting flow of the routing unit / intelligent unit of the embodiment of the application. DETAILED DESCRIPTION

[0021] The application will be further described below in combination with embodiments.

[0022] The application proposes a high-performance satellite-borne intelligent terminal, adopts a general storage and computing integrated architecture technology of "high-performance CPU+FPGA+high-computing-power GPU or NPU (GPU or NPU)+storage", a minimum unit integration technology based on three buses, and full-optical Ethernet interconnection technology, and provides strong support for satellite intelligent application and networked service, and creates a parallel high-computing-power intelligent computing platform on a satellite: satellite "supercomputing", which can be used for satellite-borne AI acceleration processing, image preprocessing, AI compression, target detection and identification processing. The system has high computing power, high integration, reconfigurability, and a standardized Ethernet interface, can be integrated on demand to form high, medium and low configurations, adapt to different application scenario requirements, and support application APP software on-orbit evolution and upgrade.

[0023] The satellite-borne intelligent terminal proposed in the application adopts a general storage and computing integrated architecture technology of "CPU+FPGA+GPU or NPU+storage", and adopts a layered open system of "hardware physical layer+hardware driver+embedded operating system+resource scheduling management layer+application service layer" on the software, which has been successfully verified on the ground and will be applied on-orbit on a satellite in the future, and adapts to the needs of rapid development of on-orbit intelligent processing of future remote sensing satellites.

[0024] With the rapid development of aerospace technology, the amount of data generated by satellites and other spacecraft increases dramatically, and higher requirements are put forward for data processing capability. The traditional on-orbit data processing mode is limited by the speed and efficiency of intelligent computing, and it is difficult to meet the needs of on-orbit high-time-efficiency and intelligent information service. High-performance GPU or NPU chips have become an important research direction in the field of on-orbit data processing due to their powerful parallel intelligent computing capability and high energy efficiency ratio. CPU, GPU, NPU and FPGA each have advantages and disadvantages, and the advantages and disadvantages of several mainstream chips are listed in the following table.

[0025]

[0026] CPU has strong management ability and certain intelligent computing ability, but the efficiency of intelligent computing is limited by the upgrade of production process and the number of CPU cores. With the rapid development of artificial intelligence technology, GPU or NPU has gradually become an ideal hardware choice for deep learning application due to its excellent parallel intelligent computing ability and rich operator ecosystem, and has broad application prospects in satellite intelligence. FPGA has the strongest real-time performance and flexibility. However, due to the size and rapid iteration of AI models, the development of FPGA is becoming more and more difficult, and the development cycle is long, which is not suitable for direct use in high-performance intelligent computing. In the face of high-performance, high-intelligence and high-energy satellite intelligent computing requirements, the combination of high-performance CPU and GPU or NPU, and the use of FPGA programmable characteristics to realize the physical layer and link layer protocol of high-speed Ethernet, will become the future development trend.

[0027] 1. System composition According to the functional positioning, the corresponding hardware unit types and internal systems are summarized, analyzed and sorted out. The satellite intelligent computing terminal architecture can be divided into three types of boards, namely routing unit, storage unit and intelligent computing unit. The layered open system of "hardware physical layer + hardware driver + embedded operating system + resource scheduling management layer + application service layer" is adopted to improve the flexibility of application APP development, allowing users to install and load application APPs, realize the customization of system functions according to requirements, and complete specific tasks. The system composition of the satellite intelligent computing terminal is shown in Figure 1 .

[0028] The routing unit, storage unit and intelligent computing unit are closely connected through three internal buses, forming an organic whole: the three buses are independent of each other and are responsible for the interaction of control flow, business flow and energy flow. Control flow (mainly through RS422 and electrical backplane): the routing unit distributes task instructions, data injection and remote control instructions to the intelligent computing unit through the control flow, and collects the telemetry state information of each unit to realize the monitoring and scheduling of the terminal system.

[0029] Business flow (mainly through all-optical Ethernet and optical backplane): when processing data, the intelligent computing unit can read the data to be processed from the storage unit at high speed through the business flow, and write the processed result data back to the storage unit at high speed through the business flow. The routing unit is responsible for managing the path and routing of all data flow.

[0030] Energy flow (through electrical backplane): the routing unit receives the whole satellite power supply and is responsible for power distribution and management of the routing unit, intelligent computing unit and storage unit. The routing unit, intelligent computing unit and storage unit provide primary power, and then convert the primary power into secondary power or tertiary power and provide it to the internal circuit.

[0031] The routing unit and the intelligent computing unit both run embedded operating systems, and the operating systems run resource scheduling management programs and application services APP. Under the scheduling of the operating systems, different application software plug-ins are run through the scheduling management of different tasks and components; different application software plug-ins are used to realize health monitoring and fault tolerance of the platform, error correction management, and intelligent computing in combination with space application scenarios. The application software plug-ins and the resource call run in the operating system, and the plug-ins access the function library resource database model and the like in the to-be-used operation through scheduling management. The operating system accesses different chip information through driving recognition. Figure 3 Both are described from the functional level and have this function. The routing unit provides interface communication with the outside, receives remote control parameters and reconstruction information input from the outside, supports the rapid transplantation and deployment of multi-user application software through CPU+FPGA+GPU or NPU, supports the flexible configuration of computing power according to different user business needs, and supports the dynamic loading of different application software. The storage unit is used for storing data and processed data, and the main stored data includes: Input data: original slice data or intermediate data to be processed obtained by the payload.

[0032] Output data: effective result data processed by the intelligent computing unit (such as detected target information, compressed images, classification results, etc.) and AI model parameters and application programs updated on-orbit.

[0033] In addition, the storage unit supports multi-user shared access to the same file, and ensures data safety and reliability through primary and backup redundant design.

[0034] The external interface of the satellite-borne intelligent computing terminal includes a power supply interface, an RS422 bus, and an all-optical Ethernet interface. The system internally includes three buses of energy flow, business flow, and control flow, which are independent of each other. The energy flow provides primary power for each unit, and each unit converts the primary power into secondary power and tertiary power and provides it to chips such as FPGA and CPU. The business flow realizes the Ethernet interconnection of each unit, and mainly transmits payload data and processing data, which are used to realize the access, distribution, and processing of high-speed data. The control flow realizes the RS422 interconnection of each unit, and mainly transmits remote control instructions and telemetry information, system monitoring and management. The data uploading network is mainly used for data uploading and software maintenance.

[0035] Preprocessed data: standardized data generated after the original payload data (such as image slices) are preprocessed through radiation correction and geometric correction.

[0036] Intelligent processing intermediate results: intermediate feature maps and temporary variables generated by the AI model when performing tasks.

[0037] Intelligent processing of final products: Detection and recognition results: structured information such as target location, category, confidence, etc.

[0038] Classification and segmentation results: such as ground-to-object classification map, cloud detection map, image segmentation results, etc.

[0039] Change detection results: change information obtained by comparing images of the same area at different times.

[0040] Compressed data: to save storage space and downstream bandwidth, image or video data compressed by AI or traditional algorithms.

[0041] Fusion data: new products obtained by fusing data from different sensors or at different times. For example, the fusion of infrared remote sensing data and microwave remote sensing data, and the fusion of data obtained by detecting the same target at different time periods.

[0042] In general, processed data covers all valuable data generated in the entire on-orbit processing chain from raw data to final information products.

[0043] Physical multiplexing: the data upload function is physically multiplexed with the existing "control flow" RS422 interface in the system.

[0044] Logical independence: in logic, the operating system divides a separate network channel or sets a specific communication protocol for the key maintenance function of "data upload" to ensure that the upload data (such as software, model parameters) can be safely, reliably, and orderly transmitted to the routing unit, and is distinguished from the conventional telemetry and remote control data stream, and is managed by priority, for example, the priority of remote control instructions is higher than that of upload data.

[0045] Workflow: upload data is sent to the routing unit by RS422 interface from the outside, and then distributed to the corresponding intelligent computing unit or storage unit by the internal RS422 (control flow) of the routing unit.

[0046] The on-board intelligent computing terminal needs to implement functions such as on-orbit maintenance and upgrade of programs, and the maintained programs include CPU programs and FPGA configuration bit streams, and the upgraded programs include AI model parameters and APP software. The external routing unit receives through the RS422 channel, and then distributes to the corresponding intelligent computing unit or storage unit through the internal RS422.

[0047] 1.2.3 Platform architecture The satellite-borne intelligent computing terminal is composed of a routing unit, a storage unit and an intelligent computing unit. The routing unit mainly realizes optical fiber data transceiving, health management, Ethernet bus communication, RS422 bus communication, data routing management, task scheduling management, remote control and telemetry, and reconstruction of uploading, centralized management of system programs, application programs and configuration programs of the intelligent computing unit, and on-off control of the internal primary power module of the platform.

[0048] The intelligent computing unit is managed based on embedded system software, and supports fast transplantation and deployment of multi-user application software, flexible configuration of computing power according to different user business requirements, and dynamic loading of different application software through heterogeneous intelligent computing and software and hardware collaborative optimization of high-performance domestic CPU+FPGA+GPU. The domestic GPU and general GPU are subjected to aerospace-level reinforcement design and verification. The storage unit realizes single-board main and backup, and has a capacity greater than 10 Tb, and is used for storing slice data and processed data, and has shared access of different users to the same file.

[0049] The routing unit receives the primary electric bus of the whole satellite, realizes the on-off of the platform routing unit through external direct instructions, and realizes the secondary power required by the single board through the power module; the primary power of the intelligent computing unit and the storage unit is interconnected by the routing unit through the connector, and the on-off of the secondary power of the single board is controlled by the indirect instructions from the routing unit.

[0050] The routing unit: the terminal contains two routing units, one main and one backup (complete cold backup), to improve the system reliability. The routing unit is mainly composed of a 7-series FPGA, a high-performance multi-core CPU, a high-computing-power GPU or NPU, an SSD solid state disk, an optical fiber interface, an RS422 bus interface, a DDR cache and the like; the routing unit mainly completes load data receiving and routing management, task scheduling management, remote control and telemetry and reconstruction of uploading, centralized management of system programs, application programs and FPGA configuration programs, and on-off control of each power module.

[0051] As shown in Figure 2 , the routing unit includes an FPGA, a CPU, an SSD, a GPU or an NPU; The CPU: as the routing processor of the system, it runs the operating system and the resource scheduling management program. It is responsible for internal task scheduling, health management, remote control and telemetry processing, centralized management of system programs, application programs and FPGA configuration programs, and sending control instructions to other units. It is connected with the FPGA, the SSD and the GPU or the NPU through the PCIE bus, and is the control and intelligent computing center of the system.

[0052] FPGA: As the interface control and data exchange hub of the system. Mainly responsible for implementing the physical layer and link layer protocol of the high-speed fiber interface, RS422 bus interface, completing the initial reception, routing exchange and distribution of payload data. At the same time, manage the DDR cache, and monitor the key state of the system. It is connected with CPU, GPU or NPU, SSD and external interface, and undertakes the scheduling task of high-speed data flow.

[0053] SSD: As the system disk and storage disk of the system. Used to store the operating system kernel, service plug-in, application program, AI model and temporary data. Its non-volatile characteristics ensure that all programs and data are not lost after power failure, ensuring that the system can be started normally and upgraded on orbit. Directly connected with CPU and FPGA, providing high-speed data storage services for the system and application. The system disk is used to store the operating system kernel, service plug-in and application program, and the storage disk is used to store the AI model and temporary data. FPGA is used to control the work of SSD, including startup, reset, power down, etc.

[0054] GPU or NPU (GPU or NPU): As a dedicated intelligent coprocessor, responsible for offloading intensive intelligent tasks from CPU, focusing on AI reasoning, parallel image processing and other intelligent algorithms that require high throughput and high energy efficiency. Interconnected with CPU through high-speed PCIE bus to realize high-speed data interaction. “CPU+SSD+GPU or NPU in the software system (CPU+SSD+GPU or NPU) Figure 3 ) together constitute the hardware resource layer. Their cooperative working mode (such as CPU calling GPU or NPU driver to execute intelligent algorithm, loading model from SSD) is realized and described in the five-layer protocol architecture shown in Figure 3 .

[0055] Intelligent unit: There are 4 intelligent units, designed with ring backup, which can support 4 units to start at the same time. The intelligent unit is composed of 7 series FPGA, high-performance multi-core CPU, 2 high-performance GPUs or NPUs, etc. GPU or NPU is designed as a module and installed on the carrier board. The intelligent unit mainly completes the running of various intelligent processing algorithms on board, and builds the system program required by various processing application APP based on the intelligent unit. Under the scheduling of the operating system, through the scheduling management of different tasks and components, different application services can be completed. At the same time, combined with the space application scene, the health monitoring and fault tolerance, error correction management of the platform software and hardware state are realized, and the evolution and update of model, parameter, algorithm, system program and application program on orbit are realized.

[0056] The internal chips of the intelligent unit as the core of computing power are divided and routed similarly, but the emphasis is different: CPU: responsible for the task management of the intelligent computing unit, communication with the routing unit, and scheduling of local FPGA and GPU or NPU resources.

[0057] GPU or NPU: the main source of computing power for the intelligent computing unit, usually equipped with 2, dedicated to running various intelligent processing APPs such as target detection, image segmentation, and deep learning algorithms.

[0058] FPGA: In addition to auxiliary interface management, it focuses on real-time data processing with extremely high requirements, such as hardware acceleration of specific image preprocessing algorithms, data format conversion, etc., and forms a complementary computing power with GPU or NPU.

[0059] Storage unit: The storage unit single board realizes main and backup, and the storage unit adopts a solid-state storage unit single board main and backup structure. The entire solid-state storage unit contains 2 module boards, which are installed on the solid-state storage unit structure after being inserted into the single machine BUS motherboard. The solid-state storage unit has a storage capacity of >10 Tb, supports multi-user storage sharing, and uses a high-reliability radiation-resistant FPGA as the data management core.

[0060] Optical backplane: The optical backplane is a kind of optical passive product that realizes complex optical signal cross-connection through optical fiber network topology connection. The optical backplane realizes the optical fiber cross-connection between the routing unit, the intelligent computing unit, and the storage unit of the intelligent terminal.

[0061] Electrical backplane: The electrical backplane realizes the interconnection of internal instructions, telemetry, and power supply, all of which are low-frequency signals.

[0062] 1.2.4 Software Scheme The software and hardware are decoupled through the development of hardware physical layer, embedded operating system, hardware driver, resource scheduling management layer, and application service layer. The 5-layer protocol architecture of the satellite-borne intelligent terminal is shown in Figure 3 The CPU, FPGA, DDR, GPU, and SSD of the routing unit and the intelligent computing unit constitute the hardware resource layer of the platform. Combined with the corresponding hardware driver, different application services can be completed through the scheduling and management of tasks and components under the scheduling of the operating system. At the same time, considering the space application scenario, the health monitoring and fault tolerance, error correction management of the platform software and hardware state are required, and the evolution and update of models, parameters, algorithms, system programs, and application programs are realized in orbit.

[0063] The satellite-borne intelligent terminal adopts a general Linux operating system, and the GPU or NPU is compatible with the mainstream general intelligent computing technology ecosystem, supporting mainstream AI frameworks such as TensorFlow, Pytorch, PaddlePaddle, and MindSpore.

[0064] As shown in Figure 3As shown, the operating system includes general mathematical tools, image processing tools, neural network model library, running scheduling module, database, load balancing module, information security module; General mathematical tools include library functions of general mathematical intelligence.

[0065] Image processing tools include functions of general image processing.

[0066] Neural network model library includes YOLO, SSD, ResNet.

[0067] The running scheduling module is responsible for dynamically allocating intelligence tasks to available intelligence resources (such as CPU cores, GPUs or NPUs), and managing the execution life cycle of the tasks. It calls the information of the load balancing module for decision making, and performs task scheduling through CPU / NPU / GPU drivers.

[0068] The database is used to store structured data such as system configuration parameters, processing result state information, and system health management data.

[0069] The load balancing module is used to monitor the load state, memory and video memory usage of each intelligence unit and its internal GPU or NPU in real time, provide decision basis for the running scheduling module, and ensure efficient and balanced use of system computing power. It is the "intelligence center" of the running scheduling module, and its data comes from the operating system kernel and hardware drivers.

[0070] The information security module is used to provide security protection for the entire system, including identity authentication and integrity check for uplink instructions and uplink programs / models, encryption and decryption of stored and transmitted data, and prevention of malicious code execution. It runs through multiple links such as system startup, data uploading, and application execution, and is the "guardian" of reliable operation of the system.

[0071] CPU driver, NPU driver, GPU driver location: These drivers are located in the driver layer and directly interact with the physical chips in the hardware resource layer.

[0072] Function: They are the bridge for the operating system to recognize, manage and call hardware resources, and "translate" general intelligence instructions from the upper layer into commands that can be executed by specific hardware.

[0073] Resource scheduling management program includes task distribution module, resource management module, task uploading module, data channel; Task distribution module: receives instructions from routing unit or task planning APP, and distributes specific tasks to idle intelligence units.

[0074] Resource management module: responsible for the identification, state tracking and allocation scheduling of intelligence, storage and network resources in the platform.

[0075] Task injection module: responsible for receiving, verifying and managing new APPs or models transmitted through the data injection network.

[0076] Data channel: a logical path that manages data transmission between APPs.

[0077] Service plug-ins are a series of reusable general-purpose function modules, such as: Data decoding / encoding plug-in: handles payload data in different formats.

[0078] General image processing plug-in: provides basic algorithms such as radiometric calibration and geometric correction.

[0079] Health monitoring plug-in: collects and reports hardware status data.

[0080] Function: They are called by the upper application service layer to avoid repeated development of basic functions by each APP, achieving "high cohesion and low coupling".

[0081] The application service layer includes intelligent task planning APP, hot area extraction APP; Cloud detection APP: automatically identifies cloud coverage areas in images.

[0082] Ship target detection APP: real-time detection and positioning of ships in marine images.

[0083] Onboard data compression APP: uses AI algorithms to efficiently compress images, reducing the amount of downlink data.

[0084] In-orbit information extraction APP: quickly generates disaster range information products for sudden events such as floods and forest fires.

[0085] Data acquisition and analysis function: this function is completed by multiple levels in coordination.

[0086] Data acquisition: the FPGA and sensor interface of the hardware resource layer complete the acquisition of raw data.

[0087] Data analysis: specific APPs in the application service layer (such as target detection APP, hot area extraction APP) call the algorithm library of the service plug-in layer and the scheduling module of the operating system layer, ultimately driving the GPU or NPU of the hardware resource layer to complete intelligent calculation.

[0088] System startup: the startup process of the onboard intelligent calculation terminal is shown in the following figure.

[0089] The specific process of powering on the onboard intelligent calculation terminal is as follows: Step one: first send the terminal start-up instruction, the FPGA configuration chip first writes the BIT file in the FLASH to the FPGA configuration area. After the FPGA configuration is successful, jump to step four, otherwise enter step two.

[0090] Step two: the FPGA configuration chip starts the second loading, rewrites the BIT file in the FLASH to the FPGA configuration area, and if the FPGA configuration is successful, jump to step four, otherwise enter step three.

[0091] Step three: the FPGA configuration chip starts the third loading, rewrites the BIT file in the FLASH to the FPGA configuration area, and if the FPGA configuration is successful, jump to step four, otherwise report the 'FPGA configuration failure' error code and enter the safe standby state; Step four: start to determine whether to start the CPU, if the CPU is started, enter step five, otherwise report the 'CPU startup failure' error code and enter the safe standby state.

[0092] Step five: start the UBOOT of the CPU, the CPU reads the UBOOT configuration data from the memory, completes the UBOOT startup, and enters step six after the CPU initialization is completed, otherwise enters step four.

[0093] Step six: start the operating system, the CPU reads the operating system kernel from the SSD to the memory, starts to execute the kernel code, and enters step seven after the operating system startup is completed, otherwise enters step four.

[0094] Step seven: start the service plug-in of the CPU, the CPU reads the service plug-in from the SSD, completes the startup of the service plug-in, and enters step eight, otherwise enters step four.

[0095] Step eight: the CPU completes the startup and performs the normal working state, receives the working mode instruction sent by the FPGA, and after the instruction is parsed, if the GPU or NPU needs to be started, enter step nine, otherwise enter step ten. In the embodiment of the application, the working mode includes preprocessing, AI compression, target detection and recognition, hotspot area extraction, etc., wherein the AI compression, target detection and cloud detection need to start the GPU or NPU. Step nine: start the GPU or NPU, power on the GPU or NPU, read the configuration file after power on, complete the startup, and enter step ten.

[0096] Step ten: the CPU completes the corresponding APP software startup according to the received RS422 instruction, and after the startup is completed, the terminal enters the working state. The APP can be: target detection and recognition APP, image segmentation APP, AI compression APP, intelligent task planning APP, hotspot area extraction APP, cloud detection APP, etc.

[0097] Although the present application has been disclosed with reference to the preferred embodiments, it is not intended to limit the present application to the preferred embodiments. Any person skilled in the art, without departing from the spirit and scope of the present application, can make possible changes and modifications to the technical solutions of the present application by using the disclosed methods and technical contents. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application, without departing from the technical solutions of the present application, shall fall within the protection scope of the technical solutions of the present application.

Claims

1. A high-performance spaceborne intelligent computing terminal, characterized in that... Includes routing unit, storage unit, and intelligent computing unit; Both the routing unit and the intelligent computing unit include a hardware physical layer, a hardware driver layer, an embedded operating system, a resource scheduling management layer, and an application service layer. The application service layer is used to configure different application service apps according to aerospace application scenarios and functional requirements. The application service apps include apps that use intelligent AI models for computation. For different application service apps, the embedded operating system calls the corresponding service plugins of the resource scheduling management layer, as well as the general task distribution and resource management modules in the operating system. Combined with the corresponding hardware drivers, it flexibly configures the computing power and hardware resources required by the intelligent AI model for multi-user application software. The routing unit is used to provide an interface for communication with the outside world, and to realize fiber optic data transmission and reception, health management, Ethernet bus communication, bus communication, data routing management, task scheduling management, remote control and telemetry, and centralized management of upper-level structures, embedded operating system programs or application programs or hardware configuration programs of the routing unit and intelligent computing unit, as well as power control functions. The intelligent computing unit is used to dynamically load the on-board intelligent AI model and its parameters to complete the operation of the on-board intelligent AI model; The storage unit is used to store the raw data of the onboard payload, the intermediate data processed by the onboard intelligent AI model, and the processed data.

2. The high-performance spaceborne intelligent computing terminal according to claim 1, characterized in that... The routing unit, storage unit, and intelligent computing unit are connected by three buses. The three buses are independent of each other and are responsible for the interaction of control flow, service flow, and energy flow, respectively. The control flow is implemented using RS422. The routing unit distributes task instructions, uploads data, and remote control instructions to the intelligent computing unit through the control flow, and collects telemetry status information of each unit to realize terminal monitoring and scheduling. The business flow is implemented using Ethernet. When the intelligent computing unit processes data, it reads the data to be processed from the storage unit through the business flow and writes the processed result data back to the storage unit through the business flow. The energy flow provides primary power to the routing unit, intelligent computing unit, and storage unit, which then converts the primary power into secondary or tertiary power for use by the internal circuitry.

3. A high-performance spaceborne intelligent computing terminal according to claim 1, characterized in that... The upload data and remote control and telemetry data streams are logically separated through different communication protocols and are subject to priority management.

4. A high-performance spaceborne intelligent computing terminal according to claim 1, characterized in that... Both the routing unit and the intelligent computing unit include FPGA, CPU, SSD solid-state drive, GPU or NPU; The FPGA module connects to the CPU, GPU or NPU, SSD solid-state drive and external interfaces. It is used to perform protocol conversion for external data interfaces, is responsible for sending and receiving external data, controls data access on the SSD solid-state drive, and completes front-end processing of payload data, including image hardware acceleration and data format conversion processing, forming a complementary computing power with the GPU or NPU. The CPU module is used to run the embedded operating system and resource scheduling management layer program, load AI models from the SSD solid-state drive, and call the GPU or NPU to perform intelligent computing; SSD solid-state drives are used to store embedded operating system kernels, service plugins, application service APP programs, AI models, and temporary data. GPUs or NPUs are interconnected with CPUs via a high-speed PCIe bus. As intelligent computing coprocessors, they are responsible for offloading intensive intelligent computing tasks from the CPU, including AI inference and parallel image processing computing.

5. A high-performance spaceborne intelligent computing terminal according to claim 1, characterized in that... The intelligent computing unit contains at least two GPUs or NPUs, which are installed in the form of modules on the board where the intelligent computing unit is located.

6. A high-performance spaceborne intelligent computing terminal according to claim 1, characterized in that, The embedded operating system is a general-purpose Linux operating system, which includes general-purpose mathematical computing tools, image processing public libraries, neural network model libraries, a runtime scheduling module, a database, and a load balancing module. General-purpose mathematical computing tools, including library functions for general-purpose mathematical computing; Image processing tools, including library functions for general image processing; Neural network model libraries, including YOLO, SSD, and ResNet; The scheduling module dynamically allocates intelligent computing tasks to available intelligent computing resources based on the monitoring results of the load balancing module, and manages the execution lifecycle of the tasks; the intelligent computing resources include CPU, GPU or NPU and FPGA; The database is used to store system configuration parameters, task metadata, status information of processing results, and system health management data. The load balancing module monitors the load status, memory and video memory usage of each GPU or NPU in real time, providing a basis for the scheduling module to make decisions.

7. A high-performance spaceborne intelligent computing terminal according to claim 6, characterized in that... The embedded operating system also includes an information security module. During the startup of the intelligent computing system, data uploading, and application execution, the information security module performs identity authentication and integrity verification on the uplink instructions, uploading programs, or model parameters, and encrypts and decrypts the stored and transmitted data to prevent malicious code execution.

8. A high-performance spaceborne intelligent computing terminal according to claim 6, characterized in that, The hardware driver layer includes CPU driver, NPU driver, and GPU driver. The embedded operating system identifies and accesses different physical chips through the CPU driver, NPU driver, and GPU driver, and interacts directly with the physical chips of the corresponding hardware resource layer.

9. A high-performance spaceborne intelligent computing terminal according to claim 1, characterized in that... The storage unit adopts a solid-state storage unit single-board main and backup structure. The entire solid-state storage unit contains two module boards. After all the module boards are installed on the solid-state storage unit structural components, they are plugged into the BUS motherboard inside the single machine.

10. The startup method of the high-performance spaceborne intelligent computing terminal according to claims 1-9, characterized in that, The steps after powering on the terminal are as follows: Step 1: First, send a power-on command to the terminal. The FPGA configuration chip first writes the BIT file in FLASH to the FPGA configuration area. If the FPGA configuration is successful, proceed to step 4; otherwise, proceed to step 2. Step 2: The FPGA configuration chip starts the second loading, rewriting the BIT file in FLASH into the FPGA configuration area. If the FPGA configuration is successful, proceed to Step 4; otherwise, proceed to Step 3. Step 3: The FPGA configuration chip starts loading for the third time, rewriting the BIT file in FLASH into the FPGA configuration area. If the FPGA configuration is successful, proceed to Step 4; otherwise, proceed to Step 1. Step 4: Determine whether to start the CPU. If the CPU is started, proceed to Step 5; otherwise, remain in Step 4. Step 5: Start U-Boot on the CPU. The CPU reads the U-Boot configuration data and completes the U-Boot startup. After the CPU initialization is complete, proceed to Step 6; otherwise, proceed to Step 4. Step Six: Start the operating system. The CPU reads the operating system kernel from the SSD, completes the operating system startup, and transfers control to the operating system before proceeding to Step Seven; otherwise, proceed to Step Four. Step 7: Start the CPU service plugin. The CPU reads the service plugin from the SSD, completes the startup of the service plugin, and proceeds to Step 8; otherwise, proceed to Step 4. Step 8: The CPU completes startup and enters normal working state, receiving working mode instructions. If the GPU or NPU needs to be started, proceed to step 9; otherwise, proceed to step 10. Step Nine: GPU or NPU startup. Power on the GPU or NPU. After powering on, the GPU or NPU reads the configuration file to complete the startup and proceeds to Step Ten. Step 10: The CPU starts the corresponding application service APP according to the received application service APP start instruction. After the start is completed, the terminal enters the working state.

Images