On-orbit real-time processing method of optical images based on satellite-borne multi-core DSP

Abstract

A method for real-time on-orbit optical image processing based on a satellite-borne multi-core DSP relates to the technical field of on-orbit optical image processing. The method is applied to a multi-core processor containing multiple supernodes and includes: obtaining data to be processed and corresponding auxiliary data; determining tasks to be processed based on the data to be processed and the auxiliary data; allocating the tasks to be processed to multiple cores of the supernode based on a hybrid granularity acceleration strategy, wherein the hybrid granularity acceleration strategy includes a coarse-grained strategy for allocation by supernode and a fine-grained strategy for allocation by core; obtaining task data corresponding to the tasks to be processed from a three-level cache framework via the cores to process the tasks to be processed; the three-level cache framework includes a first-level cache corresponding to each core, a second-level cache shared by multiple cores within the same supernode, and a third-level cache shared by multiple supernodes; and outputting processing results corresponding to the data to be processed after the multiple cores complete processing. This solution can improve the efficiency of on-orbit real-time optical image processing using a multi-core processor.

Description

Technical Field

[0001] The present application relates to the technical field of on-orbit processing of optical images, and in particular to an on-orbit real-time processing method for optical images based on a satellite-borne multi-core DSP. Background Art

[0002] Optical satellites carry multi-spectral payloads, including visible and infrared, and are unrestricted by territory or airspace, enabling global, wide-area target detection. They are currently widely used in fields such as Earth remote sensing and emergency rescue. With the advancement of optical payload technology, image resolution has reached sub-meter levels. While this allows for the acquisition of finer target features, it also leads to a dramatic increase in data rates, making real-time downlinks between satellites and the ground difficult to transmit. The traditional "on-board image acquisition—downlink from satellite to the ground—centralized processing on the ground" model is unable to meet the demands of time-sensitive missions such as emergency rescue. Therefore, on-orbit real-time processing is crucial for optical satellite systems to perform time-sensitive missions.

[0003] Real-time on-orbit optical image processing is primarily accomplished using processors such as central processing units (CPUs), field-programmable logic arrays (FPGAs), digital signal processors (DSPs), and artificial intelligence (AI). DSPs, with their low power consumption, high computing power, and flexibility, are widely used. Currently, many satellites utilize DSP chips for a variety of functions, including onboard feature extraction, fire and volcano detection, target detection, target tracking, onboard remote sensing image compression, and on-orbit satellite image processing.

[0004] Existing on-orbit real-time processing technologies for optical images generally use single-core or multi-core DSPs, which have problems such as low computing power of single-core DSPs and low efficiency of multi-core parallel computing. Therefore, how to improve the processing efficiency of on-orbit real-time processing of optical images has become an urgent problem to be solved. Summary of the Invention

[0005] To solve the above problems, this application discloses the following technical solutions:

[0006] In a first aspect, the present application provides an on-orbit real-time processing method for optical images based on a satellite-borne many-core DSP, which is applied to a many-core processor. The many-core processor includes multiple supernodes, each of which includes multiple cores. The method includes:

[0007] Obtaining data to be processed and auxiliary data corresponding to the data to be processed;

[0008] determining a plurality of tasks to be processed according to the data to be processed and the auxiliary data;

[0009] Allocating the plurality of pending tasks to the plurality of cores based on a hybrid granularity acceleration strategy, the hybrid granularity acceleration strategy comprising a coarse-grained strategy and a fine-grained strategy, the coarse-grained strategy comprising allocating the plurality of pending tasks corresponding to the same channel to the cores of the same supernode, and the fine-grained strategy comprising allocating the plurality of pending tasks corresponding to the same channel to the cores of different supernodes;

[0010] Obtaining, by the kernel, task data corresponding to the task to be processed from a three-level cache framework to process the task to be processed, wherein the three-level cache framework includes a first-level cache corresponding to each kernel, a second-level cache shared by multiple kernels in the same supernode, and a third-level cache shared by multiple supernodes;

[0011] After determining that the processing of the plurality of kernels is completed, the processing results corresponding to the data to be processed are output.

[0012] Optionally, also include:

[0013] Obtaining a remote control reconfiguration instruction, wherein the remote control reconfiguration instruction is used to indicate a target algorithm that needs to be updated;

[0014] determining a target supernode for running the target algorithm based on the remote control reconstruction instruction;

[0015] Hot reconfiguration of the target supernode is performed using a master kernel that does not belong to the target supernode.

[0016] Optionally, at least one of the following is also included:

[0017] Detecting an abnormal core in an abnormal state in the same supernode through the main core of the supernode, resetting the abnormal core, and performing hot reconstruction on the abnormal core if the reset fails;

[0018] When an abnormal supernode is found, all cores of the abnormal supernode are reset by the main core of another supernode except the abnormal supernode, and hot-reconstruct the abnormal supernode if the reset of the abnormal supernode fails;

[0019] The abnormal super node refers to a super node whose main core is in an abnormal state, and the abnormal super node reset failure includes that the main core of the abnormal super node is still in an abnormal state after the reset process.

[0020] Optionally, methods for hot-reconfiguring any supernode include:

[0021] Controlling each core of the super node to stop running;

[0022] Move the program data of each kernel of the super node to the target storage space and record the storage address in the register;

[0023] Controlling each core of the supernode to restart;

[0024] The corresponding program data is loaded again from the target storage space to each core based on the storage address recorded in the register.

[0025] Optionally, allocating the plurality of tasks to be processed to the plurality of cores based on a hybrid granularity acceleration strategy includes:

[0026] Allocating multiple to-be-processed tasks of the same channel to multiple cores of a supernode corresponding to the channel based on the coarse-grained strategy;

[0027] Determine the predicted processing latency based on the data volume of the same channel and the real-time latency of the core currently participating in the task to be processed;

[0028] When the predicted processing delay does not meet the delay condition, multiple tasks to be processed in the same channel are allocated to multiple cores of the super node corresponding to the channel and cores of other super nodes based on the fine-grained strategy.

[0029] Optionally, the method of assigning any pending task to any core includes:

[0030] Search the message buffer pool for a target message whose message valid identifier is 0 and whose message identifier is consistent with the identifier of the main kernel that assigns the task;

[0031] Writing the task to be processed, the event number and the identifier of the target core to which the task to be processed is assigned into the target message, and setting the message valid identifier of the target message to 1;

[0032] An interrupt signal is sent to the target core through a multi-core communication module to trigger the target core to obtain the task to be processed from the target message, wherein after the target core obtains the task to be processed from the target message, the message valid flag of the target message is set to 0.

[0033] Optionally, methods for obtaining task data from the three-level cache framework include:

[0034] Reading task data from the first-level cache;

[0035] In the case that the first-level cache does not obtain the task data, the task data is written into the first-level cache from the second-level cache or the third-level cache through direct memory access, and then the task data is read from the first-level cache.

[0036] Optionally, obtaining, by the kernel, task data corresponding to the task to be processed from a three-level cache framework to process the task to be processed includes:

[0037] When the task data includes matrix operation data requiring matrix operation, or when intermediate data generated by processing the task data includes matrix operation data, sending the matrix operation data to the computing array cluster so that the computing array cluster operates the matrix operation data;

[0038] Receive the calculation result of the matrix operation data fed back by the computing array cluster, and process the to-be-processed task based on the calculation result.

[0039] Optionally, sending the matrix operation data to a computing array cluster so that the computing array cluster operates on the matrix operation data includes:

[0040] Sending a calculation request corresponding to the matrix operation data to an array reconstruction center, so that the array reconstruction center reconstructs the calculation array cluster based on the calculation request. The calculation array cluster after the calculation resource reconstruction is used to calculate the matrix operation data.

[0041] Optionally, determining that the processing of the plurality of kernels is completed includes:

[0042] After the first task synchronization module of the main core receives the task completion message sent by the second task synchronization modules of the plurality of cores, it is determined that the processing of the plurality of cores is completed.

[0043] The present application provides a method for on-orbit real-time processing of optical images based on a satellite-borne multi-core DSP, which is applied to a multi-core processor containing multiple supernodes. The method includes: obtaining data to be processed and corresponding auxiliary data; determining the task to be processed based on the data to be processed and the auxiliary data; allocating the task to be processed to multiple cores of the supernode based on a hybrid granularity acceleration strategy, wherein the hybrid granularity acceleration strategy includes a coarse-grained strategy for allocation by supernode and a fine-grained strategy for allocation by core; obtaining task data corresponding to the task to be processed from a three-level cache framework through the core to process the task to be processed, wherein the three-level cache framework includes a first-level cache corresponding to each core, a second-level cache shared by multiple cores in the same supernode, and a third-level cache shared by multiple supernodes; after determining that the processing of multiple cores has been completed, outputting the processing results corresponding to the data to be processed. The above scheme can improve the efficiency of on-orbit real-time processing of optical images using a multi-core processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0045] Figure 1 This is a flow chart of a method for on-orbit real-time processing of optical images based on a satellite-borne multi-core DSP, provided in an embodiment of the present application;

[0046] Figure 2 This is a schematic diagram of the architecture of a supernode and a kernel provided in an embodiment of the present application;

[0047] Figure 3 This is a flow chart of a method for allocating multiple pending tasks to multiple cores based on a hybrid granularity acceleration strategy provided by an embodiment of the present application;

[0048] Figure 4 This is a schematic diagram of the structure of an acceleration core provided in an embodiment of the present application;

[0049] Figure 5 This is a schematic diagram of the time consumption of matrix operations performed by a kernel and a computing array cluster provided in an embodiment of the present application;

[0050] Figure 6 This is a schematic diagram of a core connection relationship for implementing an inter-core status detection mechanism provided by an embodiment of the present application;

[0051] Figure 7 This is a schematic diagram of a thermal reconstruction method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0053] The existing on-orbit real-time processing solutions for optical images have the following problems.

[0054] Under the condition of limited onboard hardware resources, it is only suitable for mission scenarios with small-scale images and a small number of targets.

[0055] Image processing involves large-scale operations such as matrix multiplication and addition, convolution, and filtering. The existing software-based computing acceleration methods are not effective, and the real-time performance of on-board processing needs to be improved.

[0056] Under space radiation environment, the existing multi-core DSP chips have poor on-orbit stability and reliability.

[0057] Compared with single-core and multi-core DSPs, multi-core DSPs have more cores and can provide stronger computing power for real-time processing of optical images by on-orbit satellites. However, multi-core DSPs have a complex structure, a large number of cores, and difficulty in scheduling between cores. How to improve the efficiency and reliability of on-orbit parallel computing has become a new challenge.

[0058] To address these challenges, this application proposes an innovative multi-core layered, directional interconnected computing architecture based on a multi-core DSP, specifically designed for real-time on-orbit processing of high-volume imagery and massive targets. This architecture proposes a hybrid granularity computing acceleration method and a hardware-software collaborative acceleration method with separate storage and computation. Furthermore, it proposes inter-core detection and thermal reconfiguration design methods tailored to space radiation environments. This solution significantly improves onboard image throughput and target capacity, ensuring stable and reliable on-orbit operation.

[0059] This application embodiment provides a method for real-time processing of optical images on-orbit based on a satellite-borne multi-core DSP. Figure 1 , which is a flowchart of the method.

[0060] The method of this embodiment may be executed by a many-core digital signal processor (DSP), which may be referred to as a many-core DSP or a many-core processor. The many-core processor may include multiple supernodes, each of which includes multiple cores.

[0061] Multi-core processors can be installed on satellites in orbit to perform real-time on-orbit processing of images captured by the satellites.

[0062] An in-orbit satellite may be configured with one or more multi-core processors, each of which may process images captured by the satellite based on the method of this embodiment.

[0063] S101: Obtain data to be processed and auxiliary data corresponding to the data to be processed.

[0064] Among them, the data to be processed may include various image data captured by the satellite where the multi-core processor is located through the camera module, including but not limited to the original image, suspected target point slices obtained based on the original image, and the trajectory list obtained by processing the suspected target point slices.

[0065] S102: Determine a plurality of tasks to be processed according to the data to be processed and the auxiliary data.

[0066] S103, allocating multiple pending tasks to multiple cores based on a hybrid granularity acceleration strategy, the hybrid granularity acceleration strategy includes a coarse-grained strategy and a fine-grained strategy, the coarse-grained strategy includes allocating multiple pending tasks corresponding to the same channel to the cores of the same supernode, and the fine-grained strategy includes allocating multiple pending tasks of the same channel to the cores of different supernodes.

[0067] S104, obtaining task data corresponding to the task to be processed from the three-level cache framework through the kernel to process the task to be processed. The three-level cache framework includes a first-level cache corresponding to each kernel, a second-level cache shared by multiple kernels in the same super node, and a third-level cache shared by multiple super nodes.

[0068] S105 , after determining that the processing of the multiple cores is completed, outputting the processing results corresponding to the data to be processed.

[0069] In some embodiments, the data to be processed may be a slice of suspected target points, and the corresponding processing result may be a target trajectory linked list.

[0070] The present application provides a method for on-orbit real-time processing of optical images based on a satellite-borne multi-core DSP, which is applied to a multi-core processor containing multiple supernodes. The method includes: obtaining data to be processed and corresponding auxiliary data; determining the task to be processed based on the data to be processed and the auxiliary data; allocating the task to be processed to multiple cores of the supernode based on a hybrid granularity acceleration strategy, wherein the hybrid granularity acceleration strategy includes a coarse-grained strategy for allocation by supernode and a fine-grained strategy for allocation by core; obtaining task data corresponding to the task to be processed from a three-level cache framework through the core to process the task to be processed, wherein the three-level cache framework includes a first-level cache corresponding to each core, a second-level cache shared by multiple cores in the same supernode, and a third-level cache shared by multiple supernodes; after determining that the processing of multiple cores has been completed, outputting the processing results corresponding to the data to be processed. The above scheme can improve the efficiency of on-orbit real-time processing of optical images using a multi-core processor.

[0071] Before executing step S101 , the many-core processor may be powered on first. After powering on, the many-core processor may first execute an initialization process, and then execute the method of the above embodiment after the initialization process is completed.

[0072] The initialization process may include the following steps:

[0073] After the many-core DSP is powered on, the master core of each supernode can execute assembly code, deploy address remapping space, and move the computing core code to be run by each core other than the master core (also known as the slave core) in the supernode to the pre-allocated physical address in the first-level cache corresponding to each slave core through Direct Memory Access (DMA) technology. After the move is completed, the master core and each slave core enter their corresponding main program code.

[0074] After entering the main program code, each core can initialize on-chip peripherals, calculation parameters, and data memory pool, start the hardware acceleration core, create interrupts, and turn on the inter-core detection flag. This completes the initialization process.

[0075] Among them, each super node can include multiple cores with the same architecture. Before the many-core processor is started, relevant personnel can pre-configure one core of each super node as the master core and the other cores as slave cores. The slave cores are mainly used to process various pending tasks issued by the master core.

[0076] All cores contained in a supernode can also be collectively referred to as computing cores.

[0077] In view of the high data rate and computational complexity of multi-target and multi-task on-orbit real-time image processing, which is difficult to meet with the computing power of a single-core DSP, the multi-core processor used in this embodiment can provide far more cores than the mainstream 8-core DSP through the two-layer computing architecture of super nodes plus computing cores.

[0078] The on-orbit processing data of optical images includes original images, slices of suspected target points, trajectory lists, etc. Due to the limited on-chip storage and SMC capacity within the supernode, existing multi-core processors, such as 8-core DSPs, generally store data in the third-generation double data rate synchronous dynamic random access memory (DDR3), and each core computing core accesses data through the data network CrossNet.

[0079] Compared to existing multi-core DSP memory architectures, some multi-core DSPs lack additional cache space between DDR3 and each core's on-chip memory. On-chip memory is extremely limited (for example, L1D memory is only 32KB), resulting in less cacheable computational data and a consequently slower execution speed. Consequently, traditional cache-based memory optimization methods require frequent data transfers between DDR3 and on-chip memory, making cache coherency maintenance difficult and resulting in low operational efficiency.

[0080] To address the above issues, based on the aforementioned two-layer computing architecture, the many-core processor of this embodiment may adopt a three-level cache framework to store data.

[0081] The three-level cache framework includes L1 cache, L2 cache, and L3 cache. L1 cache is the on-chip storage for the computing core, that is, each computing core has an independent L1 cache that is only available to that computing core. A L1 cache can include the L1 data cache L1D and the L1 instruction cache L1P.

[0082] The L2 cache is a shared cache within each supernode. Each supernode has a L2 cache that is shared by all cores within the supernode. Typically, a L2 cache has 2 megabytes (MB) of storage space. The L2 cache can be a Shared Memory Controller (SMC) cache.

[0083] The third-level cache is an external storage. In this embodiment, the third-level cache may be two groups of DDR3 shared by all super nodes in the many-core processor. Each group of DDR3 may have 2 gigabytes (GB) of storage space.

[0084] In this embodiment, multiple computing cores within a supernode can access the supernode's L2 cache (SMC) via the data network CrossNet. Any computing core within any supernode can access the L3 cache (DDR3) via the configuration network CrossNet. In terms of access speed, the L1 cache has the highest access rate, followed by the L2 cache, and the L3 cache has the lowest access rate. In terms of storage space, the L3 cache has the largest storage space, followed by the L2 cache, and the L1 cache has the smallest storage space.

[0085] In step S101 , the main core of each supernode may obtain the data to be processed and the auxiliary data corresponding to the data to be processed through other processing modules of the satellite where it is located.

[0086] Different main cores may obtain different data to be processed. For example, other processing modules may divide the image data currently to be processed into multiple channels of data to be processed, and distribute the multiple channels of data to be processed to the main cores of multiple super nodes.

[0087] by Figure 2 For example, the data to be processed and its auxiliary data of channel 1 can be distributed to the main core of supernode 1, that is, Figure 2 The core 1 shown in the figure distributes the data to be processed and its auxiliary data of channel 2 to the main core of supernode 2, that is, Figure 2 Nucleus 7 is shown.

[0088] Other processing modules may be processing modules on the satellite other than the many-core processor. These processing modules may include modules for transmitting image data captured by the camera module to the many-core processor, or modules for communication between the satellite and ground equipment. Other processing modules may be implemented using field programmable gate arrays (FPGAs) or other devices, without limitation.

[0089] The main core can receive the data to be processed and the auxiliary data in a deep storage queue manner.

[0090] The many-core processor and other processing modules can exchange data through the Serial Rapid IO (SRIO) interface.

[0091] See also Figure 2In S102, each main core can run a dynamic task planning module and a multi-core communication module. The dynamic task planning module can parse the to-be-processed data and auxiliary data according to a preset data protocol, and then generate multiple to-be-processed tasks based on the parsed data.

[0092] Among them, the method of parsing the data to be processed and the auxiliary data, and generating multiple tasks to be processed according to the parsed data can be referred to the relevant existing technology and will not be described in detail.

[0093] Among them, the dynamic task planning module and the multi-core communication module can be regarded as software modules running on the corresponding computing cores and composed of program codes.

[0094] In step S103, the dynamic task planning module can assign multiple pending tasks to multiple slave cores for processing based on a hybrid granularity acceleration strategy. After the assignment is completed, the master core can transmit the multiple pending tasks to the multiple slave cores through the multi-core communication module, so that each slave core uses its own task processing module to process the received pending tasks.

[0095] The task processing module can be regarded as a software module consisting of program codes running on the corresponding computing core.

[0096] The task processing module can read the pending tasks issued by the main kernel, which may include task types and task data, and process the current task using a three-level cache framework in accordance with the hierarchical cache processing method.

[0097] Optional, such as Figure 3 As shown, the dynamic task planning module may include the following steps to allocate multiple tasks to be processed to multiple kernel processes based on the mixed granularity acceleration strategy.

[0098] S301 : Allocate multiple to-be-processed tasks of the same channel to multiple cores of a supernode corresponding to the channel based on a coarse-grained strategy.

[0099] S302 : Determine a predicted processing delay based on the data volume of the same channel and the real-time delay of the core currently participating in the task to be processed.

[0100] S303 : When the predicted processing delay does not satisfy the delay condition, multiple tasks to be processed in the same channel are allocated to multiple cores of the super node corresponding to the channel and cores of other super nodes based on a fine-grained strategy.

[0101] It should be noted that the above steps can be repeated. For example, after any master core obtains the data to be processed, the dynamic task planning module of the master core can assign the tasks to be processed in the manner of step S301. Then, while each slave core is processing its own task to be processed, the dynamic task planning module can periodically or irregularly repeat step S302 and, based on the processing latency predicted after each execution of step S302, execute or not execute step S303.

[0102] In step S301 , after determining a plurality of tasks to be processed, the dynamic task planning module may assign the plurality of tasks to be processed to the respective slave cores contained in the same super node by default.

[0103] Combine Figure 2 In the example, core 1 can determine multiple tasks to be processed based on the data to be processed of channel 1, and assign these tasks to cores 2 to 6 of supernode 1 for processing. Core 7 can determine multiple tasks to be processed based on the data to be processed of channel 2, and assign these tasks to cores 8 to 12 of supernode 2 for processing.

[0104] In step S302 , the dynamic task planning module of each main core may run a delay prediction function, and the delay prediction function may determine a predicted processing delay according to the data volume and the real-time delay.

[0105] Then, in step S103, the dynamic task planning module can dynamically plan the number of cores required to process the data of the corresponding channel based on the predicted processing delay to address the load imbalance problem when there is a large difference in the processed data between channels, avoid the single-core single task running too long and causing the data network CrossNet used for data transmission to be blocked, and thus dynamically adjust the load of each core in the many-core processor.

[0106] The basic principle of the dynamic task planning module is that if the data volume of channel 1 is too large in a certain period of time, while the data volume of channel 2 is moderate at the same time, the module will schedule several cores in supernode 2 that are responsible for processing channel 2 to channel 1 to help process the massive data of channel 1, while ensuring that the tasks of channel 2 itself can be completed on time, thereby improving efficiency and enabling each channel to meet the timeliness requirements.

[0107] In step S302, on the one hand, each slave core can feed back its own real-time delay to the delay prediction function of the master core while processing its own task; on the other hand, before the many-core processor runs in orbit, it can statistically calculate the expected runtime delay of each slave core when processing different tasks based on the simulation test results of the algorithm under multiple targets and multiple scenarios, and record these expected runtime delays as a prediction benchmark.

[0108] On this basis, the delay prediction function can predict the predicted processing delay of each slave core based on the amount of data to be processed in the corresponding channel and the real-time delay fed back by each slave core.

[0109] The predicted processing delay of the slave core can indicate how long it is expected to take for the slave core to complete the currently assigned task. The method for determining the predicted processing delay using the delay prediction function can be found in the related prior art and will not be described in detail here.

[0110] In step S303 , the dynamic task planning module may compare the predicted processing delay with the pre-recorded expected processing delay to determine whether the predicted processing delay satisfies the delay condition.

[0111] If the predicted processing delay is less than or equal to the expected processing delay, or if the predicted processing delay is greater than the expected processing delay but the excess is less than or equal to the preset threshold, that is, the predicted processing delay is not much different from the expected processing delay, then it can be determined that the delay condition is met.

[0112] If the predicted processing delay is greater than the expected processing delay, and the excess is greater than a preset threshold, that is, the predicted processing delay is much greater than the expected processing delay, it can be determined that the delay condition is not met.

[0113] When the predicted processing delay of the slave core corresponding to a certain channel does not meet the delay condition, the dynamic task planning modules corresponding to these slave cores can execute S303 and allocate multiple tasks to be processed of the same channel to multiple cores of the super node corresponding to the channel and cores of other super nodes based on fine-grained strategies.

[0114] The slave core corresponding to a channel refers to a slave core used to process data corresponding to the channel. The dynamic task planning module corresponding to a slave core refers to a dynamic task planning module running on the master core of the supernode to which the slave core belongs.

[0115] When executing S303 , the dynamic task planning modules of different master cores may communicate with each other to determine the predicted processing delays of their respective slave cores.

[0116] by Figure 2 For example, the dynamic task planning module of core 1 and the dynamic task planning module of core 7 can both send the predicted processing delay and expected processing delay of the slave core of their own super node to each other. If the predicted processing delay of the slave core of super node 1 does not meet the delay condition, and the predicted processing delay of the slave core of super node 2 is much smaller than the corresponding expected processing delay, then, after negotiation, the dynamic task planning module of core 1 and the dynamic task planning module of core 7 can schedule part of the slave core of super node 2 to process the pending tasks corresponding to channel 1, so that the predicted processing delay of the slave core corresponding to channel 1 can meet the delay condition.

[0117] As an example, after negotiation, core 8 can be scheduled to process the tasks corresponding to channel 1. In this case, the multi-core communication module of core 1 can send inter-processor communication (IPC) instructions to core 8, and send the pending tasks corresponding to channel 1 to core 8 through the multi-core communication module, so that core 8 processes the pending tasks corresponding to channel 1. At the same time, core 7 no longer sends pending tasks to core 8. As a result, core 8 will be used to process the pending tasks corresponding to channel 1, thereby reducing the predicted processing delay of the slave core corresponding to channel 1 and improving the data processing efficiency of channel 1.

[0118] Through the above method, the dynamic task planning module can dynamically correct the original prediction benchmark during data processing, making the multi-core processor more robust when facing complex environments.

[0119] Compared to conventional multi-core processors, many-core processors have more cores, but each core has a lower clock speed. If some cores take too long to execute parallel tasks, the remaining cores will be blocked at synchronization points, resulting in overall performance degradation. Therefore, how to schedule tasks across multiple cores to achieve load balancing becomes a key factor affecting the parallel processing efficiency of many-core processors.

[0120] Furthermore, for scenarios involving real-time on-orbit processing of highly time-sensitive targets, where moving targets have fast speeds and uneven spatial distribution, traditional coarse-grained scheduling methods will lead to unbalanced loads on multiple cores, while fine-grained scheduling methods will result in excessive inter-core communication overhead.

[0121] In view of the above problems, this embodiment proposes the following Figure 3 The hardware-friendly hybrid granularity acceleration strategy shown in the figure combines coarse-grained and fine-grained task scheduling, dynamically adjusts multi-core processing resources to achieve load balancing, and improves multi-core parallel processing efficiency.

[0122] Optionally, the method for the multi-core communication module to transmit any pending task to the corresponding slave core may include:

[0123] Search the message buffer pool for a target message whose message valid identifier is 0 and whose message identifier is consistent with the identifier of the main kernel that assigns the task;

[0124] Write the pending task, event number and the identifier of the target core to which the pending task is assigned into the target message, and set the message valid identifier of the target message to 1;

[0125] An interrupt signal is sent to the target core through the multi-core communication module to trigger the target core to obtain a task to be processed from the target message, wherein after the target core obtains the task to be processed from the target message, the message valid flag of the target message is set to 0.

[0126] The above transmission process is illustrated below with the help of an example. In this example, it is assumed that the main core that assigns the task is core A, and the target core assigned the pending task is core B. That is, the multi-core communication module of core A transfers a pending task to core B for processing. In this scenario, core A first traverses the message buffer pool and finds a message with a message valid flag of 0 and a message flag equal to the flag of core A. It then fills in other data in the message, including but not limited to the event number, the pending task assigned to core B, the flag of the destination core, etc., where the flag of the destination core is the flag of core B, and sets the message valid flag to 1. Then, the message is sent to the message buffer pool and an interrupt signal is sent to core B. Then, core B responds to the interrupt, traverses all messages, finds a message with a destination core flag equal to its own flag and a message valid flag of 1, copies the content of the message and puts it into its own pending message queue. Then, it deletes the other data of the message in the message buffer pool and sets the message valid flag of the message to 0, thereby releasing the message. After that, core B can process the contents of the pending message queue one by one, obtain the pending tasks therein, and then process these tasks through its own task processing module.

[0127] It should be noted that the above method of transferring pending tasks to slave cores can also be applied to data interaction between any two cores. For example, when core A is about to send data to core B, core A first traverses the message cache pool, finds a message with a message valid identifier of 0 and a message identifier equal to the identifier of core A, fills in other data in the message, including but not limited to the event number, the data to be transmitted, the identifier of the destination core, etc., where the identifier of the destination core is the identifier of core B, and sets the message valid identifier to 1, then sends the message to the message cache pool, and sends an interrupt signal to core B. Core B then responds to the interrupt, traverses all messages, finds a message with a destination core identifier equal to its own identifier and a message valid identifier of 1, copies the content of the message and puts it into its own queue of pending messages, then deletes the other data of the message in the message cache pool, sets the message valid identifier of the message to 0, thereby releasing the message, and then core B can process the contents of the queue of pending messages one by one to obtain the data therein.

[0128] In the above embodiment, the event number can be randomly generated by the kernel each time other data is filled in.

[0129] The message buffer pool can be maintained by the multi-core communication module of each main core. All cores of the multi-core processor can access the message buffer pool, and the message buffer pool can be used to cache multiple messages. The above messages can also be called posts.

[0130] Optionally, in addition to maintaining the message buffer pool, the multi-core communication module can also be responsible for a series of register-level underlying applications such as initializing interrupts, sending interrupts, and responding to interrupts.

[0131] In step S104, the method for obtaining task data from the three-level cache framework may include:

[0132] A1, reads task data from the first-level cache;

[0133] A2: If the first-level cache does not obtain the task data, the task data is written into the first-level cache from the second-level cache or the third-level cache through direct memory access, and then the task data is read from the first-level cache.

[0134] In this embodiment, the first-level cache of each computing core may be set as the static random-access memory (SRAM) of the computing core. Specifically, the L1D of the first-level cache may be set as the SRAM.

[0135] On this basis, when any core processes a corresponding task, if it needs to obtain the corresponding task data, it can first search whether there is corresponding task data in the first-level cache corresponding to the core. If so, the task data can be read directly from the first-level cache. At the same time, the data that needs to be saved when processing the task can be directly written into the corresponding first-level cache.

[0136] If the required data is not available in the L1 cache, the task data is written to the L1 cache from the L2 cache or L3 cache via direct memory access (DMA), and then read from the L1 cache.

[0137] At the same time, DMA can also be used to move data written to the first-level cache to the third-level cache DDR3.

[0138] In the three-level cache framework, DMA is located in the data exchange center of DSP. It can connect the first-level cache and the second-level cache internally and the third-level cache externally through the relevant bus.

[0139] In step A2, for data with small amount and frequent access, the data can be directly moved from the third-level cache DDR3 to the first-level cache L1D of the core that needs the data through DMA, and then the core reads the data from the first-level cache.

[0140] For data with large data volume and low access frequency, DMA can be used to move the data from the third-level cache DDR3 to the second-level cache of the super node that needs the data. Then, according to the needs of each core, the data is moved from the second-level cache to the first-level cache L1D of the core that needs the data, and then the core reads the data from the first-level cache.

[0141] In the above embodiments, task data generally refers to any data that the kernel needs to obtain when processing a task to be processed.

[0142] Through the above solution, this embodiment achieves integrated storage and computing, with L1D responsible for efficient computation of small data volumes and DDR3 for large-capacity data storage. Compared to direct cache usage, this solution fully utilizes the high read and write bandwidth of L1D and the large storage capacity of DDR3, significantly improving data reading and computing efficiency.

[0143] In this embodiment, when the kernel obtains the task data corresponding to the task to be processed from the three-level cache framework to process the task to be processed, on the one hand, each kernel can process the assigned task to be processed, and on the other hand, each kernel can also transmit some data that requires matrix operation during the processing process to the computing array cluster for operation, thereby improving the task processing efficiency.

[0144] That is, the execution process of step S104 may include:

[0145] B1, when the task data includes matrix operation data that requires matrix operation, or when the intermediate data generated by processing the task data includes matrix operation data, sending the matrix operation data to the computing array cluster so that the computing array cluster operates the matrix operation data;

[0146] B2, receiving the calculation results of the matrix operation data fed back by the computing array cluster, and processing the tasks to be processed based on the calculation results.

[0147] The step of sending the matrix operation data to the computing array cluster to trigger the computing array cluster to operate the matrix operation data may include:

[0148] Sending a calculation request corresponding to the matrix operation data to the array reconstruction center enables the array reconstruction center to reconstruct the calculation array cluster based on the calculation request. The calculation array cluster after the calculation resource reconstruction is used to calculate the matrix operation data.

[0149] See Figure 4The computing array cluster can be implemented by FPGA and can be deployed in the acceleration core. There can be one or more acceleration cores on a satellite. The acceleration core can have a massive target data interface, which can be connected to the many-core processor through a fast configuration channel, so that the acceleration core and the many-core processor can interact with data. At the same time, the many-core processor can be connected to the many-core processor algorithm data source through the massive target transmission channel. The massive target data interface of the acceleration core can also be connected to the many-core processor algorithm data source through the massive target transmission channel, so that both the many-core processor and the acceleration core can load algorithms and data from the many-core processor algorithm data source.

[0150] The computing array cluster may include complex adders, complex multipliers, real adders, real multipliers, fixed-float converters, floating-point dividers and other devices.

[0151] In the above embodiments, the algorithm can be understood as the algorithm program code that the core or computing array cluster needs to execute. By executing these algorithm program codes, the core or computing array cluster can process data according to the corresponding algorithm and obtain corresponding processing results.

[0152] See further Figure 4 ,An acceleration core can include a computing array cluster, an array ,reconfiguration center, an array cluster controller, a DMA, a massive target data interface and a ,data center.

[0153] The computing array cluster in the acceleration core can adopt an FPGA architecture. The acceleration core can be mounted as an independent hardware peripheral on the circuit of the many-core processor, thereby accelerating specific operations such as matrix and convolution.

[0154] In step B1, if a core finds that the task data or intermediate data contains matrix operation data that requires matrix operation while processing its own pending tasks, then the core can send an operation request corresponding to the matrix operation data to the array cluster controller through the main core of the super node to which it belongs. The operation request is used to request the use of the computing array cluster to operate on the corresponding matrix operation data.

[0155] The array cluster control forwards the computing request to the array reconstruction center, and the array reconstruction center reconstructs the computing resources of the computing array cluster according to the computing request, thereby reconstructing the computing array cluster into specific computing resources suitable for processing the computing request.

[0156] As some examples, the array reconstruction center can reconfigure computing array clusters into resources for matrix addition and subtraction, matrix inversion, matrix dot multiplication, matrix conjugation, vector operations, Doppler calculations, fast Fourier transforms, and finite impulse response (FIR) filtering. It can even request customization of individual computing resources before shipment.

[0157] At the same time, the array cluster controller can cache the algorithm and matrix operation data required for matrix operation from the algorithm data source of the many-core processor to the data center of the acceleration core through DMA.

[0158] Then, the computing array cluster reconstructed into specific computing resources can load the corresponding algorithms and matrix operation data from the data center, thereby performing operations on these matrix operation data to obtain operation results.

[0159] The obtained calculation results can be stored in the data center first, and then moved to the multi-core processor algorithm data source through DMA, or can remain in the data center.

[0160] During the matrix operation of the computing array cluster, the cores of the many-core processor do not need to wait for the operation results, and can synchronously process their respective pending tasks until the core needs to use the operation results of the matrix operation data. The main core of the super node to which it belongs can send a status query request to the array cluster controller to inquire whether the corresponding matrix operation data has been completed. If the operation is completed, the operation results can be obtained from the many-core processor algorithm data source or from the data center of the acceleration core. If the operation is not completed, it is possible to wait until the operation is completed before obtaining the operation results.

[0161] The method for the array reconstruction center to reconstruct resources of the computing array cluster according to the operation request can be found in the relevant existing technology and will not be described in detail.

[0162] In the on-orbit processing algorithm for time-sensitive targets, the spatial coordinate transformation matrix must be calculated in real time based on the satellite's position, spatial attitude, motion state, and the orientation of the onboard high-resolution camera. This involves a large number of matrix operations. Matrix operations are highly complex, and directly using DSP cores is inefficient.

[0163] To address the above problems, this solution combines the computing array cluster integrated in the many-core processor and proposes the above-mentioned software and hardware collaborative acceleration method based on storage and computing separation.

[0164] The kernel can flexibly implement various functions through developers writing specific programs, such as C language programs. In contrast, the structure of the computing array cluster is fixed when it leaves the factory, so it can only process a limited number of algorithms (such as matrix operations, vector operations, Doppler, FFT, FIR, etc.). However, the computing array cluster is very efficient and can also process large amounts of data in batches. It can also be executed in parallel with the kernel without consuming the kernel's resources.

[0165] Therefore, through the above-mentioned software-hardware collaborative acceleration method based on storage and computing separation, this solution can assign matrix operations with large computational workloads to more efficient computing array clusters for parallel computing, thereby further improving the processing efficiency of on-orbit real-time image processing.

[0166] As an example, see Figure 5 and Table 1, Figure 5 Table 1 shows the latency of performing 1000 matrix operations of varying order and type using kernels and compute array clusters, respectively. Using the compute array cluster significantly reduces the time required for matrix operations compared to kernel computing. This is especially true for large-scale matrix multiplications (e.g., 9×9 matrices), where the speedup ratio reaches 93.41. Therefore, this collaborative hardware and software acceleration approach significantly improves the efficiency of matrix operations. The speedup ratio is defined as the ratio of kernel processing time divided by the compute array cluster processing time.

[0167] Table 1

[0168]

[0169] Compared to ground-based processing, on-orbit processing faces space radiation exposure, such as single-particle and cosmic ray radiation, placing higher demands on reliability design. To address this issue, this embodiment also provides a mechanism for inter-core status detection. This mechanism is used to implement core and supernode reset and hot reconfiguration, enabling software fault tolerance and fault recovery capabilities for the many-core processor.

[0170] Specifically, the inter-core status detection mechanism and the corresponding reset and hot reconfiguration capabilities can be implemented in the following ways:

[0171] C1, detects abnormal cores in the same supernode that are in abnormal state through the main core of the supernode, resets the abnormal core, and performs hot reconstruction on the abnormal core if the reset fails;

[0172] C2, when an abnormal supernode is found, all cores of the abnormal supernode are reset through the main core of another supernode except the abnormal supernode, and hot-reconstruct the abnormal supernode if the reset of the abnormal supernode fails;

[0173] The abnormal super node refers to a super node whose main core is in an abnormal state, and the abnormal super node reset failure includes that the main core of the abnormal super node is still in an abnormal state after the reset process.

[0174] The inter-core state detection mechanism can be achieved through Figure 6 The connection relationship shown is realized.

[0175] like Figure 6 As shown, within a supernode, each slave core can periodically feed its own status information back to the supernode's master core, allowing the master core to determine whether each slave core in the supernode is in an abnormal or normal state based on this status information. A core's status information includes, but is not limited to, its basic operational heartbeat and its execution information. The execution information can reflect the current stage of the pending task being processed by the core, specifically the stage at which the pending task is currently running.

[0176] In C1, if a master core detects that any one or more slave cores in the supernode are in an abnormal state, the master core will identify these slave cores in an abnormal state as abnormal cores and perform the following reset process on each abnormal core:

[0177] The pending tasks currently being processed by the abnormal core are reallocated to other slave cores, and then a reset operation is performed on the abnormal core. After the reset operation is performed, if the abnormal core is in a normal state, the reset is determined to be successful. At this time, the core can be included in the task allocation queue, and pending tasks can continue to be assigned to the core. If the abnormal core is still in an abnormal state, the reset is determined to have failed, and hot reconstruction can be performed on the abnormal core.

[0178] Therefore, this solution can avoid calculation anomalies and maintain the dynamic balance of multi-core processing.

[0179] by Figure 6 For example, core 0, which is the master core, obtains the running heartbeat and execution information of multiple slave cores in supernode 0, namely core 1 to core n, to detect the status of these cores. When it is found that core n is in an abnormal state, the pending tasks of core n are assigned to other cores, and a reset operation is performed on core n. If core n is in a normal state after the reset, the pending tasks continue to be assigned to core n. If core n is in an abnormal state after the reset, the core n is hot-reconstructed.

[0180] The number of cores contained in a supernode is not limited. As an example, a supernode can contain 7 cores. Accordingly, the cores contained in supernode 0 can be numbered from core 0 to core 6. In the above example, n can be 6.

[0181] Furthermore, the main kernels of each supernode can detect each other's status. That is, the main kernel of any supernode can send its own status information to the main kernels of other supernodes, and can receive the status information of the main kernels of other supernodes to detect the status of the main kernels of other supernodes based on the status information of the main kernels of other supernodes.

[0182] by Figure 6 For example, core x, core y and core z are the main cores of supernode 1, supernode 2 and supernode 3 respectively. Core 0, as the main core of supernode 0, can receive core x, core y and core z to detect the status of these three main cores. At the same time, core x can also receive the status information of core 0 to detect the status of core 0.

[0183] In step C2, if any master core detects that the master core of another supernode is in an abnormal state, the supernode with the master core in the abnormal state may be treated as an abnormal supernode, and all cores of the abnormal supernode may be reset.

[0184] The method for resetting all cores of the abnormal super node may be:

[0185] A normal master core in a different supernode takes over the external communication function of the master core of the abnormal supernode. After the takeover is successful, the normal master core immediately resets each core contained in the abnormal supernode. After all cores of the abnormal supernode are reset, if the master core of the abnormal supernode is in a normal state, it is determined that the reset of the abnormal supernode is successful. At this time, the master core that performed the reset operation returns the external communication function of the master core of the reset supernode to ensure the reliable operation of the chip.

[0186] After all cores of the abnormal supernode are reset, if the main core of the abnormal supernode is in an abnormal state, it is determined that the reset of the abnormal supernode has failed. At this time, the abnormal supernode can be hot-reconstructed to troubleshoot or load a new algorithm.

[0187] Still Figure 6 For example, when core 0 detects that core x is in an abnormal state, core 0 takes over the external communication function of core x, and then performs a reset operation on all cores of supernode 1. If supernode 1 is reset successfully, core 0 returns the external communication function of core x to core x. If supernode 1 fails to be reset, supernode 1 is hot reconfigured.

[0188] Optionally, methods for hot-reconfiguring any supernode include:

[0189] Control each core of the supernode to stop running;

[0190] Move the program data of each core of the super node to the target storage space and record the storage address in the register;

[0191] The various cores that control the supernode are restarted;

[0192] Based on the storage address recorded in the register, the corresponding program data is loaded again from the target storage space to each core.

[0193] See Figure 7 After receiving the hot reconstruction request, the main core that performs hot reconstruction first controls the designated core to be reconstructed to stop working through the power sleep controller (PSC), that is, controls the core to be reconstructed to power off. When hot reconstruction is performed on a super node, the core to be reconstructed here can include all the cores of this super node.

[0194] After the cores to be reconstructed stop working, the main kernel that performs hot reconstruction moves the program data running in the cores to be reconstructed from the memory to the designated running memory corresponding to the cores to be reconstructed for storage.

[0195] For example Figure 7 In the example, if all cores of supernode 1 are hot-reconfigured, the program data of each core of supernode 1 can be moved from the memory to the designated running memory of the core to be reconfigured. The designated running memory here can be the first-level cache in the three-level cache framework or a specific cache space in other levels of cache. Among them, the program data corresponding to different algorithms can be pre-stored in the memory to cope with various imaging environments and on-board troubleshooting. The memory can be a non-volatile storage Flash. The memory can include program areas corresponding to multiple supernodes, for example, including Figure 7 The program areas of supernodes 0 to 3 are shown as being used to store program data for different supernodes. The program areas of different supernodes can be understood as pre-allocated storage space in the memory. The program data of this embodiment can be understood as computer instructions executable by the kernel.

[0196] The program area corresponding to a supernode can store program data corresponding to various algorithms suitable for running on the supernode, for example, Figure 7 Algorithms 1 to p0 stored in the program area of ​​supernode 0 can be regarded as program data corresponding to p0 algorithms suitable for running on supernode 0.

[0197] The number and types of algorithms suitable for running on different supernodes can be the same or different. For example, supernode 0 is suitable for running algorithms 1 to p0, and supernode 1 is suitable for running algorithms 1 to p1.

[0198] The program area corresponding to a super node can also store the troubleshooting program corresponding to the super node. The troubleshooting programs corresponding to different super nodes can be the same or different.

[0199] After moving the program data, the main core performing hot reconstruction can write the program entry address into the register corresponding to the core to be reconstructed. After the above preparations are completed, the main core performing hot reconstruction turns on the core power supply through the PSC to load the program, thereby powering on the core to be reconstructed. At this point, the reconstruction of the core to be reconstructed is completed.

[0200] The operation of moving program data can be done by Figure 7 The configuration module is executed. The configuration module can be a FPGA board.

[0201] After the reconstruction is completed, the algorithm can be tuned more efficiently by further comparing the state parameters output by the reconstructed supernode and the unreconstructed supernode.

[0202] After the reconstruction is completed, the data that needs to be processed by the core to be reconstructed can be redistributed by the main core of the super node to which it belongs.

[0203] When performing hot reconfiguration on a supernode, it is only necessary to treat all cores of the supernode as cores to be reconfigured and then execute the above hot reconfiguration process.

[0204] When hot reconfiguration is performed on a super node, the main core performing the hot reconfiguration may first take over the external communication function of the main core of the super node being hot reconfigured.

[0205] Combine Figure 6 For example, when core 0 performs hot reconstruction on supernode 1, the above hot reconstruction process can be performed on all cores (including core x) of supernode 1 to complete the hot reconstruction of supernode 1.

[0206] The main core may obtain a hot reconstruction request when determining that the abnormal super node fails to be reset.

[0207] In combination with the above example, core 0 performs a reset process on supernode 1, and after supernode 1 fails to be reset, core 0 obtains a hot reconstruction request for hot reconstruction of supernode 1.

[0208] If it is found that several slave cores contained in a super node fail to reset, the master core of the super node can treat these slave cores that failed to reset as cores to be reconstructed and perform the above hot reconstruction process on the cores to be reconstructed, thereby hot reconstructing these slave cores that failed to reset.

[0209] Traditional on-orbit satellite DSP reconstruction requires updating the program by erasing Flash. During the erasing and writing period, all cores of the DSP are not running. However, the hot reconstruction of this solution is to reconstruct a local core while the DSP is running. It is based on a local reset, allowing the reset core to load a new set of programs, and the cores that have not been reset to run normally.

[0210] At the same time, this embodiment can add Figure 7 The additional troubleshooting area shown is used to run troubleshooting programs, providing a quicker understanding of the onboard data processing status, allowing for rapid algorithm adjustments on the ground and subsequent on-orbit updates. Using this hot reconfiguration method can enhance the DSP's flexibility and improve the hardware platform's reliability in handling complex tasks.

[0211] In step S105, any master core may determine whether the processing of multiple cores has ended in the following manner:

[0212] After the first task synchronization module of the main core receives the task completion message sent by the second task synchronization modules of the multiple cores, it determines that the processing of the multiple cores is completed.

[0213] like Figure 2 As shown, each core may include a task synchronization module, wherein the task synchronization module of the master core may be referred to as a first task synchronization module, and the task synchronization module of the slave core may be referred to as a second task synchronization module.

[0214] The task synchronization module can be regarded as a software module consisting of program codes running on the corresponding computing core.

[0215] For any task processing module of a slave core, after the task processing module has processed the pending tasks it has received, it can notify the second task synchronization module of the slave core, and then the second task synchronization module of the slave core can send a task completion message to the first task synchronization module of the main core to indicate that the pending tasks assigned to the slave core have been processed.

[0216] After the master core distributes a batch of pending tasks to multiple slave cores, if these slave cores all feedback the above-mentioned task completion message, the master core can determine that all pending tasks have been processed. At this time, the master core can collect the processing results of each slave core, and then aggregate the processing results of each slave core to generate the processing results of the pending data. Finally, the aggregated processing results are fed back to other processing modules located upstream or downstream of the many-core processor through SRIO, such as to the upstream or downstream FPGA.

[0217] Optionally, after obtaining the processing result corresponding to the data to be processed, the method of this embodiment may further include the following steps:

[0218] Obtaining a remote control reconfiguration instruction, where the remote control reconfiguration instruction is used to indicate a target algorithm that needs to be updated;

[0219] determining a target supernode for running a target algorithm based on the remote control reconfiguration instruction;

[0220] Hot reconfigure the target supernode using a master core that does not belong to the target supernode.

[0221] In the above embodiment, the satellite where the multi-core processor is located can receive the remote control reconfiguration instruction sent by the ground equipment through the controller area network (CAN) bus. The remote control reconfiguration instruction can indicate the algorithm that needs to be updated. Then, the CAN bus sends the received remote control reconfiguration instruction to the satellite through the internal bus. Figure 7 The reconstruction module shown in FIG. The reconstruction module can also be called a processing board or a processing FPGA.

[0222] After receiving the remote-controlled reconfiguration instruction, the reconfiguration module can determine a target supernode based on the remote-controlled reconfiguration instruction. Specifically, the reconfiguration module can identify a supernode currently running a target algorithm among multiple supernodes and designate that supernode as the target supernode. The target algorithm refers to the algorithm indicated in the remote-controlled reconfiguration instruction that needs to be updated.

[0223] After determining the target supernode, the reconstruction module may send reconstruction information to any master core of the many-core processor, for example, to master core 0 of supernode 0. The reconstruction information may carry the identifier of the target supernode.

[0224] After any master core receives the reconstruction information, if it determines that the target super node is its own super node, it can send a hot reconstruction request to the master core of another super node to trigger the master core of the other super node to hot reconstruct the target super node. The hot reconstruction method is described in the above embodiment and will not be repeated here.

[0225] After any master core receives the reconstruction information, if it determines that the target supernode is not its own supernode, it can forward the reconstruction information to the master core of the target supernode through IPC, and at the same time take over the external communication function of the master core of the target supernode to perform hot reconstruction of the target supernode.

[0226] As an example, see Figure 7 ,The reconstruction module determines that the target supernode is supernode 2, and then sends the reconstruction information carrying the identifier of supernode 2 to core 0. After receiving it, core 0 determines that the target supernode is supernode 2, so it sends the reconstruction information to supernode 2 and then performs hot reconstruction on supernode 2.

[0227] As another example, the reconstruction module determines that the target supernode is supernode 0, and then sends a reconstruction message carrying the identifier of supernode 0 to core 0. After receiving the message, core 0 determines that the target supernode is supernode 0 where it is located, so it sends a hot reconstruction request to the main core of supernode 1. Then, the main core of supernode 1 performs hot reconstruction on supernode 0.

[0228] Optionally, after the hot reconstruction of the target supernode is completed, the status information of the reconstruction completion can be fed back to the ground equipment so that the corresponding telemetry equipment on the ground can be updated. After the telemetry equipment is updated, the hot reconstruction is successfully completed.

[0229] The many-core processor has a large number of cores. The above embodiment can back up different programs to Flash based on the algorithm type, and then perform local hot reconstruction of the many-core processor as needed through remote control. The reconstruction is based on a supernode. The main core of another supernode will first take over the supernode to be reconstructed, and then perform hot reconstruction on it. The reconstruction does not affect the calculation of the cores of other supernodes. Different supernode combinations can be hot reconstructed as needed, giving full play to the advantages of many cores and reducing the information transmission of the satellite-to-ground link.

[0230] Need to explain, Figure 6 and Figure 4 N may represent the number of supernodes contained in a many-core processor. The value of N is not limited, for example, it may be 20, 40 or other values, depending on the structure of the many-core processor used.

[0231] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other.

[0232] For the convenience of description, the above systems or devices are described as being divided into various modules or units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0233] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.

[0234] Finally, it should be noted that, in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0235] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for real-time on-orbit processing of optical images based on a satellite-borne multi-core DSP, characterized in that: Applied to a many-core processor, the many-core processor includes a plurality of supernodes, each of the supernodes includes a plurality of cores, the method comprising: Obtaining data to be processed and auxiliary data corresponding to the data to be processed; determining a plurality of tasks to be processed according to the data to be processed and the auxiliary data; Allocating the plurality of pending tasks to the plurality of cores based on a hybrid granularity acceleration strategy, the hybrid granularity acceleration strategy comprising a coarse-grained strategy and a fine-grained strategy, the coarse-grained strategy comprising allocating the plurality of pending tasks corresponding to the same channel to the cores of the same supernode, and the fine-grained strategy comprising allocating the plurality of pending tasks corresponding to the same channel to the cores of different supernodes; Obtaining, by the kernel, task data corresponding to the task to be processed from a three-level cache framework to process the task to be processed, wherein the three-level cache framework includes a first-level cache corresponding to each kernel, a second-level cache shared by multiple kernels in the same supernode, and a third-level cache shared by multiple supernodes; After determining that the processing of the plurality of kernels is completed, the processing results corresponding to the data to be processed are output.

2. The method according to claim 1, characterized in that Also includes: Obtaining a remote control reconfiguration instruction, wherein the remote control reconfiguration instruction is used to indicate a target algorithm that needs to be updated; determining a target supernode for running the target algorithm based on the remote control reconstruction instruction; Hot reconfiguration of the target supernode is performed using a master kernel that does not belong to the target supernode.

3. The method according to claim 1, characterized in that Also includes at least one of the following: Detecting an abnormal core in an abnormal state in the same supernode through the main core of the supernode, resetting the abnormal core, and performing hot reconstruction on the abnormal core if the reset fails; When an abnormal supernode is found, all cores of the abnormal supernode are reset by the main core of another supernode except the abnormal supernode, and hot-reconstruct the abnormal supernode if the reset of the abnormal supernode fails; The abnormal super node refers to a super node whose main core is in an abnormal state, and the abnormal super node reset failure includes that the main core of the abnormal super node is still in an abnormal state after the reset process.

4. The method according to claim 2 or 3, characterized in that Methods for hot reconfiguring any supernode include: Controlling each core of the super node to stop running; Move the program data of each kernel of the super node to the target storage space and record the storage address in the register; Controlling each core of the supernode to restart; The corresponding program data is loaded again from the target storage space to each core based on the storage address recorded in the register.

5. The method according to claim 1, wherein Allocating the plurality of tasks to be processed to the plurality of cores based on a hybrid granularity acceleration strategy includes: Allocating multiple to-be-processed tasks of the same channel to multiple cores of a supernode corresponding to the channel based on the coarse-grained strategy; Determine the predicted processing latency based on the data volume of the same channel and the real-time latency of the core currently participating in the task to be processed; When the predicted processing delay does not meet the delay condition, multiple tasks to be processed in the same channel are allocated to multiple cores of the super node corresponding to the channel and cores of other super nodes based on the fine-grained strategy.

6. The method according to claim 5, characterized in that Methods for assigning any pending task to any core include: Search the message buffer pool for a target message whose message valid identifier is 0 and whose message identifier is consistent with the identifier of the main kernel that assigns the task; Writing the task to be processed, the event number and the identifier of the target core to which the task to be processed is assigned into the target message, and setting the message valid identifier of the target message to 1; An interrupt signal is sent to the target core through a multi-core communication module to trigger the target core to obtain the task to be processed from the target message, wherein after the target core obtains the task to be processed from the target message, the message valid flag of the target message is set to 0.

7. The method according to claim 1, characterized in that The methods for obtaining task data from the three-level cache framework include: Reading task data from the first-level cache; In the case that the first-level cache does not obtain the task data, the task data is written into the first-level cache from the second-level cache or the third-level cache through direct memory access, and then the task data is read from the first-level cache.

8. The method according to claim 1, characterized in that Obtaining the task data corresponding to the task to be processed from the three-level cache framework through the kernel to process the task to be processed includes: When the task data includes matrix operation data requiring matrix operation, or when intermediate data generated by processing the task data includes matrix operation data, sending the matrix operation data to the computing array cluster so that the computing array cluster operates the matrix operation data; Receive the calculation result of the matrix operation data fed back by the computing array cluster, and process the to-be-processed task based on the calculation result.

9. The method according to claim 8, characterized in that The sending of the matrix operation data to the computing array cluster so that the computing array cluster operates the matrix operation data includes: Sending a calculation request corresponding to the matrix operation data to an array reconstruction center, so that the array reconstruction center reconstructs the calculation array cluster based on the calculation request. The calculation array cluster after the calculation resource reconstruction is used to calculate the matrix operation data.

10. The method according to claim 1, characterized in that The determining that the processing of the plurality of kernels is completed includes: After the first task synchronization module of the main core receives the task completion message sent by the second task synchronization modules of the plurality of cores, it is determined that the processing of the plurality of cores is completed.

Images