Model-based definition of remote sensing data on-board processing system and device

Abstract

The present disclosure provides a model definition-based remote sensing data on-board processing system and device, which comprises a basic resource layer configured to store at least two underlying operator modules required for implementing on-board processing tasks; and a core function layer configured to store at least two processing cores required for implementing on-board processing tasks; wherein the processing cores are combined to be implemented by at least one underlying operator module through calling the underlying operator modules in the basic resource layer; in this way, when implementing on-board processing tasks based on the system, only task instructions need to be configured in an application expression layer according to application requirements of the on-board processing tasks, and the corresponding processing cores can be called from the core function layer to generate corresponding on-board processing tasks through the task instructions, thereby reducing the rewriting rate of similar algorithms, and further reducing the development cycle of the on-board processing system, and adapting to the requirements of future satellite low cost and rapid deployment.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of remote sensing satellites, and particularly relates to a model definition-based remote sensing data on-satellite processing system and device. BACKGROUND

[0002] With the breakthrough development of artificial intelligence and the rapid improvement of on-board computing and storage capabilities, remote sensing satellites are not only a data collection and distribution platform, but also a highly intelligent space information sensing and processing node. Directly generating user-required information through on-satellite intelligent processing of satellite data has become an important development trend. Realizing on-satellite processing of remote sensing data and providing valuable information on demand to the ground can effectively reduce the transmission pressure between the satellite and the ground, improve the acquisition efficiency of time-sensitive targets and event information of the satellite, shorten the service chain of products, and is of great significance for realizing direct end-to-end service from sensors to users in the future.

[0003] At present, although some units have carried out technical research work related to on-satellite processing tasks, they are all customized development for a single satellite model, and the development cycle is long, which is difficult to adapt to the requirements of low-cost and rapid deployment of future satellites. SUMMARY

[0004] Therefore, the present disclosure provides a model definition-based remote sensing data on-satellite processing system and device, which can shorten the development cycle of on-satellite processing tasks and adapt to the requirements of low-cost and rapid deployment of future satellites.

[0005] According to a first aspect of the present disclosure, a model definition-based remote sensing data on-satellite processing system is provided, comprising:

[0006] a basic resource layer configured to store at least two bottom operator modules required for implementing on-satellite processing tasks;

[0007] a core function layer configured to store at least two processing cores required for implementing on-satellite processing tasks; wherein the processing core is implemented by at least one bottom operator module through calling the bottom operator module in the basic resource layer;

[0008] an application expression layer configured to generate a corresponding on-satellite processing task from the core function layer according to a task instruction to process input remote sensing data by executing the on-satellite processing task.

[0009] In a possible implementation manner, the bottom operator module includes at least one of a neural network construction module and a general data processing module.

[0010] In a possible implementation, the neural network construction module comprises at least one of a convolution calculation submodule, a pooling calculation submodule, a summation calculation submodule, and an activation calculation submodule.

[0011] In a possible implementation, the general data processing module comprises at least one of a data preprocessing module and a matrix calculation module.

[0012] In a possible implementation, the processing core comprises at least one of a compression encoding processing core, a target detection and recognition processing core, a pose determination processing core, a stereo matching processing core, and a bundle adjustment processing core.

[0013] In a possible implementation, when the processing core is implemented by combination of at least one of the underlying operator modules, the underlying operator modules are called by the underlying resource layer based on the data processing function to be implemented by the processing core.

[0014] In a possible implementation, the application expression layer calls the on-board processing task generated by the corresponding processing core in the core function layer according to the task instruction, and the on-board processing task comprises at least one of an image compression processing task, a target detection and recognition processing task, and a satellite surveying and mapping positioning processing task.

[0015] In a possible implementation, the image compression processing task is configured to call the corresponding processing core to perform cloud amount detection on input remote sensing data according to the image compression processing task instruction based on cloud detection, and then perform compression processing on the image with a cloud amount meeting a preset standard.

[0016] According to a second aspect of the present disclosure, a model definition-based on-board remote sensing data processing device is provided, comprising the on-board remote sensing data processing system according to any one of the first aspect.

[0017] According to a third aspect of the present disclosure, an implementation method of an on-board remote sensing data processing task is provided, and the implementation method is implemented based on the system according to any one of the first aspect, and the implementation method comprises the following steps.

[0018] determining a processing core required for executing the on-board processing task from the core function layer according to the task instruction corresponding to the on-board processing task;

[0019] calling a corresponding underlying operator module from the underlying resource layer according to the processing core, so as to implement the on-board processing task by execution of the underlying operator module.

[0020] In the present disclosure, the basic resource layer of the model-defined remote sensing data on-board processing system is configured to store at least two bottom operator modules required for implementing on-board processing tasks; the core function layer is configured to store at least two processing cores required for implementing on-board processing tasks; wherein the processing core is combined by at least one bottom operator module to realize the on-board processing task by calling the bottom operator module in the basic resource layer; in this way, when implementing the on-board processing task based on the system, only the task instruction needs to be configured in the application expression layer according to the application requirement of the on-board processing task, and the corresponding processing core can be called from the core function layer to generate the corresponding on-board processing task, which reduces the rewriting rate of similar algorithms, and further reduces the development cycle of the on-board processing system, and meets the requirements of future satellite low cost and rapid deployment.

[0021] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present disclosure and serve to explain the principles of the present disclosure.

[0023] Figure 1 FIG. 1 shows a schematic block diagram of a model-defined remote sensing data on-board processing system according to an embodiment of the present disclosure;

[0024] Figure 2 FIG. 4 shows a calculation flowchart of a convolution calculation submodule according to an embodiment of the present disclosure;

[0025] Figure 3 FIG. 5 shows a calculation flowchart of a pooling calculation submodule according to an embodiment of the present disclosure;

[0026] Figure 4 FIG. 6 shows a calculation flowchart of a summation calculation submodule according to an embodiment of the present disclosure;

[0027] Figure 5 FIG. 7 shows a calculation flowchart of an activation calculation submodule according to an embodiment of the present disclosure;

[0028] Figure 6 FIG. 8 shows a schematic block diagram of a model-defined remote sensing data on-board processing system example according to an embodiment of the present disclosure;

[0029] Figure 7 FIG. 9 shows a schematic flowchart of an implementation method of a remote sensing data on-board processing task according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] Various exemplary embodiments, features, and aspects of the present disclosure will be described below in detail with reference to the accompanying drawings. Like reference numerals in the drawings denote like or similar elements. Although various aspects of the embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0031] The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0032] In addition, for the purpose of better illustrating the present disclosure, numerous specific details are set forth in the following detailed description. One skilled in the art will appreciate that the present disclosure can be practiced without certain specific details, which are provided for purposes of illustration. In some instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the pertinent aspects of the present disclosure.

[0033] <SYSTEM EMBODIMENT>

[0034] Figure 1 A schematic block diagram of a model definition based remote sensing data on-board processing system according to an embodiment of the present disclosure is shown. As shown, the remote sensing data on-board processing system 100 includes a basic resource layer 110, a core function layer 120, and an application expression layer 130. Figure 1 The basic resource layer 110 is configured to store at least two underlying operator modules required for implementing on-board processing tasks.

[0035] The basic resource layer 110 is configured to store at least two underlying operator modules required for implementing on-board processing tasks.

[0036] In constructing the remote sensing data on-board processing system of the present disclosure, it is necessary to first determine typical on-board processing tasks according to the on-board processing purposes. Specifically, there are two purposes of on-board processing, one is to reduce the pressure of satellite-ground transmission and improve the proportion of effective information in the downlink data; the other is to monitor high-value information in real time to meet the needs of high timeliness guarantee tasks. In order to reduce the pressure of satellite-ground transmission, image compression processing is a common means, therefore, image compression processing task can be taken as a typical on-board processing task. In order to monitor high-value information in real time, target detection and recognition and rapid indication positioning of remote sensing images are basic requirements, therefore, target detection and recognition processing task and satellite mapping positioning processing task can be taken as typical on-board processing tasks. Thus, the typical on-board processing tasks determined according to the on-board processing purposes include at least one of image compression processing task, target detection and recognition processing task, and satellite mapping positioning processing task.

[0037] After determining the typical on-board processing tasks, the processing algorithms of the existing typical on-board processing tasks are analyzed and summarized, and at least two common core algorithm sets that can realize certain data processing functions are extracted as common processing cores. After determining the common processing cores, the core algorithm sets of all the common processing cores are further analyzed and summarized, and at least two common basic algorithm sets that can realize the data processing functions of the common processing cores are extracted as common basic operator modules. After determining the common basic operator modules, the basic algorithm sets of the common basic operator modules are independently encapsulated to obtain at least two basic operator modules. The obtained at least two basic operator modules are stored in the basic resource layer of the remote sensing data on-board processing system, so as to complete the construction of the basic resource layer in the system based on the at least two basic operator modules. Since the basic operator modules in the basic resource layer are extracted from typical on-board processing algorithms, the rapid deployment of various on-board processing tasks can be realized through the calling of the basic operator modules, thereby reducing the rewriting rate of the same basic operator during targeted development. At the same time, through the calling of the basic operator modules, the resource duplication caused by parallel processing tasks can be avoided, and thus the size and development cost of the load system can be reduced.

[0038] In a possible implementation, the basic operator modules of the basic resource layer can include at least one of a neural network construction module and a common data processing module.

[0039] The neural network construction module is a basic operator module required for building a neural network. In a possible implementation, the neural network construction module can include at least one of a convolution calculation submodule, a pooling calculation submodule, a summation calculation submodule, and an activation calculation submodule.

[0040] In a possible implementation, the convolution calculation submodule is constructed based on a feature map weight reuse strategy. Specifically, the calculation process of the convolution calculation submodule can be as shown in FIG. 6. Figure 2 After receiving the input feature map, the convolution calculation submodule first convolves the feature map with two groups of convolution kernels respectively, then adds the convolution results of each group of convolution kernels respectively, and finally obtains two output results. By convolving the feature map with two groups of convolution kernels at the same time, the convolution calculation submodule can greatly shorten the convolution calculation time and improve the calculation efficiency.

[0041] The input of the pooling calculation submodule is an N×N feature map, where N is an integer greater than or equal to 2. In a possible implementation, the input of the pooling calculation submodule is a 2×2 feature map, which exactly meets the requirement of one basic operation. In this implementation, the calculation process of the pooling calculation submodule can be as shown in FIG. 7. Figure 3The pooling calculation submodule sends 4 values in the feature map to 3 comparators (cmp) to select the maximum value as the output result from the 4 values through the comparators. However, in order to cooperate with other neural network construction modules, the pooling calculation submodule should also output a 2*2 feature map, and therefore the output result needs to be cached. When the cached data can generate a 2*2 feature map, 4 cached data are read to generate a 2*2 feature map, and the feature map is output as the final result of the pooling. In a possible implementation, the pooling calculation submodule is configured with 4 FIFOs, each with a capacity of 2048 and a width of 8 bits. FIFO1 and FIFO2 are responsible for caching the odd and even rows of the first column, and FIFO3 and FIFO4 are responsible for caching the odd and even rows of the second column. When FIFO4 has data, it means that the cached data can generate a 2*2 feature map, at which time, 4 cached data can be read from the FIFO to generate a 2*2 feature map, which is output as the final result.

[0042] In a possible implementation, the summation calculation submodule is configured to sum the output of the previous layer and the output of the bypass layer and output to the next layer. Since the output of the previous layer and the output of the bypass layer are both 2*2 feature maps and are one-to-one corresponding, the summation calculation submodule can directly add the two feature maps. However, due to quantization, the decimal point positions of the output data of the bypass layer and the output data of the previous layer may not be consistent, so that the two numbers cannot be directly added. Therefore, before addition, the decimal point positions of the two numbers need to be aligned. The alignment principle is to align the decimal point of the larger number to the decimal point position of the smaller number, and the decimal point alignment operation can be realized by a simple shift operation. Specifically, the calculation process of the summation calculation submodule can be as shown in Figure 4 .

[0043] In a possible implementation, the activation calculation submodule can be constructed based on the Leaky ReLU activation function. Since the output data of the previous layer is not quantized, the activation calculation submodule needs to quantize the data according to the input feature map order Ein, the output feature map order Eout and the partial sum order Epsum. The calculation of the quantized data needs to align the order to the input feature map order, so it is necessary to determine whether to shift the data to the left or to the right. When the data is less than zero, only a right shift of three bits is needed to realize the activation operation. Specifically, the calculation process of the activation calculation submodule can be as shown in Figure 5 .

[0044] With the rapid development of neural network related theory, new network structures are constantly proposed, and different network structures are used for different on-board processing tasks. Based on the configuration of the neural network construction module, the corresponding neural network construction module can be called according to the demand of the on-board processing task for the neural network structure, so as to realize the flexible and rapid construction of the neural network structure.

[0045] The general data processing module is a bottom operator module required for data processing. In a possible implementation manner, the general data processing module includes at least one of a data preprocessing module and a matrix calculation module.

[0046] In a possible implementation manner, the data preprocessing module includes at least one of an input normalization processing module, an output enablement module and a polynomial generation module.

[0047] The remote sensing image data usually has a large order of magnitude, and needs to be normalized to reduce the data order of magnitude and reduce the data storage amount, therefore, the input normalization processing module is needed to normalize the data. The normalization processing module can include at least one of an elevation normalization processing module, a longitude normalization processing module and a latitude normalization processing module.

[0048] In a possible implementation manner, the calculation model of the elevation normalization processing module can be as shown in formula (1).

[0049] H=(Height-Hei_off) / Hei_scale (1)

[0050] In the formula, Height is the input elevation value, H is the normalized elevation value, Hei-off is the center offset value of the elevation value, and Hei-scale is the scale of the elevation value.

[0051] In a possible implementation manner, the calculation model of the longitude normalization processing module can be as shown in formula (2).

[0052] L=(Longitude-Lon_off) / Lon_scale (2)

[0053] In the formula, Longitude is the input longitude value, L is the normalized longitude value, Lon-off is the center offset value of the longitude value, and Lon-scale is the scale of the longitude value.

[0054] In a possible implementation manner, the calculation model of the latitude normalization processing module can be as shown in formula (3).

[0055] P=(Latitude-Lat_off) / Lat_scale (3)

[0056] wherein Latitude is the input latitude value, P is the normalized latitude value, Lat-off is the center offset value of the latitude value, and Lat-scale is the scale of the latitude value.

[0057] It should be noted that Hei-off, Hei-scale, Lon-off, Lon-scale, Lat-off, and Lat-scale in formula (1) to formula (3) are determined according to a specific task area. Specifically, the values can be configured in a preset form, or configured and updated in an uploaded form, which is not limited herein.

[0058] The output enable module is configured to output a data enable signal. Specifically, the pedometer signal and the enable signal are input to the output enable module, and the data enable signal is output. The calculation logic of the output enable module can be designed according to the calculation requirement, so that the output data enable signal meets the calculation requirement. In one possible implementation, when the input enable signal is high, the output enable module controls the switch device to select the output solution (i.e., the data enable signal) after a 2-step delay, and the data enable signal is used as an end marker as the output of the forward intersection module. In another possible implementation, when the pedometer signal is equal to 360 steps, the flag is incremented by 1, and when the flag is equal to 3 and the pedometer signal is 360 steps and the enable signal is high, the output enable module controls the switch device to select the output data enable signal after a 56-step delay. The 360 steps are the step length of a coefficient AB of a solution, and the 56 steps include the time delay of solving a linear equation set.

[0059] In the on-board processing task, polynomials are often used to form corresponding data variables for calculation. Therefore, the polynomial generation module is required to generate data variables composed of polynomials according to input values. In one possible implementation, three values are input to the polynomial generation module, and the polynomial generation module outputs a 20-bit vector composed of first-order to third-order terms of the three values in parallel, and combines the first-order and second-order terms into one signal and the third-order term into another signal for subsequent derivation design of the polynomial equation. For example, the longitude value L, the latitude value P, and the height value H are input to the polynomial generation module, and the polynomial generation module outputs a 20-bit vector [1 P LH…L 3 H 3 ] composed of first-order to third-order terms of PLH in parallel, and combines the first-order and second-order terms into one signal and the third-order term into another signal.

[0060] In a possible implementation, the matrix calculation module comprises at least one of a coefficient matrix solving module, a matrix constructing module, a matrix output valid module, a linear equation solving module, a matrix accumulating module, a vector product calculating module, and a latitude, longitude and altitude iteration module.

[0061] The coefficient matrix solving module is configured to solve the coefficient matrix. For example, the coordinates of the homonymous point pairs of the remote sensing image, the RPC parameters, and the pedometer signal are input into the coefficient matrix solving module. The coefficient matrix solving module first calculates the latitude value P, the longitude value L, and the altitude value H of the homonymous points, i.e., the PLH values of the homonymous points, based on the coordinates of the homonymous point pairs of the remote sensing image and the RPC parameters, and performs normalization processing on the PLH values of the homonymous points. Then, the coefficient matrix solving module generates a rational function polynomial coefficient vector based on the zero-order term to the third-order term of the normalized PLH values. Next, the coefficient matrix solving module establishes an error equation according to the rational function polynomial coefficient vector, and solves the coefficient matrix A and the coefficient matrix B of the error equation. Finally, when the pedometer signal meets the set requirement, the coefficient matrix A and the coefficient matrix B are output.

[0062] The matrix constructing module is configured to construct the matrix product ATA and ATB required in the solving step of the coefficient matrix A and B, and output the matrix elements constituting the ATA matrix and the ATB matrix. The AT is the transpose matrix of the matrix A. For example, the coefficient matrix A and the coefficient matrix B output by the Solve_AB module are input into the matrix constructing module. The matrix constructing module constructs the ATA matrix and the ATB matrix. Since the ATA matrix and the ATB matrix are real symmetric matrices, only the upper half matrix elements of the ATA matrix and the ATB matrix need to be output. That is, in the output ATA matrix, A 11 , A 12 , A 13 , A 22 , A 23 , A 33 , and in the output ATB matrix, B 11 , B 12 , B 13 , B 22 , B 23 , B 33 .

[0063] The matrix output valid module is configured to output an enable signal according to the input pedometer signal. For example, a high level can be output at a fixed time according to the value of the pedometer, to ensure that the output result of the matrix constructing operator is the value at the specified time.

[0064] The linear equation set solving module is configured to solve the linear equation set. In one possible implementation, the linear equation set solving module is configured to solve the linear equation set by using an improved square root method to obtain a solution with high precision while avoiding square root operation. Specifically, the upper triangular element values of the positive definite matrix A and all element values of the matrix B are input into the linear equation set solving module, and the correction X obtained by solving is output. The linear equation set solving operator includes three calculation steps: first, decomposing the positive definite matrix A according to LDL^T; second, Ly=B; and third, DL^T X=y. Wherein, L^T is the transpose matrix of L T , y is an intermediate variable in the calculation process.

[0065] In one possible implementation, the matrix D can be as shown in equation (4), and the decomposition process of the matrix A can be as shown in equation (5).

[0066]

[0067]

[0068] In the formula, a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z are variable values.

[0069]

[0070]

[0071] l ij =t ij / d j (j=1,2,...,i-1)。

[0072] The matrix accumulation module is configured to add corresponding element values of two matrices. For example, the effective values of the ATA matrix and the ATB matrix are input into the matrix accumulation module, and the accumulated values of the ATA matrix and the ATB matrix are output. In one possible implementation, in one loop calculation process, a total of 4 effective values are input, i.e. a total of 4 effective values of the ATA matrix and the ATB matrix calculated by x and y in each view, the matrix accumulation module stores the accumulated results of the 4 effective values in the FIFO at step 232 of the counter signal, and outputs the accumulated values after one step delay.

[0073] The vector product calculation module is configured to calculate the inner product of two vectors. For example, the rational function polynomial coefficient vector is multiplied by the 20-bit RPC parameter vector to obtain the numerator F and the denominator G of the solution model. The vector product calculation module calculates by directly corresponding term multiplication and addition, which is more suitable for the parallelism characteristics of FPGA hardware compared with the loop multiplication and addition used in the software end, can reduce the calculation overhead and speed up the calculation speed.

[0074] The longitude-latitude-elevation iterative model includes at least one of a longitude iterative model, a latitude iterative model and an elevation iterative model. The implementation logics of the three iterative models are similar, and the latitude iterative model is taken as an example to exemplarily illustrate the longitude-latitude-elevation iterative model. For example, the latitude and the latitude default value are input into the latitude iterative model, and after one cycle of calculation, the correction amount X0 is obtained, and the latitude is corrected based on the correction amount X0, and the corrected latitude is iterated back to the cycle for a new round of correction amount calculation.

[0075] There are a large number of repeated data processing operations in the on-board processing task. When data processing operations are needed, the corresponding general data processing module can be called according to the data processing requirements of the on-board processing task to realize flexible and rapid construction of the data processing task.

[0076] The core function layer 120 is configured to store at least two processing cores required for implementing the on-board processing task; wherein the processing core is realized by at least one bottom operator module in the basic resource layer.

[0077] In the process of constructing the bottom operator module of the on-board processing system of remote sensing data, at least two general processing cores required for constructing typical on-board processing tasks and the bottom operator modules required for implementing the data processing functions of the general processing cores have been determined. Therefore, after the construction of the bottom operator module is completed, the calling instructions of each general processing core to at least one bottom operator module are designed according to the at least one bottom operator module required for implementing the data processing functions of each general processing core, and then the calling instructions of each processing core to at least one bottom operator module are independently encapsulated to obtain at least two processing cores. Finally, the generated at least two processing cores are stored in the core function layer to complete the construction of the core function layer.

[0078] In a possible implementation manner, the processing core includes at least one of a compression encoding processing core, a target detection and recognition processing core, a pose determination processing core, a stereo matching processing core and a adjustment processing core. The compression encoding processing core can realize compression processing of remote sensing images, the target detection and recognition processing core can detect and recognize target objects in remote sensing images, the pose determination processing core can determine high-precision high-frequency attitude data of a satellite body according to installation matrices of at least two star sensors and output attitude quaternions, the stereo matching processing core can output corresponding point pairs between stereo images according to feature information or depth information extracted from remote sensing images, and the adjustment processing core can calculate spatial coordinate values of a corresponding region of a remote sensing image according to the corresponding point pairs between the stereo images and imaging parameter information of the satellite.

[0079] The application expression layer 130 is configured to call corresponding processing cores from the core function layer according to task instructions to generate corresponding on-board processing tasks, so as to process input remote sensing data by executing the on-board processing tasks.

[0080] After the construction of the basic resource layer and the core function layer of the remote sensing data on-board processing system is completed, at least two processing cores required for implementing the on-board processing tasks are determined according to specific requirements of the on-board processing tasks, and task instructions are designed according to the at least two processing cores required, and the task instructions are packaged and stored in the application expression layer, so as to complete the construction of the remote sensing data on-board processing system.

[0081] After the construction of the on-board processing tasks is completed, corresponding processing cores can be called from the core function layer according to task instructions to generate corresponding on-board processing tasks, so as to process input remote sensing data on board.

[0082] In a possible implementation, the on-board processing tasks generated by the application expression layer according to the task instructions calling corresponding processing cores from the core function layer include at least one of an image compression processing task, a target detection and recognition processing task, and a satellite surveying and mapping positioning processing task.

[0083] If the cloud ratio in the remote sensing image received on the ground is too high, it will cause waste of data resources, and in order to avoid the waste of data resources, the image compression processing task needs to have the ability to effectively eliminate images with too high cloud amount. Therefore, in a possible implementation, the image compression processing task is configured to call corresponding processing cores from the core function layer according to the image compression processing task instructions based on cloud detection to detect the cloud amount of the input remote sensing data, and then compress the images with cloud amount meeting the preset standard. In this way, the waste of data resources caused by too high cloud amount can be avoided, and the pressure of satellite-ground transmission is further reduced.

[0084] In the present disclosure, the basic resource layer of the remote sensing data on-board processing system defined based on a model is configured to store at least two bottom operator modules required for implementing on-board processing tasks; the core function layer is configured to store at least two processing cores required for implementing on-board processing tasks; wherein the processing core is implemented by at least one bottom operator module through calling the bottom operator module in the basic resource layer. In this way, when implementing on-board processing tasks based on the system, only task instructions need to be configured in the application expression layer according to the application requirements of the on-board processing tasks, and corresponding processing cores can be called from the core function layer through the task instructions to generate corresponding on-board processing tasks, which reduces the rewriting rate of similar algorithms, and further reduces the development cycle of the on-board processing system, and meets the requirements of future satellite low cost and rapid deployment.

[0085] <system example>

[0086] Figure 6 A schematic block diagram showing an example of a model definition based remote sensing data on-board processing system is shown. As shown, the remote sensing data on-board processing system includes a hardware logic layer, a basic resource layer, a core function layer, and an application expression layer. Figure 6

[0087] The hardware logic layer is a hardware entity of a remote sensing data on-board intelligent processing task underlying resource environment and related design, including system framework, prototype system, environment adaptability design, etc., providing computing resources, processing modules, communication modules, general storage, etc. required by processing tasks, carrying all resources and algorithm functions of the upper layer. In a possible implementation manner, the hardware entity can be constructed based on an FPGA core processor, and can also extend to support components such as NPU and DSP. The hardware entity can also be other hardware entities that can carry all resources and algorithms of the upper layer, which are not limited here.

[0088] The general data processing module and the neural network construction module are included in the basic resource layer. The general data processing module and the neural network construction module are consistent with the above description, and will not be described here.

[0089] The compression encoding processing core, the target detection and recognition processing core, the pose determination processing core, the stereo matching processing core, and the adjustment processing core are included in the core function layer. Each processing core has been described in the above description, and will not be described here.

[0090] The image compression processing task based on cloud detection, the multi-source data target detection and recognition processing task, and the satellite efficient mapping positioning processing task are included in the application expression layer. The image compression processing task based on cloud detection is configured to perform cloud amount detection on input remote sensing data, and then perform compression processing on images with cloud amount meeting a preset standard. The multi-source data target detection and recognition processing task is configured to perform high-value target detection and recognition processing on input multi-source data (such as visible light data, SAR data, or infrared data). The satellite efficient mapping positioning processing task is configured to perform fast and efficient high-precision pose determination, stereo matching, or adjustment processing on input mapping satellite data.

[0091] <Device Embodiment>

[0092] The present disclosure also provides a model definition based remote sensing data on-board processing device, including the remote sensing data on-board processing system of any one of the system embodiments.

[0093] <Implementation Method Embodiment>

[0094] Figure 7 A schematic flow chart showing an implementation method of a remote sensing data on-board processing task is shown. As shown, Figure 7 ​As shown, the method comprises steps S1100-S1200.

[0095] S1100, determining a processing core required for performing the on-satellite processing task from the core function layer according to a task instruction corresponding to the on-satellite processing task.

[0096] S1200, calling a corresponding bottom operator module from the basic resource layer according to the processing core, so as to realize the on-satellite processing task through execution of the bottom operator module.

[0097] In the present disclosure, when implementing the on-satellite processing task of remote sensing data, only the processing core required for the on-satellite processing task needs to be determined, and the calling instruction of the required processing core is designed and encapsulated according to the required processing core, so that the on-satellite processing task can be realized, the research and development cost of the on-satellite processing task is reduced, the research and development time is reduced, and the rapid deployment requirement of the on-satellite processing task can be met.

[0098] The above has described the embodiments of the present disclosure, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles, practical application, or technical improvement of the technology in the market of the embodiments, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A model definition based remote sensing data on-board processing system, characterized in that, Comprise: A basic resource layer configured to store at least two underlying operator modules required for implementing on-board processing tasks, the underlying operator module being a basic algorithm set that can implement general processing core data processing functions, the underlying operator module comprising at least one of a neural network construction module and a general data processing module, the neural network construction module comprising at least one of a convolution calculation submodule, a pooling calculation submodule, a summation calculation submodule and an activation calculation submodule, and the general data processing module comprising at least one of a data preprocessing module and a matrix calculation module; A core function layer configured to store at least two processing cores required for implementing certain data processing functions for implementing on-board processing tasks; wherein the processing core is implemented by at least one underlying operator module in the basic resource layer, and is implemented by combination of at least one underlying operator module, and the processing core comprises at least one of a compression encoding processing core, a target detection and recognition processing core, a pose determination processing core, a stereo matching processing core and a bundle adjustment processing core; An application expression layer configured to generate corresponding on-board processing tasks by calling corresponding processing cores from the core function layer according to task instructions, so as to process input remote sensing data by executing the on-board processing tasks, and the on-board processing tasks generated by the application expression layer according to task instructions calling corresponding processing cores from the core function layer comprise at least one of an image compression processing task, a target detection and recognition processing task and a satellite positioning processing task. In constructing the remote sensing data on-board processing system, first, typical on-board processing tasks are determined according to on-board processing purposes; the processing algorithms of the typical on-board processing tasks are analyzed and summarized, and at least two general core algorithm sets that can implement certain data processing functions are extracted as general processing cores; the core algorithm sets of all general processing cores are analyzed and summarized, and at least two general basic algorithm sets that can implement general processing core data processing functions are extracted as general underlying operator modules; after the general underlying operator modules are determined, the basic algorithm sets of each general underlying operator module are independently encapsulated to obtain at least two underlying operator modules. In implementing on-board processing tasks based on the system, first, at least two processing cores required for implementing on-board processing tasks are determined according to specific requirements of on-board processing tasks, and then task instructions are designed according to the at least two processing cores required, and the task instructions are encapsulated and stored in the application expression layer, and corresponding processing cores are called from the core function layer to generate corresponding on-board processing tasks based on the task instructions in the application expression layer, and then the input remote sensing data is processed by executing the on-board processing tasks.

2. The system of claim 1, wherein, When the processing core is implemented by at least one underlying operator module in the basic resource layer, the underlying operator module is called by the basic resource layer based on the data processing function to be implemented by the processing core.

3. The system of claim 1, wherein, The image compression processing task is configured to call a corresponding processing core from the core function layer according to an image compression processing task instruction based on cloud detection to perform cloud amount detection on input remote sensing data, and then perform compression processing on images with a cloud amount meeting a preset standard.

4. A model definition based remote sensing data on-board processing device, characterized in that, The remote sensing data on-board processing system comprises the remote sensing data on-board processing system according to any one of claims 1-3.

5. A method for implementing a remote sensing data on-board processing task, characterized in that, The system implementation method according to any one of claims 1-3, the implementation method comprising: determining a processing core required for executing the on-board processing task from the core function layer according to a task instruction corresponding to the on-board processing task; calling a corresponding bottom operator module from the basic resource layer according to the processing core, so as to implement the on-board processing task through execution of the bottom operator module.

Images