Task processing method, device and equipment based on satellite-borne data and medium

Abstract

The invention discloses a task processing method, device and equipment based on satellite-borne data and a medium, an on-satellite reasoning module outputs network intermediate features and geographic coordinates, a ground processing module dynamically calls a downstream task model in a downstream task model library according to the geographic coordinates for final reasoning, the calculation load of an on-satellite chip is reduced, and the calculation efficiency is improved. A special model for targeted training can be called in different regions to realize self-adaptive interpretation, a user does not need to download an original image to the ground, real-time and diversified processing of a remote sensing image can be realized, and the user experience is improved. The task processing method based on the satellite-borne data enables a task processing system based on the satellite-borne data to objectively and efficiently extract target information in a satellite-borne environment with limited resources, a special product is formed, and time and labor cost are reduced.

Description

Technical Field

[0001] The present invention relates to the field of task processing based on satellite-borne data, and in particular to a task processing method, device, equipment and medium based on satellite-borne data. Background Art

[0002] Hyperspectral imaging data is usually acquired by hyperspectral imagers carried on high-altitude platforms (such as satellites and drones) or ground platforms. These imagers are able to image in multiple narrow continuous spectral channels and simultaneously obtain the image and the spectral information corresponding to each pixel.

[0003] In related technologies, mission processing systems based on satellite-borne data rely on specific interpretation tasks for model generation. Different models must be trained for different interpretation tasks during the model generation phase. If only one model is deployed onboard, subsequent updates to new models are difficult and have low fault tolerance. Deploying multiple models strains the computing resources of the onboard chips and creates subsequent model selection issues. Furthermore, the sheer volume of hyperspectral satellite data makes real-time interpretation difficult due to the limited computing power and storage resources of onboard chips, leading to data transmission delays or the forced discarding of high-value data. Furthermore, traditional satellite interpretation models lack geographic adaptability, unable to adjust their algorithms adaptively based on geographic location. Summary of the Invention

[0004] The present invention aims to address at least one of the technical problems existing in the prior art. To this end, it proposes a task processing method based on satellite-borne data. This method combines downsampling and upsampling to reduce the burden of model generation and onboard deployment, significantly shortening the development cycle of remote sensing interpretation models. Furthermore, users no longer need to download the original images to the ground, enabling real-time and diversified processing of remote sensing images, reducing time and labor costs.

[0005] In a first aspect, an embodiment of the present invention provides a task processing method based on satellite onboard data, the method being applied to a target satellite, the method comprising:

[0006] Acquiring real-time spectral image data and geographic coordinate information corresponding to the spectral image data;

[0007] Slicing the spectral image data to obtain a plurality of image slice data;

[0008] Performing image coding processing on each of the image slice data through a pre-trained target image coding neural network to obtain an image slice code;

[0009] The image slice code and the geographic coordinate information are sent to a ground control center so that the ground control center performs the following operations on the image slice code and the geographic coordinate information: calling a target downstream task model that matches the task requirement information and the geographic coordinate information from a preset downstream task model library; wherein the downstream task model library includes multiple preset downstream task models, each of which corresponds to a preset coordinate information; performing task processing on the image slice code according to the target downstream task model to obtain a task processing result.

[0010] According to some embodiments of the present invention, before performing image coding processing on each of the image slice data using the pre-trained target image coding neural network to obtain the image slice code, the target image coding neural network is further pre-trained, specifically including:

[0011] Acquire a remote sensing dataset; wherein the remote sensing dataset includes a plurality of training remote sensing images;

[0012] Based on the plurality of training remote sensing images, unsupervised model training is performed on the original image coding neural network to obtain the target image coding neural network.

[0013] According to some embodiments of the present invention, the spectral image data is segmented to obtain a plurality of image slice data, including:

[0014] Performing sliding window slicing on the spectral image data at a preset overlap rate;

[0015] The sliced ​​spectral image data is subjected to image dimension reduction processing to obtain a plurality of image slice data.

[0016] According to some embodiments of the present invention, the target image coding neural network includes an image block embedding layer, a first linear layer, a second linear layer, an encoding layer, and a decoding layer. The unsupervised model training of the original image coding neural network based on the plurality of training remote sensing images to obtain the target image coding neural network includes:

[0017] For each of the training remote sensing images, inputting the training remote sensing image into the image block embedding layer to obtain a block embedding feature tensor;

[0018] The first linear layer performs downsampling processing on the block embedding feature tensor to obtain a compressed feature tensor;

[0019] Performing a nonlinear transformation on the compressed feature tensor through an activation function to obtain an activated feature tensor;

[0020] The second linear layer upsamples the activated feature tensor to obtain a reconstructed feature tensor;

[0021] The encoding layer performs global context modeling on the reconstructed feature tensor input to obtain a global context encoding tensor;

[0022] The global context coding tensor is input into the decoding layer for image reconstruction and the parameters of the image coding neural network are updated to obtain the target image coding neural network.

[0023] According to some embodiments of the present invention, the method is also applied to a ground control center, and the method includes:

[0024] Obtaining mission requirement information, image slice codes and geographic coordinate information sent from a target satellite; wherein the image slice codes and geographic coordinate information are generated in the target satellite through the following steps: obtaining real-time spectral image data and geographic coordinate information corresponding to the spectral image data; segmenting the spectral image data to obtain a plurality of image slice data; and performing image coding processing on each image slice data using a pre-trained target image coding neural network to obtain an image slice code;

[0025] Calling a target downstream task model that matches the task requirement information and the geographic coordinate information from a preset downstream task model library; wherein the downstream task model library includes a plurality of preset downstream task models, each of which corresponds to one of the preset coordinate information;

[0026] The image slice encoding is task-processed according to the target downstream task model to obtain a task processing result.

[0027] According to some embodiments of the present invention, before calling the target downstream task model that matches the task requirement information and the geographic coordinate information from the preset downstream task model library, the method further includes presetting the downstream task model library, specifically including:

[0028] Building a downstream task dataset database based on the remote sensing dataset;

[0029] According to the target image encoding neural network and the downstream task dataset, the downstream task network is trained in a supervised training mode, and the training results are saved in the downstream task model library;

[0030] The on-board deployment module is separated from the target image encoding neural network, and the on-board deployment module is deployed into an on-board chip.

[0031] According to some embodiments of the present invention, the target downstream task model is a target detection model, and performing task processing on the image slice encoding according to the target downstream task model to obtain a task processing result includes:

[0032] Performing target object recognition on the image slice encoding using the target detection model to obtain object position information of the target object in the spectral image data;

[0033] generating a target object identifier based on the object position information;

[0034] The number of target objects corresponding to the target objects in the spectral image data is determined according to the number of the target object identifiers.

[0035] In the second aspect, an embodiment of the present invention also proposes a task processing device based on satellite onboard data, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the task processing method based on satellite onboard data as described in the first aspect.

[0036] In a third aspect, an embodiment of the present invention further proposes an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the task processing method based on satellite-borne data as described in the first aspect is implemented.

[0037] In a fourth aspect, an embodiment of the present invention further proposes a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the task processing method based on satellite-borne data as described in the first aspect.

[0038] The task processing method based on satellite-borne data according to an embodiment of the present invention has at least the following beneficial effects:

[0039] In the image coding neural network, two linear layers and activation functions are placed between the image block embedding layer and the Transformer encoder layer to form a combined upsampling and downsampling processing pipeline. This breaks through the traditional single-path feature transfer model, avoiding the loss of detail information caused by direct downsampling and the redundant computation caused by simple upsampling. With a small number of parameters and a simple structure, this provides infrastructure support for the lightweight deployment of onboard modules, eliminating the need for large video memory and high-performance chips. Secondly, the onboard inference module outputs intermediate network features and geographic coordinates. The ground processing module dynamically calls downstream task models from the downstream task model library based on the geographic coordinates for final inference, reducing the computational load on the onboard chip. Different regions can call on specialized, specifically trained models for adaptive interpretation. Users no longer need to download the original imagery to the ground, enabling real-time and diversified processing of remote sensing imagery. This satellite-based data-based task processing method enables the task processing system to objectively and efficiently extract target information in a resource-constrained onboard environment, creating thematic products and reducing time and labor costs.

[0040] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0042] Figure 1 A flowchart of a task processing method based on satellite-borne data provided by an embodiment of the present invention;

[0043] Figure 2 A detailed flow chart of step S200 in the task processing method based on satellite-borne data provided in an embodiment of the present invention;

[0044] Figure 3 A detailed flow chart of pre-training the target image encoding neural network in step S300 in the task processing method based on satellite-borne data provided by an embodiment of the present invention;

[0045] Figure 4 A detailed flow chart of step S900 in the task processing method based on satellite-borne data provided in an embodiment of the present invention;

[0046] Figure 5 A detailed flow chart of step S700 in the task processing method based on satellite-borne data provided in an embodiment of the present invention;

[0047] Figure 6A detailed flowchart of presetting the downstream task model library in step S600 in the task processing method based on satellite onboard data provided by an embodiment of the present invention;

[0048] Figure 7 A schematic diagram of an image coding neural network module in a task processing method based on satellite-borne data provided by an embodiment of the present invention;

[0049] Figure 8 A schematic diagram of a downstream task network in a task processing method based on satellite-borne data provided by an embodiment of the present invention;

[0050] Figure 9 A module diagram of a task processing method based on satellite-borne data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0052] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on the present invention.

[0053] In the description of the present invention, "several" means more than one, "plurality" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0054] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0055] In the description of the present invention, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the exemplary expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0056] Intelligent remote sensing interpretation technology has broad applications in a wide range of fields, including crop monitoring and yield estimation and forest carbon sequestration. Related research and exploration has long garnered significant attention. As a specialized form of imagery, remote sensing imagery has been the subject of extensive research by scholars both domestically and internationally using deep learning methods for intelligent remote sensing interpretation as early as 2013. This research covers a wide range of application scenarios, including target and scene retrieval, object detection, object classification, change detection, and 3D reconstruction.

[0057] Satellite-acquired spectral imagery is a type of remote sensing imagery. It is acquired through hyperspectral sensors onboard satellite platforms and combines wide-area coverage with sophisticated spectral analysis capabilities. Its technical characteristics, such as high spectral resolution, make it irreplaceable in scenarios requiring quantitative analysis.

[0058] Traditionally, models for onboard modules have primarily consisted of simple structures such as convolutional and linear layers, such as the YOLO series. While the YOLO series boasts a simple structure and a small number of parameters, it still requires quantization on satellite chips, inevitably resulting in some loss of accuracy.

[0059] The self-attention mechanism is a technology used to process sequence data. Its core is to dynamically capture the dependencies between elements within the sequence and model the global context information by calculating the similarity weights between elements.

[0060] The Transformer encoding layer, comprised of a self-attention mechanism, is the core framework of large-scale model technology. Within the Transformer encoding layer, the self-attention mechanism interacts with every element in the sequence, forming an nxn attention matrix. This computational complexity leads to a significant amount of computation when the sequence length n is large. The self-attention mechanism uses the pairwise correlation between two inputs as a weighted average to map each input to the output. Therefore, within the Transformer encoding layer, the output is dependent on all previous inputs, not just the last one. This allows the Transformer encoding layer to parallelize the computation of all self-attention mechanisms. However, the computational complexity and parallel nature of the Transformer encoding layer make onboard deployment difficult, making it unsuitable for deployment in onboard chips.

[0061] In deep learning, unsupervised learning and supervised learning are two of the most common and important concepts. Supervised learning involves training a model using a dataset containing inputs and corresponding outputs (i.e., labels or answers), enabling the model to learn the mapping relationship from inputs to outputs. Unsupervised learning processes unlabeled datasets and aims to discover hidden structures or patterns in the data, such as clustering and dimensionality reduction, without knowing the correct answers in advance.

[0062] In related technologies, mission processing systems based on satellite-borne data rely on specific interpretation tasks for model generation. Different models must be trained for different interpretation tasks during the model generation phase. If only one model is deployed on a satellite, subsequent updates to new models are difficult and have low fault tolerance. Deploying multiple models strains the computing resources of the onboard chips and creates subsequent model selection issues. Furthermore, the sheer volume of hyperspectral satellite data makes real-time interpretation difficult due to the limited computing power and storage resources of onboard chips, leading to data transmission delays or the forced discarding of high-value data. Furthermore, traditional satellite interpretation models lack geographic adaptability, unable to adjust their algorithms adaptively based on geographic location.

[0063] Based on this, the present invention provides a method, apparatus, device, and medium for task processing based on satellite onboard data. These methods provide infrastructure support for lightweight deployment of onboard modules, eliminating the need for large video memory and high-performance chips. Furthermore, users no longer need to download the original images to the ground, enabling real-time and diversified processing of remote sensing images. This method enables task processing systems based on satellite onboard data to objectively and efficiently extract target information in resource-constrained onboard environments, creating thematic products and reducing time and labor costs.

[0064] The method of the embodiment of the present invention is further described below based on the accompanying drawings.

[0065] First, as Figure 1 and Figure 9 The task processing method based on satellite-borne data of the present invention is applied to a target satellite, and the specific method includes but is not limited to the following steps:

[0066] S100: Acquire real-time spectral image data and geographic coordinate information corresponding to the spectral image data;

[0067] S200: Slice the spectral image data to obtain a plurality of image slice data;

[0068] S300: performing image coding processing on each image slice data through a pre-trained target image coding neural network to obtain an image slice code;

[0069] S400: Send the image slice code and geographic coordinate information to the ground control center, so that the ground control center performs the following operations on the image slice code and geographic coordinate information: call the target downstream task model that matches the task requirement information and geographic coordinate information from the preset downstream task model library; wherein the downstream task model library includes multiple preset downstream task models, and each preset downstream task model corresponds to a preset coordinate information; perform task processing on the image slice code according to the target downstream task model to obtain the task processing result.

[0070] Specifically, the core of the image coding neural network of this embodiment is the MAE model, which is a neural network model based on self-supervised learning. Through pre-training of the target image coding neural network, it can be migrated to various downstream tasks, including image classification, target detection, and image generation, etc. Migration means that the features extracted by the encoder of the MAE model can be used for various downstream tasks. When migrating to downstream tasks, the encoder of the MAE model and the decoding head of the relevant downstream tasks constitute the network structure of the downstream task network, wherein the parameters of the MAE encoder are fixed and do not participate in training. During training, only the network parameters of the downstream task network are trained.

[0071] Specifically, in step S100, the satellite acquires spectral image data in real time through the onboard sensor. The spectral image data includes the number of bands and spatial size. In this embodiment, the spatial size of the spectral image data acquired by the satellite is 5056×5056 pixels, and the number of bands is 32 bands, which can provide finer spectral resolution and support material composition identification. In step S300, the onboard deployment module part ( Figure 7 or Figure 8 The image slice data is subjected to image coding processing to obtain image slice coding.

[0072] Specifically, if Figure 2 , Figure 1 The step S200 shown in FIG. 1 also includes but is not limited to the following steps:

[0073] S201: Slicing the spectral image data in a sliding window with a preset overlap ratio;

[0074] S202: Performing image dimension reduction processing on the sliced ​​spectral image data to obtain a plurality of image slice data.

[0075] It can be understood that the overlap rate refers to the ratio of the overlapping area between adjacent image slices (sliding windows) to the window size, and the calculation formula is:

[0076]

[0077] Specifically, a sliding window slice is applied to the input spectral image data at a preset overlap ratio of 0.5, dividing large data into smaller blocks. A higher overlap ratio generates more slices, improving data integrity, continuity, and diversity. For large features (such as lakes and urban agglomerations), a higher overlap ratio ensures that the feature is fully covered by at least one slice, avoiding missed detections due to misaligned slices. If the last sliding window crosses the boundary, coordinates are retracted to ensure that the feature remains within the boundary.

[0078] Specifically, PCA (principal components analysis) dimensionality reduction is performed on the sliced ​​data. The sliced ​​data is first treated as a two-dimensional matrix (spatial dimension × number of bands) and normalized to ensure uniform mean and variance across all bands. The covariance matrix for all bands is then calculated, taking care to avoid statistical bias caused by slice boundaries. After eigendecomposition, the first four eigenvectors are selected to construct a new low-dimensional spatial basis. Matrix multiplication is then performed on the original band data at each spatial location with these eigenvectors to obtain four principal component bands, compressing the spectral image data to four bands. This reduces the number of bands in the image data to four, thereby preserving key information while reducing data dimensionality and computational complexity.

[0079] Specifically, if Figure 3 Before performing step S300, the target image coding neural network in step S300 is pre-trained, including but not limited to the following steps:

[0080] S800: Acquire a remote sensing dataset; wherein the remote sensing dataset includes a plurality of training remote sensing images;

[0081] S900: Based on multiple training remote sensing images, perform unsupervised model training on the original image coding neural network to obtain the target image coding neural network.

[0082] Specifically, before pre-training, the sizes of multiple training remote sensing images are uniformly processed into a fixed size a. It is understood that in step S201, the spectral image data is sliced ​​using the fixed size a used in pre-training as a standard to ensure that the input spectral image data is compatible with the pre-trained target image encoding neural network, thereby maintaining data consistency.

[0083] Specifically, if Figure 4 、 Figure 7 and Figure 9 The image coding neural network includes an image block embedding layer, a first linear layer, a second linear layer, an encoding layer, and a decoding layer. Step S200 specifically further includes the following steps:

[0084] S901: For each training remote sensing image, input the training remote sensing image into the image block embedding layer to obtain a block embedding feature tensor;

[0085] S902: The first linear layer performs downsampling processing on the block embedding feature tensor to obtain a compressed feature tensor;

[0086] S903: Performing a nonlinear transformation on the compressed feature tensor through an activation function to obtain an activated feature tensor;

[0087] S904: The second linear layer upsamples the activated feature tensor to obtain a reconstructed feature tensor;

[0088] S905: The encoding layer performs global context modeling on the reconstructed feature tensor input to obtain a global context encoding tensor;

[0089] S906: Input the global context coding tensor into the decoding layer to perform image reconstruction and update the parameters of the image coding neural network to obtain the target image coding neural network.

[0090] S907: Separate the on-board deployment module from the image coding neural network, and deploy the on-board deployment module into the on-board chip.

[0091] Specifically, if Figure 7The image block embedding layer is the Patch embedding layer, the first linear layer is the downsampling layer, the second linear layer is the upsampling layer, and the encoding layer is the Transformer layer. In step S901, the input training remote sensing image size is B*H*W*C (Batch×height×width×number of channels). The model input is generally an RGB image, so the number of channels C is usually 3. The image block embedding layer reduces the spatial dimension H*W of the image to (H / 16)*(W / 16), that is, each dimension is reduced by 16 times, while the number of channels is increased by 256 times. That is, the data volume of the output block embedding feature tensor remains basically unchanged.

[0092] Specifically, if Figure 4 、 Figure 7 and Figure 9 In step S907, after pre-training the target image coding neural network, the image block embedding layer and the first linear layer are separated from the image coding neural network as an onboard deployment module. The onboard deployment module is then deployed to the onboard chip of the target satellite. Because the image slice codes obtained by the target satellite are calculated based on the onboard deployment module separated in step S907, the data structure of the image slice codes obtained on the target satellite is consistent with the compressed feature tensor data structure in step S902.

[0093] Specifically, if Figure 4 、 Figure 7 and Figure 9 In step S903, the compressed feature tensor is nonlinearly transformed using an activation function. By introducing a nonlinear mapping, the linearly projected block embedding feature tensor is mapped into a nonlinear space, enabling the model to fit arbitrarily complex functions and enhancing local feature modeling capabilities. In this embodiment, the activation function is the Tanh function (hyperbolic tangent function).

[0094] Specifically, if Figure 4 、 Figure 7 and Figure 9 In step S904, the second linear layer upsamples the activated feature tensor to obtain a reconstructed feature tensor, which is essentially to restore the compressed feature dimension through the fully connected layer to provide high-dimensional semantic information for subsequent global modeling and image reconstruction.

[0095] Specifically, if Figure 4 、 Figure 7 and Figure 9In step S905, the Transformer encoding layer includes a self-attention mechanism. The input reconstructed feature tensor captures the long-range dependencies between image blocks through the dynamic weight distribution of the self-attention mechanism. For example, in remote sensing images, the encoding layer can identify the spatial topological relationship between roads and buildings, or the spectral correlation between water bodies and vegetation. The multi-head design allows information to be extracted from multiple semantic subspaces (such as local texture and global shape) in parallel, and finally fused into a unified global context encoding tensor. The encoding layer using the self-attention mechanism can automatically identify the relationship between different areas in the image, such as determining whether a certain area is a water body or farmland, and establish their association with the surrounding environment.

[0096] Specifically, if Figure 4 、 Figure 7 and Figure 9 In step S906, after the decoding layer receives the global context coding tensor, it gradually restores the spatial resolution through deconvolution or bilinear interpolation, and combines the features of different stages of the coding layer with jump connections (such as U-Net architecture) to compensate for the loss of details in the downsampling process. For example, in the low-light enhancement task, the global context coding tensor is enlarged to the original image size through the transposed convolution layer, and the dense residual block (DRB) is introduced to enhance the local texture recovery ability. In the reconstruction process, multi-objective loss functions (such as mean square error MSE and perceptual loss) are jointly optimized. The former constrains pixel-level consistency, and the latter extracts high-level semantic feature differences through a pre-trained network to ensure the visual authenticity and spectral continuity of the reconstructed image.

[0097] Second, as Figure 1 and Figure 9 The task processing method based on satellite-borne data of the present invention is also applied to a ground control center. The specific method includes but is not limited to the following steps:

[0098] S500: Acquire mission requirement information, image slice codes and geographic coordinate information sent from the target satellite; wherein the image slice codes and geographic coordinate information are in the target satellite and are generated through steps S100 to S400;

[0099] S600: Calling a target downstream task model that matches the task requirement information and the geographic coordinate information from a preset downstream task model library; wherein the downstream task model library includes a plurality of preset downstream task models, each of which corresponds to a preset coordinate information;

[0100] S700: Perform task processing on the image slice encoding according to the target downstream task model to obtain a task processing result.

[0101] Specifically, if Figure 6Before step S600, the downstream task model library is also preset, including but not limited to the following steps:

[0102] S1000: Build a downstream task dataset database based on remote sensing datasets;

[0103] S1100: Based on the target image encoding neural network and the downstream task dataset, the downstream task network is trained in a supervised training mode, and the training results are saved to the downstream task model library;

[0104] Specifically, in step S1100, Figure 8 The downstream task network includes the MAE encoder part of the image coding neural network and the decoding head of the relevant downstream tasks. The on-board deployment module part (image block embedding layer and the first linear layer, i.e., Figure 7 or Figure 8 The training results include a variety of downstream task models, which can be further applied to various downstream tasks such as image classification, object detection, and image generation. In addition, the decoding heads of related downstream tasks include Figure 8 The detection head and classification head in the network are as follows: the detection head contains a series of convolutional layers and deconvolution layers to generate detection results. It is responsible for predicting the bounding box regression value of each anchor box and the confidence of the target existence. The classification head uses global average pooling to classify each feature map and outputs the probability distribution of each category by reducing the dimension of the feature map.

[0105] Specifically, in step S600, if the current geographic coordinate information corresponds to a river area and the task requirement information is ship target detection, the ship target detection model is called.

[0106] Specifically, if Figure 5 Step S700 also includes but is not limited to the following steps:

[0107] S701: performing target object recognition on the image slice code using a target detection model to obtain object position information of the target object in the spectral image data;

[0108] S702: Generate a target object identifier based on the object position information;

[0109] S703: Determine the number of target objects corresponding to the target objects in the spectral image data according to the number of target object identifiers.

[0110] It can be understood that if the target downstream task model in step S700 is the called ship target detection model, the image slice code of step S500 is input into the downstream task network of step S1100, and the image result is obtained by reasoning in combination with the remaining part of the ship target detection model, and then the image result is task processed, that is, the number of ships in the area is estimated based on the number of frames detected in the image.

[0111] According to an embodiment of the present invention, a task processing method based on satellite onboard data forms a combined upsampling and downsampling processing channel in an image coding neural network by setting two linear layers and activation functions between the image block embedding layer and the Transformer encoder layer. This breaks through the traditional single-path feature transfer mode, avoids the loss of detail information caused by direct downsampling and the redundant calculation caused by simple upsampling, and has a small number of parameters and a simple structure. It provides infrastructure support for the lightweight deployment of onboard deployment modules, eliminating the need for large video memory and high-performance chips. Secondly, the onboard inference module outputs the network intermediate features and geographic coordinates. The ground processing module dynamically calls the downstream task model in the downstream task model library based on the geographic coordinates for final inference, reducing the computational load of the onboard chip. Different regions can call specialized models trained specifically for them to achieve adaptive interpretation. Users do not need to download the original image to the ground, and can achieve real-time and diversified processing of remote sensing images. The task processing method based on satellite onboard data enables the task processing system based on satellite onboard data to objectively and efficiently extract target information in a resource-constrained onboard environment, forming thematic products and reducing time and labor costs.

[0112] In the third aspect, an embodiment of the present invention also proposes a task processing device based on satellite onboard data, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the task processing method based on satellite onboard data as described in any one of the first and second aspects.

[0113] In a fourth aspect, an embodiment of the present invention further proposes an electronic device, characterized in that it includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the task processing method based on satellite-borne data of the above-mentioned first and second aspects.

[0114] In a fifth aspect, an embodiment of the present invention also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the task processing method based on satellite-borne data of the above-mentioned first and second aspects.

[0115] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory optionally includes a memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of the above-mentioned networks include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0116] Those skilled in the art will appreciate that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data) and is volatile and non-volatile, removable, and non-removable. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage, or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0117] While the embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to the embodiments described above. Various modifications may be made within the scope of knowledge possessed by a person skilled in the art without departing from the spirit of the present invention. Furthermore, the embodiments of the present invention and the features thereof may be combined with one another unless there is a conflict.

Claims

1. A task processing method based on satellite-borne data, characterized in that: The method is applied to a target satellite, and the method comprises: Acquiring real-time spectral image data and geographic coordinate information corresponding to the spectral image data; Slicing the spectral image data to obtain a plurality of image slice data; Performing image coding processing on each of the image slice data through a pre-trained target image coding neural network to obtain an image slice code; The image slice code and the geographic coordinate information are sent to a ground control center so that the ground control center performs the following operations on the image slice code and the geographic coordinate information: calling a target downstream task model that matches the task requirement information and the geographic coordinate information from a preset downstream task model library; wherein the downstream task model library includes multiple preset downstream task models, each of which corresponds to a preset coordinate information; performing task processing on the image slice code according to the target downstream task model to obtain a task processing result.

2. The method according to claim 1, characterized in that Before performing image coding processing on each of the image slice data by using the pre-trained target image coding neural network to obtain the image slice code, the target image coding neural network is also pre-trained, specifically including: Acquire a remote sensing dataset; wherein the remote sensing dataset includes a plurality of training remote sensing images; Based on the plurality of training remote sensing images, unsupervised model training is performed on the original image coding neural network to obtain the target image coding neural network.

3. The method according to claim 1, characterized in that The spectral image data is segmented to obtain a plurality of image slice data, including: Performing sliding window slicing on the spectral image data at a preset overlap rate; The sliced ​​spectral image data is subjected to image dimension reduction processing to obtain a plurality of image slice data.

4. The method according to claim 2, characterized in that The image coding neural network includes an image block embedding layer, a first linear layer, a second linear layer, an encoding layer, and a decoding layer. The original image coding neural network is trained in an unsupervised model based on the plurality of training remote sensing images to obtain the target image coding neural network, including: For each of the training remote sensing images, inputting the training remote sensing image into the image block embedding layer to obtain a block embedding feature tensor; The first linear layer performs downsampling processing on the block embedding feature tensor to obtain a compressed feature tensor; Performing a nonlinear transformation on the compressed feature tensor through an activation function to obtain an activated feature tensor; The second linear layer upsamples the activated feature tensor to obtain a reconstructed feature tensor; The encoding layer performs global context modeling on the reconstructed feature tensor input to obtain a global context encoding tensor; The global context coding tensor is input into the decoding layer for image reconstruction and the parameters of the image coding neural network are updated to obtain the target image coding neural network.

5. The task processing method based on satellite-borne data according to claim 1, characterized in that: The method is also applied to a ground control center, and includes: Obtaining mission requirement information, image slice codes and geographic coordinate information sent from a target satellite; wherein the image slice codes and geographic coordinate information are generated in the target satellite through the following steps: obtaining real-time spectral image data and geographic coordinate information corresponding to the spectral image data; segmenting the spectral image data to obtain a plurality of image slice data; and performing image coding processing on each image slice data using a pre-trained target image coding neural network to obtain an image slice code; Calling a target downstream task model that matches the task requirement information and the geographic coordinate information from a preset downstream task model library; wherein the downstream task model library includes a plurality of preset downstream task models, each of which corresponds to one of the preset coordinate information; The image slice encoding is task-processed according to the target downstream task model to obtain a task processing result.

6. The method according to claim 5, characterized in that Before calling the target downstream task model that matches the task requirement information and the geographic coordinate information from the preset downstream task model library, the method further includes presetting the downstream task model library, specifically including: Building a downstream task dataset database based on the remote sensing dataset; According to the target image encoding neural network and the downstream task dataset, the downstream task network is trained in a supervised training mode, and the training results are saved in the downstream task model library; The on-board deployment module is separated from the target image encoding neural network, and the on-board deployment module is deployed into an on-board chip.

7. The method according to claim 5, characterized in that The target downstream task model is a target detection model, and performing task processing on the image slice encoding according to the target downstream task model to obtain a task processing result includes: Performing target object recognition on the image slice encoding using the target detection model to obtain object position information of the target object in the spectral image data; generating a target object identifier based on the object position information; The number of target objects corresponding to the target objects in the spectral image data is determined according to the number of the target object identifiers.

8. A task processing device based on satellite-borne data, characterized in that: comprising at least one control processor and a memory for communicatively coupling with the at least one control processor; The memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the task processing method based on satellite-borne data as described in any one of claims 1 to 7.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method for processing tasks based on satellite-borne data as claimed in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the task processing method based on satellite-borne data according to any one of claims 1 to 7.

Images