Satellite-ground-cloud-edge collaborative ground target recognition method and device based on deep network

Abstract

The present invention discloses a method and device for satellite-ground-cloud-edge collaborative ground target recognition based on a deep network. The device includes a ground cloud service node and a satellite edge node, wherein the satellite edge node includes a computing control module, an on-board communication module, a satellite data acquisition module, a data preprocessing module, a data labeling module, and an on-board recognition processing module. The method is as follows: first, a ground target recognition task support environment is constructed, and an existing target recognition task model is loaded. If the current model matches the requirements of the recognition task, the recognition task is executed; if it does not match, the target data of various recognition tasks are collected as standard training samples; then a target recognition model is constructed, and the model is trained using standard training samples; then the trained target recognition model is sent to the satellite edge node; finally, the satellite edge node executes the target recognition task and returns the target recognition result to the ground cloud service node. The present invention improves the timeliness and accuracy of satellite ground target recognition.

Description

Technical Field

[0001] The present invention belongs to the field of satellite intelligent application technology, and in particular to a method and device for satellite-ground-cloud-edge collaborative ground target recognition based on a deep network. Background Art

[0002] With the development of artificial intelligence, computing power is shifting from central clouds to the edge and devices themselves. The coordination of cloud, edge, and device computing power is becoming increasingly important. Applying cloud-edge collaboration to spatial information systems is an inevitable trend. Cloud-edge collaboration will evolve toward on-demand scheduling of computing power across three levels: cloud, edge, and device. The goal of this collaboration is to improve computing resource utilization. Networks will evolve from cloud-network convergence to computing-network convergence, providing a reliable and efficient on-demand network for computing services.

[0003] Existing satellite-based target recognition methods mostly rely on storage and transmission to the ground. Due to satellite orbital constraints, imagery captured outside of the country or outside tracking and control stations is only transmitted to ground stations during transit for processing. Ground stations, equipped with sophisticated computing equipment and a rich library of target models, enable rapid and comprehensive target recognition. However, their primary drawback is their poor time-to-target effectiveness, effectively identifying only fixed targets and lacking timely information for moving targets. Furthermore, existing recognition methods have limited capabilities. Deep learning networks can improve both the accuracy and speed of target recognition. Summary of the Invention

[0004] The purpose of the present invention is to provide a satellite-ground-cloud-edge collaborative ground target recognition method and device based on deep network with high timeliness and accuracy.

[0005] The technical solution to achieve the purpose of the present invention is: a satellite-ground-cloud-edge collaborative ground target recognition method based on a deep network, comprising the following steps:

[0006] Step 1: Build a satellite-ground-cloud-edge collaborative ground target recognition task support environment and load the existing target recognition task model;

[0007] Step 2: The ground cloud service node confirms the recognition task model according to the requirements of the recognition task. If the current recognition task model matches, it proceeds to step 7 to perform the recognition task; if the current recognition task model does not meet the requirements of the recognition task, it proceeds to step 3;

[0008] Step 3: Collect target data for various recognition tasks, including large-scale ground equipment recognition and large-scale maritime vessel recognition, store them in a target database, pre-process and label the target data, and generate standard training samples.

[0009] Step 4: Build a target recognition model based on onboard resource constraints and configure the training parameters of the target recognition model;

[0010] Step 5: The ground cloud service node trains the target recognition model and tests the accuracy of the trained target recognition model prediction;

[0011] Step 6: The ground cloud service node sends the trained target recognition model to the satellite edge node through the satellite-to-ground communication link;

[0012] Step 7: The satellite edge node performs the target recognition task and returns the target recognition result and real-time target status to the ground cloud service node;

[0013] Step 8: The ground cloud service node controls the satellite edge node to collect target area data and update the mission database.

[0014] A satellite-ground-cloud-edge collaborative ground target recognition device based on a deep network includes a ground cloud service node and a satellite edge node, wherein the satellite edge node includes a computing control module, an on-board communication module, a satellite data acquisition module, a data preprocessing module, a data annotation module and an on-board recognition processing module;

[0015] The ground cloud service node uses a high-performance server cluster for business process control, data transmission and reception control, model training optimization, and model database support;

[0016] The computing control module adopts a heterogeneous computing platform based on CPU+GPU, which is used for onboard business process control, data transmission and reception control, and image processing control;

[0017] The onboard communication module is used to receive communication signals from other satellites and implement signal demodulation and deframing, and then transmit them to the calculation and control module; at the same time, it receives data from the calculation and control module, implements signal framing and modulation, and then sends it to the ground cloud service node or other satellite edge nodes;

[0018] The satellite data acquisition module is used to obtain satellite images to be identified;

[0019] The data preprocessing module is used to preprocess the satellite image to be identified and generate a standard image for annotation;

[0020] The data annotation module is used to mark the target content in the pre-processed satellite image for training by the model training module;

[0021] The on-board recognition processing module is used to input the pre-processed standard satellite image into the uploaded target recognition model and output the recognition result.

[0022] Compared with the existing technology, the present invention has the following significant advantages: (1) It performs ground target identification based on satellite-ground-cloud-edge collaboration, obtains target data in real time through satellite edge nodes, and improves the timeliness of target identification; (2) It improves the accuracy of target identification by coordinating target identification and acquisition tasks in the target area through ground cloud service nodes and satellite edge nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a satellite-ground-cloud-edge collaborative ground target recognition method based on a deep network of the present invention.

[0024] Figure 2 It is a flow chart of the final construction of the target recognition model of the present invention.

[0025] Figure 3 This is a structural block diagram of a satellite-ground-cloud-edge collaborative ground target recognition device based on a deep network in the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Combine Figure 1 , a satellite-ground-cloud-edge collaborative ground target recognition method based on deep network, including the following steps:

[0028] Step 1: Build a satellite-ground-cloud-edge collaborative ground target recognition task support environment and load the existing target recognition task model;

[0029] Step 2: The ground cloud service node confirms the recognition task model according to the requirements of the recognition task. If the current recognition task model matches, it proceeds to step 7 to perform the recognition task; if the current recognition task model does not meet the requirements of the recognition task, it proceeds to step 3;

[0030] Step 3: Collect target data for various recognition tasks, including large-scale ground equipment recognition and large-scale maritime vessel recognition, store them in a target database, pre-process and label the target data, and generate standard training samples.

[0031] Step 4: Build a target recognition model based on onboard resource constraints and configure the training parameters of the target recognition model;

[0032] Step 5: The ground cloud service node trains the target recognition model and tests the accuracy of the trained target recognition model prediction;

[0033] Step 6: The ground cloud service node sends the trained target recognition model to the satellite edge node through the satellite-to-ground communication link;

[0034] Step 7: The satellite edge node performs the target recognition task and returns the target recognition result and real-time target status to the ground cloud service node;

[0035] Step 8: The ground cloud service node controls the satellite edge node to collect target area data and update the mission database.

[0036] As a specific implementation method, the construction of the satellite-ground-cloud-edge collaborative ground target recognition task support environment described in step 1 is as follows:

[0037] Step 1.1: Build a satellite-ground-cloud-edge collaborative ground target recognition mission support environment, including building a mission operating environment and building a cloud-edge collaborative communication link;

[0038] Step 1.2: Build the task running environment and deploy it on the ground cloud service node and satellite edge node;

[0039] The task operation environment of the ground cloud service node consists of a server cluster, which is used for business process control, data transmission and reception control, model training optimization, and model database support. It is also equipped with a task model library and a target database. The task model library is used to manage existing target recognition models that can be directly used. The target database is used to store various target data, including target data of large ground equipment and large marine vessels. The stored target data is used for retraining or adjusting the target recognition model in the future.

[0040] The mission operating environment of the satellite edge node is built based on software virtualization technology, which is used for cross-platform deployment and migration of mission systems, on-board mission resource management, control model sending and receiving, mission start and stop, message preparation, and presentation of mission execution results and observation parameters.

[0041] Step 1.3: Build a cloud-edge collaborative communication link, including an uplink from the ground cloud service node to the satellite edge node, a downlink from the satellite edge node to the ground cloud service node, and parallel links between satellite edge nodes. These links are used for the transmission of satellite-to-ground control commands, recognition models, target data and status information, as well as relay transmission between the ground cloud service node and non-directly connected satellite edge nodes.

[0042] As a specific implementation method, the ground cloud service node in step 2 confirms the recognition task model according to the requirements of the recognition task. If the current recognition task model matches, the process proceeds to step 7 to execute the recognition task; if the current recognition task model fails to meet the requirements of the recognition task, the process proceeds to step 3, as follows:

[0043] After receiving a target recognition task, the ground cloud service node extracts the target identifier and searches for it in the cloud service task model library, which stores readily available target recognition models. Once these models are added to the satellite edge node, the target recognition task can be executed. If a target model matching the identifier is successfully retrieved during the search, the model is added to the satellite edge node in the corresponding area, and the edge node is controlled to execute the recognition task and obtain the recognition result. If a target model matching the identifier cannot be retrieved, model construction is performed.

[0044] As a specific implementation method, the target data of various recognition tasks are collected in step 3, stored in the target database, pre-processed and annotated, and standard training samples are generated. Figure 2 , as follows:

[0045] Step 3.1, collect target data of various recognition tasks and store them in the target database;

[0046] The following two methods are used to collect target data for various recognition tasks:

[0047] The first method is to collect data on the target to be identified through channels such as public network information collection, professional agency services, and historical data accumulation of the system, and store it in the target database of the ground cloud service node;

[0048] In the second method, the ground cloud service node sends instructions to the satellite edge nodes within a set range based on the code of the target area to be identified, and collects the data of the target to be identified in real time. After the collection is completed, it is stored in the target database of the ground cloud service node;

[0049] Step 3.2: The ground cloud service node obtains the sensor parameters of the satellite edge node, determines the preprocessing indicators of the target data to be identified based on the sensor parameters, and performs preprocessing operations on the target data to be identified, as follows:

[0050] The ground cloud service node obtains the sensor parameters of the satellite edge node, which include the optical sensor resolution, frame, and sampling frequency. The preprocessing indicators of the target data are determined based on the sensor parameters. The preprocessing indicators include resolution and frame. The target data to be identified is normalized and the image is sharpened to highlight texture features.

[0051] Step 3.3: Send the pre-processed target data to the target data labeling link, select and label all targets in the target data, and store the target data as model training samples in the target database for the next step of training the target recognition model.

[0052] As a specific implementation method, in step 4, according to the on-board resource constraints, a target recognition model is built and training parameters of the target recognition model are configured as follows:

[0053] Step 4.1: The ground cloud service node obtains the software and hardware resource parameters of the satellite edge node and constructs a target recognition model that adapts to the hardware constraints of the satellite edge node, including a data preprocessing layer, an internal hidden layer, and an output layer. The target recognition model should have the characteristics of low computational complexity, low frame rate requirements, small model size, and high accuracy.

[0054] The target recognition model uses the MobileNet SSD model as the basic model;

[0055] Step 4.2: Design the network structure and parameters of the target recognition model, including the number of model layers, internal hidden layers, number of units in each layer, number of network layer iterations, optimizer, evaluation function, activation function selection, and Dropout layer.

[0056] Step 4.3: Based on the computing speed and sample size of the ground cloud service node, determine the network update frequency, number of training iterations, and evaluation function of the target recognition model.

[0057] As a specific implementation, the ground cloud service node in step 5 trains the target recognition model and tests the accuracy of the trained target recognition model prediction as follows:

[0058] Step 5.1: Use the target training samples pre-processed in step 3 to train the target recognition model built in step 4 to obtain a trained target recognition model;

[0059] Step 5.2: Test the accuracy of the trained object recognition model predictions.

[0060] As a specific implementation, the satellite edge node in step 7 performs the target recognition task and returns the target recognition result and the target real-time status to the ground cloud service node, as follows:

[0061] The ground cloud service node establishes a connection with the satellite edge node, clearly identifies the satellite edge node that performs the target recognition task, and transmits the trained target recognition model to the corresponding satellite edge node. The satellite edge node performs the target recognition task, dynamically adjusts the sensor parameters based on different orbital altitudes, and returns the target recognition results and real-time status of the target according to the requirements of the ground cloud service node; the ground cloud service node changes the target recognition model requirements through the cloud-edge collaborative communication link and completes the target recognition and collection tasks in the target area.

[0062] Combine Figure 3The present invention provides a satellite-ground-cloud-edge collaborative ground target recognition device based on a deep network, comprising a ground cloud service node and a satellite edge node, wherein the satellite edge node comprises a computing control module, an on-board communication module, a satellite data acquisition module, a data preprocessing module, a data labeling module and an on-board recognition processing module;

[0063] The ground cloud service node uses a high-performance server cluster for business process control, data transmission and reception control, model training optimization, and model database support;

[0064] The computing control module adopts a heterogeneous computing platform based on CPU+GPU, which is used for onboard business process control, data transmission and reception control, and image processing control;

[0065] The onboard communication module, which includes an antenna, a radio frequency module, a baseband processing module, and an information processing module, is used to receive communication signals from other satellites, demodulate and deframe the signals, and then transmit them to the computing and control module. It also receives data from the computing and control module, frames and modulates the signals, and then sends them to the ground cloud service node or other satellite edge nodes. This module exchanges data with the computing and control module through LVDS and RapidIO interfaces, and the exchanged data types include commands, status, services, target indications, and images.

[0066] The satellite data acquisition module is used to obtain satellite images to be identified;

[0067] The data preprocessing module is used to preprocess the satellite image to be identified and generate a standard image for annotation;

[0068] The data annotation module is used to mark the target content in the pre-processed satellite image for training by the model training module;

[0069] The on-board recognition processing module is used to input the pre-processed standard satellite image into the uploaded target recognition model and output the recognition result.

[0070] Example 1

[0071] In this embodiment, optical images of a large transport vehicle, a large ship, and a large transport aircraft are used as target data to be identified. This type of image data is stored in the target database of the ground cloud service node. The ground cloud service node retrieves the sensor parameters and computing controller parameters of the satellite edge node in the mission execution area, obtaining its optical sensor resolution of 4000*3000, the model storage space capacity P, and the computing controller memory parameters. Then, based on the sensor resolution and model storage space, the optical image data is preprocessed. The resolution of all images of the transport vehicle is normalized to 800*600. The target images are grayscale processed and the images are cropped to the sensor image size. Finally, the preprocessed image data is divided into two groups, A and B, with an 8:2 ratio. Both groups of data are sent to the target data annotation stage. The targets in Group A are boxed and labeled, and the data in Group B is labeled. After completion, Group A is used as the model training sample, and Group B is stored as the model test sample in the target database, pending the next step of training the target recognition model.

[0072] In this example, the MobileNet SSD model was used as the basic model for object recognition. This model features fast recognition speed, compact size, and high computational efficiency. The model consists of a data preprocessing layer, internal hidden layers, and an output layer. MobileNet is responsible for feature extraction, comprising 28 layers, while SSD is responsible for sample classification. The preprocessing layer uses an 800*600*3 structure, and the internal hidden layers utilize a convolutional network structure with a 3*3 kernel size and a Softmax heuristic function.

[0073] In this embodiment, after importing the target recognition basic model, based on the obtained sensor data, the sensor data in this embodiment is an optical sensor with a resolution of 600*800, and the training computing node has an operating power of 2.4GHz. The training samples use Group A data in the target database, and are trained in 3 categories. The training step size is 6 times, and the training is 1000 rounds.

[0074] In this example, after the target recognition model training is complete, it is run on Group B data. In this example, a 98% pass recognition rate is used as the passing recognition rate. Four sets of tests are performed. If the average target recognition accuracy reaches 98%, the model is considered qualified. If it fails to meet the requirement, it is retrained. Once the model passes the test, it is uploaded to the satellite boundary point.

[0075] In this embodiment, during the target recognition task, the ground cloud service node first establishes a link with the satellite edge node performing the task, with other satellite edge nodes acting as relays. After uploading the target model to the satellite edge node, the ground cloud service node sends target recognition instructions, including sampling and transmitting back the target area, and performing uninterrupted recognition of the target area. Upon receiving the instructions, the task satellite edge node activates the virtualized computing environment, calls the recognition sensor to perform a photography task on the target area, and transmits the captured data back to the ground cloud server via the satellite link. The ground cloud service node then sends the recognition task instructions to the satellite edge node, which then performs the recognition task in real time and returns the recognized image and recognition result information.

Claims

1. A satellite-ground-cloud-edge collaborative ground target recognition method based on deep network, characterized by: The following steps are involved: Step 1: Build a satellite-ground-cloud-edge collaborative ground target recognition task support environment and load the existing target recognition task model; Step 2: The ground cloud service node confirms the recognition task model according to the requirements of the recognition task. If the current recognition task model matches, it proceeds to step 7 to perform the recognition task; if the current recognition task model does not meet the requirements of the recognition task, it proceeds to step 3; Step 3: Collect target data for various recognition tasks, including large-scale ground equipment recognition and large-scale maritime vessel recognition, store them in a target database, pre-process and label the target data, and generate standard training samples. Step 4: Build a target recognition model based on onboard resource constraints and configure the training parameters of the target recognition model; Step 5: The ground cloud service node trains the target recognition model and tests the accuracy of the trained target recognition model prediction; Step 6: The ground cloud service node sends the trained target recognition model to the satellite edge node through the satellite-to-ground communication link; Step 7: The satellite edge node performs the target recognition task and returns the target recognition result and real-time target status to the ground cloud service node; Step 8: The ground cloud service node controls the satellite edge node to collect target area data and update the mission database.

2. The satellite-ground-cloud-edge collaborative ground target recognition method based on deep network according to claim 1 is characterized in that: The construction of the satellite-ground-cloud-edge collaborative ground target recognition task support environment described in step 1 is as follows: Step 1.1: Build a satellite-ground-cloud-edge collaborative ground target recognition mission support environment, including building a mission operating environment and building a cloud-edge collaborative communication link; Step 1.2: Build the task running environment and deploy it on the ground cloud service node and satellite edge node; The task operation environment of the ground cloud service node consists of a server cluster, which is used for business process control, data transmission and reception control, model training optimization, and model database support. It is also equipped with a task model library and a target database. The task model library is used to manage existing target recognition models that can be directly used. The target database is used to store various target data, including target data of large ground equipment and large marine vessels. The stored target data is used for retraining or adjusting the target recognition model in the future. The mission operating environment of the satellite edge node is built based on software virtualization technology, which is used for cross-platform deployment and migration of mission systems, on-board mission resource management, control model sending and receiving, mission start and stop, message preparation, and presentation of mission execution results and observation parameters. Step 1.3: Build a cloud-edge collaborative communication link, including an uplink from the ground cloud service node to the satellite edge node, a downlink from the satellite edge node to the ground cloud service node, and parallel links between satellite edge nodes. These links are used for the transmission of satellite-to-ground control commands, recognition models, target data and status information, as well as relay transmission between the ground cloud service node and non-directly connected satellite edge nodes.

3. The satellite-ground-cloud-edge collaborative ground target recognition method based on deep network according to claim 1 is characterized in that: As described in step 3, target data for various recognition tasks are collected and stored in the target database. The target data are preprocessed and labeled to generate standard training samples, as follows: Step 3.1, collect target data of various recognition tasks and store them in the target database; Step 3.2: The ground cloud service node obtains the sensor parameters of the satellite edge node, determines the preprocessing indicators of the target data to be identified based on the sensor parameters, and performs preprocessing operations on the target data to be identified; Step 3.3: Send the pre-processed target data to the target data labeling link, select and label all targets in the target data, and store the target data as model training samples in the target database for the next step of training the target recognition model.

4. The method for satellite-ground-cloud-edge collaborative ground target recognition based on deep network according to claim 3 is characterized in that: The target data of various recognition tasks described in step 3.1 are collected and stored in the target database, as follows: The following two methods are used to collect target data for various recognition tasks: The first method is to collect data on the target to be identified through channels such as public network information collection, professional agency services, and historical data accumulation of the system, and store it in the target database of the ground cloud service node; In the second method, the ground cloud service node sends instructions to the satellite edge nodes within a set range nearby according to the coding of the target area to be identified, collects data of the target to be identified in real time, and stores it in the target database of the ground cloud service node after the collection is completed.

5. The method for satellite-ground-cloud-edge collaborative ground target recognition based on deep network according to claim 3 is characterized in that: The ground cloud service node described in step 3.2 obtains the sensor parameters of the satellite edge node, determines the preprocessing index of the target data to be identified based on the sensor parameters, and performs preprocessing operations on the target data to be identified, as follows: The ground cloud service node obtains the sensor parameters of the satellite edge node, which include the optical sensor resolution, frame, and sampling frequency. The preprocessing indicators of the target data are determined based on the sensor parameters. The preprocessing indicators include resolution and frame. The target data to be identified is normalized and the image is sharpened to highlight the texture features.

6. The satellite-ground-cloud-edge collaborative ground target recognition method based on deep network according to claim 1 is characterized in that According to the on-board resource constraints described in step 4, build a target recognition model and configure the training parameters of the target recognition model as follows: Step 4.1: The ground cloud service node obtains the software and hardware resource parameters of the satellite edge node and builds a target recognition model that adapts to the hardware constraints of the satellite edge node, including a data preprocessing layer, an internal hidden layer, and an output layer. Step 4.2: Design the network structure and parameters of the target recognition model, including the number of model layers, internal hidden layers, number of units in each layer, number of network layer iterations, optimizer, evaluation function, activation function, and dropout layer; Step 4.3: Based on the computing speed and sample size of the ground cloud service node, determine the network update frequency, number of training iterations, and evaluation function of the target recognition model.

7. The method for satellite-ground-cloud-edge collaborative ground target recognition based on deep network according to claim 6 is characterized in that: The object recognition model described in step 4.1 uses the MobileNet SSD model as the base model.

8. The method for satellite-ground-cloud-edge collaborative ground target recognition based on deep network according to claim 1 is characterized in that: The ground cloud service node described in step 5 trains the target recognition model and tests the accuracy of the trained target recognition model predictions as follows: Step 5.1: Use the target training samples pre-processed in step 3 to train the target recognition model built in step 4 to obtain a trained target recognition model; Step 5.2: Test the accuracy of the trained object recognition model predictions.

9. The satellite-ground-cloud-edge collaborative ground target recognition method based on deep network according to claim 1 is characterized in that: The satellite edge node described in step 7 performs the target recognition task and returns the target recognition result and the target real-time status to the ground cloud service node, as follows: The ground cloud service node establishes a connection with the satellite edge node, identifies the satellite edge node that performs the target recognition task, and transmits the trained target recognition model to the corresponding satellite edge node. The satellite edge node performs the target recognition task, dynamically adjusts the sensor parameters based on different orbital altitudes, and returns the target recognition results and real-time target status according to the requirements of the ground cloud service node. The ground cloud service node changes the target recognition model requirements and completes the target recognition and collection tasks in the target area through the cloud-edge collaborative communication link.

10. A satellite-ground-cloud-edge collaborative ground target recognition device based on deep network, characterized in that: It includes ground cloud service nodes and satellite edge nodes, wherein the satellite edge nodes include a computing control module, an on-board communication module, a satellite data acquisition module, a data preprocessing module, a data annotation module and an on-board identification processing module; The ground cloud service node uses a high-performance server cluster for business process control, data transmission and reception control, model training optimization, and model database support; The computing control module adopts a heterogeneous computing platform based on CPU+GPU, which is used for onboard business process control, data transmission and reception control, and image processing control; The onboard communication module is used to receive communication signals from other satellites and implement signal demodulation and deframing, and then transmit them to the calculation and control module; at the same time, it receives data from the calculation and control module, implements signal framing and modulation, and then sends it to the ground cloud service node or other satellite edge nodes; The satellite data acquisition module is used to obtain satellite images to be identified; The data preprocessing module is used to preprocess the satellite image to be identified and generate a standard image for annotation; The data annotation module is used to mark the target content in the pre-processed satellite image for training by the model training module; The on-board recognition processing module is used to input the pre-processed standard satellite image into the uploaded target recognition model and output the recognition result.

Images