A remote sensing satellite monitoring system and method

Abstract

The embodiment of the application discloses a remote sensing satellite monitoring system and method, the system comprises a terminal, generates monitoring task data according to input data, decodes identification result semantic data and calls a target model library to carry out target positioning and scene display; a satellite-ground communication module, interacts monitoring task data and identification result semantic data, sends algorithm update data and attitude and orbit control signals and receives image data; a satellite-borne computer, decodes monitoring task data and generates monitoring signals and target identification signals according to satellite time data and satellite attitude and orbit data; a satellite-borne camera, receives monitoring signals and scans to obtain image data when a satellite passes through a region to be monitored within a monitoring time; a satellite-borne processing module, receives target identification signals, satellite attitude and orbit data and image data, processes image data to obtain target identification result data and encodes identification result semantic data; a satellite-borne attitude and orbit control module, collects satellite attitude and orbit data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of remote sensing satellite application services. More particularly, it relates to a remote sensing satellite monitoring system and method. BACKGROUND

[0002] In the traditional remote sensing satellite data acquisition process, it takes 2 hours to several days to transmit the images collected from the satellite to the ground and to receive and preprocess them on the ground. The time limit in hours has become a pain point for the development of the remote sensing application industry. From ship tracking to city emergency management to environmental dynamic monitoring, the pursuit of timeliness is the core development direction of current remote sensing application enterprises.

[0003] The development of China's satellite internet system has laid a space network infrastructure for the future development of low-orbit satellites and constellations in the direction of networking and intelligentization. Future low-orbit constellations rely on laser links to form a high-speed interconnected constellation network. Therefore, research on high-performance distributed intelligent processing units on board can enable flexible deployment of real-time task planning and resource scheduling algorithms, efficient collaborative real-time applications for remote sensing satellites, and is a future development trend. Under the condition that satellite remote sensing data real-time acquisition, on-board computing power, and real-time transmission of interpretation results are guaranteed, domestic and foreign research institutions are continuously developing research and practice on various real-time processing functions of on-board remote sensing data.

[0004] For example, abroad, the E01 satellite launched by the United States has an on-board processor that can complete automatic identification of areas of interest, regional change detection, cloud judgment, and invalid data rejection on orbit; the TechSat21 plan developed by the U.S. Air Force Laboratory AFRL realizes on-board SAR moving target detection (MTI) and geographic positioning; the Jet Propulsion Laboratory (JPL) of the United States realizes real-time imaging of SAR on the RS satellite, completes real-time search and rescue target retrieval and water body change detection, and realizes real-time ground subsidence information acquisition by interferometric processing of interferometric SAR data; the BIRD satellite launched by Germany has an on-board processor that can complete "hot spot detection" such as vegetation fires, volcanic activity, and oil well fires on orbit; the Pleiades-HR of France uses a FPGA-based MVP modular processor to realize on-board processing, including radiation correction, geometric correction, and image compression, etc.

[0005] The Haisi-1 satellite launched by China in December 2020 carried an intelligent processing module developed by the Shijiazhuang team to carry out on-orbit preprocessing and target detection of SAR data. The Gaofen-3 02 satellite launched by China in November 2021 added an on-board real-time processor (e.g., based on an FPGA+DSP architecture) to achieve on-board SAR satellite preprocessing (e.g., interferometric processing and phase unwrapping), shortening the ground image acquisition process. In February 2022, the "Chaohu-1" satellite developed by Changsha Tianyi Space Technology Research Institute Co., Ltd. was successfully launched, with functions such as SAR on-orbit image compression, preprocessing, and AI target recognition. The core project "Jilin-1" satellite constellation developed by Changguang Satellite Technology Co., Ltd. has undergone three upgrades of the on-board intelligent processing platform, and the on-board computing power has been greatly improved. Based on various optical satellite on-board intelligent processing algorithms, real-time processing of 500 square kilometers of mid-infrared images per second on board is achieved, and high-temperature hotspots are identified. Real-time registration processing can be performed on video images captured by the satellite to extract stationary aircraft targets and calculate the speed of moving aircraft and vehicle targets. The "Luojia-3 01" satellite developed by Wuhan University was launched on January 15, 2023, carrying a Long March 2D (Y71) launch vehicle. Through software installed on a 5G mobile terminal, the user can select the satellite observation position and imaging mode, generate a task request, and push it to the cloud service center. Then, through the ground control station network, the command is uploaded. When the satellite executes the task, it collects video, performs on-orbit intelligent processing and on-board compression, and then transmits the data back to the Wuhan ground station through satellite-ground communication, and then sends it to the user after demodulation and analysis.

[0006] The above application results fully demonstrate that China's remote sensing satellites have made great progress in on-board intelligent processing, and various application modes are still being explored. In particular, a remote sensing monitoring system that is simpler, easier to understand, and has a faster and more direct task uploading channel for user remote sensing monitoring task planning, rapid geographic positioning of targets, more realistic display of on-board target recognition content and surrounding environment, and convenient user understanding of the corresponding topological relationship between ground objects is urgently needed. SUMMARY

[0007] The purpose of the present application is to provide a remote sensing satellite monitoring system and method to solve at least one of the problems in the prior art.

[0008] To achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows:

[0009] The first aspect of the application provides a remote sensing satellite monitoring system, which comprises a ground terminal for generating remote sensing monitoring task data according to user input data and sending the data to a satellite-ground communication module, and decoding identification result semantic data sent by the satellite-ground communication module and calling a plurality of model databases for at least one target identification, positioning and multi-dimensional display of environment;

[0010] The satellite-ground communication module is used for interacting remote sensing monitoring task data and identification result semantic data, and sending algorithm update data and attitude and orbit control signals to a satellite-borne processing module and a satellite-borne attitude and orbit control module, and receiving ground remote sensing image data sent by a satellite-borne camera;

[0011] The satellite-borne computer is used for decoding remote sensing monitoring task data and generating remote sensing monitoring signals and target identification signals according to satellite time data and satellite attitude and orbit data, and sending the signals to the satellite-borne camera and the satellite-borne processing module, and collecting satellite operation state information and sending the information to the ground terminal via the satellite-ground communication module to update satellite state information;

[0012] The satellite-borne camera is used for receiving remote sensing monitoring signals and scanning to obtain ground remote sensing image data when the satellite passes through a remote sensing area to be monitored within a monitoring time, and sending the ground remote sensing image data to the satellite-borne processing module;

[0013] The satellite-borne processing module is used for receiving target identification signals, satellite attitude and orbit data and ground remote sensing image data, processing the ground remote sensing image data to obtain target identification result data, and encoding the target identification result data to obtain identification result semantic data and sending the data to the satellite-ground communication module;

[0014] The satellite-borne attitude and orbit control module is used for collecting satellite attitude and orbit data and sending the data to the satellite-borne processing module and the satellite-borne computer.

[0015] Optionally, the satellite-ground communication module comprises a ground communication module for receiving remote sensing monitoring task data and sending the data to a satellite-borne communication module, and receiving identification result semantic data sent by the satellite-borne communication module and sending the data to the ground terminal;

[0016] The satellite-borne communication module is used for receiving remote sensing monitoring task data and sending the data to the satellite-borne computer, and receiving identification result semantic data sent by the satellite-borne processing module and sending the data to the ground communication module;

[0017] The ground station is used for sending algorithm update data of the satellite-borne processing module to the satellite-borne processing module, sending satellite attitude and orbit control signals to the satellite-borne attitude and orbit control module, and receiving ground remote sensing image data sent by the satellite-borne camera.

[0018] Optionally, the ground terminal comprises a remote sensing monitoring task autonomous planning unit;

[0019] The remote sensing monitoring task autonomous planning unit is used to generate remote sensing monitoring tasks in response to the remote sensing monitoring area, monitoring time and target type set by the user, and to encode the remote sensing monitoring tasks to generate remote sensing monitoring task data.

[0020] Optionally, the onboard processing module includes a target recognition and localization unit;

[0021] The target identification and positioning unit is used to perform radiometric correction processing, land-sea judgment processing, image slicing processing, target identification processing, and target positioning processing on ground remote sensing image data to obtain target identification result data.

[0022] Optionally, the ground station includes a satellite-borne processing algorithm uploading unit;

[0023] The satellite-borne processing algorithm uploading unit is used to process ground remote sensing image data to obtain simulated target recognition result data, and to determine whether the simulated target recognition result data and the recognition result semantic data are consistent.

[0024] If there is a discrepancy, the algorithm of the onboard processing module stored in the onboard processing algorithm uploading unit is debugged and sent to the onboard processing module.

[0025] Optionally, the ground terminal further includes a target multi-dimensional scene display unit;

[0026] The target multi-dimensional scene display unit is used to visualize the target multi-dimensional map based on image simulation and dynamic simulation algorithms and multiple model databases and target recognition result data.

[0027] Optionally, the target identification result data includes target type, confidence level, center point longitude, center point latitude, length, width, orientation, and static / dynamic status.

[0028] Optionally, the target multi-dimensional scene display unit includes a target type two-dimensional and three-dimensional model database, a global Internet remote sensing satellite base map model database, a global digital elevation model database, and a local area three-dimensional geographic scene model database.

[0029] Optionally, the target recognition and localization unit performs target recognition and target localization based on deep learning.

[0030] The first aspect of the present invention provides a remote sensing satellite monitoring method, which includes using a ground terminal to encode and generate remote sensing monitoring task data based on user input data and send it to a satellite-to-ground communication module, and decoding the semantic data of the identification results sent by the satellite-to-ground communication module and calling multiple model databases to perform multi-dimensional display of at least one target identification, positioning and environment.

[0031] Using the satellite-to-ground communication module, remote sensing monitoring mission data and semantic data of recognition results are exchanged, and algorithm update data and attitude and orbit control signals are sent to the satellite processing module and the satellite attitude and orbit control module, and ground remote sensing image data sent by the satellite camera are received.

[0032] Using the onboard computer, remote sensing monitoring mission data is decoded, and remote sensing monitoring signals and target identification signals are generated autonomously based on satellite time data and satellite attitude and orbit data. These signals are then sent to the onboard camera and onboard processing module, and satellite operation status information is collected and sent to the ground terminal via the satellite-to-ground communication module to update the satellite status information.

[0033] Using a satellite-borne camera, remote sensing monitoring signals are received, and when the satellite passes over the area to be monitored during the monitoring period, ground remote sensing image data is scanned and acquired, and the ground remote sensing image data is sent to the satellite-borne processing module.

[0034] Using the onboard processing module, target recognition signals, satellite attitude and orbit data and ground remote sensing image data are received. The ground remote sensing image data is processed to obtain target recognition result data, and the target recognition result data is encoded to obtain recognition result semantic data and sent to the satellite-to-ground communication module.

[0035] The satellite attitude and orbit control module is used to collect satellite attitude and orbit data and send it to the satellite processing module and the satellite computer.

[0036] The beneficial effects of this invention are as follows:

[0037] This invention provides a remote sensing satellite monitoring system that simplifies and clarifies the planning of remote sensing monitoring tasks for users, and enables faster and more direct task assignment. It also enables rapid geographic positioning of targets and provides a more realistic display of the identified satellite targets and their surrounding environment, facilitating users' immediate and deeper understanding of the corresponding topological relationships between ground features. Attached Figure Description

[0038] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0039] Figure 1 This diagram illustrates an application of the remote sensing satellite monitoring system provided in an embodiment of the present invention.

[0040] Figure 2 This diagram illustrates another application of the remote sensing satellite monitoring system provided in this embodiment of the invention.

[0041] Figure 3 This diagram illustrates the process of processing onboard remote sensing image data from a low-Earth orbit satellite in the onboard processing module of the remote sensing satellite monitoring system provided in this embodiment of the invention.

[0042] Figure 4 The diagram illustrates the process of generating image slices and parallel target recognition processing from remote sensing image data streams in the onboard processing module of the remote sensing satellite monitoring system provided in this embodiment of the invention.

[0043] Figure 5 This diagram illustrates the ground debugging and on-orbit testing flowchart of the processing algorithm of the onboard processing module of the remote sensing satellite monitoring system provided in this embodiment of the invention.

[0044] Figure 6 This diagram illustrates the geographical extent of a ground-based remote sensing image of the remote sensing satellite monitoring system provided in an embodiment of the present invention.

[0045] Figure 7 This diagram illustrates a local target area in a ground-based remote sensing image of the remote sensing satellite monitoring system provided in an embodiment of the present invention.

[0046] Figure 8 This diagram illustrates a three-dimensional target model of the remote sensing satellite monitoring system provided in an embodiment of the present invention. Detailed Implementation

[0047] To more clearly illustrate the present invention, the following description, in conjunction with embodiments and accompanying drawings, further explains the invention. Similar components in the drawings are indicated by the same reference numerals. Those skilled in the art should understand that the specific description below is illustrative rather than restrictive and should not be construed as limiting the scope of protection of the present invention.

[0048] Currently, remote sensing satellites have made significant progress in onboard intelligent processing, and their various application modes are still being explored. In particular, there is an urgent need to develop a remote sensing monitoring system that is simpler, clearer, and easier to understand, with faster and more direct task assignment channels for user remote sensing monitoring mission planning, rapid target geolocation, more realistic display of onboard target identification content and surrounding environment, and facilitates users' immediate and deeper understanding of the corresponding topological relationships between ground features.

[0049] In view of this, one embodiment of the present invention provides a remote sensing satellite monitoring system, which includes a ground terminal for encoding remote sensing monitoring task data based on user input data and sending it to a satellite-to-ground communication module, and for decoding the semantic data of the recognition results sent by the satellite-to-ground communication module and calling multiple model databases to perform multi-dimensional display of at least one target identification, localization, and environment display; a satellite-to-ground communication module for interacting with the remote sensing monitoring task data and the semantic data of the recognition results, and for sending algorithm update data and attitude and orbit control signals to the onboard processing module and the onboard attitude and orbit control module, and for receiving ground remote sensing image data sent by the onboard camera; and an onboard computer for decoding the remote sensing monitoring task data and performing autonomous task planning and generating remote sensing monitoring signals based on satellite time data and satellite attitude and orbit data. The system receives target identification signals and sends them to the onboard camera and onboard processing module. It also collects satellite operational status information and sends it to the ground terminal via the satellite-to-ground communication module to update the satellite status information. The onboard camera receives remote sensing monitoring signals and scans the ground remote sensing image data when the satellite passes over the area to be monitored during the monitoring period. It then sends the ground remote sensing image data to the onboard processing module. The onboard processing module receives target identification signals, satellite attitude and orbit data, and ground remote sensing image data. It processes the ground remote sensing image data to obtain target identification result data and encodes the target identification result data to obtain semantic data of the identification result, which is then sent to the satellite-to-ground communication module. The onboard attitude and orbit control module collects satellite attitude and orbit data and sends it to the onboard processing module and the onboard computer.

[0050] In a specific example, such as Figure 1 As shown, the remote sensing satellite monitoring system provided by this invention can be a real-time remote sensing satellite application service system. Furthermore, in this real-time remote sensing satellite application service system, users can plan satellite remote sensing monitoring tasks through a visual interface on a handheld tablet computer (PAD) or other terminal device, and can instantly obtain the interpretation results information of intelligent satellite remote sensing monitoring.

[0051] In a specific example, the PAD terminal automatically generates short message command data (e.g., remote sensing monitoring task data) based on the remote sensing monitoring area, monitoring time, and target type specified by the user. This data is then sent to the ground user's BeiDou short message terminal (e.g., ground communication module), then to the onboard BeiDou short message integrated machine (e.g., onboard communication module), and finally to the onboard integrated computer (e.g., onboard computer or satellite service computer). The onboard integrated computer, combined with the onboard measured satellite orbit parameters, calculates and generates remote sensing monitoring signals and target identification commands (e.g., target identification signals), which are then transmitted to the onboard intelligent processing equipment (e.g., onboard processing module) and the onboard camera for execution. The satellite-ground communication module includes the ground user short message terminal and the onboard BeiDou short message integrated machine.

[0052] Furthermore, when the satellite passes over the monitoring area (e.g., the remote sensing monitoring area) at a specified time (e.g., the monitoring time), the satellite line scan camera (e.g., the onboard camera) is turned on to scan and acquire ground remote sensing image data. The onboard intelligent processing equipment processes the image data (e.g., ground remote sensing image data) in real time to generate target recognition results. The target recognition results include target type, target status, and geographical location. Taking a ship target as an example, it includes ship type, recognition confidence level, ship orientation, static and dynamic status, pixel coordinates, and geographical coordinates.

[0053] Furthermore, the target identification results need to be encoded and encrypted into short messages, which are then sent to ground equipment (such as PAD terminals) via the satellite-borne Beidou integrated machine for decoding. The PAD terminal can then call the target model library based on the short message data to locate and visualize the target and its surrounding scene in two-dimensional and three-dimensional maps.

[0054] Furthermore, the onboard computer is used to periodically transmit satellite operational status information to ground equipment via the onboard BeiDou integrated machine. This information is used to correct the satellite's orbital position displayed on the ground, and the PAD terminal can update and visualize the satellite status information. Additionally, the satellite operational status information also includes satellite battery level, temperature, and deformation information.

[0055] In a specific example, such as Figure 2 As shown, the remote sensing satellite monitoring system provided by this invention can be a one-click remote sensing application service system. Further, the PAD terminal can be used to generate remote sensing monitoring tasks and also to parse and visualize the semantic information of on-board identification results (e.g., semantic data of identification results); the ground user BeiDou short message terminal and the on-board BeiDou short message terminal can be used to send remote sensing monitoring tasks to the satellite computer and also to receive target identification semantic short message information processed on-board; the on-board intelligent processing equipment performs both remote sensing data preprocessing and target identification, semantic generation, target positioning, and short message information generation. The satellite computer receives remote sensing meta-task instructions (e.g., remote sensing monitoring task data) from the on-board BeiDou short message integrated machine to perform autonomous task planning, and controls the satellite camera (e.g., on-board camera) and the on-board intelligent processing equipment (e.g., on-board processing module) to execute ground remote sensing image acquisition and target identification tasks within the planned specified time. Satellite attitude and orbit control equipment (such as onboard attitude and orbit control modules) transmits the acquired satellite attitude and orbit information to onboard intelligent processing equipment for onboard target geolocation and to the satellite mission computer for high-precision onboard autonomous mission planning.

[0056] The embodiments of the present invention enable users to plan remote sensing monitoring tasks in a simpler, clearer, and easier-to-understand way, and to upload tasks more quickly and directly; to achieve rapid geographic positioning of targets; and to display the satellite-borne target identification content and surrounding environment more realistically, so as to facilitate users to understand the corresponding topological relationships between ground features more quickly and deeply.

[0057] In a specific example, a real-time remote sensing satellite application service system includes hardware devices related to software functions such as PAD terminals, ground user BeiDou short message terminals, onboard BeiDou short message integrated machines, satellite service computers, satellite cameras, intelligent processing equipment, satellite attitude and orbit control equipment, satellite platforms, and other payloads; its software units include a remote sensing mission autonomous planning unit (e.g., a remote sensing monitoring mission autonomous planning unit), an onboard intelligent target recognition and rapid positioning unit (e.g., a target recognition and positioning unit), a satellite-to-ground data communication and encoding / decoding unit (e.g., a ground communication unit, an onboard communication unit, and a ground station unit), an onboard intelligent processing algorithm uploading module (e.g., an onboard processing algorithm uploading unit), and a target two-dimensional / three-dimensional scene display unit (e.g., a target multi-dimensional scene display unit).

[0058] In one possible implementation, the ground terminal includes a remote sensing monitoring task autonomous planning unit; the remote sensing monitoring task autonomous planning unit is used to generate remote sensing monitoring tasks in response to the remote sensing monitoring area, monitoring time and target type set by the user, and to encode the remote sensing monitoring tasks to generate remote sensing monitoring task data.

[0059] In a specific example, the autonomous remote sensing monitoring task planning unit enables users to perform low-Earth orbit satellite remote sensing monitoring task analysis and planning via a PAD terminal. Furthermore, the user first generates a remote sensing monitoring task through the interface of the autonomous remote sensing monitoring task planning unit on the PAD terminal device, then sends it to the ground user's BeiDou short message terminal, then to the onboard navigation short message integrated machine, and finally to the onboard computer for autonomous task planning. The task planning results are then executed by the onboard camera and intelligent processing equipment within a specified time window, controlled by the onboard computer.

[0060] It should be noted that since the PAD terminal's mission planning results for the satellite are based on the satellite's historical orbit data, which has a certain degree of bias, and the satellite's computer will use the current orbit data to recalculate the specific execution time for each mission, the execution time and the time planned on the ground may have a certain degree of bias. Therefore, the satellite has a certain degree of autonomy in mission planning.

[0061] Furthermore, users can simply draw and delineate the area to be monitored (e.g., the remote sensing monitoring area) on a two-dimensional map through the interface of the remote sensing monitoring task self-planning unit software on the PAD terminal, and input the monitoring time window range (e.g., monitoring time) and specify the target type information of the remote sensing task (e.g., target type).

[0062] In a specific example, due to the limitations of short message communication encoding, this embodiment sets the maximum number of identifiable target types to 255, and supports the simultaneous monitoring of up to 16 targets.

[0063] In a specific example, the remote sensing monitoring task is transformed into semantic text as follows:

[0064] Area: (E109.486,N21.7518)(E109.727,N21.7015)(E109.675,N21.4886)(E109.434,N21.5388);

[0065] Time: 2023-06-16 11:25:34, 2023-06-28 11:25:34;

[0066] Target: Plain, Carrier, Warship,….

[0067] Where Area represents the remote sensing monitoring area; Time represents the monitoring time; Target represents the target type; Plain represents aircraft; Carrier represents aircraft carrier; Warship represents strategic ship; E represents east longitude; and N represents north latitude.

[0068] In a specific example, the PAD terminal and the satellite computer calculate the satellite transit time in different time periods based on the semantic text information above, and determine the onboard camera's power-on and power-off times to control the camera's operating timing.

[0069] In a specific example, the satellite's operational status information is collected by the satellite's onboard BeiDou integrated device and periodically sent to ground equipment. This information is used to correct the satellite's orbital position displayed on the ground terminal, and the PAD terminal can update and visualize the satellite's status information.

[0070] In a specific example, since the satellite computer can obtain more accurate satellite orbit parameters and satellite time information, the satellite mission planning results calculated by the satellite computer are used as the basis for execution; while the mission planning results executed by the ground PAD terminal are only used as a data source to assist users in visual interpretation, allowing users to roughly understand the effective transit time of the low-Earth orbit satellite constellation.

[0071] This embodiment can reduce the amount of data exchanged between the PAD terminal and the satellite during the satellite mission planning process, and also improve the accuracy and effectiveness of satellite autonomous mission planning.

[0072] The remote sensing monitoring task autonomous planning unit in this embodiment enables users to plan satellite remote sensing monitoring tasks directly on a PAD mobile device. Users can outline the area to be monitored, set a monitoring time window, and generate a remote sensing monitoring task through the PAD terminal interface. The PAD terminal then automatically edits and generates a short message, which is sent to the ground user's BeiDou short message terminal, then to the onboard navigation short message integrated unit, and finally to the satellite-based computer for autonomous task planning. The task planning result is then controlled by the satellite-based computer to execute the onboard camera and onboard intelligent processing equipment within the specified time window.

[0073] This embodiment utilizes a map-based visual interaction on the PAD terminal to plan remote sensing monitoring tasks, shielding users from complex satellite orbit calculations or transit time assessments. This makes the entire process simpler, clearer, and easier to understand, and the task assignment channel is faster and more direct.

[0074] In one possible implementation, the spaceborne processing module includes a target identification and positioning unit; the target identification and positioning unit is used to perform radiometric correction processing, image slicing processing, land-sea judgment processing, target identification processing, and target positioning processing on ground remote sensing image data to obtain target identification result data.

[0075] In a specific example, the data processing object of the low-orbit remote sensing satellite is the on-board Level 0 remote sensing data generated in real time by the satellite camera (e.g., the onboard camera). The onboard intelligent processing module processes the on-board Level 0 remote sensing data using a radiometric correction algorithm to generate a Level 1 remote sensing data stream, and processes the Level 1 remote sensing data stream through land-sea judgment, intelligent target recognition (e.g., target recognition processing), and target positioning algorithms (e.g., target positioning processing).

[0076] In a specific example, radiometric correction includes spectral segmentation, strip noise removal, and satellite attitude and orbit control parameter calculation; image tiling and land / sea determination include steps such as ephemeris and attitude parameter analysis, image tiling, geographic extent calculation, and land / sea determination of the tiling.

[0077] This embodiment includes an onboard intelligent target recognition and positioning unit (e.g., a target recognition and positioning unit). The object of low-orbit remote sensing satellite data processing is the onboard Level 0 remote sensing data generated in real time by the satellite camera. The onboard intelligent target recognition processing realizes the generation of a radiometrically corrected Level 1 remote sensing data stream from the onboard Level 0 remote sensing data, sea-land judgment, intelligent target recognition, and intelligent target positioning algorithms. That is, it realizes real-time target recognition and positioning processing of remote sensing monitoring data on the satellite, and can identify the target's geographical location, orientation, and static and dynamic status information.

[0078] In one possible implementation, the target recognition and localization unit performs target recognition and target localization based on deep learning.

[0079] In a specific example, intelligent object recognition includes a deep neural network-based object detection and recognition algorithm model, which possesses powerful feature learning and transfer learning capabilities. Furthermore, the YOLO series models perform well on small object detection and recognition tasks, with single-stage object detection completing the entire object detection and recognition process in one stage, essentially meeting the requirements of real-time systems.

[0080] Furthermore, this embodiment uses YOLOv5 as a single-stage model and employs Mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling to enhance the detection effect on small targets, maintaining accuracy while ensuring lightweight design.

[0081] In a specific example, the image slicing and intelligent target recognition processes can be completed using a multi-threaded parallel computing approach.

[0082] In a specific example, the on-board intelligent target recognition and processing algorithm runs on the AI ​​chip of the on-board intelligent processing board. Furthermore, the flowchart for separately generating image tiles from the remote sensing data stream and the parallel target recognition processing is shown below. Figure 3 and Figure 4 As shown.

[0083] This embodiment uses the YoLoV5 deep learning algorithm to identify target type, location, orientation, and static / dynamic status information, and directly achieves rapid geolocation of the target based on the attitude and orbit parameters collected by the onboard attitude and orbit control equipment.

[0084] In one possible implementation, the space-to-ground communication module includes a ground communication module for receiving remote sensing monitoring mission data and sending it to the spaceborne communication module, and for receiving semantic data of recognition results sent by the spaceborne communication module and sending it to a ground terminal; a spaceborne communication module for receiving remote sensing monitoring mission data and sending it to the spaceborne computer, and for receiving semantic data of recognition results sent by the spaceborne processing module and sending it to the ground communication module; and a ground station for sending algorithm update data from the spaceborne processing module to the spaceborne processing module, sending satellite attitude and orbit control signals to the spaceborne attitude and orbit control module, and receiving ground remote sensing image data sent by the spaceborne camera.

[0085] In a specific example, the satellite-to-ground communication module implements satellite-to-ground data communication functions, which are achieved by the remote sensing satellite ground station, ground user BeiDou short message terminals (e.g., ground communication modules), and onboard BeiDou short message terminals (e.g., onboard communication modules). Furthermore, the satellite-based computer and PAD terminal implement short message data encoding and decoding functions.

[0086] In a specific example, the data transmitted by the satellite-to-ground communication module includes BeiDou short message data, which comprises planning mission instructions (e.g., remote sensing monitoring mission data), satellite control instructions (e.g., satellite attitude and orbit control signals), and on-board target identification results (e.g., semantic data of the identification results). Furthermore, the satellite-to-ground communication module transmits remote sensing image data (e.g., ground remote sensing image data) and data uploaded to the onboard processing algorithm program (e.g., algorithm update data from the onboard processing module). It is important to note that ground user BeiDou short message terminals can only transmit short message data.

[0087] In a specific example, the real-time remote sensing application system uses two communication methods between the satellite and the ground: one is BeiDou short message communication, and the other is remote sensing satellite ground station communication. The BeiDou short message communication method is primarily for ordinary users, enabling the uploading of remote sensing monitoring task planning information and the downloading of target identification results. The ground station communication method is primarily for administrators, and in addition to providing functions for ordinary users, it also includes onboard intelligent processing algorithm updates, remote sensing data downloading, and satellite attitude and orbit control.

[0088] In a specific example, the ground station communication method is the same as the remote sensing satellite ground communication method; the short message communication method uses the encoding of remote sensing monitoring tasks and monitoring results for uploading and downloading. Furthermore, the short message encoding method is designed as shown in Table 1.

[0089] Table 1. Coding Rules for Remote Sensing Monitoring Task Upload

[0090]

[0091] In a specific example, due to the length limitation of onboard short message communication, the number of targets involved in the target identification results sent in each short message is also limited. Taking the 154-byte limit as an example, the target identification result downlink encoding rules are shown in Table 2. A single short message can send 6 target identification attribute information. Furthermore, if it is necessary to send more than a number of target attributes, it needs to be encoded into multiple short messages for downlink. After receiving the data on the ground, the targets can be sorted according to their identification time based on their target numbers.

[0092] In a specific example, to help the PAD terminal understand the approximate geographical extent of satellite-captured images, the first target (e.g., target number 0x00) stores the geographical extent information of the entire satellite image. This involves recording the geographical coordinates of the center point of the entire satellite image, its pixel dimensions, and tilt angle, and transmitting this information to the PAD terminal for mapping the remote sensing monitoring area.

[0093] Table 2. Encoding Rules for Target Recognition Results Download

[0094]

[0095]

[0096] In this embodiment, the satellite-to-ground data communication module transmits user remote sensing monitoring mission planning information and onboard intelligent target identification information via BeiDou short message communication. The BeiDou short message communication method is for ordinary users, enabling the uploading of remote sensing monitoring mission planning information and the downloading of target identification results. The ground station communication method is for administrator users, and in addition to providing functions for ordinary users, it also includes functions for updating onboard intelligent processing algorithms, downloading remote sensing data, and controlling satellite attitude and orbit. The short message communication method uses encoding of remote sensing monitoring missions and monitoring results for both uploading and downloading.

[0097] This embodiment implements short message encoding of remote sensing monitoring planning tasks and target identification results to achieve corresponding satellite-to-ground communication.

[0098] In one possible implementation, the ground station includes a spaceborne processing algorithm uploading unit; the spaceborne processing algorithm uploading unit is used to process ground remote sensing image data to obtain simulated target recognition result data, and determine whether the simulated target recognition result data and the recognition result semantic data are consistent; if they are inconsistent, the algorithm of the spaceborne processing module stored in the spaceborne processing algorithm uploading unit is debugged and sent to the spaceborne processing module.

[0099] In a specific example, the real-time remote sensing application system has the capability to upload ground-based debugging algorithms to the intelligent satellite processing module. The algorithm uploading and on-orbit testing process at the satellite's intelligent processing chip is as follows: Figure 5 As shown.

[0100] In a specific example, since the satellite-based processing algorithm uploading function requires a large satellite-to-ground transmission rate, it needs to be completed with the help of a satellite ground receiving and processing station (e.g., a ground station).

[0101] In a specific example, the operating system and the updatable algorithm for onboard intelligent processing in the onboard processing module are logically separated. The algorithm has hot-swappable plug-in capability, which facilitates updates and replacements.

[0102] In a specific example, after receiving remote sensing image data and onboard intelligent processing results (e.g., semantic data of recognition results), the ground station first needs to simulate the raw remote sensing data (e.g., remote sensing image data) acquired onboard to conduct an onboard intelligent processing simulation experiment to reproduce the intelligent processing results (e.g., simulated target recognition result data) and see if they are consistent with those processed by the onboard processing module. If they are inconsistent, ground debugging and testing of the onboard processing algorithm are then conducted to check whether the algorithm's overall performance, such as accuracy, efficiency, and size, has improved, or whether it has solved the actual target detection problem. If improvements are confirmed, the data is then uploaded to the onboard processing chip in the onboard processing module via the ground station for on-orbit performance testing. If they are consistent, no debugging or testing is required.

[0103] This embodiment enables the ground to continuously test and update processing algorithms and software to improve the on-board intelligent target recognition capability of the on-board processing module, and simultaneously improves the on-orbit processing capability by updating the on-board processing algorithm of the on-board processing module through the high-speed on-board structural channel of the ground station.

[0104] The onboard intelligent processing algorithm uploading unit (e.g., the onboard processing algorithm uploading unit) in this embodiment realizes the uploading and updating of onboard intelligent processing algorithms. The operating system and the uploading algorithms of the onboard intelligent processing unit are logically separated, and the algorithms have hot-swappable plug-in capabilities for easy updates and replacements. By acquiring remote sensing data and onboard intelligent processing results from satellite images using a satellite ground receiving and processing station, and simulating the raw remote sensing data acquired onboard, onboard intelligent processing experiments can be conducted. This allows for the reproduction of intelligent processing results, ground debugging and testing of the onboard processing algorithm, and verification of the algorithm's comprehensive performance, including accuracy, efficiency, and size.

[0105] In one possible implementation, the ground terminal further includes a target multi-dimensional scene display unit; the target multi-dimensional scene display unit is used to visualize the target multi-dimensional map based on image simulation and dynamic simulation algorithms and multiple model databases and according to target recognition result data.

[0106] In a specific example, the target multi-dimensional scene display unit includes target two-dimensional and three-dimensional scene demonstration units. Furthermore, the target two-dimensional and three-dimensional scene demonstration units primarily rely on PAD terminals capable of connecting to an open internet environment to realize the simulation and visualization demonstration of satellite image real-time processing and target recognition results. This function adds image simulation and dynamic simulation algorithms to provide two-dimensional and three-dimensional map visualizations of the identified targets, allowing for a more intuitive display of the real-time target status and surrounding geographical information that can be obtained through satellite-based intelligent real-time processing and ground-based historical scene rendering.

[0107] In one possible implementation, the target identification result data includes target type, confidence level, center point longitude, center point latitude, length, width, orientation, and static / dynamic status.

[0108] In a specific example, on the PAD terminal, a target foreground image is overlaid based on a pre-installed geographic information base map and reconstructed from the semantic information of the BeiDou short message. Furthermore, for the on-board processing results, the semantic information of the BeiDou short message includes target type, confidence level, center point longitude, center point latitude, length, width, orientation, and static / dynamic status; or it may include no target detected, in which case only the geographic range information of the acquired image is sent.

[0109] In a specific example, when displayed on a ground-based PAD terminal, in addition to acquiring target short message information (such as target recognition result data), it is also necessary to call the specified two-dimensional and three-dimensional models in the target model library based on the target type information, and generate target two-dimensional and three-dimensional model texture styles based on the target's length, width, orientation, and static / dynamic status information. Furthermore, using Internet remote sensing map data and the latest target recognition results, entirely new two-dimensional and three-dimensional geographic scene information is constructed.

[0110] In a specific example, the association between global and local geographic scenes is illustrated as follows: Figure 6-8 As shown, Figure 7 and Figure 8 The image shows a simulated two-dimensional and three-dimensional scene model of a target using the latest target recognition results, overlaid with historical two-dimensional and three-dimensional target surrounding environment data. Figure 6 This is a schematic diagram showing the geographic extent of the latest imagery acquired by low-Earth orbit satellites, in which historical remote sensing data is used during the demonstration of on-board processing results. Figure 7 This is a schematic diagram of a local target area in a remote sensing image, where the rectangle shows the target texture effect; Figure 8 This diagram illustrates the construction of a 3D model of a target using the latest target recognition results. The rectangle within the diagram shows the simulation and overlay effect of the 3D target model.

[0111] In one possible implementation, the target multi-dimensional scene display unit includes a target type two-dimensional and three-dimensional model database, a global Internet remote sensing satellite base map model database, a global digital elevation model database, and a local area three-dimensional geographic scene model database.

[0112] This embodiment adds support for known target type 2D and 3D model libraries, global Internet remote sensing satellite base maps, global 30-meter resolution digital elevation models (DEMs), and local area 3D geographic scene models to the target multi-dimensional scene display unit. This allows for a more realistic display of target recognition content and surrounding environment on the PAD terminal, making it easier for users to understand the corresponding topological relationships between ground features more deeply.

[0113] The target 2D and 3D scene display unit (e.g., target multi-dimensional scene display unit) in this embodiment achieves ground simulation visualization based on the real-time target recognition results of spaceborne images rapidly transmitted to the ground. This unit calls upon existing 2D and 3D module library models, overlays global IoT maps and global DEM data, and relies on real-time dynamic simulation algorithms to provide visualization of the identified target on 2D and 3D maps, more intuitively displaying the real-time target status and surrounding geographical environment information that can be obtained by onboard real-time intelligent processing.

[0114] This embodiment utilizes known target type 2D and 3D model libraries, global Internet remote sensing satellite base maps, global DEMs, and local regional 3D geographic scene models to more realistically display satellite target identification content and surrounding environment on the PAD terminal, facilitating users to instantly and more deeply understand the corresponding topological relationships between ground features.

[0115] Another embodiment of the present invention provides a remote sensing satellite monitoring method, which includes using a ground terminal to encode and generate remote sensing monitoring task data based on user input data and send it to a satellite-to-ground communication module; decoding the semantic data of the recognition results sent by the satellite-to-ground communication module and calling multiple model databases to perform multi-dimensional display of at least one target's identification, location, and environment; using the satellite-to-ground communication module to interact with the remote sensing monitoring task data and the semantic data of the recognition results, and sending algorithm update data and attitude and orbit control signals to the onboard processing module and the onboard attitude and orbit control module, and receiving ground remote sensing image data sent by the onboard camera; using an onboard computer to decode the remote sensing monitoring task data and perform autonomous task planning based on satellite time data and satellite attitude and orbit data to generate remote sensing monitoring signals and target information. The system identifies targets and sends corresponding signals to the onboard camera and processing module. It also collects satellite operational status information and sends it to the ground terminal via the satellite-to-ground communication module to update the satellite status information. Using the onboard camera, it receives remote sensing monitoring signals and scans the monitored area when the satellite passes over it during the monitoring period to acquire ground remote sensing image data, which is then sent to the onboard processing module. The onboard processing module receives target identification signals, satellite attitude and orbit data, and ground remote sensing image data. It processes the ground remote sensing image data to obtain target identification result data and encodes the target identification result data to obtain semantic data, which is then sent to the satellite-to-ground communication module. The onboard attitude and orbit control module collects satellite attitude and orbit data and sends it to the onboard processing module and the onboard computer.

[0116] The embodiments of the present invention enable users to plan remote sensing monitoring tasks in a simpler, clearer, and easier-to-understand way, and to upload tasks more quickly and directly; to achieve rapid geographic positioning of targets; and to display the satellite-borne target identification content and surrounding environment more realistically, so as to facilitate users to understand the corresponding topological relationships between ground features more quickly and deeply.

[0117] In the description of this invention, it should be noted that the terms "upper," "lower," etc., indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Unless otherwise expressly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication between two elements. For those skilled in the art, the specific meaning of the above terms in this invention can be understood according to the specific circumstances.

[0118] It should also be noted that in the description of this invention, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0119] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. For those skilled in the art, other variations or modifications can be made based on the above description. It is impossible to exhaustively list all the implementation methods here. All obvious variations or modifications derived from the technical solutions of the present invention are still within the protection scope of the present invention.

Claims

1. A remote sensing satellite monitoring system, characterized in that, The system includes The ground terminal is used to encode and generate remote sensing monitoring task data based on user input data and send it to the satellite-ground communication module, as well as to decode the semantic data of the recognition results sent by the satellite-ground communication module and call multiple model databases to identify, locate and display the environment of at least one target. The satellite-to-ground communication module is used to exchange remote sensing monitoring task data and semantic data of recognition results, and to send algorithm update data and attitude and orbit control signals to the satellite processing module and the satellite attitude and orbit control module, as well as to receive ground remote sensing image data sent by the satellite camera. The satellite-to-ground communication module includes a ground user short message terminal and an on-board Beidou short message integrated machine. The ground user short message terminal is a ground user Beidou short message terminal. The onboard computer is used to decode remote sensing monitoring mission data and perform autonomous mission planning based on satellite time data and satellite attitude and orbit data to generate remote sensing monitoring signals and target recognition signals, which are then sent to the onboard camera and onboard processing module. It also collects satellite operating status information and sends it to the ground terminal via the satellite-to-ground communication module to update the satellite status information. The spaceborne camera is used to receive remote sensing monitoring signals and scan the ground remote sensing image data when the satellite passes over the area to be monitored during the monitoring time, and then send the ground remote sensing image data to the spaceborne processing module. The onboard processing module receives target recognition signals, satellite attitude and orbit data, and ground remote sensing image data. It processes the ground remote sensing image data to obtain target recognition result data, encodes the target recognition result data to obtain semantic data, and sends it to the satellite-to-ground communication module. The onboard processing module includes a target recognition and positioning unit. This unit performs radiometric correction, land-sea identification, image slicing, target recognition, and target positioning processing on the ground remote sensing image data to obtain target recognition result data. The onboard attitude and orbit control module is used to collect satellite attitude and orbit data and send it to the onboard processing module and the onboard computer.

2. The remote sensing satellite monitoring system according to claim 1, characterized in that, The satellite-to-ground communication module includes The ground communication module is used to receive remote sensing monitoring mission data and send it to the spaceborne communication module, and to receive the semantic data of the recognition results sent by the spaceborne communication module and send it to the ground terminal. The spaceborne communication module is used to receive remote sensing monitoring mission data and send it to the spaceborne computer, and to receive the semantic data of the recognition results sent by the spaceborne processing module and send it to the ground communication module. The ground station is used to send algorithm update data from the onboard processing module to the onboard processing module, send satellite attitude and orbit control signals to the onboard attitude and orbit control module, and receive ground remote sensing image data sent by the onboard camera.

3. The remote sensing satellite monitoring system according to claim 2, characterized in that, The ground terminal includes a remote sensing monitoring task autonomous planning unit; The remote sensing monitoring task autonomous planning unit is used to generate remote sensing monitoring tasks in response to the remote sensing monitoring area, monitoring time and target type set by the user, and to encode the remote sensing monitoring tasks to generate remote sensing monitoring task data.

4. The remote sensing satellite monitoring system according to claim 2, characterized in that, The ground station includes a satellite-borne processing algorithm uploading unit; The satellite-borne processing algorithm uploading unit is used to process ground remote sensing image data to obtain simulated target recognition result data, and to determine whether the simulated target recognition result data and the recognition result semantic data are consistent. If there is a discrepancy, the algorithm of the onboard processing module stored in the onboard processing algorithm uploading unit is debugged and sent to the onboard processing module.

5. The remote sensing satellite monitoring system according to claim 4, characterized in that, The ground terminal also includes a target multi-dimensional scene display unit; The target multi-dimensional scene display unit is used to visualize the target multi-dimensional map based on image simulation and dynamic simulation algorithms and multiple model databases and target recognition result data.

6. The remote sensing satellite monitoring system according to claim 5, characterized in that, The target identification result data includes target type, confidence level, center point longitude, center point latitude, length, width, orientation, and static / dynamic status.

7. The remote sensing satellite monitoring system according to claim 6, characterized in that, The target multi-dimensional scene display unit includes a target type two-dimensional and three-dimensional model database, a global Internet remote sensing satellite base map model database, a global digital elevation model database, and a local area three-dimensional geographic scene model database.

8. The remote sensing satellite monitoring system according to claim 7, characterized in that, The target recognition and localization unit performs target recognition and target localization based on deep learning.

9. A remote sensing satellite monitoring method, characterized in that, The method includes Using a ground terminal, remote sensing monitoring task data is generated by encoding user input data and sent to the satellite-to-ground communication module. Semantic data of the recognition results sent by the satellite-to-ground communication module is decoded and multiple model databases are called to identify, locate, and display the environment of at least one target. Using the space-to-ground communication module, remote sensing monitoring task data and semantic data of recognition results are exchanged, and algorithm update data and attitude and orbit control signals are sent to the onboard processing module and the onboard attitude and orbit control module, and ground remote sensing image data sent by the onboard camera are received. The space-to-ground communication module includes a ground user short message terminal and an onboard Beidou short message integrated machine. The ground user short message terminal is a ground user Beidou short message terminal. Using the onboard computer, remote sensing monitoring mission data is decoded, and remote sensing monitoring signals and target identification signals are generated autonomously based on satellite time data and satellite attitude and orbit data. These signals are then sent to the onboard camera and onboard processing module, and satellite operation status information is collected and sent to the ground terminal via the satellite-to-ground communication module to update the satellite status information. Using a satellite-borne camera, remote sensing monitoring signals are received, and when the satellite passes over the area to be monitored during the monitoring period, ground remote sensing image data is scanned and acquired, and the ground remote sensing image data is sent to the satellite-borne processing module. The onboard processing module receives target recognition signals, satellite attitude and orbit data, and ground remote sensing image data. It processes the ground remote sensing image data to obtain target recognition result data, encodes the target recognition result data to obtain semantic data, and sends it to the satellite-to-ground communication module. The onboard processing module includes a target recognition and positioning unit. This unit performs radiometric correction, land-sea identification, image slicing, target recognition, and target positioning processing on the ground remote sensing image data to obtain the target recognition result data. The satellite attitude and orbit control module is used to collect satellite attitude and orbit data and send it to the satellite processing module and the satellite computer.

Images