Distributed remote sensing method, device and storage medium based on satellite Internet

Abstract

The present application discloses a distributed remote sensing method, device and storage medium based on satellite Internet, including: receiving a remote sensing image acquisition request corresponding to a target area sent by a user terminal through satellite Internet, and uploading a remote sensing image acquisition task to a first remote sensing satellite through satellite Internet and intersatellite links; respectively collecting remote sensing images corresponding to the target area; based on the remote sensing images, generating a spatial confidence map corresponding to a first feature map, and determining a target feature image block in the first feature map; receiving target feature image blocks sent by each second remote sensing satellite, splicing and fusing multiple target feature image blocks corresponding to each second remote sensing satellite and the first feature map to generate a second feature map; transmitting the second feature map to the user terminal in real time through satellite Internet and intersatellite links. The technical effect of improving the satellite's ability to distribute remote sensing information and the distributed computing ability of remote sensing tasks is achieved.

Description

Technical Field

[0001] The present application relates to the field of satellite Internet technology, and in particular to a distributed remote sensing method, device and storage medium based on satellite Internet. Background Art

[0002] With the continuous development of satellite technology, satellites are widely used in economic construction, disaster prevention and mitigation, environmental monitoring and other fields. However, due to the limited number of satellites and the limited layout of ground stations, the timeliness of satellite services is greatly restricted. Among them, the above-mentioned satellite services include satellite remote sensing, satellite communications and satellite computing.

[0003] For example, at the satellite remote sensing level: due to the limitations of satellite data transmission capacity and the layout of gateway stations, user shooting needs cannot be immediately reported to remote sensing satellites, and satellite images cannot be obtained immediately. The information acquisition chain is complex and long, which restricts the ability to distribute remote sensing information. When encountering temporary or emergency observation needs, the conflict of network resources is more obvious.

[0004] In terms of satellite communications: real-time remote sensing missions generate massive amounts of earth observation data, and time-sensitive remote sensing users require high-speed inter-satellite and satellite-to-ground communications to complete the efficient transmission of remote sensing information. However, inter-satellite laser communications, especially inter-orbital inter-satellite laser communication terminals, are difficult to effectively guarantee long-term stable high-speed transmission of massive data. Therefore, how to reduce the demand for communication in real-time remote sensing missions is the key to the success of low-orbit integrated remote sensing constellations.

[0005] In terms of satellite computing: With the increase in optical remote sensing payload capacity, the video and image data throughput rate generated by remote sensing satellites is 1~10Gbps, while low-orbit satellites with limited size, weight, and power consumption have weak computing capabilities. If all data is transmitted to ground cloud servers, there will be a large delay, which is difficult to meet the real-time requirements of delay-sensitive tasks in strong confrontation scenarios. In addition, since my country's earth stations and data transmission stations cannot achieve global coverage, how to use limited computing resources to provide efficient computing services is an important technical guarantee for real-time remote sensing tasks.

[0006] The publication number is CN117856876A, and the name is a high-orbit and low-orbit inter-satellite distributed cooperative communication system and method. The system includes: a high-orbit satellite; a low-orbit communication unit, installed on a high-orbit satellite, used to send synchronization signals, cooperative relay timing tables and phase synchronization completion identification information to the high-orbit communication unit, and receive data transmission requests, test signals and data to be transmitted sent by the high-orbit communication unit; a cooperative relay calculation unit, installed on a high-orbit satellite, used to calculate the cooperative relay timing table; multiple low-orbit satellites; a high-orbit communication unit, installed on a low-orbit satellite, used to send data transmission requests, perform signal synchronization tests, send test signals, and send data to be transmitted; an inter-satellite interconnection unit, installed on a low-orbit satellite, used to transmit data between low-orbit satellites.

[0007] The publication number is CN113644957A, and the name is a space-based information relay transmission method for satellite internet networks, comprising: when it is determined that the user satellite is unable to execute the task request, based on the idle resources of the relay satellite released by the satellite internet network operation and maintenance control center, a relay link application is submitted to the satellite internet network operation and maintenance control center to determine the available relay transmission link; the user satellite control center tracks the user satellite that needs to perform the relay transmission task with the high-orbit satellite in the relay transmission link through control instructions, and executes the relay transmission task after completing the tracking; after the high-orbit satellite receives the remote sensing data sent by the user satellite, it sends the remote sensing data to the ground station to complete the data landing of the user satellite.

[0008] With regard to the technical problems existing in the above-mentioned prior art, due to the limitations of satellite data transmission capacity and the constraints of the layout of gateway stations, users' shooting needs cannot be immediately reported to remote sensing satellites, satellite images cannot be immediately acquired, and the information acquisition chain is complicated and lengthy, which restricts the distribution capacity of remote sensing information and the distributed computing capacity of remote sensing tasks. No effective solution has been proposed so far. Summary of the invention

[0009] The embodiments of the present disclosure provide a distributed remote sensing method, device and storage medium based on satellite Internet, so as to at least solve the technical problems existing in the prior art that are limited by the satellite data transmission capacity and the constraints of the layout of gateway stations, the user's shooting needs cannot be immediately reported to the remote sensing satellite, the satellite images cannot be immediately acquired, and the information acquisition chain is complicated and lengthy, which restricts the distribution capability of remote sensing information and the distributed computing capability of remote sensing tasks.

[0010] According to one aspect of an embodiment of the present disclosure, a distributed remote sensing method based on satellite Internet is provided, including: a gateway station receives a remote sensing image acquisition request for a target area sent by a user terminal through the satellite Internet, and uploads a remote sensing image acquisition task to a first remote sensing satellite through the satellite Internet and an intersatellite link, wherein the remote sensing image acquisition task corresponds to the remote sensing image acquisition request; the first remote sensing satellite broadcasts the remote sensing image acquisition task to multiple second remote sensing satellites, and the first remote sensing satellite and the multiple second remote sensing satellites respectively collect remote sensing images corresponding to the target area; the first remote sensing satellite and each second remote sensing satellite generate a remote sensing image corresponding to the first feature map based on the remote sensing image The first remote sensing satellite receives the target feature image blocks sent by each second remote sensing satellite, and uses the multiple target feature image blocks corresponding to each second remote sensing satellite and its own first feature map for splicing and fusion, so as to generate a second feature map corresponding to the target area; and the first remote sensing satellite transmits the second feature map corresponding to the remote sensing image to the user terminal in real time through the satellite Internet and the intersatellite link.

[0011] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is running, a processor executes any one of the methods described above.

[0012] According to another aspect of the embodiment of the present disclosure, a distributed remote sensing device based on satellite Internet is also provided, including: an acquisition request sending module, which is used for the gateway station to receive the remote sensing image acquisition request for the target area sent by the user terminal through the satellite Internet, and to upload the remote sensing image acquisition task to the first remote sensing satellite through the satellite Internet and the inter-satellite link, wherein the remote sensing image acquisition task corresponds to the remote sensing image acquisition request; a remote sensing image acquisition module, which is used for the first remote sensing satellite to broadcast the remote sensing image acquisition task to multiple second remote sensing satellites, and the first remote sensing satellite and the multiple second remote sensing satellites respectively acquire remote sensing images corresponding to the target area; a first feature map generation module, which is used for the first remote sensing satellite and each second remote sensing satellite to generate, based on the remote sensing image, a remote sensing image acquisition task corresponding to the remote sensing image acquisition request; a remote sensing image acquisition module, which is used for the first remote sensing satellite to broadcast the remote sensing image acquisition task to multiple second remote sensing satellites, and the first remote sensing satellite and the multiple second remote sensing satellites respectively acquire remote sensing images corresponding to the target area; and a first feature map generation module, which is used for the first remote sensing satellite and each second remote sensing satellite to generate, based on the remote sensing image, a remote sensing image acquisition task corresponding to the target area. A spatial confidence map corresponding to the first feature map, and determining a target feature image block in the first feature map, wherein the spatial confidence map includes confidence scores corresponding to each feature image block in the first feature map, and the target feature image block is used to indicate a feature image block whose confidence score is greater than a preset threshold; a second feature map generation module, used for the first remote sensing satellite to receive the target feature image blocks sent by each second remote sensing satellite, and to splice and fuse multiple target feature image blocks corresponding to each second remote sensing satellite and its own first feature map, so as to generate a second feature map corresponding to the target area; and a feature map transmission module, used for the first remote sensing satellite to transmit the second feature map corresponding to the remote sensing image to the user terminal in real time through the satellite Internet and the intersatellite link.

[0013] According to another aspect of the embodiment of the present disclosure, there is also provided a distributed remote sensing device based on satellite Internet, including: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: a gateway station receives a remote sensing image acquisition request for a target area sent by a user terminal through satellite Internet, and uploads a remote sensing image acquisition task to a first remote sensing satellite through satellite Internet and an intersatellite link, wherein the remote sensing image acquisition task corresponds to the remote sensing image acquisition request; the first remote sensing satellite broadcasts the remote sensing image acquisition task to multiple second remote sensing satellites, and the first remote sensing satellite and the multiple second remote sensing satellites respectively collect remote sensing images corresponding to the target area; the first remote sensing satellite and each second remote sensing satellite The satellite generates spatial confidence maps corresponding to the first feature map based on the remote sensing image, and determines the target feature image block in the first feature map, wherein the spatial confidence map includes confidence scores corresponding to each feature image block in the first feature map, and the target feature image block is used to indicate the feature image block whose confidence score is greater than a preset threshold; the first remote sensing satellite receives the target feature image blocks sent by each second remote sensing satellite, and uses multiple target feature image blocks corresponding to each second remote sensing satellite and its own first feature map for splicing and fusion, so as to generate a second feature map corresponding to the target area; and the first remote sensing satellite transmits the second feature map corresponding to the remote sensing image to the user terminal in real time through the satellite Internet and the inter-satellite link.

[0014] The present application provides a distributed remote sensing method based on satellite Internet. First, the gateway receives a remote sensing image acquisition request for a target area sent by a user terminal through satellite Internet, and uploads the remote sensing image acquisition task to the first remote sensing satellite through satellite Internet and intersatellite link. Then, the first remote sensing satellite broadcasts the remote sensing image acquisition task to multiple second remote sensing satellites, and the first remote sensing satellite and multiple second remote sensing satellites respectively collect remote sensing images corresponding to the target area. Further, the first remote sensing satellite and each second remote sensing satellite generate a spatial confidence map corresponding to the first feature map based on the remote sensing image, and determine the target feature image block in the first feature map. After that, the first remote sensing satellite receives the target remote sensing image block sent by each second remote sensing satellite, and uses the multiple target feature image blocks corresponding to each second remote sensing satellite and its own first feature map for splicing and fusion, thereby generating a second feature map corresponding to the target area. Finally, the first remote sensing satellite transmits the second feature map corresponding to the remote sensing image to the user terminal in real time through satellite Internet and intersatellite link.

[0015] With reference to the above contents, it can be seen that since the present application directly utilizes the signal gateway and uploads the remote sensing image acquisition request to the first remote sensing satellite through satellite Internet and intersatellite links, compared with the traditional user terminal first sending the remote sensing image acquisition request to the operation and control system, the operation and control system sends the remote sensing image acquisition request to the measurement and control system, and the measurement and control system then uploads the satellite control command to the first remote sensing satellite, it can simplify the information transmission process, improve the efficiency of information flow, get rid of the limitations of the ground station network, and reduce the difficulty of coordinating ground station network resources.

[0016] In addition, referring to the above content, it can be seen that the present application not only uses one remote sensing satellite to obtain the remote sensing image corresponding to the target area, but also uses multiple remote sensing satellites (i.e., including a first remote sensing satellite and multiple second remote sensing satellites) to jointly obtain the target feature image blocks corresponding to the target area, and finally uses the first remote sensing satellite to splice and fuse the target feature image blocks corresponding to each second remote sensing satellite and its own first feature map, so as to generate a second feature map corresponding to the target area (corresponding to the remote sensing image of the target area), thereby ensuring that the overall picture of the target area is fully captured and satisfying the remote sensing image acquisition request of the user terminal.

[0017] The above operation achieves the technical effect of being able to instantly upload the user terminal's shooting needs to the remote sensing satellite, improving the efficiency of information transmission, ensuring the completion of the shooting of remote sensing images of the target area, and further improving the satellite's distribution capability for remote sensing information and the distributed computing capability of remote sensing tasks.

[0018] This solves the technical problems existing in the prior art, such as the limitation of satellite data transmission capacity and the constraints of gateway station layout, the inability to immediately upload user shooting needs to remote sensing satellites, the inability to immediately obtain satellite images, and the complex and lengthy information acquisition chain, which restricts the distribution capacity of remote sensing information and the distributed computing capacity of remote sensing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0020] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to Embodiment 1 of the present application;

[0021] Figure 2A is a schematic diagram of a distributed remote sensing system based on satellite Internet according to Example 1 of the present application;

[0022] Figure 2Bis a schematic diagram of another distributed remote sensing system based on satellite Internet according to Example 1 of the present application;

[0023] Figure 3A According to the embodiment 1 of the present application Figure 2A A corresponding modular schematic diagram of the first remote sensing satellite;

[0024] Figure 3B According to the embodiment 1 of the present application Figure 2B The corresponding modular schematic diagram of the gateway station;

[0025] Figure 4 is a flow chart of the distributed remote sensing method based on satellite Internet according to Example 1 of the present application;

[0026] Figure 5 is a schematic diagram of a distributed remote sensing device based on satellite Internet according to Example 2 of the present application;

[0027] Figure 6 This is a schematic diagram of a distributed remote sensing device based on satellite Internet as described in Example 3 of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present disclosure.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0030] Example 1

[0031] According to this embodiment, a method embodiment of distributed remote sensing based on satellite Internet is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] The method embodiment provided in this embodiment can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computing device for implementing a distributed remote sensing method based on satellite Internet. Figure 1 As shown, the computing device may include one or more processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, the transmission device, and the input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0033] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computing device. As involved in the embodiments of the present disclosure, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0034] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the distributed remote sensing method based on satellite Internet in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the distributed remote sensing method based on satellite Internet of the above-mentioned application program is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computing device. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of the computing device.

[0037] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing devices described above.

[0038] Figure 2A is a schematic diagram of a distributed remote sensing system based on satellite Internet according to this embodiment. Figure 2AAs shown, the system includes: a user terminal 10, a remote sensing application system 20, a gateway station 30 and a plurality of remote sensing satellites. The plurality of remote sensing satellites include, for example, a first remote sensing satellite 40, a plurality of second remote sensing satellites 501~50m and a relay satellite 60. And the first remote sensing satellite 40 is a low-orbit satellite, and the low-orbit satellite is determined as the master satellite, so that remote sensing task allocation and information fusion operations are completed on the first remote sensing satellite 40. In addition, in this embodiment, the plurality of second remote sensing satellites 501~50m are determined as slave satellites, and the plurality of second remote sensing satellites 501~50m are used to collect remote sensing images corresponding to the target area and generate target feature image blocks. It is worth noting that Figure 2A The first remote sensing satellite 40, the plurality of second remote sensing satellites 501-50m and the relay satellite 60 are a temporarily formed satellite group, which is mainly used for multi-satellite coordinated observation tasks. Before receiving the coordinated observation task, the first remote sensing satellite 40, the plurality of second remote sensing satellites 501-50m and the relay satellite 60 are used to perform other tasks.

[0039] Of course, the present application only exemplarily lists the first remote sensing satellite 40, multiple second remote sensing satellites 501~50m and the relay satellite 60. Those skilled in the art need to understand that other collaborative observation tasks may correspond to other temporarily formed satellite groups, which are not specifically limited here.

[0040] Further, refer to Figure 2A As shown, the user terminal 10 is connected to the gateway station 30 through the remote sensing application system 20, and the gateway station 30 is connected to the first remote sensing satellite 40. Therefore, the user terminal 10 can upload the remote sensing image acquisition request to the first remote sensing satellite 40 through the remote sensing application system 20 and the gateway station 30. In addition, the user terminal 10 can also directly access the satellite Internet through the low-orbit satellite terminal, and directly upload the remote sensing image acquisition request to the first remote sensing satellite 40.

[0041] When the first remote sensing satellite 40 receives the remote sensing image acquisition task, it plans and makes decisions on the remote sensing task based on the remote sensing image acquisition task and the orbital position information of each satellite in the low-orbit constellation, and directly transmits the remote sensing image acquisition task to multiple second remote sensing satellites 501~50m through inter-satellite links or transmits the remote sensing image acquisition task to multiple second remote sensing satellites 501~50m through relay satellites 60, so that the first remote sensing satellite 40 and the multiple second remote sensing satellites 501~50m jointly perform the distributed shooting task.

[0042] When the first remote sensing satellite 40 and the plurality of second remote sensing satellites 501-50m have finished shooting, the first remote sensing satellite 40 and the plurality of second remote sensing satellites 501-50m perform image processing on the collected remote sensing images (including generating a spatial confidence map, information packaging, and constructing a communication map). Afterwards, the plurality of second remote sensing satellites 501-50m send the target feature image blocks to the first remote sensing satellite 40 (the transmission path includes direct transmission and transmission through the relay satellite 60). Finally, the first remote sensing satellite 40 performs information fusion and generates a second feature map with complete features.

[0043] Figure 2B is a schematic diagram of another distributed remote sensing system based on satellite Internet according to an embodiment of the present application. Figure 2B As shown, the system includes: a user terminal 10, a remote sensing application system 20, a gateway station 30 and a plurality of remote sensing satellites. The plurality of remote sensing satellites include, for example, a plurality of second remote sensing satellites 501-50m. In this case, remote sensing task allocation and information fusion operations are all completed on the cloud computing center 70 of the gateway station 30. There is no distinction between master and slave satellites in the low-orbit constellation, and all are distributed remote sensing nodes.

[0044] It is worth noting that Figure 2B The plurality of second remote sensing satellites 501-50m in the satellite group are temporarily formed and are mainly used for multi-satellite coordinated observation tasks. Before receiving the coordinated observation task, the plurality of second remote sensing satellites 501-50m are used to perform other tasks.

[0045] Of course, the present application only lists multiple second remote sensing satellites 501~50m for example. Those skilled in the art need to understand that other collaborative observation tasks may correspond to other temporarily formed satellite groups, which is not specifically limited here.

[0046] Further, refer to Figure 2B As shown, the user terminal 10 is connected to the gateway station 30 through the remote sensing application system 20, and the gateway station 30 is connected to the multiple second remote sensing satellites 501-50m. Therefore, the user terminal 10 can send the remote sensing image acquisition task to the gateway station 30 through the remote sensing application system 20. The cloud computing center 70 in the gateway station 30 analyzes and plans the remote sensing image acquisition task, and then directly injects the remote sensing image acquisition task to the multiple second remote sensing satellites 501-50m through the feeder link or sends the remote sensing image acquisition task to the multiple second remote sensing satellites 501-50m through the intersatellite link, so that the multiple second remote sensing satellites 501-50m perform the distributed shooting task.

[0047] In addition, the user terminal 10 can also directly access the satellite Internet through the low-orbit satellite terminal, and directly upload the remote sensing image acquisition request to any second remote sensing satellite among the plurality of second remote sensing satellites 501-50m. Any second remote sensing satellite among the plurality of second remote sensing satellites transmits the remote sensing image acquisition request to the gateway station 30 via the intersatellite link and the feeder circuit. The cloud computing center 70 of the gateway station 30 completes the analysis and planning of the remote sensing image acquisition task, and then directly uploads the remote sensing image acquisition task corresponding to the remote sensing image acquisition request to each second remote sensing satellite 501-50m through the feeder link or through the intersatellite link.

[0048] It is worth noting that, when the gateway station 30 receives the remote sensing images sent by each of the second remote sensing satellites 501-50m, the most complete and clear first feature map can be determined based on the ephemeris information and orbital position information of each of the second remote sensing satellites 501-50m. For example, the remote sensing image collected by the second remote sensing satellite that is close to the target area and has no obvious occlusion factors during shooting is the first feature map.

[0049] After the shooting is completed, each second remote sensing satellite 501-50m transmits the collected remote sensing image to the gateway station 30 via the intersatellite link and the feeder link. After the gateway station 30 completes the image processing (ie, generating the spatial confidence map) and the information fusion, the second feature map is generated.

[0050] Figure 3A According to the embodiment of the present application Figure 2A The corresponding modular schematic diagram of the first remote sensing satellite. Figure 3A As shown, in the case where the first remote sensing satellite 40 is the main satellite, the first remote sensing satellite 40 includes: a satellite-to-ground communication module, a mission broadcast module, an inter-satellite communication module, a remote sensing image shooting module, a spatial confidence map generation module, an information packaging module, a communication map construction module and a feature map fusion module.

[0051] The satellite-to-ground communication module is connected to the user terminal 10 through the gateway 30 and the remote sensing application system 20 to receive the remote sensing image acquisition task. The task broadcast module is connected to the inter-satellite communication module, and the inter-satellite communication module is connected to each second remote sensing satellite 501-50m to broadcast the remote sensing image acquisition task to multiple second remote sensing satellites 501-50m.

[0052] The remote sensing image shooting module is connected in communication with the satellite-to-ground communication module, and is used to collect remote sensing images corresponding to the target area in response to the remote sensing image acquisition task. The spatial confidence map generation module is connected in communication with the remote sensing image shooting module, and is used to generate the corresponding first feature map and spatial confidence map based on the remote sensing image, and determine the target feature image block in the first feature map based on the spatial confidence map. The information packaging module is connected in communication with the spatial confidence map generation module and the intersatellite communication module, and is used to generate the first data packet and the second data packet based on the target feature image block and the request map sent by each second remote sensing satellite 501~50m. The communication map construction module is connected in communication with the information packaging module and the intersatellite communication module, and is used to construct a communication map and transmit the communication map to the intersatellite communication module.

[0053] The feature map fusion module is connected to the intersatellite communication module for receiving target feature image blocks corresponding to each second remote sensing satellite 501-50m through the intersatellite communication module. The feature map fusion module is also connected to the spatial confidence map generation module and the satellite-to-ground communication module for fusing the first feature map and the target feature image blocks corresponding to each second remote sensing satellite 501-50m using a multi-head attention mechanism, and transmitting the generated second feature map to the user terminal 10.

[0054] Figure 3B According to the embodiment of the present application Figure 2B The corresponding modular diagram of the gateway station. Figure 3B As shown, the gateway station 30 includes a gateway station communication module, a mission planning module, a satellite-to-ground communication module, a remote sensing image receiving module, a spatial confidence map generation module, a decision module, and a feature map fusion module.

[0055] The gateway communication module is connected to the user terminal 10 through the remote sensing application system 20 to receive the remote sensing image acquisition request. The task planning module is connected to the gateway communication module to generate the corresponding remote sensing image acquisition task and determine each second remote sensing satellite 501-50m based on the remote sensing image acquisition request and the satellite ephemeris information. The satellite-to-ground communication module is connected to the task planning module to annotate the remote sensing image acquisition task to each second remote sensing satellite 501-50m.

[0056] The remote sensing image receiving module is connected to the satellite-to-ground communication module for receiving remote sensing images collected by each second remote sensing satellite 501-50m. The spatial confidence map generation module is connected to the remote sensing image receiving module for generating a spatial confidence map and a first feature map corresponding to each second remote sensing satellite 501-50m, and determining the target feature image block in the first feature map. The decision module is connected to the spatial confidence map generation module for determining the first feature map for fusion when the spatial confidence map generation module generates the first feature map corresponding to each second remote sensing satellite 501-50m. The first feature map for fusion has more complete and clear features than other first feature maps.

[0057] The feature map fusion module is connected in communication with the spatial confidence map generation module, and is used to receive the first feature map for fusion and the target feature image blocks corresponding to each first feature map. The feature map fusion module is also connected in communication with the gateway communication module, and is used to fuse the first feature map for fusion and the target feature image blocks corresponding to each second remote sensing satellite 501-50m using a multi-head attention mechanism, and transmit the generated second feature map to the user terminal 10.

[0058] It should be noted that the user terminal 10, remote sensing application system 20, gateway station 30, first remote sensing satellite 40, multiple second remote sensing satellites 501~50m, relay satellite 60 and cloud computing center 70 of the gateway station 30 in the system can all be applied to the hardware structure described above.

[0059] Under the above operating environment, according to the first aspect of this embodiment, a distributed remote sensing method based on satellite Internet is provided, which comprises Figure 2A However, it should be clear to those skilled in the art that this embodiment is only based on Figure 2A The operations performed by each implementation entity in the Figure 2B The operations performed by the various implementation entities in can also implement this method, and the same operation examples will not be repeated again. Figure 4 A schematic diagram showing the process of the method is shown in FIG. Figure 4 As shown, the method includes:

[0060] S402: The gateway receives a remote sensing image acquisition request for a target area sent by a user terminal through the satellite Internet, and uploads a remote sensing image acquisition task to a first remote sensing satellite through the satellite Internet and an intersatellite link, wherein the remote sensing image acquisition task corresponds to the remote sensing image acquisition request;

[0061] S404: the first remote sensing satellite broadcasts the remote sensing image acquisition task to the plurality of second remote sensing satellites, and the first remote sensing satellite and the plurality of second remote sensing satellites respectively collect remote sensing images corresponding to the target area;

[0062] S406: The first remote sensing satellite and each second remote sensing satellite respectively generate a spatial confidence map corresponding to the first feature map based on the remote sensing image, and determine a target feature image block in the first feature map, wherein the spatial confidence map includes confidence scores corresponding to each feature image block in the first feature map, and the target feature image block is used to indicate a feature image block whose confidence score is greater than a preset threshold;

[0063] S408: The first remote sensing satellite receives the target feature image blocks sent by each second remote sensing satellite, and uses a plurality of target feature image blocks corresponding to each second remote sensing satellite and its own first feature map to perform splicing and fusion, thereby generating a second feature map corresponding to the target area; and

[0064] S410: The first remote sensing satellite transmits the second feature map corresponding to the remote sensing image to the user terminal in real time through the gateway station and based on the satellite Internet and the inter-satellite link.

[0065] Specifically, refer to Figure 2A As shown, first, the user terminal 10 sends a remote sensing image acquisition request for the target area to the gateway station 30 through the remote sensing application system 20. Thus, the gateway station 30 responds to the remote sensing image acquisition request for the target area sent by the user terminal 10, and uploads the remote sensing image acquisition task to the first remote sensing satellite 40 through the satellite Internet and the intersatellite link (S402). Among them, the remote sensing image acquisition task corresponds to the remote sensing image acquisition request. It is worth noting that the gateway station 30 in this embodiment, for example, includes a measurement and control system and an operation and control system, so that the user terminal 10 only needs to communicate with the gateway station 30 through the satellite Internet, and the remote sensing image acquisition request can be uploaded to the gateway station 30 through the remote sensing application system 20 in real time, and the first remote sensing satellite 40 can receive the remote sensing image acquisition task in real time.

[0066] Therefore, unlike the traditional method in which the user terminal 10 first sends the remote sensing image acquisition request to the operation and control system, and the operation and control system then sends the satellite control instructions and the remote sensing image acquisition task to the measurement and control system, and finally the measurement and control system uploads the remote sensing image acquisition task to the first remote sensing satellite 40, the present application constructs a communication-remote sensing-computing integrated architecture system based on satellite Internet. The user terminal 10, the signal gateway station 30 and the first remote sensing satellite 40 are all seamlessly connected to the satellite Internet, the information transmission process is streamlined, and user services are more efficient.

[0067] When the first remote sensing satellite receives the remote sensing image acquisition task sent by the gateway, the first remote sensing satellite broadcasts the remote sensing image acquisition task to the plurality of second remote sensing satellites, and the first remote sensing satellite and the plurality of second remote sensing satellites respectively collect remote sensing images corresponding to the target area (S404).

[0068] Specifically, on the one hand, due to the constraints of factors such as the transit time and the field of view of the remote sensing camera, the detection range of the first remote sensing satellite 40 is limited. Therefore, for a relatively large target area, the first remote sensing satellite 40 cannot complete the comprehensive shooting of the target area within one orbital cycle. On the other hand, since the first remote sensing satellite 40 is affected by cloud cover and other conditions during its transit, it is impossible to capture the entire target area using only the first remote sensing satellite 40. Therefore, in order to solve the above two problems, it is necessary to coordinate the first remote sensing satellite 40 and multiple second remote sensing satellites 501~50m to complete the remote sensing detection of the target area, thereby ensuring that the complete remote sensing image of the target area can be collected.

[0069] When the first remote sensing satellite and each second remote sensing satellite respectively collect remote sensing images for the target area, a spatial confidence map corresponding to the first feature map is generated respectively, and a target feature image block in the first feature map is determined (S406). The spatial confidence map includes confidence scores corresponding to each feature image block in the first feature map, and the target feature image block is used to indicate a feature image block whose confidence score is greater than a preset threshold.

[0070] Specifically, when the first remote sensing satellite 40 and each of the second remote sensing satellites 501~50m collect remote sensing images, the remote sensing images are encoded and decoded, and a first feature map and a spatial confidence map corresponding to the first feature map are generated. Among them, the spatial confidence map reflects the importance of each feature image block in the first feature map, and the confidence score of the feature image block with clear features is higher, and the confidence score of the feature image block with blurred or missing features is lower. Therefore, the first remote sensing satellite 40 and each of the second remote sensing satellites 501~50m can use the feature image block with a confidence score greater than a preset threshold as the target feature image block, and each of the second remote sensing satellites 501~50m sends the target feature image block to the first remote sensing satellite 40. The above content will be described in detail later, so it will not be repeated here.

[0071] Therefore, since each second remote sensing satellite 501~50m in the present application does not send the collected remote sensing images directly to the first remote sensing satellite 40, but pre-determines the target feature image blocks with clearer features and sends the target feature image blocks to the first remote sensing satellite 40, it is possible to help the first remote sensing satellite 40 restore the full-scale information that is missing due to limited viewing angle, self-occlusion of the target area, and space illumination, thereby further achieving the technical effect of saving communication resources of the inter-satellite link and improving data transmission efficiency.

[0072] Further, when the first remote sensing satellite receives the target feature image blocks sent by each second remote sensing satellite, it uses the multiple target feature image blocks corresponding to each second remote sensing satellite and its own first feature map to perform splicing and fusion, thereby generating a second feature map corresponding to the target area (S408). Specifically, when the first remote sensing satellite 40 receives the target feature image blocks sent by each second remote sensing satellite 501~50m, it uses a multi-head attention mechanism to splice and fuse the target feature image blocks sent by each second remote sensing satellite 501~50m and its own first feature map, thereby obtaining an enhanced second feature map. The above content will be described in detail later, so it will not be repeated here.

[0073] Finally, when the first remote sensing satellite generates a second characteristic map corresponding to the target area, the second characteristic map is transmitted to the user terminal in real time through the gateway based on satellite Internet and intersatellite link (S410).

[0074] As described in the background technology, for satellite remote sensing, due to the limitation of satellite data transmission capacity and the layout of gateway stations, users’ shooting needs cannot be immediately reported to remote sensing satellites, and satellite images cannot be obtained immediately. The information acquisition chain is complex and long, which restricts the ability to distribute remote sensing information. When encountering temporary or emergency observation needs, the conflict of network resources is more obvious.

[0075] In terms of satellite communications: real-time remote sensing missions generate massive amounts of earth observation data, and time-sensitive remote sensing users require high-speed inter-satellite and satellite-to-ground communications to complete the efficient transmission of remote sensing information. However, inter-satellite laser communications, especially inter-orbital inter-satellite laser communication terminals, are difficult to effectively guarantee long-term stable high-speed transmission of massive data. Therefore, how to reduce the demand for communication in real-time remote sensing missions is the key to the success of low-orbit integrated remote sensing constellations.

[0076] In terms of satellite computing: With the increase in optical remote sensing payload capacity, the video and image data throughput rate generated by remote sensing satellites is 1~10Gbps, while low-orbit satellites with limited size, weight, and power consumption have weak computing capabilities. If all data is transmitted to ground cloud servers, there will be a large delay, which is difficult to meet the real-time requirements of delay-sensitive tasks in strong confrontation scenarios. In addition, since my country's earth stations and data transmission stations cannot achieve global coverage, how to use limited computing resources to provide efficient computing services is an important technical guarantee for real-time remote sensing tasks.

[0077] In view of this, the present application provides a distributed remote sensing method based on satellite Internet. And referring to the above-mentioned contents, it can be known that since the present application directly uses the signal gateway and injects the remote sensing image acquisition request to the first remote sensing satellite through the satellite Internet and the intersatellite link, compared with the traditional user terminal first sending the remote sensing image acquisition request to the operation and control system, the operation and control system sends the remote sensing image acquisition request to the measurement and control system, and the measurement and control system then injects the satellite control instruction to the first remote sensing satellite, it can simplify the information transmission process, improve the efficiency of information flow, get rid of the limitations of the ground station network, and reduce the difficulty of ground station network resource coordination.

[0078] In addition, referring to the above content, it can be seen that the present application not only uses one remote sensing satellite to obtain the remote sensing image corresponding to the target area, but also uses multiple remote sensing satellites (i.e., including a first remote sensing satellite and multiple second remote sensing satellites) to jointly obtain the target feature image blocks corresponding to the target area, and finally uses the first remote sensing satellite to splice and fuse the target feature image blocks corresponding to each second remote sensing satellite and its own first feature map, so as to generate a second feature map corresponding to the target area (corresponding to the remote sensing image of the target area), thereby ensuring that the overall picture of the target area is fully captured and satisfying the remote sensing image acquisition request of the user terminal.

[0079] The above operation achieves the technical effect of being able to instantly upload the user terminal's shooting needs to the remote sensing satellite, improving the efficiency of information transmission, ensuring the completion of the shooting of remote sensing images of the target area, and further improving the satellite's distribution capability for remote sensing information and the distributed computing capability of remote sensing tasks.

[0080] This solves the technical problems existing in the prior art, such as the limitation of satellite data transmission capacity and the constraints of gateway station layout, the inability to immediately upload user shooting needs to remote sensing satellites, the inability to immediately obtain satellite images, and the complex and lengthy information acquisition chain, which restricts the distribution capacity of remote sensing information and the distributed computing capacity of remote sensing tasks.

[0081] Optionally, the first remote sensing satellite and each second remote sensing satellite generate a spatial confidence map corresponding to the first feature map based on the remote sensing image, and determine the target feature image block in the first feature map, including: the first remote sensing satellite and each second remote sensing satellite respectively determine the spatial confidence map corresponding to the first feature map, and determine the confidence score corresponding to each feature image block in the first feature map based on the spatial confidence map; the first remote sensing satellite and each second remote sensing satellite determine whether the confidence score is greater than a preset threshold; and when the confidence score is greater than the preset threshold, the first remote sensing satellite and each second remote sensing satellite determine the feature image block as the target feature image block. Further optionally, the first remote sensing satellite and each second remote sensing satellite respectively determine a spatial confidence map corresponding to the first feature map, and determine the confidence scores corresponding to each feature image block in the first feature map based on the spatial confidence map, including: the first remote sensing satellite and each second remote sensing satellite determine the first feature map corresponding to the remote sensing image through image encoding, and determine the spatial confidence map corresponding to the first feature map through image decoding; and the first remote sensing satellite and each second remote sensing satellite determine the confidence scores corresponding to each feature image block based on the spatial confidence map.

[0082] Specifically, first, the first remote sensing satellite 40 and each of the second remote sensing satellites 501-50m respectively determine the spatial confidence map corresponding to the first feature map. The generation process of the spatial confidence map corresponding to the first feature map can be implemented by image encoding and image decoding. Q i Remote sensing images X i Take image encoding and decoding as an example:

[0083] For example, the first remote sensing satellite Q i The remote sensing image input in the kth communication cycle is X i , then the calculation formula of the first feature map corresponding to the remote sensing image generated by image encoding is as follows:

[0084]

[0085] in, The first remote sensing satellite Q i The first feature map corresponding to the remote sensing image, represents the encoder, The first remote sensing satellite Q i Corresponding remote sensing images, H represents the length of the remote sensing image, W represents the width of the remote sensing image, and D represents the channel of the remote sensing image.

[0086] Afterwards, the first remote sensing satellite generates a spatial confidence map corresponding to the first feature map through image decoding. The calculation formula is as follows:

[0087]

[0088] in, The first remote sensing satellite Q i The spatial confidence map corresponding to the first feature map of Represents a decoder, The first remote sensing satellite Q i The first feature map corresponding to the remote sensing image.

[0089] Thus, based on the above operation, the first remote sensing satellite Q i It can be determined that the first characteristic graph Corresponding spatial confidence map Similarly, based on the same operation as above, each second remote sensing satellite Q j It can be determined that the first characteristic graph Corresponding spatial confidence map , which will not be elaborated here.

[0090] Furthermore, since the spatial confidence map reflects the importance of different feature image blocks in the first feature map, the first remote sensing satellite Q i and the second remote sensing satellites Q j Based on its own spatial confidence map, it is possible to determine the confidence scores corresponding to each feature image block in the first feature map.

[0091] For example, the first remote sensing satellite Q i Based on the generated spatial confidence map , determine the image block with the feature The corresponding confidence score , and the feature image block The corresponding confidence score , ..., and feature image blocks The corresponding confidence score Among them, the first feature map Contains multiple feature image blocks .

[0092] The second remote sensing satellite Q 1 Based on the generated spatial confidence map , determine the image block with the feature The corresponding confidence score , and the feature image block The corresponding confidence score , ..., and feature image blocks The corresponding confidence score Among them, the first feature map Contains multiple feature image blocks .

[0093] And so on.

[0094] The second remote sensing satellite Q m Based on the generated spatial confidence map , determine the image block with the feature The corresponding confidence score , and the feature image block The corresponding confidence score , ..., and feature image blocks The corresponding confidence score Among them, the first feature map Contains multiple feature image blocks .

[0095] The first remote sensing satellite Q i and the second remote sensing satellites Q j When the confidence scores corresponding to the characteristic image blocks are determined respectively, the first remote sensing satellite Q i and the second remote sensing satellites Q j Determine whether the confidence score corresponding to each feature image block is greater than a preset threshold. And if the confidence score is greater than the preset threshold, the first remote sensing satellite Q i and the second remote sensing satellites Q j The feature image block is determined as the target feature image block.

[0096] First Remote Sensing Satellite Q i For example: the first remote sensing satellite Q i Determine and each feature image block The corresponding confidence score With preset threshold y k For example, the first remote sensing satellite Q i Determine the image patch with features The corresponding confidence score Greater than the preset threshold y k , then the first remote sensing satellite will feature image blocks Determine the target feature image block.

[0097] Similarly, each second remote sensing satellite Q j The target feature image block may also be determined based on the above method, which will not be described in detail here.

[0098] Thus, by using target feature image blocks with higher confidence scores and eliminating feature image blocks with lower confidence scores, the technical effect of further saving communication resources and improving data transmission efficiency is achieved while ensuring that the user terminal receives the second feature map of the complete target area.

[0099] Optionally, it also includes: the first remote sensing satellite determines the first data packet based on its own spatial confidence map when determining the target feature image block, wherein the first data packet includes the target feature image block that needs to be requested from each second remote sensing satellite; the first remote sensing satellite determines the second data packet based on the request map sent by each second remote sensing satellite, wherein the request map is used to indicate that each second remote sensing satellite requests the target feature image block sent by the first remote sensing satellite, and the second data packet includes the target feature image block that needs to be sent to each second remote sensing satellite; and the first remote sensing satellite constructs a communication map and determines whether to communicate with each second remote sensing satellite. Further optionally, the operation of the first remote sensing satellite constructing a communication map and determining whether to communicate with each second remote sensing satellite includes: in the first round of communication, the first remote sensing satellite is respectively connected to each second remote sensing satellite in communication; and in the subsequent communication of the first round of communication, the first remote sensing satellite determines whether to communicate with each second remote sensing satellite according to its own spatial confidence map and the request map corresponding to each second remote sensing satellite.

[0100] Specifically, when the first remote sensing satellite 40 and the plurality of second remote sensing satellites 501-50m determine the target feature image blocks, it is necessary to further determine which image information to send. The data packet encapsulated by the first remote sensing satellite 40 includes a first data packet and a second data packet. The first data packet is used to indicate the target feature image blocks that need to be requested from each second remote sensing satellite 501-50m, and the second data packet is used to indicate the target feature image blocks that need to be sent to each second remote sensing satellite 501-50m.

[0101] Among them, the first remote sensing satellite Q i The corresponding calculation formula for the first data packet is as follows:

[0102]

[0103] in, The first remote sensing satellite Q i The corresponding first data packet, The first remote sensing satellite Q i The spatial confidence map corresponding to the first feature map of FIG. And wherein the first data packet is negatively correlated with the spatial confidence map.

[0104] Among them, the first remote sensing satellite Q i The corresponding calculation formula for the second data packet is as follows:

[0105]

[0106] in, The first remote sensing satellite Q i The corresponding target feature image block, that is, the first remote sensing satellite Q i The corresponding second data packet, Represents the partial area of ​​the first feature map that needs to be transmitted, which is a binary matrix. The first remote sensing satellite Q i The first feature map corresponding to the remote sensing image of the first remote sensing satellite Q i The length of the corresponding remote sensing image, W, represents the length of the first remote sensing satellite Q i The width of the corresponding remote sensing image, D, represents the width of the first remote sensing satellite Q i The corresponding channel of the remote sensing image.

[0107] The data contained in the second data packet can be selected by a binary matrix, and the calculation formula of the data contained in the second data packet is as follows:

[0108]

[0109] in, Represents the partial area in the first feature map that needs to be transmitted, which is a binary matrix. In the case of round 0 communication, the first remote sensing satellite Q i Only the spatial confidence map corresponding to the first feature map needs to be considered. represents the request graph corresponding to the jth second remote sensing satellite, and when k=1, Indicates that in the first round of communication, a fully connected undirected communication needs to be established. That is, in the first round of communication, the first remote sensing satellite establishes a communication connection with all the second remote sensing satellites. When k=2, This indicates that in the second round of communication, the first remote sensing satellite establishes a communication connection with the second remote sensing satellite that sent the request map in the first round of communication. Indicates that in subsequent communication rounds (i.e., the first communication round, the second communication round, ...), the first remote sensing satellite Q i It is necessary to consider both the spatial confidence map corresponding to the first feature map and the request map sent by each second remote sensing satellite.

[0110] Thus, in the kth round of communication, the first remote sensing satellite Q i The data information that needs to be packaged is finally determined as .in, Indicates the first data packet, represents the second data packet. And because the first remote sensing satellite Q i Only the first data packet which is negatively correlated with the confidence and the second data packet which is spatially sparse but perceptually critical are transmitted, thus greatly reducing the first remote sensing satellite Q i and the second remote sensing satellites Q j The communication bandwidth required for collaborative detection and further save the first remote sensing satellite Q i and the second remote sensing satellites Q j 's computing power.

[0111] Similarly, each second remote sensing satellite Q j The specific operation method for determining the first data packet and the second data packet is as described above and will not be repeated here.

[0112] After the first remote sensing satellite Q i and the second remote sensing satellites Q j When the information is packaged, a sparsely connected communication graph needs to be further constructed to determine when to communicate with which satellite. This can avoid wasting precious bandwidth resources by reducing unnecessary communications.

[0113] In the first round of communications, the first remote sensing satellite Q i There are no other second remote sensing satellites Q jTherefore, it is necessary to establish a fully connected unidirectional communication and broadcast its own data information to each second remote sensing satellite. Q j .

[0114] In the follow-up communication of the first round of communication, the first remote sensing satellite Q i According to its own spatial confidence map and the second remote sensing satellite Q j The corresponding request graph further determines whether it is consistent with each second remote sensing satellite Q j Communications connection. Only the first remote sensing satellite Q i and the second remote sensing satellites Q j Communications are established only when there is a need for information exchange, aiming to minimize the amount of data transmitted between satellites, thereby achieving more efficient and accurate collaborative target perception.

[0115] Among them, according to the binary matrix The maximum value of the first remote sensing satellite Q i and the second remote sensing satellites Q j The necessity of communication between is the adjacency matrix of the communication topology of the kth communication cycle, then The elements are:

[0116]

[0117] in, Represents the adjacency matrix. Represents the maximum value of a binary matrix.

[0118] And if the adjacency matrix If an element in is 1, it means that the first remote sensing satellite corresponding to the element Q i and the Second Remote Sensing Satellite Q j If there is a need for communication between them, this communication link will be activated. The communication graph constructed in this way is sparse and efficient, which can save communication resources while ensuring effective information transmission.

[0119] Thus, by constructing a sparsely connected communication graph and determining when the first remote sensing satellite communicates with which second remote sensing satellite, the technical effect of reducing unnecessary communications, avoiding wasting precious bandwidth resources, and minimizing the amount of data transmitted between the first remote sensing satellite and each second remote sensing satellite is achieved.

[0120] Optionally, the first remote sensing satellite receives the target feature image blocks sent by each second remote sensing satellite, and uses multiple target feature image blocks corresponding to each second remote sensing satellite and its own first feature map to splice and fuse them, so as to generate a second feature map corresponding to the target area. The operation includes: the first remote sensing satellite receives the target feature image blocks sent by each second remote sensing satellite, and uses a multi-head attention mechanism to splice and fuse the target feature image blocks corresponding to each second remote sensing satellite and its own first feature map, so as to determine the second feature map corresponding to the target area. Further optionally, the first remote sensing satellite receives the target feature image blocks sent by each second remote sensing satellite, and uses a multi-head attention mechanism to splice and fuse the target feature image blocks corresponding to each second remote sensing satellite and its own first feature map, so as to determine the second feature map corresponding to the target area. The operation includes: the first remote sensing satellite inputs the target feature image blocks corresponding to each second remote sensing satellite and its own first feature map of the confidence score corresponding to each target feature image block into the feedforward neural network model; and the first remote sensing satellite determines the second feature map corresponding to the target area output by the feedforward neural network model.

[0121] Specifically, when the first remote sensing satellite 40 receives the target feature image blocks sent by the second remote sensing satellites 501~50m, it can further adopt a multi-head attention mechanism to fuse the target feature image blocks sent by the second remote sensing satellites 501~50m and update its own first feature map, so as to obtain an enhanced second feature map.

[0122] Among them, the spatial confidence map in the data packet sent by each second remote sensing satellite 501~50m reflects the perception level of each second remote sensing satellite 501~50m to the target area from different perspectives, providing prior information for the attention learning mechanism. The multi-head attention mechanism can calculate the scaled dot product attention weights from different channel inputs separately and in parallel. The input information includes the target feature image block corresponding to the first remote sensing satellite 40, the target feature image block sent by each second remote sensing satellite 501~50m, and the confidence score corresponding to the target feature image block of each second remote sensing satellite 501~50m. The first remote sensing satellite 40 inputs the attention weights and sparse features of each channel into a feedforward neural network for information fusion, and outputs the enhanced second feature map.

[0123] The first remote sensing satellite Q i According to its own first characteristic map and each second remote sensing satellite Q j The sent target feature image patch can calculate the scaled dot product attention weight:

[0124]

[0125] in, represents the scaled dot product attention weight, represents a multi-head attention mechanism that can fuse multi-source image features at the spatial position of each target feature image block. The first remote sensing satellite Q i The first feature map corresponding to the remote sensing image, Indicates each second remote sensing satellite Q j The second data packet sent (including the target feature image block) Indicates each second remote sensing satellite Q j The spatial confidence map sent represents the second remote sensing satellite Q j Key image areas under the perspective of the first remote sensing satellite Q i Contributes to the calculation of attention weights.

[0126] After the kth round of communication, a more complete and comprehensive second feature map can be obtained by fusion of multi-source image information using a feedforward neural network:

[0127]

[0128] in, The first remote sensing satellite Q i The corresponding second feature map, represents the scaled dot product attention weight, Indicates each second remote sensing satellite Q j The second data packet sent.

[0129] Therefore, by utilizing the multi-head attention mechanism to fuse the target feature image blocks sent by each second remote sensing satellite and update its own first feature map, the technical effect of being able to output a complete and enhanced second feature map is achieved.

[0130] Therefore, according to the first aspect of this embodiment, the technical effect of being able to instantly upload the shooting needs of the user terminal to the remote sensing satellite, improving the efficiency of information transmission, ensuring the completion of the shooting of remote sensing images of the target area, and further improving the satellite's distribution capabilities for remote sensing information and the distributed computing capabilities of remote sensing tasks is achieved.

[0131] In addition, reference Figure 1As shown, according to the second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0132] Therefore, according to this embodiment, the technical effect of being able to instantly upload the shooting needs of the user terminal to the remote sensing satellite, improving the efficiency of information transmission, ensuring the completion of the shooting of remote sensing images of the target area, and further improving the satellite's distribution capabilities for remote sensing information and the distributed computing capabilities of remote sensing tasks is achieved.

[0133] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0135] Example 2

[0136] Figure 5 The satellite Internet-based distributed remote sensing device 500 according to this embodiment is shown, and the device 500 corresponds to the method according to embodiment 1. Figure 5As shown, the device 500 includes: an acquisition request sending module 510, which is used for the gateway to receive the remote sensing image acquisition request for the target area sent by the user terminal through the satellite Internet, and to upload the remote sensing image acquisition task to the first remote sensing satellite through the satellite Internet and the intersatellite link, wherein the remote sensing image acquisition task corresponds to the remote sensing image acquisition request; a remote sensing image acquisition module 520, which is used for the first remote sensing satellite to broadcast the remote sensing image acquisition task to multiple second remote sensing satellites, and the first remote sensing satellite and the multiple second remote sensing satellites respectively acquire remote sensing images corresponding to the target area; a first feature map generation module 530, which is used for the first remote sensing satellite and each second remote sensing satellite to generate a spatial confidence map corresponding to the first feature map based on the remote sensing image. a spatial confidence map, and determines a target feature image block in the first feature map, wherein the spatial confidence map includes confidence scores corresponding to each feature image block in the first feature map, and the target feature image block is used to indicate a feature image block whose confidence score is greater than a preset threshold; a second feature map generation module 540, which is used for the first remote sensing satellite to receive the target feature image blocks sent by each second remote sensing satellite, and to splice and fuse multiple target feature image blocks corresponding to each second remote sensing satellite and its own first feature map, so as to generate a second feature map corresponding to the target area; and a feature map transmission module 550, which is used for the first remote sensing satellite to transmit the second feature map corresponding to the remote sensing image to the user terminal in real time through the satellite Internet and the intersatellite link.

[0137] Optionally, the first feature map generation module 530 includes: a confidence score determination module, which is used for the first remote sensing satellite and each second remote sensing satellite to respectively determine the spatial confidence map corresponding to the first feature map, and determine the confidence score corresponding to each feature image block in the first feature map based on the spatial confidence map; a first determination module, which is used for the first remote sensing satellite and each second remote sensing satellite to determine whether the confidence score is greater than a preset threshold; and a target feature image block determination module, which is used for the first remote sensing satellite and each second remote sensing satellite to determine the feature image block as a target feature image block when the confidence score is greater than the preset threshold.

[0138] Optionally, the confidence score determination module includes: a spatial confidence map generation module, which is used for the first remote sensing satellite and each second remote sensing satellite to determine the first feature map corresponding to the remote sensing image through image encoding, and to determine the spatial confidence map corresponding to the first feature map through image decoding; and a confidence score determination submodule, which is used for the first remote sensing satellite and each second remote sensing satellite to determine the confidence scores corresponding to each feature image block based on the spatial confidence map.

[0139] Optionally, the device 500 also includes: a first data packet determination module, which is used for the first remote sensing satellite to determine the first data packet based on its own spatial confidence map when determining the target feature image block, wherein the first data packet includes the target feature image blocks that need to be requested from each second remote sensing satellite; a second data packet determination module, which is used for the first remote sensing satellite and based on the request map sent by each second remote sensing satellite to determine the second data packet, wherein the request map is used to indicate that each second remote sensing satellite requests the target feature image block sent by the first remote sensing satellite, and the second data packet includes the target feature image block that needs to be sent to each second remote sensing satellite; and a second remote sensing satellite determination module, which is used for the first remote sensing satellite to construct a communication map and determine whether to communicate with each second remote sensing satellite.

[0140] Optionally, the second remote sensing satellite determination module includes: a first round communication determination module, which is used to determine whether the first remote sensing satellite is connected to each second remote sensing satellite for communication in the first round of communication; and a subsequent communication determination module, which is used to determine whether the first remote sensing satellite is connected to each second remote sensing satellite for communication in subsequent communications after the first round of communication based on its own spatial confidence map and a request map corresponding to each second remote sensing satellite.

[0141] Optionally, the second feature map generation module 540 includes: a second feature map generation submodule, which is used for the first remote sensing satellite to receive the target feature image blocks sent by each second remote sensing satellite, and use a multi-head attention mechanism to splice and fuse the target feature image blocks corresponding to each second remote sensing satellite and its own first feature map, so as to determine the second feature map corresponding to the target area.

[0142] Optionally, the second feature map generation submodule includes: a first feature map input module, which is used for the first remote sensing satellite to input the first feature map of the target feature image blocks of each second remote sensing satellite and the confidence score corresponding to each target feature image block into the feedforward neural network model; and a second feature map determination module, which is used for the first remote sensing satellite to determine the second feature map corresponding to the target area output by the feedforward neural network model.

[0143] Therefore, according to this embodiment, the technical effect of being able to instantly upload the shooting needs of the user terminal to the remote sensing satellite, improving the efficiency of information transmission, ensuring the completion of the shooting of remote sensing images of the target area, and further improving the satellite's distribution capabilities for remote sensing information and the distributed computing capabilities of remote sensing tasks is achieved.

[0144] Example 3

[0145] Figure 6 The satellite Internet-based distributed remote sensing device 600 according to this embodiment is shown, and the device 600 corresponds to the method according to embodiment 1. Figure 6As shown, the device 600 includes: a processor 610; and a memory 620, connected to the processor 610, for providing the processor 610 with instructions for processing the following processing steps: the gateway receives a remote sensing image acquisition request for a target area sent by a user terminal through the satellite Internet, and uploads a remote sensing image acquisition task to a first remote sensing satellite through the satellite Internet and an intersatellite link, wherein the remote sensing image acquisition task corresponds to the remote sensing image acquisition request; the first remote sensing satellite broadcasts the remote sensing image acquisition task to multiple second remote sensing satellites, and the first remote sensing satellite and the multiple second remote sensing satellites respectively collect remote sensing images corresponding to the target area; the first remote sensing satellite and each second remote sensing satellite, based on the remote sensing image, Generate spatial confidence maps corresponding to the first feature map respectively, and determine the target feature image block in the first feature map, wherein the spatial confidence map includes confidence scores corresponding to each feature image block in the first feature map, and the target feature image block is used to indicate a feature image block whose confidence score is greater than a preset threshold; the first remote sensing satellite receives the target feature image blocks sent by each second remote sensing satellite, and uses multiple target feature image blocks corresponding to each second remote sensing satellite and its own first feature map for splicing and fusion, so as to generate a second feature map corresponding to the target area; and the first remote sensing satellite transmits the second feature map corresponding to the remote sensing image to the user terminal in real time through the satellite Internet and the intersatellite link.

[0146] Optionally, the first remote sensing satellite and each second remote sensing satellite generate a spatial confidence map corresponding to the first feature map based on the remote sensing image, and determine the target feature image block in the first feature map, including: the first remote sensing satellite and each second remote sensing satellite respectively determine the spatial confidence map corresponding to the first feature map, and determine the confidence score corresponding to each feature image block in the first feature map based on the spatial confidence map; the first remote sensing satellite and each second remote sensing satellite determine whether the confidence score is greater than a preset threshold; and when the confidence score is greater than the preset threshold, the first remote sensing satellite and each second remote sensing satellite determine the feature image block as the target feature image block.

[0147] Optionally, the first remote sensing satellite and each second remote sensing satellite respectively determine a spatial confidence map corresponding to the first feature map, and determine the confidence scores corresponding to each feature image block in the first feature map based on the spatial confidence map, including: the first remote sensing satellite and each second remote sensing satellite determine the first feature map corresponding to the remote sensing image through image encoding, and determine the spatial confidence map corresponding to the first feature map through image decoding; and the first remote sensing satellite and each second remote sensing satellite determine the confidence scores corresponding to each feature image block based on the spatial confidence map.

[0148] Optionally, the device 600 also includes: when the first remote sensing satellite determines the target feature image block, based on its own spatial confidence map, determines a first data packet, wherein the first data packet includes the target feature image blocks that need to be requested from each second remote sensing satellite; the first remote sensing satellite determines a second data packet based on a request map sent by each second remote sensing satellite, wherein the request map is used to indicate that each second remote sensing satellite requests the target feature image block sent by the first remote sensing satellite, and the second data packet includes the target feature image blocks that need to be sent to each second remote sensing satellite; and the first remote sensing satellite constructs a communication map and determines whether to communicate with each second remote sensing satellite.

[0149] Optionally, the first remote sensing satellite constructs a communication map and determines whether to communicate with each second remote sensing satellite, including: in a first round of communication, the first remote sensing satellite is respectively connected to each second remote sensing satellite for communication; and in subsequent communications after the first round of communication, the first remote sensing satellite determines whether to communicate with each second remote sensing satellite based on its own spatial confidence map and a request map corresponding to each second remote sensing satellite.

[0150] Optionally, the first remote sensing satellite receives the target feature image blocks sent by each second remote sensing satellite, and uses multiple target feature image blocks corresponding to each second remote sensing satellite and its own first feature map to splice and fuse them, so as to generate a second feature map corresponding to the target area. The operation includes: the first remote sensing satellite receives the target feature image blocks sent by each second remote sensing satellite, and uses a multi-head attention mechanism to splice and fuse the target feature image blocks corresponding to each second remote sensing satellite and its own first feature map, so as to determine the second feature map corresponding to the target area.

[0151] Optionally, the first remote sensing satellite receives the target feature image blocks sent by each second remote sensing satellite, and uses a multi-head attention mechanism to splice and fuse the target feature image blocks corresponding to each second remote sensing satellite and its own first feature map, so as to determine the second feature map corresponding to the target area. The operation includes: the first remote sensing satellite inputs the target feature image blocks corresponding to each second remote sensing satellite and its own first feature map of the confidence score corresponding to each target feature image block into a feedforward neural network model; and the first remote sensing satellite determines the second feature map corresponding to the target area output by the feedforward neural network model.

[0152] Therefore, according to this embodiment, the technical effect of being able to instantly upload the shooting needs of the user terminal to the remote sensing satellite, improving the efficiency of information transmission, ensuring the completion of the shooting of remote sensing images of the target area, and further improving the satellite's distribution capabilities for remote sensing information and the distributed computing capabilities of remote sensing tasks is achieved.

[0153] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0154] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0156] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0159] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A distributed remote sensing method based on satellite Internet, characterized in that: include: The gateway station receives a remote sensing image acquisition request for a target area sent by a user terminal through the satellite Internet, and uploads a remote sensing image acquisition task to a first remote sensing satellite through the satellite Internet and an intersatellite link, wherein the remote sensing image acquisition task corresponds to the remote sensing image acquisition request; The first remote sensing satellite broadcasts the remote sensing image acquisition task to multiple second remote sensing satellites, and the first remote sensing satellite and the multiple second remote sensing satellites respectively collect remote sensing images corresponding to the target area; The first remote sensing satellite and each of the second remote sensing satellites respectively generate a spatial confidence map corresponding to the first feature map based on the remote sensing image, and determine a target feature image block in the first feature map, wherein the spatial confidence map includes confidence scores corresponding to each feature image block in the first feature map, and the target feature image block is used to indicate a feature image block whose confidence score is greater than a preset threshold; The first remote sensing satellite receives the target feature image blocks sent by the second remote sensing satellites, and uses the multiple target feature image blocks corresponding to the second remote sensing satellites and its own first feature map to perform splicing and fusion, thereby generating a second feature map corresponding to the target area; as well as The first remote sensing satellite transmits the second feature map corresponding to the remote sensing image to the user terminal in real time through the gateway station based on satellite Internet and inter-satellite links.

2. The method according to claim 1, characterized in that The first remote sensing satellite and each second remote sensing satellite generate a spatial confidence map corresponding to the first feature map based on the remote sensing image, and determine the target feature image block in the first feature map, including: The first remote sensing satellite and each second remote sensing satellite respectively determine a spatial confidence map corresponding to the first feature map, and determine a confidence score corresponding to each feature image block in the first feature map based on the spatial confidence map; The first remote sensing satellite and each of the second remote sensing satellites determine whether the confidence score is greater than a preset threshold; and When the confidence score is greater than the preset threshold, the first remote sensing satellite and each of the second remote sensing satellites determine the feature image block as the target feature image block.

3. The method according to claim 2, characterized in that The operation of respectively determining a spatial confidence map corresponding to the first feature map by the first remote sensing satellite and each second remote sensing satellite, and determining a confidence score corresponding to each feature image block in the first feature map based on the spatial confidence map includes: The first remote sensing satellite and each second remote sensing satellite determine a first feature map corresponding to the remote sensing image through image encoding, and determine a spatial confidence map corresponding to the first feature map through image decoding; and The first remote sensing satellite and each second remote sensing satellite determine the confidence score corresponding to each characteristic image block based on the spatial confidence map.

4. The method according to claim 3, characterized in that Also includes: When the first remote sensing satellite determines the target feature image block, the first remote sensing satellite determines a first data packet based on its own spatial confidence map, wherein the first data packet includes the target feature image blocks that need to be requested from each of the second remote sensing satellites; The first remote sensing satellite determines a second data packet based on the request graph sent by each of the second remote sensing satellites, wherein the request graph is used to indicate the target feature image blocks that each of the second remote sensing satellites requests the first remote sensing satellite to send, and the second data packet includes the target feature image blocks that need to be sent to each of the second remote sensing satellites; as well as The first remote sensing satellite constructs a communication graph and determines whether to communicate with each of the second remote sensing satellites.

5. The method according to claim 4, characterized in that The operation of the first remote sensing satellite constructing a communication graph and determining whether to communicate with each of the second remote sensing satellites includes: In a first round of communication, the first remote sensing satellite is respectively connected to the second remote sensing satellites for communication; and In subsequent communications of the first round of communications, the first remote sensing satellite determines whether to establish communication connection with each of the second remote sensing satellites based on its own spatial confidence map and the request map corresponding to each of the second remote sensing satellites.

6. The method according to claim 5, characterized in that The operation of the first remote sensing satellite receiving the target feature image blocks sent by each second remote sensing satellite, and using a plurality of target feature image blocks corresponding to each second remote sensing satellite and its own first feature map to perform splicing and fusion to generate a second feature map corresponding to the target area includes: The first remote sensing satellite receives the target feature image blocks sent by the second remote sensing satellites, and uses a multi-head attention mechanism to splice and fuse the target feature image blocks corresponding to the second remote sensing satellites and its own first feature map, so as to determine the second feature map corresponding to the target area.

7. The method according to claim 6, characterized in that The first remote sensing satellite receives the target feature image blocks sent by the second remote sensing satellites, and uses a multi-head attention mechanism to splice and fuse the target feature image blocks corresponding to the second remote sensing satellites and its own first feature map, thereby determining an operation of a second feature map corresponding to the target area, including: The first remote sensing satellite inputs the target feature image blocks corresponding to the second remote sensing satellites, the confidence scores corresponding to the target feature image blocks, and its own first feature map into the feedforward neural network model; and The first remote sensing satellite determines a second feature map output by the feedforward neural network model and corresponding to the target area.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 7.

9. A distributed remote sensing device based on satellite Internet, characterized in that: include: an acquisition request sending module, configured to receive, at the gateway station, a remote sensing image acquisition request for a target area sent by a user terminal through the satellite Internet, and to upload a remote sensing image acquisition task to a first remote sensing satellite through the satellite Internet and an intersatellite link, wherein the remote sensing image acquisition task corresponds to the remote sensing image acquisition request; A remote sensing image acquisition module, configured for the first remote sensing satellite to broadcast the remote sensing image acquisition task to a plurality of second remote sensing satellites, and for the first remote sensing satellite and the plurality of second remote sensing satellites to respectively acquire remote sensing images corresponding to the target area; A first feature map generation module is used for the first remote sensing satellite and each second remote sensing satellite to respectively generate a spatial confidence map corresponding to the first feature map based on the remote sensing image, and determine a target feature image block in the first feature map, wherein the spatial confidence map includes confidence scores corresponding to each feature image block in the first feature map, and the target feature image block is used to indicate a feature image block whose confidence score is greater than a preset threshold; A second feature map generating module is used for the first remote sensing satellite to receive the target feature image blocks sent by the second remote sensing satellites, and to splice and fuse the multiple target feature image blocks corresponding to the second remote sensing satellites and its own first feature map, so as to generate a second feature map corresponding to the target area; as well as The feature map transmission module is used for the first remote sensing satellite to transmit the second feature map corresponding to the remote sensing image to the user terminal in real time through the satellite Internet and the inter-satellite link.

10. A distributed remote sensing device based on satellite Internet, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: The gateway station receives a remote sensing image acquisition request for a target area sent by a user terminal through the satellite Internet, and uploads a remote sensing image acquisition task to a first remote sensing satellite through the satellite Internet and an intersatellite link, wherein the remote sensing image acquisition task corresponds to the remote sensing image acquisition request; The first remote sensing satellite broadcasts the remote sensing image acquisition task to multiple second remote sensing satellites, and the first remote sensing satellite and the multiple second remote sensing satellites respectively collect remote sensing images corresponding to the target area; The first remote sensing satellite and each of the second remote sensing satellites respectively generate a spatial confidence map corresponding to the first feature map based on the remote sensing image, and determine a target feature image block in the first feature map, wherein the spatial confidence map includes confidence scores corresponding to each feature image block in the first feature map, and the target feature image block is used to indicate a feature image block whose confidence score is greater than a preset threshold; The first remote sensing satellite receives the target feature image blocks sent by the second remote sensing satellites, and uses the multiple target feature image blocks corresponding to the second remote sensing satellites and its own first feature map to perform splicing and fusion, thereby generating a second feature map corresponding to the target area; as well as The first remote sensing satellite transmits the second feature map corresponding to the remote sensing image to the user terminal in real time through satellite Internet and inter-satellite link.

Images