A star-machine-ground multi-element cooperative reconnaissance method, system, device and medium

Abstract

The application discloses a kind of star-machine-ground multi-element cooperative reconnaissance method, system, equipment and medium, it is related to unmanned vehicle target reconnaissance field.The method is obtained by remote sensing control system satellite to acquire reconnaissance image, and suspected target is pre-detected based on artificial intelligence algorithm and generates rough coordinate and target image slice;Satellite image is grid processed, and the sub-grid where target exists is marked and corresponding cluster unmanned vehicle marshalling is distributed;After unmanned vehicle marshalling flies to the sub-grid, according to rough coordinate search and based on the target image slice, lightweight template matching is completed to achieve target identification and information return;Finally, the return image is fused with satellite image by ground terminal to form global situation information.The application greatly reduces the demand of airborne computing power, converts blind search into fixed-point verification, and realizes efficient and accurate reconnaissance under limited bandwidth.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) target reconnaissance, specifically to a multi-dimensional collaborative reconnaissance method, system, equipment, and medium involving satellite, aircraft, and ground. Background Technology

[0002] The statements in this section are provided only as background information in connection with this disclosure and may not constitute prior art.

[0003] Satellite imaging reconnaissance, as one of the core means of modern strategic reconnaissance, has advantages such as large coverage area, clear image resources, and no risk of damage. However, it also has problems such as long revisit cycles, poor real-time performance, long data transmission time, and difficulty in balancing high resolution and large-area coverage. On the other hand, swarm drones are intelligent swarm systems composed of a large number of low-cost, miniaturized drones. They have decentralized systems, autonomous decision-making, and modular deployment capabilities, and can efficiently complete reconnaissance and surveillance tasks.

[0004] However, existing swarm drone reconnaissance also has significant shortcomings: on the one hand, limited by size and cost, swarm drones have limited endurance, instantaneous reconnaissance coverage, and real-time processing capabilities, making it impossible to run high-computing-power intelligent identification systems on the airborne end. Completing full-domain traversal and target identification requires a significant amount of time resources, failing to meet the timeliness requirements of modern warfare. On the other hand, limited by the bandwidth of the swarm network link, the swarm cannot simultaneously conduct wide-area or local reconnaissance and transmit massive amounts of raw images or videos back in real time. Furthermore, when the swarm conducts blind wide-area searches, the probability of being detected is high, and the survival risk is extremely high. Therefore, how to rationally utilize available resources such as satellites, swarm drones, and ground terminals to establish a collaborative mechanism to improve operational reconnaissance efficiency is a technical challenge that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to address the current limitations of swarm drones, such as poor endurance and low onboard computing power (inability to run large-scale intelligent recognition models) due to size and cost constraints, which prevent them from independently completing wide-area searches and rapid target identification. Furthermore, swarm drones face high detection rates and survival risks during blind wide-area searches. This invention provides a satellite-aircraft-ground multi-element collaborative reconnaissance method, system, equipment, and medium. Based on satellite pre-imaging to screen suspected targets and generate "target image slices," combined with precise grid-based dispatching from ground stations, the invention guides swarm drones to the corresponding grid and completes target identification through extremely lightweight "template matching." This satellite-aircraft-ground multi-element collaborative architecture significantly improves swarm combat reconnaissance efficiency and enables multi-element linkage for rapid wide-area searches and precise local identification, even with limited computing power, endurance, and link bandwidth for drones.

[0006] The technical solution of the present invention is as follows: A multi-element collaborative reconnaissance method based on a satellite-aircraft-ground system is disclosed. The system includes an on-orbit reconnaissance satellite, a portable remote sensing satellite control system, multiple swarm drones, and a ground intelligence fusion processing terminal integrated into a swarm ground station. The swarm drone group comprises multiple swarm drones. The method includes the following steps: The portable remote sensing satellite control system remotely schedules the on-orbit reconnaissance satellite to acquire satellite reconnaissance images of the area to be reconnaissance and transmits them down. After receiving the satellite reconnaissance image, the portable remote sensing satellite control system performs pre-detection and marking of suspected targets in the satellite reconnaissance image based on artificial intelligence algorithms, and generates preliminary intelligence containing the target's rough coordinates and target image slices; The satellite reconnaissance image of the area to be reconnoitered is processed into multiple sub-grids, and the sub-grids containing the suspected target are marked. The cluster ground station performs task allocation based on the prior intelligence, constructs the multiple cluster UAV groups, and assigns a corresponding cluster UAV group to each labeled subgrid for close-range reconnaissance. The cluster of UAVs flew to their respective assigned subgrids, searched for the suspected targets based on the rough coordinates of the targets, and performed template matching through the target image slices to complete target identification. They obtained the optical reconnaissance images and accurate positioning coordinates of the suspected targets and transmitted the reconnaissance information back to the cluster ground station. The ground intelligence fusion processing terminal fuses the optical reconnaissance images, accurate positioning coordinates, and satellite reconnaissance images transmitted back by each of the clustered UAVs to form a global target situational intelligence for target confirmation.

[0007] Furthermore, the portable remote sensing satellite control system remotely schedules the on-orbit reconnaissance satellite to acquire and transmit satellite reconnaissance images of the area to be reconnaissance, including: The portable remote sensing satellite control system sends on-site scheduling and uploading commands to the on-orbit reconnaissance satellite. According to the above instructions, adjust the lens orientation of the reconnaissance camera carried by the on-orbit reconnaissance satellite to align with the area to be reconnaissance, complete the capture of the satellite reconnaissance image of a specific width, and transmit it in real time.

[0008] Furthermore, after receiving the satellite reconnaissance image, the portable remote sensing satellite control system performs pre-detection and marking of suspected targets within the satellite reconnaissance image based on artificial intelligence algorithms, generating preliminary intelligence including approximate target coordinates and target image slices, including: The portable remote sensing satellite control system automatically decrypts, decompresses, and corrects the received satellite reconnaissance images sequentially. A deep learning target detection algorithm is used to extract and label the coordinates of the suspected target in the corrected satellite reconnaissance image to obtain the rough coordinates of the target; Based on the marked suspected target, a corresponding target image slice is generated in the satellite reconnaissance image, and the rough coordinates of the target are combined with the target image slice to form the preliminary intelligence as a priori intelligence support.

[0009] Furthermore, the satellite reconnaissance image of the area to be reconnoitered is processed into multiple sub-grids, and the sub-grids containing the suspected target are marked, including: The satellite reconnaissance image of the area to be reconnoitered is discretized into multiple square grids with specific side lengths to obtain the multiple sub-grids; Extract information from the prior intelligence and label the subgrid containing the suspected target as the target subgrid with features; The cluster ground station allocates tasks based on the pre-information, constructs multiple cluster UAV groups, and assigns a corresponding cluster UAV group to each labeled subgrid for close-range reconnaissance, including: The multiple clustered UAVs are grouped together by coordinating internal resources using a task allocation algorithm, which may include a genetic algorithm or an auction algorithm. Tasks are distributed to all the target subgrids that have been marked with features, ensuring that each target subgrid is assigned to a swarm of drones for close-range reconnaissance.

[0010] Furthermore, the swarm of drones flies to their respective assigned sub-grids and searches for the suspected target based on the target's approximate coordinates, including: Each of the aforementioned clustered UAV groups pre-loads the target image slices as matching templates before flight; Each of the aforementioned swarm drones is organized according to the principle of the shortest total flight path, and a path planning algorithm is used to complete the reconnaissance coverage of the corresponding assigned subgrids; wherein, the path planning algorithm includes any one of ant colony algorithm, A algorithm or particle swarm algorithm; During the reconnaissance coverage process, the target is guided to the vicinity of the suspected target for close-range search based on the target's rough coordinates.

[0011] Furthermore, during the close-range reconnaissance process of the swarm of drones, the procedure also includes handling any unforeseen targets not included in the prior intelligence: The swarm of UAVs acquires image information of the encountered target through its carried optical reconnaissance pod, and obtains the target-to-object distance between the encountered target and the swarm of UAVs currently performing the reconnaissance mission in the swarm of UAVs. The coordinates of the encountered target are calculated based on the geometric positional relationship between the encountered target and the cluster of UAVs currently performing the reconnaissance mission; The image information and coordinates of the encountered target are transmitted back to the cluster ground station as reconnaissance information, and the encountered target is added to the target list.

[0012] Furthermore, the ground-based intelligence fusion processing terminal fuses the optical reconnaissance images, accurate positioning coordinates, and satellite reconnaissance images transmitted back by each of the aforementioned cluster of UAVs to form a comprehensive target situational intelligence for target confirmation, including: The ground intelligence fusion processing terminal uses a multimodal image fusion algorithm to perform multi-dimensional fusion processing on the optical reconnaissance images, accurate positioning coordinates, and satellite reconnaissance images of each of the suspected targets. Generate global target situation intelligence containing the distribution of targets across the entire domain. Based on the global target situation intelligence, perform unified statistics and numbering on all targets discovered by reconnaissance to complete target confirmation.

[0013] This invention also proposes a space-aircraft-ground multi-element collaborative reconnaissance system that performs the method described above, the system comprising: On-orbit reconnaissance satellites are used to acquire satellite reconnaissance images of the area to be reconnaissance and transmit them down. A portable remote sensing satellite control system is used to remotely schedule the on-orbit reconnaissance satellite; and after receiving the satellite reconnaissance images, it performs pre-detection and marking of suspected targets in the satellite reconnaissance images based on artificial intelligence algorithms, generating preliminary intelligence containing rough coordinates of the targets and image slices of the targets; A ground intelligence fusion processing terminal, integrated into a cluster ground station, is used to perform gridding processing on the satellite reconnaissance images of the area to be reconnoitered, forming multiple sub-grids, and marking the sub-grids containing the suspected targets; and to allocate tasks according to the prior intelligence, assigning a corresponding cluster UAV group to each marked sub-grid for close-range reconnaissance. Multiple swarm drones are grouped together, the swarm drone group containing multiple swarm drones; used to fly to their respective assigned sub-grids, search for the suspected target based on the target's rough coordinates, and perform template matching through the target image slices to complete target identification, obtain the optical reconnaissance image and accurate positioning coordinates of the suspected target and transmit them back; The ground intelligence fusion processing terminal is also used to fuse the optical reconnaissance images, the accurate positioning coordinates, and the satellite reconnaissance images transmitted back by each of the clustered UAVs to form a global target situational intelligence for target confirmation.

[0014] The present invention also proposes an electronic device, comprising: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the instructions stored in the memory to perform the method described above.

[0015] The present invention also proposes a computer-readable storage medium for storing instructions that, when executed, cause the method described above to be implemented.

[0016] Compared with existing technologies, the advantages of this invention are: 1. Significantly reduces the onboard computing power requirements of drones. This invention utilizes the high-altitude wide-area view of satellites and the high computing power of ground stations to perform artificial intelligence pre-detection on large-scale satellite images in advance, screening out suspected targets in one go and generating "target image slices". After the drone flies to the target area, it only needs to call this slice and perform a lightweight "template matching" with the real-time image to complete the confirmation. This mechanism perfectly avoids running complex and large-scale deep learning recognition models on low-cost drones, completely solving the pain point of limited onboard computing power.

[0017] 2. Significantly improves search efficiency and survival rate. This invention introduces a gridded processing and feature annotation mechanism for satellite imagery, combined with a task allocation algorithm to accurately dispatch tasks to swarm drones. It transforms the traditional blind "wide-area, full-domain traversal search" of drone swarms into a "local, fixed-point, close-range verification" method based on a map. This not only greatly saves valuable drone endurance and time resources but also effectively reduces the survival risk of drone swarms being discovered due to prolonged, large-scale exposure.

[0018] 3. Saves link bandwidth and improves situational awareness accuracy. This invention only needs to transmit target image slices and precise positioning coordinates, without requiring the cluster to simultaneously transmit massive video streams, thus overcoming the bottleneck of limited link bandwidth in group networks. Simultaneously, the ground intelligence fusion processing terminal utilizes a multimodal image fusion algorithm to fuse high-resolution local images captured by UAVs with wide-area geographic images captured by satellites, forming a comprehensive target situational intelligence that considers both macro and micro perspectives, significantly improving the accuracy of battlefield target confirmation and the comprehensiveness of situational awareness.

[0019] 4. Strong dynamic adaptability and high ability to prevent missed detection. In response to the dynamic changes in the battlefield environment, this invention additionally designs a dynamic target detection and coordinate calculation mechanism. This allows the UAV to use its optical pod and target range to calculate the geometric coordinates of newly appearing targets in real time, effectively compensating for the information lag caused by the revisit cycle in satellite reconnaissance and ensuring the absolute integrity of reconnaissance coverage. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0021] Figure 1 This is a diagram illustrating the architecture of the space-aircraft-ground multi-element collaborative reconnaissance system provided in this embodiment of the invention. Figure 2 This is a schematic diagram of reconnaissance grid segmentation and annotation provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the allocation and mapping relationship of clustered UAV reconnaissance tasks provided in an embodiment of the present invention; Figure 4 This is a flowchart of the satellite-aircraft-ground collaborative reconnaissance strategy provided in this embodiment of the invention; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0023] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0024] Example 1 To address the limitations of existing swarm drones, such as poor endurance, limited instantaneous reconnaissance coverage, and restricted real-time processing capabilities (making it impossible to run high-computing-power intelligent identification systems on board), which are constrained by size and cost, thus hindering their ability to independently perform wide-area, comprehensive searches and rapid target identification, and resulting in high detection rates and survival risks when operating in groups, this embodiment provides a space-aircraft-ground multi-element collaborative reconnaissance method. This method is applied to a space-aircraft-ground multi-element collaborative reconnaissance system, which includes an on-orbit reconnaissance satellite, a portable remote sensing satellite control system, multiple swarm drone groups (each swarm drone group comprising multiple swarm drones), and a ground intelligence fusion processing terminal integrated into a swarm ground station. For the specific communication and hierarchical architecture of the system, please refer to [link to relevant documentation]. Figure 1 The diagram shows the composition of a multi-dimensional collaborative reconnaissance system involving space, aircraft, and ground.

[0025] Specifically, the complete workflow of the aforementioned space-aircraft-ground multi-element collaborative reconnaissance method includes the following steps: Step S101: The on-orbit reconnaissance satellite is remotely scheduled through the portable remote sensing satellite control system to acquire satellite reconnaissance images of the area to be reconnaissance and transmit them.

[0026] Specifically, the portable remote sensing satellite control system sends on-site scheduling and uploading commands to the on-orbit reconnaissance satellite; according to the uploading commands, the on-orbit reconnaissance satellite adjusts the lens orientation of its onboard reconnaissance camera to align with the area to be reconnoitered. As a preferred implementation, the reconnaissance camera is controlled to capture and transmit satellite reconnaissance images of a specific swath width, particularly, for example, 100km × 100km, in real time. This step fully leverages the advantages of satellites, such as large coverage area and no risk of damage, to quickly acquire a global field of view.

[0027] Step S102: After receiving the satellite reconnaissance image, the portable remote sensing satellite control system performs pre-detection and marking of suspected targets in the satellite reconnaissance image based on artificial intelligence algorithms, and generates preliminary intelligence containing the target's rough coordinates and target image slices.

[0028] This step is a core prerequisite for reducing the onboard computing power requirements of drones. The specific implementation process includes: First, the portable remote sensing satellite control system automatically receives, decrypts, decompresses, and corrects the received wide-swath satellite reconnaissance images in sequence. Subsequently, the system calls upon high-performance ground computing resources and employs a deep learning target detection algorithm (preferably, a high-precision artificial intelligence algorithm such as YOLOv5) to automatically extract and mark the coordinates of the suspected target within the corrected satellite reconnaissance image, thereby obtaining the rough coordinates of the target. Finally, based on the marked location of the suspected target in the satellite reconnaissance image, a corresponding target image slice is generated by cropping according to a preset pixel frame. The system combines the target's approximate coordinates with the target image slice to form the preliminary intelligence, which serves as preliminary intelligence support for swarm UAVs to carry out battlefield reconnaissance missions.

[0029] Step S103: The satellite reconnaissance image of the area to be reconnoitered is processed into a grid to form multiple sub-grids, and the sub-grids containing the suspected target are marked.

[0030] Considering that the endurance and reconnaissance capabilities of swarm drones cannot achieve a complete search and traversal of a 100km × 100km target area, this step discretizes the satellite reconnaissance image of the area to be reconnoitered into multiple square grids with specific side lengths. Preferably, the side length is set to 20km to obtain the multiple sub-grids (a 20km range closely matches the single-squad reconnaissance coverage capability of conventional micro-drones). Subsequently, the coordinate information in the preliminary intelligence is extracted, and the sub-grids containing the suspected targets are used as target sub-grids for feature annotation. This transforms the blind wide-area global search into a precise local pinpoint verification. The specific reconnaissance grid segmentation and annotation effect is as follows: Figure 2 As shown.

[0031] Step S104: The cluster ground station performs task allocation based on the prior intelligence, constructs multiple cluster UAV groups, and assigns a corresponding cluster UAV group to each labeled subgrid for close-range reconnaissance.

[0032] Specifically, the cluster ground station utilizes task allocation algorithms to coordinate internal resources and construct multiple cluster UAV formations. These task allocation algorithms include mature optimization algorithms such as genetic algorithms or auction algorithms to maximize resource utilization. The system automatically distributes tasks to all feature-annotated target sub-grids, ensuring that each target sub-grid containing a suspected target is assigned a cluster UAV formation for close-range reconnaissance, avoiding waste of computing power and range. The specific cluster UAV reconnaissance task allocation grid mapping relationship is as follows: Figure 3 As shown.

[0033] Step S105: The cluster of UAVs flies to their respective assigned sub-grids, searches for the suspected target based on the target's rough coordinates, and performs template matching through the target image slices to complete target identification, obtains the optical reconnaissance image and accurate positioning coordinates of the suspected target, and transmits the reconnaissance information back to the cluster ground station.

[0034] This is the key step in solving the technical problem of this invention. The specific execution logic is as follows: Before each of the aforementioned clustered UAV groups takes flight, the target image slices generated in S102 are pre-binded into the onboard computer as matching templates. After takeoff, each of the aforementioned clustered UAVs, based on the principle of the shortest total route, uses a path planning algorithm (such as ant colony algorithm, A algorithm, or particle swarm algorithm) to complete the reconnaissance coverage route planning for its respective assigned subgrid; During reconnaissance coverage, the UAV is guided directly to the vicinity of the suspected target based on the target's approximate coordinates for close-range search. Once it reaches the target's airspace, the UAV directly calls upon pre-loaded "target image slices" and performs lightweight "template matching" with images captured in real-time by the onboard optical reconnaissance pod. (Since only slice features need to be compared, there is no need to run complex deep learning networks, thus perfectly avoiding the pain point of low computing power in swarm UAVs.) This quickly completes the target identification and confirmation, and acquires high-definition optical reconnaissance images and accurate positioning coordinates for downlink.

[0035] In this embodiment, it should also be noted that during the close-range reconnaissance process of the swarm of drones, the step of processing encountered targets not included in the prior intelligence is also included: In response to the dynamic nature of the battlefield environment, for newly emerging targets (i.e., unintended targets) not captured by pre-imaged satellites, the swarm of UAVs acquires image information of the unintended targets through its onboard optical reconnaissance pods. It then uses onboard ranging equipment (such as laser rangefinders) or visual estimation to determine the target-to-object distance between the unintended target and the UAV currently performing the reconnaissance mission within the swarm. Subsequently, combining the UAV's real-time pose with the geometric positional relationship, the accurate coordinates of the unintended target are calculated. Finally, the image information and coordinates of the unintended target are transmitted back to the swarm ground station as reconnaissance information, and the unintended target is dynamically added to the target list.

[0036] Step S106: The ground intelligence fusion processing terminal fuses the optical reconnaissance images, accurate positioning coordinates and satellite reconnaissance images transmitted back by each of the clustered UAVs to form a global target situational intelligence for target confirmation.

[0037] Specifically, the ground intelligence fusion processing terminal employs a multimodal image fusion algorithm, preferably a deep learning-based Multi-Modal-Fusion algorithm, to fuse the acquired optical reconnaissance images (providing high-resolution microscopic details), accurate positioning coordinates, and satellite reconnaissance images (providing macroscopic geographic context) of each suspected target in a multi-dimensional manner. This ultimately generates a comprehensive target situational intelligence containing the distribution of targets across the entire region. Based on this comprehensive target situational intelligence, all targets discovered through reconnaissance are uniformly counted and numbered, assisting ground operators or commanders in completing the final target screening and confirmation. This completes the comprehensive reconnaissance coverage and precise reconnaissance mission for the entire target area. The complete workflow of this embodiment's satellite-aircraft-ground collaborative reconnaissance strategy can be found in [link to documentation]. Figure 4 As shown.

[0038] Example 2 Based on the same inventive concept as Embodiment 1, this embodiment provides a space-aircraft-ground multi-element collaborative reconnaissance system. This system is an architecture combining physical hardware and virtual modules to execute the space-aircraft-ground multi-element collaborative reconnaissance method described in Embodiment 1.

[0039] Combination Figure 1 The diagram shows the composition of a space-aircraft-ground multi-element collaborative reconnaissance system. This system mainly comprises four interconnected core subsystems: an on-orbit reconnaissance satellite, a portable remote sensing satellite control system, multiple swarm drone formations (each swarm drone formation containing multiple swarm drones), and a ground intelligence fusion processing terminal integrated into a swarm ground station. The functions and collaborative relationships of each component are as follows: In-orbit reconnaissance satellites are used to acquire and transmit satellite reconnaissance images of the area to be reconnaissance.

[0040] Specifically, the on-orbit reconnaissance satellite is equipped with a reconnaissance camera and a satellite-to-ground communication payload. It can receive on-site dispatch and uploading instructions from the ground, and adjust the camera lens orientation to align with the area to be reconnoitered according to the instructions. It can complete the wide-area acquisition of satellite reconnaissance images with a specific swath width (e.g., 100km×100km) and transmit the image data to the ground receiving end in real time.

[0041] A portable remote sensing satellite control system is used to remotely schedule the on-orbit reconnaissance satellite; and after receiving the satellite reconnaissance images, it performs pre-detection and marking of suspected targets in the satellite reconnaissance images based on artificial intelligence algorithms, generating preliminary intelligence containing rough coordinates of the targets and image slices of the targets.

[0042] As one of the "preprocessing and command centers" in the system, this control system first automatically decrypts, decompresses, and corrects the received wide-swath satellite reconnaissance images. Then, leveraging its high computing power, it runs deep learning target detection algorithms (such as YOLO v5) to filter out suspected targets in the images, obtain their approximate coordinates, and extract corresponding target image slices. The "preliminary intelligence" output by this module provides a priori basis for subsequent close-range reconnaissance by the UAV, effectively eliminating the computational burden of wide-area blind search and highly complex image recognition for the UAV itself.

[0043] A ground intelligence fusion processing terminal, integrated into a cluster ground station, is used to perform gridding processing on the satellite reconnaissance images of the area to be reconnoitered, forming multiple sub-grids, and marking the sub-grids containing the suspected targets; and to allocate tasks according to the prior intelligence, assigning a corresponding cluster UAV group to each marked sub-grid for close-range reconnaissance.

[0044] Specifically, the terminal discretizes the area to be reconnoitered (e.g., a 100km × 100km satellite image) into several sub-grids with specific side lengths (e.g., 20km). Figure 2 (As shown). Subsequently, task allocation algorithms such as genetic algorithms or auction algorithms are used to coordinate internal UAV resources and assign reconnaissance tasks to target subgrids marked with suspected targets (e.g., Figure 3 (As shown). Through the grid-based dispatching mechanism of this module, efficient focusing of local reconnaissance resources is achieved.

[0045] Multiple drones are grouped together to fly to their respective assigned sub-grids, search for suspected targets based on the rough coordinates of the target, perform template matching through the target image slices to complete target identification, acquire optical reconnaissance images and accurate positioning coordinates of the suspected targets, and transmit them back.

[0046] After receiving their mission and loading the target image slices, each swarm of UAVs uses a path planning algorithm (ant colony algorithm, A algorithm, etc.) to fly to the mission grid along the shortest route. Within the grid, the swarm of UAVs only needs to call a low-computing-power "template matching" program to compare the real-time images acquired by the onboard optical reconnaissance pods with the carried "slices" to quickly complete target identification. Furthermore, for encountered targets not in the pre-existing intelligence, the swarm can also calculate their coordinates using geometric positional relationships by acquiring image information and target distance, and then transmit these coordinates back.

[0047] In addition, the ground intelligence fusion processing terminal is also used to fuse the optical reconnaissance images, the accurate positioning coordinates and the satellite reconnaissance images transmitted back by each of the clustered UAVs to form a global target situational intelligence for target confirmation.

[0048] Specifically, the terminal uses a built-in multi-modal image fusion algorithm (such as the Multi-Modal-Fusion deep learning algorithm) to fuse the local high-precision coordinates and optical reconnaissance images transmitted by the UAV with the global reconnaissance images transmitted by the satellite in multiple dimensions, generating intuitive global target situational intelligence, thereby assisting ground command personnel in completing the final target numbering, statistics and confirmation.

[0049] The satellite-aircraft-ground multi-element collaborative reconnaissance system provided in this embodiment achieves efficient and accurate reconnaissance coverage of the entire battlefield under the constraints of limited link bandwidth and low computing power and short endurance of UAVs through a three-level collaborative architecture of satellite pre-imaging slicing, ground station grid-based intelligent dispatching, and lightweight template matching of swarm UAVs.

[0050] More specific work interaction processes and implementation details for each part of the system have been described in detail in the multi-factor collaborative reconnaissance method in Example 1, and will not be repeated here.

[0051] Based on the same technical concept, embodiments of the present invention also provide an electronic device that can implement the satellite-aircraft-ground multi-element collaborative reconnaissance method provided in the above embodiments of the present invention. In one embodiment, the electronic device can be a server, a terminal device, or other electronic equipment. Figure 5 As shown, the electronic device may include: At least one processor and a memory connected to the at least one processor. In this embodiment of the invention, the specific connection medium between the processor and the memory is not limited. Figure 5 The example used is the connection between the processor and memory via a bus. The bus... Figure 5 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. Buses can be divided into address buses, data buses, control buses, etc., but for ease of representation, [the specific bus type is not shown here]. Figure 5 The processor is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, a processor can also be called a controller; there are no restrictions on the name.

[0052] In this embodiment of the invention, the memory stores instructions executable by at least one processor. By executing the instructions stored in the memory, the at least one processor can perform the aforementioned space-aircraft-ground multi-element collaborative reconnaissance method. The processor can implement... Figure 5 The functions of each module in the device shown.

[0053] The processor is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory and calling data stored in memory, it can monitor the device's various functions and process data, thereby enabling overall monitoring of the device.

[0054] In an alternative design, the processor may include one or more processing units. The processor may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may also not be integrated into the processor. In some embodiments, the processor and memory may be implemented on the same chip; in some embodiments, they may also be implemented separately on separate chips.

[0055] The processor can be a general-purpose processor, such as a CPU, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the space-aircraft-ground multi-element collaborative reconnaissance method disclosed in the embodiments of this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0056] Memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia cards, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), and electrically erasable programmable read-only memory (EPROM). Only memory (EEPROM), magnetic storage, magnetic disks, optical disks, etc. A memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in embodiments of this invention can also be a circuit or any other device capable of performing storage functions for storing program instructions and / or data.

[0057] By designing and programming the processor, the code corresponding to the space-aircraft-ground multi-element collaborative reconnaissance method described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute the steps of the method described in the foregoing embodiments during operation. How to design and program the processor is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0058] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform a space-aircraft-ground multi-element collaborative reconnaissance method described above.

[0059] In some alternative embodiments, the present invention also provides that various aspects of a space-aircraft-ground multi-element collaborative reconnaissance method can also be implemented in the form of a program product, which includes program code that, when the program product is run on a device, causes the control device to perform the steps in a space-aircraft-ground multi-element collaborative reconnaissance method according to various exemplary embodiments of the present invention as described above.

[0060] It should be noted that although several units or sub-units of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the invention, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units. Furthermore, although the operation of the method of the invention is described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0061] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can be implemented in one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs) containing computer-usable program code. The form of a computer program product implemented on ROM, optical memory, etc.

[0062] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a server, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0063] Program code for performing the operations of this invention can be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0064] In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0067] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0068] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. A multi-element collaborative reconnaissance method involving space, air, and ground, characterized in that: It is applied to a multi-element collaborative reconnaissance system consisting of an on-orbit reconnaissance satellite, a portable remote sensing satellite control system, multiple clustered UAVs, and a ground intelligence fusion processing terminal integrated into a clustered ground station. The swarm drone group comprises multiple swarm drones; the method includes the following steps: The portable remote sensing satellite control system remotely schedules the on-orbit reconnaissance satellite to acquire satellite reconnaissance images of the area to be reconnaissance and transmits them down. After receiving the satellite reconnaissance image, the portable remote sensing satellite control system performs pre-detection and marking of suspected targets in the satellite reconnaissance image based on artificial intelligence algorithms, and generates preliminary intelligence containing the target's rough coordinates and target image slices; The satellite reconnaissance image of the area to be reconnoitered is processed into multiple sub-grids, and the sub-grids containing the suspected target are marked. The cluster ground station performs task allocation based on the prior intelligence, constructs the multiple cluster UAV groups, and assigns a corresponding cluster UAV group to each labeled subgrid for close-range reconnaissance. The cluster of UAVs flew to their respective assigned subgrids, searched for the suspected targets based on the rough coordinates of the targets, and performed template matching through the target image slices to complete target identification. They obtained the optical reconnaissance images and accurate positioning coordinates of the suspected targets and transmitted the reconnaissance information back to the cluster ground station. The ground intelligence fusion processing terminal fuses the optical reconnaissance images, accurate positioning coordinates, and satellite reconnaissance images transmitted back by each of the clustered UAVs to form a global target situational intelligence for target confirmation.

2. The satellite-aircraft-ground multi-element collaborative reconnaissance method according to claim 1, characterized in that, The portable remote sensing satellite control system remotely schedules the on-orbit reconnaissance satellite to acquire and transmit satellite reconnaissance images of the area to be reconnoitered, including: The portable remote sensing satellite control system sends on-site scheduling and uploading commands to the on-orbit reconnaissance satellite. According to the above instructions, adjust the lens orientation of the reconnaissance camera carried by the on-orbit reconnaissance satellite to align with the area to be reconnaissance, complete the capture of the satellite reconnaissance image of a specific width, and transmit it in real time.

3. The satellite-aircraft-ground multi-element collaborative reconnaissance method according to claim 1, characterized in that, After receiving the satellite reconnaissance image, the portable remote sensing satellite control system performs pre-detection and marking of suspected targets within the image based on artificial intelligence algorithms, generating preliminary intelligence including approximate target coordinates and target image slices, including: The portable remote sensing satellite control system automatically decrypts, decompresses, and corrects the received satellite reconnaissance images sequentially. A deep learning target detection algorithm is used to extract and label the coordinates of the suspected target in the corrected satellite reconnaissance image to obtain the rough coordinates of the target; Based on the marked suspected target, a corresponding target image slice is generated in the satellite reconnaissance image, and the rough coordinates of the target are combined with the target image slice to form the preliminary intelligence as a priori intelligence support.

4. The satellite-aircraft-ground multi-element collaborative reconnaissance method according to claim 1, characterized in that, The satellite reconnaissance image of the area to be reconnoitered is processed into multiple sub-grids, and the sub-grids containing the suspected target are marked, including: The satellite reconnaissance image of the area to be reconnoitered is discretized into multiple square grids with specific side lengths to obtain the multiple sub-grids; Extract information from the prior intelligence and label the subgrid containing the suspected target as the target subgrid with features; The cluster ground station allocates tasks based on the pre-information, constructs multiple cluster UAV groups, and assigns a corresponding cluster UAV group to each labeled subgrid for close-range reconnaissance, including: The multiple clustered UAVs are grouped together by coordinating internal resources using a task allocation algorithm, which may include a genetic algorithm or an auction algorithm. Tasks are distributed to all the target subgrids that have been marked with features, ensuring that each target subgrid is assigned to a swarm of drones for close-range reconnaissance.

5. The satellite-aircraft-ground multi-element collaborative reconnaissance method according to claim 1, characterized in that, The swarm of drones flies to their respective assigned sub-grids and searches for the suspected targets based on the approximate coordinates of the target, including: Each of the aforementioned clustered UAV groups pre-loads the target image slices as matching templates before flight; Each of the aforementioned swarm of UAVs operates on the principle of minimizing the total flight path, employing a path planning algorithm to achieve reconnaissance coverage of the corresponding assigned subgrids; wherein, the path planning algorithm includes any one of ant colony algorithm, A algorithm, or particle swarm algorithm; During the reconnaissance coverage process, the target is guided to the vicinity of the suspected target for close-range search based on the target's rough coordinates.

6. The satellite-aircraft-ground multi-element collaborative reconnaissance method according to claim 1, characterized in that, During the close-range reconnaissance process conducted by the swarm of drones, the procedure also includes handling encountered targets not included in the prior intelligence: The swarm of UAVs acquires image information of the encountered target through its carried optical reconnaissance pod, and obtains the target-to-object distance between the encountered target and the swarm of UAVs currently performing the reconnaissance mission in the swarm of UAVs. The coordinates of the encountered target are calculated based on the geometric positional relationship between the encountered target and the cluster of UAVs currently performing the reconnaissance mission; The image information and coordinates of the encountered target are transmitted back to the cluster ground station as reconnaissance information, and the encountered target is added to the target list.

7. The satellite-aircraft-ground multi-element collaborative reconnaissance method according to claim 1, characterized in that, The ground-based intelligence fusion processing terminal fuses the optical reconnaissance images, accurate positioning coordinates, and satellite reconnaissance images transmitted back by each of the aforementioned clustered UAVs to form a comprehensive target situational intelligence for target confirmation, including: The ground intelligence fusion processing terminal uses a multimodal image fusion algorithm to perform multi-dimensional fusion processing on the optical reconnaissance images, accurate positioning coordinates, and satellite reconnaissance images of each of the suspected targets. Generate global target situation intelligence containing the distribution of targets across the entire domain. Based on the global target situation intelligence, perform unified statistics and numbering on all targets discovered by reconnaissance to complete target confirmation.

8. A space-aircraft-ground multi-element collaborative reconnaissance system, characterized in that, The system, which performs the method described above, includes: On-orbit reconnaissance satellites are used to acquire satellite reconnaissance images of the area to be reconnaissance and transmit them down. A portable remote sensing satellite control system is used to remotely schedule the on-orbit reconnaissance satellite; and after receiving the satellite reconnaissance images, it performs pre-detection and marking of suspected targets in the satellite reconnaissance images based on artificial intelligence algorithms, generating preliminary intelligence containing rough coordinates of the targets and image slices of the targets; A ground intelligence fusion processing terminal, integrated into a cluster ground station, is used to perform gridding processing on the satellite reconnaissance images of the area to be reconnoitered, forming multiple sub-grids, and marking the sub-grids containing the suspected targets; and to allocate tasks according to the prior intelligence, assigning a corresponding cluster UAV group to each marked sub-grid for close-range reconnaissance. Multiple swarm drones are grouped together, the swarm drone group containing multiple swarm drones; used to fly to their respective assigned sub-grids, search for the suspected target based on the target's rough coordinates, and perform template matching through the target image slices to complete target identification, obtain the optical reconnaissance image and accurate positioning coordinates of the suspected target and transmit them back; The ground intelligence fusion processing terminal is also used to fuse the optical reconnaissance images, the accurate positioning coordinates, and the satellite reconnaissance images transmitted back by each of the clustered UAVs to form a global target situational intelligence for target confirmation.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which executes the instructions stored in the memory to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store instructions that, when executed, cause the method as described in any one of claims 1-7 to be implemented.

Images