Multi-uav cooperative processing method and device

Abstract

This specification provides embodiments of a method and apparatus for collaborative processing of multiple unmanned aerial vehicles (UAVs). One method includes: calculating the geographic coordinates of the target location point based on measurement data from an image containing the target location point; performing an encoding retrieval on a processing area determined based on the geographic coordinates according to an index table to obtain an attribute encoding set; and determining the spatial encoding and grid image of the processing grid for the target location point in the image; reading the UAV identifier from each attribute encoding in the attribute encoding set and determining the corresponding collaborative UAV based on the UAV identifier; and sending collaborative processing data containing the spatial encoding, the grid image, and the corresponding attribute encoding to the collaborative UAV to perform collaborative processing of the target location point.

Description

Technical Field

[0001] This document relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a method and apparatus for collaborative processing of multiple UAVs. Background Technology

[0002] With the continuous development and maturation of UAV equipment and airborne sensors, UAVs are widely used in target perception and tracking, surveying and mapping, resource investigation and other scenarios, and play an important role in engineering fields such as earth science, remote sensing and search and rescue. With the improvement of the computing power of edge computing devices, the ability of airborne data processing and analysis has been significantly improved, and it has become a research hotspot in fields such as airborne real-time remote sensing observation and airborne computer vision. Therefore, how to achieve collaborative management of airborne data is a challenge faced by researchers. Summary of the Invention

[0003] This specification provides one or more embodiments of a multi-UAV collaborative processing method. The multi-UAV collaborative processing method is applied to a requesting UAV in a UAV swarm, and includes: calculating the geographic coordinates of the target location point based on measurement data from an image containing the target location point; performing an attribute code retrieval on a processing area determined based on the geographic coordinates according to an index table to obtain an attribute code set; and determining the spatial code and grid image of the processing grid for the target location point in the image; reading the UAV identifier from each attribute code in the attribute code set, and determining the corresponding collaborative UAV based on the UAV identifier; and sending collaborative processing data containing the spatial code, the grid image, and the corresponding attribute code to the collaborative UAV to perform collaborative processing of the target location point.

[0004] This specification provides one or more embodiments of another collaborative processing method for multiple drones, applied to collaborative drones in a drone swarm, comprising: receiving collaborative processing data sent by a requesting drone in the drone swarm; the collaborative processing data includes a spatial code and a grid image of a target location point in a processing grid containing the target location point, as well as an attribute code. Based on the attribute code, the target image image is read, and the target image image is cropped based on the spatial code to obtain a target grid image. The grid image and the target grid image are matched to obtain the image location point coordinates of the target location point in the target grid image. Based on the image location point coordinates, the collaborative processing result of the target location point is calculated and sent to the requesting drone.

[0005] This specification provides one or more embodiments of a multi-UAV collaborative processing device, in which a requesting UAV operates within a UAV swarm, includes: a coordinate calculation module configured to calculate the geographic coordinates of the target location point based on measurement data from an image containing the target location point; an encoding retrieval module configured to perform encoding retrieval on a processing area determined based on the geographic coordinates according to an index table, obtain an attribute encoding set, and determine the spatial encoding and grid image of the processing grid for the target location point in the image; a UAV determination module configured to read UAV identifiers from each attribute encoding in the attribute encoding set and determine the corresponding collaborative UAV based on the UAV identifiers; and a collaborative processing data sending module configured to send collaborative processing data containing the spatial encoding, the grid image, and the corresponding attribute encoding to the collaborative UAV for collaborative processing of the target location point.

[0006] This specification provides one or more embodiments of another multi-UAV collaborative processing device, operating within a UAV swarm, comprising: a collaborative processing data receiving module configured to receive collaborative processing data sent by a requesting UAV in the UAV swarm; the collaborative processing data includes a spatial code and a grid image of a target location point within an image containing the target location point, as well as an attribute code; an image cropping module configured to read the target image based on the attribute code and crop the target image based on the spatial code to obtain a target grid image; a matching processing module configured to perform matching processing on the grid image and the target grid image to obtain the image location coordinates of the target location point in the target grid image; and a collaborative processing result calculation module configured to calculate the collaborative processing result of the target location point based on the image location coordinates and send it to the requesting UAV.

[0007] This specification provides one or more embodiments of a multi-UAV collaborative processing device, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to: calculate the geographic coordinates of the target location point based on measurement data from an image containing the target location point; perform an encoding retrieval on a processing area determined based on the geographic coordinates according to an index table to obtain an attribute encoding set; and determine the spatial encoding and grid image of the processing grid for the target location point in the image; read the UAV identifier from each attribute encoding in the attribute encoding set and determine the corresponding collaborative UAV based on the UAV identifier; and send collaborative processing data containing the spatial encoding, the grid image, and the corresponding attribute encoding to the collaborative UAV to perform collaborative processing of the target location point.

[0008] This specification provides one or more embodiments of another multi-UAV cooperative processing device, comprising: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to: receive cooperative processing data sent by a requesting UAV in the UAV swarm; the cooperative processing data includes a spatial code and a grid image of a target location point in an image containing the target location point, and an attribute code; based on the attribute code, read the target image image and crop the target image image based on the spatial code to obtain a target grid image; perform matching processing on the grid image and the target grid image to obtain the image location point coordinates of the target location point in the target grid image; and calculate the cooperative processing result of the target location point based on the image location point coordinates and send it to the requesting UAV.

[0009] This specification provides one or more embodiments of a storage medium for storing computer-executable instructions that, when executed by a processor, implement the following process: calculating the geographic coordinates of the target location point based on measurement data from an image containing the target location point; performing an encoding retrieval on a processing area determined based on the geographic coordinates according to an index table to obtain an attribute encoding set; and determining the spatial encoding and grid image of the processing grid for the target location point in the image; reading the UAV identifier from each attribute encoding in the attribute encoding set and determining the corresponding cooperative UAV based on the UAV identifier; and sending cooperative processing data containing the spatial encoding, the grid image, and the corresponding attribute encoding to the cooperative UAV to perform cooperative processing of the target location point.

[0010] This specification provides one or more embodiments of another storage medium for storing computer-executable instructions that, when executed by a processor, implement the following process: receiving collaborative processing data sent by a requesting drone in the drone swarm; the collaborative processing data includes a spatial code and a grid image of a processing grid containing an image of a target location point, as well as an attribute code. Based on the attribute code, reading the target image and cropping the target image based on the spatial code to obtain a target grid image. Matching the grid image and the target grid image to obtain the image location coordinates of the target location point in the target grid image. Based on the image location coordinates, calculating the collaborative processing result of the target location point and sending it to the requesting drone. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a collaborative processing method for multiple unmanned aerial vehicles (UAVs) provided in one or more embodiments of this specification.

[0013] Figure 2 A schematic diagram of an empty index table provided for one or more embodiments of this specification;

[0014] Figure 3 A schematic diagram of a meshing result provided for one or more embodiments of this specification;

[0015] Figure 4 A schematic diagram illustrating the composition structure of an image attribute encoding provided in one or more embodiments of this specification;

[0016] Figure 5 A schematic diagram illustrating an index table update process provided for one or more embodiments of this specification;

[0017] Figure 6 A schematic diagram illustrating the processing of a captured geographic image provided in one or more embodiments of this specification;

[0018] Figure 7 A schematic diagram illustrating a process for processing received information provided in one or more embodiments of this specification;

[0019] Figure 8 A schematic diagram illustrating the process of sending collaborative task request information as provided in one or more embodiments of this specification;

[0020] Figure 9 An index table and an image coverage situation diagram are provided for one or more embodiments of this specification;

[0021] Figure 10 A schematic diagram illustrating an attribute encoding set query process provided in one or more embodiments of this specification;

[0022] Figure 11 A schematic diagram illustrating a collaborative processing method for multiple UAVs applied in a collaborative positioning scenario, provided by one or more embodiments of this specification.

[0023] Figure 12A timing diagram illustrating the processing of a multi-UAV collaborative processing method for a collaborative positioning scenario, provided in one or more embodiments of this specification.

[0024] Figure 13 This document provides a flowchart of the query processing in a collaborative processing method for multiple UAVs applied in a collaborative positioning scenario, as illustrated in one or more embodiments of this specification.

[0025] Figure 14 A flowchart illustrating another collaborative processing method for multiple unmanned aerial vehicles (UAVs) provided in one or more embodiments of this specification;

[0026] Figure 15 A flowchart illustrating a collaborative tracking method for multiple unmanned aerial vehicles (UAVs) provided for one or more embodiments of this specification.

[0027] Figure 16 A flowchart illustrating another collaborative tracking method for multiple unmanned aerial vehicles (UAVs) provided in one or more embodiments of this specification.

[0028] Figure 17 A flowchart illustrating an image processing method for multiple unmanned aerial vehicles (UAVs) provided for one or more embodiments of this specification.

[0029] Figure 18 A schematic diagram illustrating an embodiment of a multi-UAV collaborative processing device provided in one or more embodiments of this specification;

[0030] Figure 19 A schematic diagram of another embodiment of a multi-UAV collaborative processing device provided in one or more embodiments of this specification;

[0031] Figure 20 A schematic diagram of the structure of a multi-UAV collaborative processing device provided for one or more embodiments of this specification;

[0032] Figure 21 This is a schematic diagram of another multi-UAV collaborative processing device provided in one or more embodiments of this specification. Detailed Implementation

[0033] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0034] This specification provides one or more embodiments of a collaborative processing method for multiple unmanned aerial vehicles (UAVs) as follows:

[0035] In practical applications, low-altitude (mission altitude 100m-1000m) UAVs are characterized by miniaturization, high flexibility, and low cost. They can provide a variety of optical image data and radar data with higher temporal and spatial resolution, and are an effective supplement to remote sensing for Earth observation. However, due to limited flight time, number of sensors, and computing power, the Earth perception capability of a single UAV is limited.

[0036] In the process of processing data through drone swarms, the collaborative management and sharing of airborne imagery data among multiple drones is difficult. This greatly limits the development of multi-drone collaborative perception and data processing. The main reasons for the difficulties in the collaborative management and application of airborne imagery data include:

[0037] (1) The limited communication bandwidth of drone swarms makes it difficult to transmit and share airborne image data in real time within the drone swarm.

[0038] In practical applications, the communication bandwidth of drones using self-organizing network technology is about 20Mbps. With the support of 5G communication networks, the communication rate can be increased to 1Gbps. However, the amount of airborne image data is large, and existing communication networks are difficult to achieve real-time transmission of image data within a drone swarm. For example, the maximum video bitrate of the DJI Phantom 4 Pro drone is 100Mbps. Even with a 5G communication network, it can only support video data transmission from a maximum of 10 drones, which limits the number of drones in a drone swarm.

[0039] (2) There is a lack of effective collaborative management solutions for airborne image data among drones in drone swarms.

[0040] Even if the communication bandwidth can meet the requirements for image data transmission, the image data will be repeatedly stored by drones in the drone swarm after transmission, wasting the drone's onboard storage resources.

[0041] Based on this, this embodiment proposes a collaborative management scheme for airborne image data applicable to multiple UAVs based on a global discrete grid system, and applies it to collaborative image target processing.

[0042] This embodiment provides a collaborative processing method for multiple drones. When collaborative processing of a target location point is required, the requesting drone in the drone swarm performs encoding retrieval of the processing area containing the target location point according to an index table to obtain an attribute encoding set. It also determines the spatial encoding and grid image of the processing grid in the acquired image containing the target location point, identifies the cooperating drone corresponding to each attribute encoding in the attribute encoding set, and sends a collaborative processing request containing the spatial encoding, grid image, and corresponding attribute encoding to the cooperating drone to perform collaborative processing of the target location point. This reduces the requirement for drone communication bandwidth and improves the efficiency of joint processing of image images acquired by drones in the drone swarm.

[0043] Reference Figure 1 The multi-drone collaborative processing method provided in this embodiment is applied to requesting drones in a drone swarm, specifically including steps S102 to S108.

[0044] In this embodiment, the drones in the drone swarm communicate with each other through a drone swarm communication network. The drone swarm communication network typically adopts a self-organizing network or a star topology. The multi-drone collaborative processing method provided in this embodiment is applicable to the aforementioned self-organizing network and star topology, and can also be applied to other types of communication networks.

[0045] Step S102: Calculate the geographic coordinates of the target location point based on the measurement data of the image containing the target location point.

[0046] The imagery includes airborne imagery captured by the UAV; the measurement data includes POS data of the imagery and / or the UAV's altitude; the geographic coordinates of the target location include the ground coordinates of the target location. In practice, if the imagery contains the target location, the geographic coordinates of the target location are calculated based on the measurement data of the imagery containing the target location.

[0047] The collaborative processing method for multiple UAVs in this embodiment is based on an efficient image data indexing and querying method suitable for airborne environments. Furthermore, by uniformly encoding image metadata, it provides an encoding basis for the image data indexing of UAVs in the UAV swarm. Therefore, before specifically describing the collaborative processing process of target location points, this embodiment first provides a detailed explanation of the encoding of image data and the construction of image data indexes.

[0048] In this embodiment, based on the global discrete grid system, the image data of the UAV swarm is encoded, combined, and indexed, providing a unified and efficient spatial encoding, calculation, and analysis method. This helps to achieve collaborative management, processing, and analysis of multi-UAV imagery on an onboard computer with limited hardware resources.

[0049] To achieve collaborative management of drone swarms, the following preparations should be made for the drones before flight:

[0050] (1) Set up the flight mission and flight plan for the drone swarm, complete the multi-drone path planning before flight, and import historical imagery and ground data of the flight area into the computing platforms of the drones and users.

[0051] Optional, ground data includes DEM data. DEM data represents the bare ground surface, after removing all natural and architectural features from the ground.

[0052] (2) Determine the appropriate image gridding level for the image coverage area based on the size of the flight area, the parameters of the UAV camera, and the performance of the onboard computer.

[0053] The image meshing hierarchy includes the hierarchy in which the meshes representing the image are meshed are located.

[0054] To improve the accuracy of the determined image meshing level, in one optional implementation of this embodiment, the image meshing level is calculated in the following manner:

[0055] Calculate the grid level of the geographic image based on the image area and the grid area of ​​the geographic image.

[0056] The image meshing level is calculated based on the mesh level and the preset depth.

[0057] Optionally, the geographic imagery includes images acquired by a drone. The grid hierarchy includes the level at which the grid is located when the image area is comparable to the grid area.

[0058] Specifically, the image gridding level is calculated based on the image area, the layer grid area, and the preset depth.

[0059] Optionally, the image area refers to the area of ​​the image captured by the drone; the hierarchical grid area refers to the area of ​​the grid at the corresponding level after calculation based on the preset hierarchical grid scale. It should be noted that the grid size is the same at each level; the preset depth is a preset value.

[0060] in, L G Indicates the image gridding level. Indicates the image area S img With grid area S G The corresponding mesh level, d represents the preset depth, specifically expressed as

[0061] For example, the image area is approximately 0.01 square kilometers, d = 3. Combining the directional and northward scales of the grids from levels 16 to 31 shown in Table 1, we know that the grid area of ​​level 19 is 0.063408 × 0.062801 = 0.00398 square kilometers, and the grid area of ​​level 18 is 0.126817 × 0.125601 = 0.0159 square kilometers. Since 0.00398 < 0.01 < 0.0159, the calculated...

[0062]

[0063]

[0064] Table 1. Grid scales from level 16 to level 31 (unit: meters)

[0065] It should be noted that, in order to meet the needs of global UAV imagery data organization and management, a spatial grid subdivision method based on Web Mercator projection and a spatial grid coding method based on multi-scale Morton curves are adopted. Taking the area near point (113°E, 34°N) as an example, Table 1 lists the directional and northward scales of the grids from level 16 to level 31. It can be seen that the Earth grid subdivision model corresponding to the spatial grid subdivision method based on Web Mercator projection includes multi-scale grids from global scale to centimeter scale, and therefore can be applied to most imagery data application scenarios.

[0066] (3) Establish and test a multi-UAV networking communication network, as well as communication links between UAVs and / or ground stations and users.

[0067] (4) Create empty index tables on the computing platforms of the UAV, ground station, and user terminal respectively.

[0068] Optionally, the index table adopts a key-value structure, using spatial encoding as the primary key and attribute encoding as the value, thereby creating spatial, temporal, and attribute indexes for the image. This index table can be stored in computer memory or a lightweight database, such as SQLite (relational database management system) and Redis (Remote Dictionary Server). An empty index representation is as follows... Figure 2 As shown.

[0069] The above details the configuration process and content of drones in a drone swarm before flight. The following section uses a requesting drone as an example to illustrate the execution process of drone flight within a drone swarm. It should be noted that all drones in the swarm perform the following operations during flight. It should also be noted that the encoding and meshing in this embodiment are based on a global discrete grid system.

[0070] In one optional implementation of this embodiment, the drone is requested to perform the following operation during flight:

[0071] (1) Acquire geographic images and perform spatial encoding and attribute encoding on the geographic images to obtain corner spatial codes, range spatial codes, and image attribute codes.

[0072] During the actual execution process, according to the tasks set before the flight, the camera module acquires geographic images, encodes the geographic images to obtain image codes, and converts the vector polygons of the covered area of ​​the image into spatial codes to realize the spatial association between the image and the grid.

[0073] Image coding includes spatial coding and attribute coding. Spatial coding (SCode) is used to record the geographic location information of the image coverage area, the four corner points (endpoints) of the image, or the image target. Attribute coding (ACode) is used to record the acquisition time, source, type, image gridding level, and other attributes of the image.

[0074] In one optional implementation of this embodiment, the spatial encoding process includes:

[0075] Calculate the corner coordinates of the corner points in the geographic image;

[0076] Based on the corner point's position coordinates and spatial coding level, calculate the corner point's coding index;

[0077] The corner spatial code of the corner point is calculated based on the coding index and the spatial coding level.

[0078] Optionally, the encoding index includes row and column numbers.

[0079] In the specific implementation process, for corner spatial coding, the corner position coordinates in the geographic image are first calculated, then the corner coding index is calculated based on the corner position coordinates and spatial coding level, and finally the corner spatial code is calculated based on the coding index and spatial coding level.

[0080] In calculating the corner coordinates, the corner coordinates in the geographic image are calculated based on the measurement data of the geographic image. Specifically, coordinate transformation is performed based on the measurement data of the geographic image to obtain the latitude and longitude coordinates of the corner. Alternatively, the corner coordinates can be calculated based on the measurement data of the geographic image and preset DEM data. Specifically, the test data is corrected according to ground data to obtain corrected measurement data, and coordinate transformation is performed based on the corrected measurement data to obtain the latitude and longitude coordinates of the corner. Optionally, the measurement data of the geographic image is obtained from the UAV's POS system; additionally, the UAV's altitude can be obtained based on the UAV's barometer.

[0081] If the geographical location of point A in the calculated geographic image is (lng, lat), in the process of calculating the corner spatial code of point A, the longitude row and column number I and the latitude row and column number J are calculated first; where:

[0082]

[0083]

[0084] After obtaining the longitude row and column number I and the latitude row and column number J, the corner spatial code is calculated based on the longitude row and column number, the latitude row and column number, and the default spatial coding level L=31:

[0085]

[0086] Where L=31 is the smallest level code with the smallest grid area, so that L=31 can make the positional accuracy of the calculated corner spatial code better than 2 cm; the row and column numbers are calculated using the tms mode; This represents the cross-biting operation in the Morton encoding method.

[0087] In addition to the determination of corner spatial encoding as described above, in an optional implementation of this embodiment, the spatial encoding process further includes:

[0088] The geographic image is processed by gridding based on the image gridding hierarchy to obtain at least one single-scale grid.

[0089] The at least one single-scale grid is aggregated to obtain at least one multi-scale grid, and the spatial code of each grid in the at least one multi-scale grid is used as the range spatial code.

[0090] Specifically, the geographic image is gridded based on the image gridding hierarchy to obtain at least one single-scale grid. In order to improve the efficiency of spatial coding processing and reduce the number of spatial codes obtained, after obtaining at least one single-scale grid, the at least one single-scale grid is aggregated to obtain at least one multi-scale grid. Then, the spatial codes of each grid in the at least one multi-scale grid are used as the range spatial codes of the geographic image.

[0091] Optionally, a single-scale grid means that all grids are of the same size; a multi-scale grid means that it contains grids of multiple scales with inconsistent sizes. In one optional implementation of this embodiment, in the process of aggregating at least one single-scale grid to obtain at least one multi-scale grid, the single-scale grids that satisfy the quadtree relationship are first identified, and then aggregating these single-scale grids to obtain at least one multi-scale grid. That is, single-scale grids with the same parent node (quadtree node) are aggregated to obtain a multi-scale grid.

[0092] like Figure 3 The meshing result shown is Figure 3 (a) represents a single-scale grid obtained after meshing according to the image meshing hierarchy; the single-scale grid is then aggregated to obtain, as shown below. Figure 3 (b) shows the multi-scale meshing results.

[0093] After obtaining the multi-scale grid, the spatial code of each grid in the multi-scale grid is used as the extent spatial code of the geographic image. In an optional implementation of this embodiment, the spatial code of each grid is obtained in the following way:

[0094] Calculate the spatial code of the maximum level, and based on the spatial code of the maximum level, calculate the spatial code of each level;

[0095] The grid level of each grid is determined, and the spatial code of each grid is read based on the grid level.

[0096] Specifically, based on a 31-level spatial coding system, the next level's coding is obtained by averaging the values ​​of every four adjacent grid cells using a quadtree between spatial codes. This process is repeated for all levels of spatial coding. After obtaining all levels of spatial coding, the spatial code of each grid cell is read according to its corresponding grid level in the multi-scale grid. In calculating the 31st level spatial coding, based on the characteristics of Z-shaped coding, the horizontal and vertical coordinates of the grid cells are converted to integer codes using a cross-biting method. For example, the calculation steps for the single-scale integer code corresponding to grid coordinates (20, 17) are as follows: the binary code bits corresponding to grid coordinates (20, 17) are (10100, 10001); cross-biting yields 1100010010; resulting in the single-scale integer code 786. The above single-scale integer coding cannot simultaneously express multi-scale information. Since the multi-scale integer coding is based on single-scale integer coding, shifting the single-scale integer code value one bit to the left yields the spatial code of the highest level (31st level) in the multi-scale integer coding. The spatial encoding of each grid is recorded by a 64-bit unsigned integer, which has integer properties, recursion, and high clustering with spatial scaling.

[0097] In addition to performing spatial encoding on the geographic image to obtain corner spatial encoding and range spatial encoding, this embodiment also performs attribute encoding on the geographic image to obtain image attribute encoding.

[0098] To improve the effectiveness of image attribute codes obtained through attribute encoding processing and ensure the unique identification function of image attribute codes for geographic imagery, this embodiment provides an optional implementation in which the attribute encoding processing includes:

[0099] Read the attribute data from the metadata of the geographic image;

[0100] The geographic image is subjected to attribute encoding processing based on the attribute data, image gridding level and preset encoding algorithm to obtain the attribute encoding of the geographic image.

[0101] Optionally, the attribute data includes at least one of the following: image acquisition time, image frame number, drone number, and image type.

[0102] Optionally, the image attribute encoding in this embodiment includes the image acquisition time tm, image frame number fr, UAV number id, image data type ty, and image meshing level L. G A hybrid encoding consisting of six fields, including the other attribute ot; such as Figure 4 As shown, tm, fr, id, and ty can be extracted from the metadata of geographic imagery.

[0103] Among them, tm is used to record the timestamp of the geographic image, with a maximum precision of 1 second; fr is used to record which frame the image belongs to within a certain second. In shooting or recording mode, the imaging rate of the UAV image sensor is usually no higher than 60fps, so fr can be equal to 6 bits; id is used to record the UAV number. UAV swarms performing tasks such as ground observation and target tracking usually use a single-frequency communication network, and the UAVs' local area network IP addresses belong to the same network segment. Therefore, the UAV number can be uniquely identified by the last byte of the IPv4 address. Optionally, id can be recorded by 8 bits; ty is used to record the image type. For example, 0, 1, and 2 are used to represent visible light, infrared, and hyperspectral images, respectively. Ot is a reserved field for other attributes, such as the scene category of the image. If this field is not used, ot takes the value of 0.

[0104] If the attribute data of a geographic image is known, the image attribute code of the geographic image can be calculated using simple bitwise operations and addition operations:

[0105] ACode=(tm<<32)+(fr<<26)+(id<<18)+(ty<<15)+(L G <<10)+ot

[0106] To achieve highly efficient attribute encoding processing.

[0107] The attribute encoding process described above ensures that the resulting image attribute codes have integer characteristics, facilitating storage and retrieval. Furthermore, each geographic image has a unique attribute code. The first 32 bits of the image attribute code are a timestamp, allowing for the creation of a time index for the geographic image based on the timestamp and the sorting of the image attribute codes, enabling time-based image lookups. Additionally, the hexadecimal string of the image data encoding can be used as the image file name, imbuing it with rich attribute information and establishing a correspondence between image attribute codes and image files. In this case, since the storage path of image files is usually fixed, users or programs can easily extract the corresponding image file using the image attribute code.

[0108] It should be noted that in this embodiment, the obtained image attribute codes correspond one-to-one with the geographic image images; the geographic image images and the range spatial codes have a one-to-many relationship, that is, the geographic image images include several multi-scale grids, and the range spatial codes include several.

[0109] (2) Update the index table based on the range spatial encoding and the image attribute encoding, and compress the corner spatial encoding and the image attribute encoding to obtain compressed encoding.

[0110] In this embodiment, the geographic imagery is encoded to update the index table stored by the UAV based on spatial range encoding and image attribute encoding. Therefore, after obtaining the spatial range encoding and image attribute encoding, the index table is updated based on these encodings.

[0111] In one optional implementation of this embodiment, the process of updating the index table based on range spatial encoding and image attribute encoding is implemented in the following way:

[0112] Check whether the primary key of the index table contains the range space code;

[0113] If so, the image attribute encoding is mapped to the key value of the range space encoding as the primary key;

[0114] If not, the index table will be updated with the range space code as the primary key and the image attribute code as the key value.

[0115] Specifically, in the process of updating the index table based on range space encoding and image attribute encoding, if the primary key in the stored index table already contains range space encoding, then the image attribute encoding is mapped to the key value of the corresponding primary key. If the primary key does not contain range space encoding, then a mapping relationship between range space encoding and image attribute encoding is established, and based on the mapping relationship, the index table is updated with the range space encoding as the primary key and the image attribute encoding as the key value.

[0116] like Figure 5 As shown in (a), the image attribute encoding of the geographic image is ACode. n+1 The corresponding range space encodings include SCode2, SCode3, SCode4, SCode5, and SCode6. m-1 SCode m SCode m+1 SCode m+2 The index table shows that SCode2 and SCode already exist. m These two primary keys will then be ACode n+1 Mapping to SCode2 and SCode m The key values ​​of these two primary keys, such as Figure 5 As shown in (b), SCode3, SCode4, SCode5, and SCode do not exist in the index table. m-1 SCode m+1 SCode m+2 These primary keys are then created as SCode3, SCode4, SCode5, and SCode, respectively. m-1 SCode m+1SCode m+2 With ACode n+1 The mapping relationship, and respectively using SCode3, SCode4, SCode5, SCode m-1 SCode m+1 SCode m+2 Primary key, ACode n+1 Update the index table for key-value pairs, such as Figure 5 As shown in (b). It should be noted that, Figure 5 (b) In the key column, ... represents other primary keys, such as SCode6, and in the value column, ... represents other key values, such as ACode1. An index can be built using B-numbers and their variants to create a one-dimensional spatial grid encoding in the key column.

[0117] After obtaining the corner spatial code, range spatial code, and image attribute code, to achieve data sharing, the requesting UAV needs to share the collected geographic impact image data with other candidate UAVs. To overcome the disadvantage of limited communication bandwidth, only the codes are shared with candidate UAVs, rather than directly sharing the image data. To further overcome the disadvantage of limited communication bandwidth, since the range spatial code is relatively large, the corner spatial code and image attribute code are shared with candidate UAVs, enabling them to update their stored index tables based on these codes. Furthermore, to further conserve communication bandwidth, the corner spatial code and image attribute code are compressed to obtain compressed codes. These compressed codes include compressed image codes.

[0118] (3) The compressed code is sent to the candidate drones in the drone swarm other than the requesting drone to update the index table of each candidate drone.

[0119] After the corner spatial encoding and image attribute encoding are compressed to obtain the compressed code, the compressed code is sent to the candidate drones in the drone swarm other than the requesting drone to update the index table of each candidate drone.

[0120] In one optional implementation of this embodiment, any candidate drone other than the requested drone performs the following operation:

[0121] Receive the compressed code sent by the requesting drone and decompress the compressed code to obtain the corner spatial code and the image attribute code;

[0122] The corner spatial code is parsed to obtain the range spatial code;

[0123] Based on the range spatial encoding and the image attribute encoding, the index table of any candidate UAV is updated.

[0124] Specifically, candidate drones (excluding the requesting drone) receive the compressed code sent by the requesting drone and decompress it to obtain corner spatial codes and image attribute codes. The corner spatial codes are then parsed to obtain range spatial codes. Based on the range spatial codes and image attribute codes, the index table of candidate drones is updated. Knowing the corner spatial codes allows determination of the coverage area of ​​the corresponding geographic image, and the range spatial codes of the geographic image can be determined based on the coverage area.

[0125] This is equivalent to determining the image based on four points and then performing spatial coding on the image to obtain the range spatial code. The determination method for the range spatial code is similar to that described above; please refer to the relevant content mentioned earlier.

[0126] It should be noted that updating the index table of candidate drones is related to... Figure 5 The process of updating the index table for the requesting drone is similar and can be referred to the above-mentioned content; this embodiment is not limited thereto. It should also be noted that the candidate drone also acquires geographic imagery and performs the above-mentioned processes during flight, and the requesting drone also receives the compressed code sent by the candidate drone and updates its stored index table.

[0127] In summary, such as Figure 6 The diagram illustrates the processing of captured geographic images. During flight, the camera module of any UAV captures images and generates standard and compressed image codes using the image encoding generation function. Based on these, the index update function updates the stored index table using the standard image codes, while the information sending function sends the compressed image codes. The standard image codes include range spatial encoding and image attribute encoding, while the compressed image codes include compressed corner spatial encoding and compressed image attribute encoding.

[0128] like Figure 7The diagram illustrates the processing of received information. After receiving information from other drones in the drone swarm, the communication module of any drone uses the information parsing function to parse and process the received information, obtaining the range spatial code and image attribute code. It then uses the index update function to update the stored index table based on the range spatial code and image attribute code. Furthermore, if the parsed information includes collaborative task request information, the task execution function is used to execute the task based on the collaborative task request information, and the information sending function returns the task execution result to the drone that sent the received information. Additionally, the collaborative task request result can also be obtained after parsing.

[0129] In addition, the drone also includes a request information generation function. For example... Figure 8 The diagram illustrates the process of sending collaborative task request information. After receiving a data processing task, any UAV determines whether the data processing task requires other UAVs to perform collaboratively. If so, it generates collaborative task request information using the request information generation function and sends the collaborative task request information to the collaborating UAVs using the information sending function.

[0130] It is evident that the drones in a drone swarm include at least the following functions: image encoding generation, index updating, information transmission, information parsing, task execution, and request information generation.

[0131] The image encoding generation function implements the aforementioned spatial encoding and attribute encoding processes, generating standard and compressed image encodings. Furthermore, this function modifies image file names to hexadecimal attribute encoding values, establishing a link between image files and image attribute encodings. The standard image encoding is used to update the index table; the compressed image encoding is shared with other endpoints (drones, ground stations, and / or users) using the information transmission function. Optionally, the compressed image encoding includes one image attribute encoding and four corner point spatial encodings, occupying a total of 40 bytes. Latitude and longitude are typically recorded using double (8 bytes) or Decimal (10,7) (10 bytes) data types. Therefore, when the spatial information of an image is recorded using latitude and longitude, the image information includes one attribute encoding and four pairs of latitude and longitude coordinates, occupying at least 72 bytes. It is evident that image encoding occupies less storage space, which helps reduce the bandwidth requirements for drone communication.

[0132] The index update function updates the standard image encoding according to... Figure 5 Insert the index table as shown to update the index table. In addition, after updating the index table, the image coverage status map also needs to be updated synchronously.

[0133] The information sending function determines which receivers are available based on the type of information to be sent, and then transmits the information to the receivers via the drone swarm's communication network. Optionally, if the information type is an encoding type, the interface is determined to be another drone besides the one being sent; if the information type is a collaborative task request type, the interface is determined to be the drone corresponding to the drone identifier in the attribute information to be sent; if the information type is a collaborative task result type, the receiver is determined to be the drone that sent the collaborative task request information.

[0134] The information parsing function is used to parse the received information (compressed image encoding, collaborative task request information, and collaborative task request results). Optionally, for compressed image encoding, the above spatial encoding process is used to convert the compressed image encoding into its standard version; the collaborative task request information or collaborative task request results can be converted into their formats.

[0135] The task execution function is used to perform corresponding tasks based on the collaborative task request information, using stored images, and to generate collaborative task request results, which are then sent to the information sending function.

[0136] The request information generation function is used to generate collaborative task request information based on various collaborative needs.

[0137] The functions of the drones in the drone swarm have been described in detail above. It should be noted that, in addition to the functions described above, the drones in the drone swarm may also be configured with other communication functions, which are not limited in this embodiment.

[0138] The above only describes the data interaction between drones in a drone swarm. The interaction process between drones in a drone swarm and the ground station and user terminal is similar to the above process. If interaction is required, it can be done in the manner described above. In addition, the user terminal can establish communication with the central node or ground station to receive various data and computing services provided by the drone swarm in real time, and command the drone swarm to complete tasks such as image capture.

[0139] In this embodiment, to more intuitively view the image coverage situation, the image coverage status map is displayed and updated according to the index table; in an optional implementation method provided in this embodiment, the image coverage status map is obtained in the following way:

[0140] An initial image coverage status map is rendered based on historical imagery and a global discrete grid system; the image grid in the initial image coverage status map is the first display state.

[0141] Read at least one range spatial code from the index table, update the display state of the image grid corresponding to the at least one range spatial code in the initial image coverage situation map to the second display state, and obtain the intermediate image coverage situation map;

[0142] Read the attribute codes mapped by each range spatial code in at least one range spatial code in the index table, associate the geographic image image associated with the attribute code with the corresponding image grid, and obtain an image coverage status map.

[0143] In the specific execution process, during the rendering of the image coverage status map, the initial image coverage status map is first rendered as a first display state based on historical imagery (i.e., historical image tiles) and the global discrete grid system. Then, the spatial encoding of the range in the index table is read, and the display state of the image grid corresponding to the spatial encoding in the initial image coverage status map is updated to a second display state, obtaining an intermediate image coverage status map. To provide a more intuitive understanding of the coverage situation, the attribute encoding mapped to each spatial encoding in the index table is read, and the associated geographic imagery is read. The geographic imagery is then associated with the corresponding image gridry for display, obtaining the image coverage status map. Optionally, the first display state includes a grayscale display state; the second display state includes a color display state.

[0144] Specifically, if a grid is already covered by UAV imagery, the historical image tiles within that grid are displayed in color, and the latest acquired image is overlaid on that grid. Otherwise, the historical image tiles are covered by a semi-transparent gray grid. It should be noted that the image coverage situation map in this embodiment is constructed based on a global discrete grid system, and uses historical image tiles as the foundation for its construction. This not only results in a good visualization effect for the image coverage situation map, but also provides assistance for image coverage analysis, ground feature change detection, and UAV imagery positioning in GNSS (Global Navigation Satellite System) denied environments.

[0145] like Figure 9 The index table and image coverage status map shown are as follows, in which, Figure 9 In the image coverage map of (b), areas not covered by UAV imagery are represented by semi-transparent gray grids; areas covered by UAV imagery are represented by color-coded grids. Figure 9 SCode in (a) m For example, Figure 9 The spatial encoding of the grid marked in (b) is SCode. m In the index table, SCode mThis primary key is associated with ACode1, ACode2, ..., ACode... n-1 and ACode n These key-value pairs are then associated with the grid of the marker for rendering, where the ACode closest to the current time is used. n It should be placed in the first position, and then arranged in order.

[0146] As can be seen, when updating the key values ​​of the index table, they are stored in ascending order based on the timestamps contained in the attribute encoding.

[0147] It should be noted that the image coverage situation map can be updated and stored after each update of the index table, or it can be constructed based on the index table when a situation map viewing request is detected. This embodiment does not limit this.

[0148] The above describes the encoding and processing of geographic imagery, as well as the related content on collaborative airborne data management based on index tables. Furthermore, the aforementioned encoding and processing techniques and collaborative airborne data management can be applied to specific tasks to be performed.

[0149] In specific implementation, taking the task to be processed for the target location as an example, during the task processing, the requesting drone in the drone swarm acquires the collected image containing the target location, and calculates the geographical coordinates of the target location based on the measurement data of the image.

[0150] Optionally, the requesting drone may include a central node drone that establishes communication with the user terminal.

[0151] It should be noted that both the imagery and the aforementioned geographic imagery are images collected by drones, and the processing methods are similar to those for the aforementioned geographic imagery. Please refer to the relevant content above.

[0152] That is, after acquiring an image containing the target location point, and calculating the geographic coordinates of the target location point based on the measurement data of the image, steps S104 to S108 are executed. Meanwhile, spatial encoding and attribute encoding are performed on the image to obtain corner spatial encoding, range spatial encoding, and image attribute encoding. The index table is updated based on the range spatial encoding and the image attribute encoding. The corner spatial encoding and the image attribute encoding are then compressed to obtain compressed encoding. The compressed encoding is sent to candidate drones in the drone swarm other than the requesting drone to update the index tables of each candidate drone.

[0153] In the specific execution process, in order to improve the calculated geographic coordinates of the target location point, in an optional implementation method provided in this embodiment, the following operations are performed during the calculation of the geographic coordinates of the target location point based on the measurement data of the image:

[0154] Based on the measurement data, coordinate transformation is performed to obtain the latitude and longitude coordinates of the target location point;

[0155] or,

[0156] Based on ground data, the measurement data is corrected to obtain corrected measurement data;

[0157] Based on the measurement correction data, coordinate transformation is performed to obtain the latitude and longitude coordinates of the target location point.

[0158] Specifically, similar to the calculation process of the latitude and longitude coordinates of the aforementioned corner point, in the process of calculating the latitude and longitude coordinates of the target location point, if there is no ground data, the coordinates are transformed based on the POS data obtained by the POS system and the UAV flight altitude obtained by the barometer to obtain the latitude and longitude coordinates of the target location point; if there is ground data, the geographical coordinates of the target location point are calculated based on the POS data and the ground data.

[0159] Step S104: According to the index table, the processing area determined based on the geographic location coordinates is encoded and retrieved to obtain the attribute encoding set, and the spatial encoding and grid image of the processing grid of the target location point in the image are determined.

[0160] In this embodiment, to improve the accuracy and comprehensiveness of the task processing results for the target location point, a processing area is determined based on the geographic coordinates of the target location point, and task processing for the target location point is performed based on the processing area. The processing area includes a defined range containing the target location point. The processing grid includes the grid where the target location point is located; the grid image includes the grid where the target location point is located, obtained by cropping. It should be noted that the level corresponding to the processing grid can be preset or randomly determined; this embodiment does not impose any limitations on this.

[0161] Optionally, the index table consists of at least one index with a range space encoding as the primary key and an attribute encoding as the key value; a primary key maps to at least one key value, and the at least one key value includes the attribute encoding of each drone in the drone swarm that is mapped to the primary key.

[0162] In specific implementation, after calculating and obtaining the geographic coordinates of the target location, the processing area of ​​the target location is first determined. Then, the processing area is encoded and retrieved according to the index table to obtain the attribute code set. Therefore, in order to improve the effectiveness and comprehensiveness of task processing for the target location, in an optional implementation of this embodiment, the following operation is performed during the process of encoding and retrieving the processing area determined based on the geographic coordinates according to the index table to obtain the attribute code set:

[0163] The processing area is obtained by dividing the region using the geographical coordinates as the region center and a preset length as the division length;

[0164] The processing area is encoded and retrieved according to the index table to obtain an attribute encoding set.

[0165] Specifically, the region is divided with the target location point as the center and a preset length as the radius to obtain a processing area. Then, the processing area is encoded and retrieved according to an index table to obtain an attribute code set. Optionally, the preset length is equal to the maximum value of the target positioning plane error in the process of determining the geographic location coordinates.

[0166] To improve the accuracy of the determined attribute code set, in one optional implementation of this embodiment, the following operations are performed during the process of obtaining the attribute code set by performing an encoding retrieval of the processing area according to the index table:

[0167] Spatial encoding is performed on the processing area to obtain the regional spatial encoding of the processing area, and the encoding interval of each grid spatial encoding in the regional spatial encoding is calculated according to a preset interval algorithm;

[0168] Read the primary key located in the encoding range from the index table, and read the key value mapped by the primary key;

[0169] The key values ​​are filtered, and the attribute code set is constructed based on the filtered key values.

[0170] Specifically, in the process of encoding and retrieving the processing area determined by the geographic location coordinates according to the index table to obtain the set of encoded attributes, the area is first divided with the geographic location coordinates as the center and a preset length as the radius to obtain the processing area of ​​the target location point. Then, based on the above spatial encoding processing method, the processing area is spatially encoded to obtain the regional spatial code of the processing area. Then, the encoding interval of each grid spatial code in the regional spatial code is calculated according to the preset interval algorithm. The primary key located in the encoding interval is read from the index table, and the attribute encoding set is constructed based on the key value mapped by the primary key.

[0171] The regional spatial coding is similar to the range spatial coding described above; since the regional spatial coding includes the spatial coding of each grid in the multi-scale grid within the region, the coding interval of each grid is calculated for each grid spatial coding.

[0172] like Figure 10 As shown, based on the POS data of the image where the target location point P is located and the UAV flight altitude or DEM data, the geographic coordinates of the target location point P are calculated as (lng1, lat1, alt1) using the single-image positioning method. A circular area with the target location point P as the center and the maximum target positioning plane error of the single-image positioning method as the radius is determined as the processing area of ​​the target location point P. Spatial coding is then performed on the processing area to obtain the interval spatial code Ω of the processing area. s ={…,SCode i Then calculate SCode according to the following formula. i (SCode i ∈Ω s The encoding interval U = {…, U i , ...}:

[0173]

[0174]

[0175]

[0176] L = 31. After obtaining the encoding range, query the index table for all keys within the range of U to obtain the primary key set. Then, extract the key values ​​corresponding to the primary key set and delete the duplicates to obtain the attribute encoding set, which contains the image attribute encodings within the processing area.

[0177] It should be noted that, based on the attribute code set, it is possible to query the time and multiple attributes of the data. In addition, the primary key set and attribute code set can be used to query the image coverage status within the processing area.

[0178] like Figure 10 The attribute code set query process shown first involves gridding the processing area to obtain the regional spatial code of the processing area, which is composed of the grid spatial codes of each grid. Then, the code interval is calculated to obtain the code interval. Finally, the index table is queried to find the key that belongs to the code interval. Then, the value corresponding to the key is extracted, and duplicate values ​​are deleted to obtain the final attribute code set.

[0179] In the specific execution process, in order to improve the collaborative processing effect of multiple UAVs, on the one hand, the collaborative UAVs are determined by determining the attribute encoding set; on the other hand, the spatial encoding and grid image of the processing grid of the target location point in the image are determined and sent to the collaborative UAVs for auxiliary task processing. In an optional implementation of this embodiment, the spatial encoding and grid image of the processing grid of the target location point in the image are determined in the following way:

[0180] In the image, the spatial encoding of the processing grid containing the target location point is determined;

[0181] Based on the spatial encoding, the image corresponding to the processing grid is cropped from the image to become the grid image.

[0182] Specifically, the spatial encoding of the processing grid containing the target location points is determined, and the image corresponding to the processing grid is cropped from the image to form a grid image.

[0183] Using the previous example, the spatial code of the processing grid where the non-rigid target location point P is located is SCode. The grid corresponding to SCode is cropped from the image to form the grid image Patch1 containing the target location point P.

[0184] It should be noted that step S104 can also be replaced by performing an encoding retrieval on the processing area determined based on the geographic coordinates according to the index table to obtain an attribute encoding set, and / or determining the spatial encoding and grid image of the processing grid of the target location point in the image, thus forming a new implementation method with other processing steps provided in this embodiment. For example, during cooperative localization processing, the encoding retrieval on the processing area determined based on the geographic coordinates according to the index table to obtain an attribute encoding set, and determining the spatial encoding and grid image of the processing grid of the target location point in the image; during cooperative tracking processing, determining the spatial encoding and grid image of the processing grid of the target location point in the image is sufficient. Furthermore, determining the spatial encoding and grid image of the processing grid of the target location point in the image in step S104 can also be replaced by determining the spatial encoding and grid image of the processing grid of the processing area in the image.

[0185] Step S106: Read the drone identifier from each attribute code in the attribute code set, and determine the corresponding cooperative drone based on the drone identifier.

[0186] The above-mentioned index table is used to encode and retrieve the processing area determined based on the geographic coordinates to obtain the attribute code set, as well as to determine the spatial code and grid image of the processing grid of the target location point in the image. In this step, based on the obtained attribute code set, the attribute code set is parsed to determine the cooperative UAV.

[0187] In practice, each attribute code in the attribute code set records the drone identifier. Therefore, the collaborative drone is determined based on the drone identifier in each attribute code in the attribute code set. For example, the obtained attribute code set {…, ACode} n ...}, where, for ACode n Decode the data to obtain the drone ID = n.

[0188] Step S108: Send collaborative processing data containing the spatial code, the grid image, and the corresponding attribute code to the collaborative UAV to perform collaborative processing of the target location point.

[0189] After determining the collaborative drone as described above, in this step, system processing data containing spatial encoding, grid image and corresponding attribute encoding is sent to the collaborative drone to perform collaborative processing of the target location point.

[0190] Continuing with the previous example, send a message containing ACode to the collaborative drone with drone ID n. n Data processing is performed collaboratively by SCode and Patch1.

[0191] In order to improve the accuracy of the collaborative processing results during the specific implementation process, in an optional implementation method provided in this embodiment, the collaborative processing includes:

[0192] Receive the collaborative processing data sent by the requesting drone;

[0193] Based on the attribute encoding contained in the collaborative processing data, the target image is read, and based on the spatial encoding contained in the collaborative processing data, the target image is cropped to obtain a target grid image.

[0194] The grid image and the target grid image are matched to obtain the coordinates of the target location point in the target grid image.

[0195] Based on the coordinates of the image location points, the collaborative processing result of the target location point is calculated and sent to the requesting drone.

[0196] Optionally, during the process of calculating the collaborative processing result of the target location point based on the image location point coordinates and sending it to the requesting drone, the image location point coordinates are transformed to obtain the collaborative location coordinates of the target location point, and the collaborative location coordinates are sent to the requesting drone as the collaborative processing result.

[0197] or,

[0198] Based on the coordinates of the image location points, stereo positioning is performed to obtain the cooperative location coordinates of the target location point, and the cooperative location coordinates are sent to the requesting drone as the cooperative processing result.

[0199] Specifically, after receiving the collaborative processing data, the collaborative UAV first reads the associated target image based on the attribute encoding in the collaborative processing data. Then, it performs image cropping on the target image based on spatial encoding to obtain the target grid image. In order to improve the efficiency of image matching and reduce the consumption of computing resources on the UAV's onboard computer, the grid image and the target grid image are matched to obtain the image position coordinates of the target location point in the target grid image. Based on the image position coordinates, the geographic location coordinates of the target location point are calculated, and the calculated geographic location coordinates are sent to the requesting UAV as the collaborative processing result.

[0200] Continuing with the previous example, when drone n receives the collaborative processing data, it first determines the ACode... n Read the associated target image, and then crop the target grid image Patch onto the target image using the spatial grid corresponding to the SCode. n Patch1 and Patch n Image matching is performed to obtain the coordinates (x, y) of the corresponding image point (P) of the target location point P in the target image. n y n ), using the single-image positioning method to obtain (x n y n The corresponding geographic coordinates (lng) n lat n alt n Alternatively, based on the coordinates of corresponding image points, a stereo positioning method can be used to achieve (lng) n lat n alt n The calculation of (lng) is performed, and finally the calculated result is obtained. n lat n alt n Send to the requesting drone.

[0201] In the specific execution process, after receiving the collaborative processing result sent by the cooperating drone, in order to obtain the positioning coordinates of the target location point, in an optional implementation of this embodiment, after sending collaborative processing data to the system for collaborative processing of the target location point, the following operations are also performed:

[0202] Receive the collaborative position coordinates returned by the collaborative drone after collaborative processing;

[0203] The location coordinates of the target location point are calculated based on the cooperative location coordinates and the geographic location coordinates.

[0204] Specifically, the system receives the collaborative position coordinates returned by the collaborative drone after collaborative processing, and calculates the average value of the collaborative position coordinates and the geographical position coordinates as the positioning coordinates of the target location point.

[0205] It should be noted that the geographical coordinates obtained after the collaborative processing by the aforementioned collaborative drones are the collaborative position coordinates.

[0206] In summary, the multi-UAV collaborative processing method provided in this embodiment is based on the encoding of UAV images and the collaborative management of airborne image data to achieve collaborative task processing among UAVs in a UAV swarm. Among them, the encoding of UAV images can convert the metadata of the images into a unified image code. Compared with the original image metadata, the image code occupies less storage space, reduces the requirements for UAV communication bandwidth, and uses the image attribute code as the unique identifier of the image, which is conducive to the effective identification and retrieval of image files.

[0207] Compared with R*-Tree, it improves the efficiency of index generation, index update and index query, and can well meet the needs of creating and updating image indexes with high dynamics and large data volume in airborne environment, as well as the high frequency of query needs, thus ensuring the effectiveness and feasibility of this embodiment.

[0208] Based on the above, the image-target cooperative localization method is used to calculate the location coordinates of the target point, which improves the accuracy of the calculated location coordinates and reduces the error of the obtained location coordinates.

[0209] The following example uses a multi-UAV cooperative processing method provided in this embodiment in a cooperative positioning scenario as an example, combined with... Figures 11 to 13 The collaborative processing method for multiple UAVs provided in this embodiment will be further explained by referring to [reference needed]. Figure 12 The collaborative processing method for multiple UAVs applied to collaborative positioning scenarios includes the following steps.

[0210] Step S1202: If an image containing the target location point is detected based on the target recognition algorithm, the pixel coordinates of the target location point are obtained.

[0211] Step S1204: Calculate the geographic coordinates of the target location point based on pixel coordinates, DEM data, and POS data of the image.

[0212] In addition, steps S1202 to S1204 can be replaced by, when an image containing a target location point is detected based on the target recognition algorithm, calculating the geographic coordinates of the target location point based on the DEM data and the POS data of the image; or, it can be replaced by, when an image containing a target location point is detected, calculating the geographic coordinates of the target location point based on the DEM data and the POS data of the image.

[0213] Step S1206: Determine the query area of ​​the target location point with the target location point as the center and the preset length as the radius.

[0214] Step S1208: Perform query processing on the query area to obtain the query results.

[0215] Optionally, query processing can be achieved by performing the following steps:

[0216] Step S1208-2: Perform spatial encoding processing on the query area to obtain the regional spatial encoding of the query area;

[0217] Step S1208-4: Calculate the coding interval corresponding to each grid spatial code in the regional spatial coding according to the preset algorithm;

[0218] Step S1208-6: Obtain the primary key set by finding the primary keys in the index table that are within the encoding range;

[0219] Step S1208-8: Extract the attribute codes of each primary key mapping in the primary key set, and perform deduplication on the extracted attribute codes to obtain the attribute code set.

[0220] Step S1208-10: Determine the spatial encoding of the processing grid where the query region is located.

[0221] Step S1210: Crop the grid image from the image based on the spatial encoding in the query results.

[0222] Step S1212: Parse each attribute code in the attribute code set in the query results, and determine the collaborative drone based on the drone identifier contained in each attribute code obtained through parsing.

[0223] Step S1214: Generate cooperative positioning requests for each cooperative UAV based on attribute encoding, grid image and spatial encoding, and send them to each cooperative UAV.

[0224] like Figure 11 As shown, UAV1 needs to locate target P. The pixel coordinates (x1, y1) of target P in the image are obtained through a target recognition algorithm. First, based on the POS data of the image containing target P and the UAV's flight altitude or DEM data, the geographic coordinates (lng1, lat1, alt1) of target P are calculated using a single-image positioning method. Using target P as the center and the maximum value of the target positioning plane error of the single-image positioning method as the radius, the region is divided to obtain the processing area. The processing area is then queried. If the query result is empty, the process ends; if the query result is not empty, the query result is processed to obtain the attribute encoding set {…, ACode}. i ...}, the grid code SCode of the processing grid where the processing area is located, the image is cropped according to the grid code SCode to obtain the grid image Patch1, the cooperative UAV is determined based on the attribute codes in the attribute code set, and a generator containing the grid code SCode, the grid image Patch1 and the corresponding attribute code ACode is generated. n The system sends a collaborative positioning request to the corresponding collaborative UAVn via the UAV swarm communication network.

[0225] Step S1228: Receive the cooperative position coordinates sent by the cooperative drone.

[0226] Step S1230: The geographic coordinates and cooperative coordinates of the target location point are fused to obtain the positioning coordinates of the target location point.

[0227] like Figure 11 As shown, UAV1 receives the cooperative position coordinates (lng) n lat n alt n After that, set (lng1, lat1, alt1) and (lng n lat n alt n The positioning results are fused to obtain the positioning coordinates, and the process ends.

[0228] Steps S1202 to S1214 and S1230 to S1232 provided in this embodiment can be executed by the requesting drone in the drone swarm. It should be noted that the process of the requesting drone in the drone swarm executing steps S1202 to S1214 and S1230 to S1232 is coordinated with the process of steps S1216 to S1228 provided in the following embodiment, which can be executed by the cooperating drone in the drone swarm. Therefore, when reading this embodiment, please refer to the relevant content of steps S1216 to S1228 below. Correspondingly, when reading steps S1216 to S1228 below, please also refer to steps S1202 to S1214 and S1230 to S1232 provided in this embodiment.

[0229] One or more embodiments of another multi-UAV collaborative processing method provided in this specification are as follows:

[0230] The multi-drone collaborative processing method provided in this embodiment can be applied to collaborative drones in a drone swarm. The multi-drone collaborative processing method provided in the above embodiment can be applied to requesting drones in a drone swarm. The multi-drone collaborative processing method for collaborative drones in a drone swarm provided in this embodiment and the multi-drone collaborative processing method for requesting drones in a drone swarm provided in the above embodiment cooperate with each other during execution. When reading this embodiment, you can refer to the relevant content of the above embodiment. When reading the above embodiment, you can also refer to the relevant content of this embodiment.

[0231] Reference Figure 14 The multi-UAV collaborative processing method provided in this embodiment can be applied to collaborative UAVs in a UAV swarm, specifically including steps S1402 to S1408.

[0232] Step S1402: Receive collaborative processing data sent by the requesting drone in the drone swarm.

[0233] Optionally, the collaborative processing data includes the spatial encoding and grid image of the target location point in the processing grid of the image containing the target location point, as well as the attribute encoding.

[0234] Step S1404: Based on the attribute encoding, read the target image and crop the target image based on the spatial encoding to obtain the target grid image.

[0235] Step S1406: Perform matching processing on the grid image and the target grid image to obtain the image position coordinates of the target location point in the target grid image.

[0236] Step S1408: Based on the coordinates of the image location points, calculate the collaborative processing result of the target location point and send it to the requesting drone.

[0237] In one optional implementation of this embodiment, the step of calculating the collaborative processing result of the target location point based on the image location point coordinates and sending it to the requesting drone includes:

[0238] The coordinates of the image location points are transformed to obtain the cooperative location coordinates of the target location point, and the cooperative location coordinates are sent to the requesting drone as the cooperative processing result.

[0239] or,

[0240] Based on the coordinates of the image location points, stereo positioning is performed to obtain the cooperative location coordinates of the target location point, and the cooperative location coordinates are sent to the requesting drone as the cooperative processing result.

[0241] The following example uses a multi-UAV cooperative processing method provided in this embodiment in a cooperative positioning scenario as an example, combined with... Figures 11 to 13 The collaborative processing method for multiple UAVs provided in this embodiment will be further explained by referring to [reference needed]. Figure 12 The collaborative processing method for multiple UAVs applied to collaborative positioning scenarios includes the following steps.

[0242] Step S1216: Receive the cooperative positioning request sent by the requesting drone.

[0243] Step S1218: Read the associated target image based on the attribute encoding contained in the cooperative localization request.

[0244] Step S1220: Based on the grid coding contained in the cooperative localization request, the target image is cropped to obtain a target grid image.

[0245] Step S1222: Perform image matching between the grid image and the target grid image to obtain the coordinates of the corresponding image point of the target location point in the target image.

[0246] Step S1224: Calculate the cooperative position coordinates of the target location points based on the coordinates of the corresponding image points, the POS data of the target image, and the DEM data.

[0247] Step S1226: Send the cooperative position coordinates to the requesting drone. For example... Figure 11 As shown, after receiving the cooperative positioning request, UAVn first uses the ACode... n Extract the corresponding target image, and then crop the target grid image Patch onto the image using the non-rigid spatial grid corresponding to SCode. nThen Patch1 and Patch n Matching to obtain the coordinates (x, y) of the corresponding image point of target P in the target image. n ,y n ), using the single-image positioning method to obtain (x n ,y n The corresponding geographic coordinates (lng) n lat n alt n The coordinates are used as cooperative position coordinates and returned to UAV1 via the UAV swarm communication network.

[0248] This specification provides one or more embodiments of a cooperative tracking method for multiple unmanned aerial vehicles (UAVs) as follows:

[0249] Reference Figure 15 The multi-UAV collaborative tracking processing method provided in this embodiment can be applied to requesting UAVs in a UAV swarm, specifically including steps S1502 to S1508.

[0250] Step S1502: Obtain the pixel coordinates of the target location point in the image.

[0251] Step S1504: Determine the query area of ​​the target location point with the target location point as the center and a preset length as the radius.

[0252] Step S1506: Determine the spatial encoding of the processing grid where the query region is located, and crop the grid image in the image based on the spatial encoding.

[0253] Step S1508: Send a cooperative tracking request containing the pixel coordinates, the spatial code, and the grid image to the cooperative drones other than the requesting drone in the drone swarm for cooperative tracking processing.

[0254] In one optional implementation of this embodiment, it further includes:

[0255] The system receives the coordinates of the same image point sent by the cooperative drone, merges the coordinates of the same image point with the pixel coordinates, and obtains the cooperative tracking result.

[0256] It should be noted that the relevant content in the multi-UAV cooperative tracking processing method provided in this embodiment is similar to the relevant content in the multi-UAV cooperative processing method provided in the above embodiments. When reading this embodiment, please refer to the relevant content in the above embodiments. This embodiment will not repeat the details here.

[0257] This specification provides one or more embodiments of a cooperative tracking method for multiple unmanned aerial vehicles (UAVs) as follows:

[0258] The multi-UAV cooperative tracking processing method provided in this embodiment can be applied to cooperative UAVs in a UAV swarm. The multi-UAV cooperative tracking processing method provided in the above embodiment can be applied to requesting UAVs in a UAV swarm. The multi-UAV cooperative tracking processing method provided in this embodiment, which can be applied to cooperative UAVs in a UAV swarm, and the multi-UAV cooperative tracking processing method provided in the above embodiment, which can be applied to requesting UAVs in a UAV swarm, cooperate with each other during execution. When reading this embodiment, you can refer to the relevant content of the above embodiment. When reading the above embodiment, you can also refer to the relevant content of this embodiment.

[0259] Reference Figure 16 The multi-UAV cooperative tracking processing method provided in this embodiment can be applied to cooperative UAVs in a UAV swarm, specifically including steps S1602 to S1608.

[0260] Step S1602: Receive the collaborative tracking request sent by the requesting drone.

[0261] Optionally, the collaborative tracking request includes the pixel coordinates of the target location point in the image, the spatial encoding of the processing grid in which the query region of the target location point is located, and the grid image corresponding to the spatial encoding in the image.

[0262] Step S1604: If a target image containing the spatial code is acquired, a target grid image is cropped from the target image according to the spatial code.

[0263] Step S1606: Perform image matching between the target grid image and the grid image to obtain the coordinates of the corresponding image point of the target location point in the target image.

[0264] Step S1608: Send the coordinates of the image point with the same name to the requesting drone.

[0265] This specification provides one or more embodiments of an image processing method for multiple unmanned aerial vehicles (UAVs) as follows:

[0266] The image processing method for multiple UAVs provided in this embodiment is similar to the collaborative processing method for multiple UAVs provided in the above embodiments. Please refer to the relevant content of the above embodiments when reading this embodiment, and this embodiment will not be repeated here; you can also refer to the relevant content of this embodiment when reading the above embodiments.

[0267] Reference Figure 17 The image processing method for multiple drones provided in this embodiment can be applied to any drone in a drone swarm, specifically including steps S1702 to S1706.

[0268] Step S1702: Acquire image data and encode the image data to obtain image encoding.

[0269] In one optional implementation of this embodiment, the image is encoded to obtain an image code, including:

[0270] The image is spatially encoded to obtain corner spatial codes and range spatial codes; and,

[0271] The image is subjected to attribute encoding processing to obtain attribute codes.

[0272] Step S1704: Update the index table based on the range spatial encoding and attribute encoding in the image encoding.

[0273] Step S1706: Send the corner spatial code and the attribute code in the image encoding to the candidate drones other than any of the drones in the drone swarm to update the index table of the candidate drones.

[0274] In one optional implementation of this embodiment, sending the corner spatial code and the attribute code from the image encoding to candidate drones other than any of the drones in the drone swarm includes:

[0275] The corner point spatial encoding and the attribute encoding are compressed to obtain compressed encoding;

[0276] The compressed code is sent to the candidate drone.

[0277] In one optional implementation of this embodiment, the following operations are also performed:

[0278] Receive the target corner spatial code and target attribute code sent by any candidate UAV;

[0279] The spatial encoding of the target corner point is parsed to obtain the spatial encoding of the target range;

[0280] The index table is updated based on the target range spatial encoding and the target attribute encoding.

[0281] Optionally, receive the target corner spatial code and target attribute code sent by any candidate UAV, including:

[0282] Receive compressed codes sent by any candidate UAV and decompress the compressed codes to obtain the target corner spatial code and the target attribute code.

[0283] This specification provides one or more embodiments of a collaborative processing device for multiple unmanned aerial vehicles (UAVs) as follows:

[0284] In the above embodiments, a method for collaborative processing of multiple drones requesting drones in a drone swarm is provided. Correspondingly, a device for collaborative processing of multiple drones requesting drones operating in a drone swarm is also provided, which will be described below with reference to the accompanying drawings.

[0285] Reference Figure 18 The diagram illustrates an embodiment of a collaborative processing device for multiple unmanned aerial vehicles (UAVs) provided in this embodiment.

[0286] Since the apparatus embodiments correspond to the method embodiments, the descriptions are relatively simple. For relevant parts, please refer to the corresponding descriptions of the method embodiments provided above. The apparatus embodiments described below are merely illustrative.

[0287] This embodiment provides a collaborative processing device for multiple unmanned aerial vehicles (UAVs), including:

[0288] The coordinate calculation module 1802 is configured to calculate the geographic coordinates of the target location point based on measurement data from an image containing the target location point;

[0289] The encoding retrieval module 1804 is configured to perform encoding retrieval on the processing area determined based on the geographic location coordinates according to the index table, obtain an attribute encoding set, and determine the spatial encoding and grid image of the processing grid of the target location point in the image.

[0290] The drone identification module 1806 is configured to read the drone identifier from each attribute code in the attribute code set and determine the corresponding cooperative drone based on the drone identifier;

[0291] The collaborative processing data sending module 1808 is configured to send collaborative processing data containing the spatial code, the grid image, and the corresponding attribute code to the collaborative UAV for collaborative processing of the target location point.

[0292] One or more embodiments of another multi-UAV collaborative processing device provided in this specification are as follows:

[0293] In the above embodiments, a multi-drone collaborative processing method for collaborative drones in a drone swarm is provided. Correspondingly, a multi-drone collaborative processing device is also provided, which will be described below with reference to the accompanying drawings.

[0294] Reference Figure 19 The diagram illustrates an embodiment of a collaborative processing device for multiple unmanned aerial vehicles (UAVs) provided in this embodiment.

[0295] Since the apparatus embodiments correspond to the method embodiments, the descriptions are relatively simple. For relevant parts, please refer to the corresponding descriptions of the method embodiments provided above. The apparatus embodiments described below are merely illustrative.

[0296] This embodiment provides a collaborative processing device for multiple unmanned aerial vehicles (UAVs), including:

[0297] The collaborative processing data receiving module 1902 is configured to receive collaborative processing data sent by requesting drones in the drone swarm; the collaborative processing data includes the spatial encoding and grid image of the target location point in the processed grid of the image containing the target location point, as well as the attribute encoding;

[0298] The image cropping module 1904 is configured to read the target image based on the attribute encoding, and crop the target image based on the spatial encoding to obtain a target grid image;

[0299] Matching processing module 1906 is configured to perform matching processing on the grid image and the target grid image to obtain the image position coordinates of the target position point in the target grid image;

[0300] The collaborative processing result calculation module 1908 is configured to calculate the collaborative processing result of the target location point based on the coordinates of the image location point and send it to the requesting drone.

[0301] This specification provides one or more embodiments of a collaborative processing device for multiple unmanned aerial vehicles (UAVs) as follows:

[0302] Corresponding to the above-described method for collaborative processing of multiple drones requesting drones in a drone swarm, based on the same technical concept, one or more embodiments of this specification also provide a collaborative processing device for multiple drones, which is used to execute the above-described collaborative processing method for multiple drones. Figure 20 This is a schematic diagram of the structure of a multi-UAV collaborative processing device provided for one or more embodiments of this specification.

[0303] This embodiment provides a collaborative processing device for multiple unmanned aerial vehicles (UAVs), comprising:

[0304] like Figure 20As shown, the collaborative processing device for multiple UAVs can vary significantly due to differences in configuration or performance. It may include one or more processors 2001 and memory 2002, with memory 2002 storing one or more application programs or data. Memory 2002 can be temporary or persistent storage. The application programs stored in memory 2002 may include one or more modules (not shown), each module comprising a series of computer-executable instructions from the collaborative processing device. Furthermore, processor 2001 may be configured to communicate with memory 2002, executing the series of computer-executable instructions stored in memory 2002 on the collaborative processing device. The collaborative processing device for multiple UAVs may also include one or more power supplies 2003, one or more wired or wireless network interfaces 2004, one or more input / output interfaces 2005, one or more keyboards 2006, etc.

[0305] In one specific embodiment, the multi-UAV collaborative processing device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the multi-UAV collaborative processing device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0306] Based on measurement data from imagery containing the target location point, calculate the geographic coordinates of the target location point;

[0307] The processing area determined based on the geographic coordinates is encoded and retrieved according to the index table to obtain the attribute encoding set, and the spatial encoding and grid image of the processing grid of the target location point in the image are determined.

[0308] Read the drone identifier from each attribute code in the attribute code set, and determine the corresponding cooperative drone based on the drone identifier;

[0309] The collaborative drone is sent collaborative processing data containing the spatial code, the grid image, and the corresponding attribute code to perform collaborative processing of the target location point.

[0310] One or more embodiments of another multi-UAV collaborative processing device provided in this specification are as follows:

[0311] Corresponding to the above-described method for collaborative processing of multiple drones in a drone swarm, based on the same technical concept, one or more embodiments of this specification also provide a multi-drone collaborative processing device, which is used to execute the above-described multi-drone collaborative processing method. Figure 21 This is a schematic diagram of the structure of a multi-UAV collaborative processing device provided for one or more embodiments of this specification.

[0312] This embodiment provides a collaborative processing device for multiple unmanned aerial vehicles (UAVs), comprising:

[0313] like Figure 21 As shown, the collaborative processing device for multiple UAVs can vary significantly due to differences in configuration or performance. It may include one or more processors 2101 and memory 2102, with memory 2102 storing one or more application programs or data. Memory 2102 can be temporary or persistent storage. The application programs stored in memory 2102 may include one or more modules (not shown), each module comprising a series of computer-executable instructions from the collaborative processing device. Furthermore, processor 2101 may be configured to communicate with memory 2102, executing the series of computer-executable instructions stored in memory 2102 on the collaborative processing device. The collaborative processing device for multiple UAVs may also include one or more power supplies 2103, one or more wired or wireless network interfaces 2104, one or more input / output interfaces 2105, one or more keyboards 2106, etc.

[0314] In one specific embodiment, the multi-UAV collaborative processing device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the multi-UAV collaborative processing device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0315] Receive collaborative processing data sent by requesting drones in the drone swarm; the collaborative processing data includes the spatial encoding and grid image of the target location point in the processed grid of the image containing the target location point, as well as the attribute encoding;

[0316] Based on the attribute encoding, the target image is read, and based on the spatial encoding, the target image is cropped to obtain a target grid image;

[0317] The grid image and the target grid image are matched to obtain the coordinates of the target location point in the target grid image.

[0318] Based on the coordinates of the image location points, the collaborative processing result of the target location point is calculated and sent to the requesting drone.

[0319] This specification provides one or more embodiments of a storage medium as follows:

[0320] Corresponding to the above-described method for collaborative processing of multiple drones requesting drones in a drone swarm, based on the same technical concept, one or more embodiments of this specification also provide a storage medium.

[0321] The storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed by a processor, implement the following process:

[0322] Based on measurement data from imagery containing the target location point, calculate the geographic coordinates of the target location point;

[0323] The processing area determined based on the geographic coordinates is encoded and retrieved according to the index table to obtain the attribute encoding set, and the spatial encoding and grid image of the processing grid of the target location point in the image are determined.

[0324] Read the drone identifier from each attribute code in the attribute code set, and determine the corresponding cooperative drone based on the drone identifier;

[0325] The collaborative drone is sent collaborative processing data containing the spatial code, the grid image, and the corresponding attribute code to perform collaborative processing of the target location point.

[0326] It should be noted that the embodiment of a storage medium in this specification and the embodiment of a collaborative processing method for multiple UAVs in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0327] One or more embodiments of another storage medium provided in this specification are as follows:

[0328] Corresponding to the above-described method for collaborative processing of multiple drones in a drone swarm, based on the same technical concept, one or more embodiments of this specification also provide a storage medium.

[0329] The storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed by a processor, implement the following process:

[0330] Receive collaborative processing data sent by requesting drones in the drone swarm; the collaborative processing data includes the spatial encoding and grid image of the target location point in the processed grid of the image containing the target location point, as well as the attribute encoding;

[0331] Based on the attribute encoding, the target image is read, and based on the spatial encoding, the target image is cropped to obtain a target grid image;

[0332] The grid image and the target grid image are matched to obtain the coordinates of the target location point in the target grid image.

[0333] Based on the coordinates of the image location points, the collaborative processing result of the target location point is calculated and sent to the requesting drone.

[0334] It should be noted that the embodiments of another storage medium in this specification and the embodiments of another collaborative processing method for multiple UAVs in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0335] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, please refer to each other. Each embodiment focuses on describing the differences from other embodiments. For example, the device embodiment, equipment embodiment, and storage medium embodiment are all similar to the method embodiment, so the description is relatively simple. For reading the relevant content of the device embodiment, equipment embodiment, and storage medium embodiment, please refer to the description of the method embodiment.

[0336] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0337] In the 1930s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement to the methodology cannot be implemented using a hardware physical module. For example, a Programmable Logic Device (PLD) (e.g., a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0338] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0339] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0340] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0341] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0342] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0343] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0344] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0345] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0346] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0347] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0348] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0349] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0350] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0351] The above description is merely an embodiment of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. A collaborative processing method for multiple drones, applied to requesting drones in a drone swarm, the method comprising: Based on measurement data from imagery containing the target location point, calculate the geographic coordinates of the target location point; The processing area determined based on the geographic coordinates is encoded and retrieved according to the index table to obtain the attribute encoding set, and the spatial encoding and grid image of the processing grid of the target location point in the image are determined. Read the drone identifier from each attribute code in the attribute code set, and determine the corresponding cooperative drone based on the drone identifier; The collaborative drone is sent collaborative processing data containing the spatial code, the grid image, and the corresponding attribute code to perform collaborative processing of the target location point; The method further includes: determining the appropriate image gridding level for image coverage based on the flight area size, UAV camera parameters, and onboard computer performance. The image gridding level is calculated in the following manner: The This indicates the image meshing level, the Represents the image area With grid area The corresponding grid level, where d represents the preset depth, and the image area refers to the area of ​​the image captured by the UAV; the grid area refers to the area of ​​the grid at the corresponding level obtained after calculation based on the preset grid level scale.

2. The multi-UAV collaborative processing method according to claim 1 further includes: Geographic images are acquired, and spatial encoding and attribute encoding are performed on the geographic images to obtain corner spatial encoding, range spatial encoding and image attribute encoding. The index table is updated based on the range spatial encoding and the image attribute encoding, and the corner spatial encoding and the image attribute encoding are compressed to obtain compressed encoding; The compressed code is sent to candidate drones in the drone swarm other than the requesting drone to update the index table of each candidate drone.

3. The multi-UAV collaborative processing method according to claim 2, wherein the spatial coding processing includes: Calculate the corner coordinates of the corner points in the geographic image; Based on the corner point's position coordinates and spatial coding level, calculate the corner point's coding index; The corner spatial code of the corner point is calculated based on the coding index and the spatial coding level.

4. The multi-UAV collaborative processing method according to claim 3, wherein the spatial coding processing further includes: The geographic image is processed by gridding based on the image gridding hierarchy to obtain at least one single-scale grid. The at least one single-scale grid is aggregated to obtain at least one multi-scale grid, and the spatial code of each grid in the at least one multi-scale grid is used as the range spatial code.

5. The collaborative processing method for multiple UAVs according to claim 4, wherein the spatial encoding of each grid is obtained in the following manner: Calculate the spatial code of the maximum level, and based on the spatial code of the maximum level, calculate the spatial code of each level; The grid level of each grid is determined, and the spatial code of each grid is read based on the grid level.

6. The collaborative processing method for multiple UAVs according to claim 2, wherein the attribute encoding processing includes: Read the attribute data from the metadata of the geographic image; The geographic image is subjected to attribute encoding processing based on the attribute data, image gridding level and preset encoding algorithm to obtain the attribute encoding of the geographic image. The attribute data includes at least one of the following: image acquisition time, image frame number, drone number, and image type.

7. The multi-UAV collaborative processing method according to claim 2, wherein updating the index table based on the range spatial encoding and the image attribute encoding comprises: Check whether the primary key of the index table contains the range space code; If so, the image attribute encoding is mapped to the key value of the range space encoding as the primary key; If not, the index table will be updated with the range space code as the primary key and the image attribute code as the key value.

8. The multi-UAV collaborative processing method according to claim 2, wherein any candidate UAV other than the requesting UAV performs the following operation: Receive the compressed code sent by the requesting drone and decompress the compressed code to obtain the corner spatial code and the image attribute code; The corner spatial code is parsed to obtain the range spatial code; Based on the range spatial encoding and the image attribute encoding, the index table of any candidate UAV is updated.

9. The collaborative processing method for multiple UAVs according to claim 1, wherein the step of performing an attribute code set by encoding and retrieving the processing area determined based on the geographic location coordinates according to the index table includes: The processing area is obtained by dividing the region using the geographical coordinates as the region center and a preset length as the division length; The processing area is encoded and retrieved according to the index table to obtain an attribute encoding set.

10. The collaborative processing method for multiple UAVs according to claim 9, wherein the step of performing an encoding retrieval of the processing region according to the index table to obtain an attribute encoding set includes: Spatial encoding is performed on the processing area to obtain the regional spatial encoding of the processing area, and the encoding interval of each grid spatial encoding in the regional spatial encoding is calculated according to a preset interval algorithm; Read the primary key located in the encoding range from the index table, and read the key value mapped by the primary key; The key values ​​are filtered, and the attribute code set is constructed based on the filtered key values.

11. The multi-UAV collaborative processing method according to claim 1, wherein determining the spatial encoding and grid image of the processing grid of the target location point in the image image comprises: In the image, the spatial encoding of the processing grid containing the target location point is determined; Based on the spatial encoding, the image corresponding to the processing grid is cropped from the image to become the grid image.

12. The multi-UAV cooperative processing method according to claim 1, wherein the cooperative processing includes: Receive the collaborative processing data sent by the requesting drone; Based on the attribute encoding contained in the collaborative processing data, the target image is read, and based on the spatial encoding contained in the collaborative processing data, the target image is cropped to obtain a target grid image. The grid image and the target grid image are matched to obtain the coordinates of the target location point in the target grid image. Based on the coordinates of the image location points, the collaborative processing result of the target location point is calculated and sent to the requesting drone.

13. The multi-UAV cooperative processing method according to claim 1, after the step of sending cooperative processing data including the spatial code, the grid image, and the corresponding attribute code to the cooperative UAV for cooperative processing of the target location point, further includes: Receive the collaborative position coordinates returned by the collaborative drone after collaborative processing; The location coordinates of the target location point are calculated based on the cooperative location coordinates and the geographic location coordinates.

14. The multi-UAV collaborative processing method according to claim 1 further includes: An initial image coverage status map is rendered based on historical imagery and a global discrete grid system; the image grid in the initial image coverage status map is the first display state. Read at least one range spatial code from the index table, update the display state of the image grid corresponding to the at least one range spatial code in the initial image coverage situation map to the second display state, and obtain the intermediate image coverage situation map; Read the attribute codes mapped by each range spatial code in at least one range spatial code in the index table, associate the geographic image image associated with the attribute code with the corresponding image grid, and obtain an image coverage status map.

15. A collaborative processing method for multiple drones, applied to collaborative drones in a drone swarm, the method comprising: Receive collaborative processing data sent by requesting drones in the drone swarm; The collaborative processing data includes the spatial encoding and grid image of the target location point in the processing grid of the image containing the target location point, as well as the attribute encoding; Based on the attribute encoding, the target image is read, and based on the spatial encoding, the target image is cropped to obtain a target grid image; The grid image and the target grid image are matched to obtain the coordinates of the target location point in the target grid image. Based on the coordinates of the image location points, the collaborative processing result of the target location points is calculated and sent to the requesting drone; The method further includes: the requesting UAV determining a suitable image gridding level for the image coverage area based on the flight area size, UAV camera parameters, and onboard computer performance; the image gridding level is calculated in the following manner: The This indicates the image meshing level, the Represents the image area With grid area The corresponding grid level, where d represents the preset depth, and the image area refers to the area of ​​the image captured by the UAV; the grid area refers to the area of ​​the grid at the corresponding level obtained after calculation based on the preset grid level scale.

16. A multi-drone collaborative processing device, comprising a requesting drone operating within a drone swarm, including: The coordinate calculation module is configured to calculate the geographic coordinates of the target location point based on measurement data from an image containing the target location point; The encoding retrieval module is configured to perform encoding retrieval on the processing area determined based on the geographic coordinates according to the index table, obtain an attribute encoding set, and determine the spatial encoding and grid image of the processing grid of the target location point in the image. The drone identification module is configured to read the drone identifier from each attribute code in the attribute code set, and determine the corresponding cooperative drone based on the drone identifier; The collaborative processing data sending module is configured to send collaborative processing data, including the spatial code, the grid image, and the corresponding attribute code, to the collaborative UAV for collaborative processing of the target location point; The encoding retrieval module is further configured to: determine the appropriate image gridding level for image coverage based on the flight area size, UAV camera parameters, and onboard computer performance. The image gridding level is calculated in the following manner: The This indicates the image meshing level, the Represents the image area With grid area The corresponding grid level, where d represents the preset depth, and the image area refers to the area of ​​the image captured by the UAV; the grid area refers to the area of ​​the grid at the corresponding level obtained after calculation based on the preset grid level scale.

17. A collaborative processing device for multiple unmanned aerial vehicles (UAVs), comprising collaborative UAVs operating in a UAV swarm, including: The collaborative processing data receiving module is configured to receive collaborative processing data sent by requesting drones in the drone swarm. The collaborative processing data includes the spatial encoding and grid image of the target location point in the processing grid of the image containing the target location point, as well as the attribute encoding; The image cropping module is configured to read the target image based on the attribute encoding, and crop the target image based on the spatial encoding to obtain a target grid image; The matching processing module is configured to perform matching processing on the grid image and the target grid image to obtain the image position coordinates of the target position point in the target grid image; The collaborative processing result calculation module is configured to calculate the collaborative processing result of the target location point based on the coordinates of the image location point and send it to the requesting drone; The requesting drone determines the appropriate image gridding level for the image coverage area based on the size of the flight area, the drone's camera parameters, and the performance of its onboard computer. The image gridding level is calculated in the following manner: The This indicates the image meshing level, the Represents the image area With grid area The corresponding grid level, where d represents the preset depth, and the image area refers to the area of ​​the image captured by the UAV; the grid area refers to the area of ​​the grid at the corresponding level obtained after calculation based on the preset grid level scale.

18. A collaborative processing device for multiple unmanned aerial vehicles (UAVs), comprising: processor; And a memory configured to store computer-executable instructions, which, when executed, cause the processor to implement the multi-UAV collaborative processing method according to any one of claims 1-15.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store computer-executable instructions, which, when executed by a processor, implement the multi-UAV collaborative processing method as described in any one of claims 1-15.

Images