A three-dimensional situation fusion method based on distributed perception of UAVs

Abstract

A three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles (UAVs) is disclosed, comprising the following steps: independent data acquisition; shared data association; distributed data registration; and cross-linked situation fusion. The method uses UAVs to acquire environmental information using onboard sensors, combining two-dimensional grid information with three-dimensional elevation information to construct a three-dimensional environmental map. The UAVs then calculate the similarity of three-dimensional feature points and perform situation fusion using feature extraction and a sliding window matching algorithm. This method optimizes UAV flight control, improving flight safety and mission execution efficiency.

Description

Technical Field

[0001] The present invention relates to the field of distributed information fusion, and in particular to a three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles. Background Art

[0002] With the development of drone technology, its applications in military reconnaissance, environmental monitoring, disaster relief, and other fields are becoming increasingly widespread. However, the perception capabilities of a single drone are limited, making it difficult to fully capture detailed information in complex environments. To address this, distributed drone perception systems have emerged. Through the coordinated operation of multiple drones, they achieve comprehensive environmental perception and information sharing.

[0003] Traditional drone perception systems primarily rely on a single sensor for data collection, resulting in incomplete information acquisition and low processing efficiency. Furthermore, drones often require real-time updates of environmental information during missions to ensure flight safety and mission accuracy. Existing technologies have limitations in data fusion and situational awareness, particularly when multiple drones are operating collaboratively. Efficient and accurate information fusion and situational awareness remain a hot topic and a challenge in current research. Summary of the Invention

[0004] The present invention proposes a three-dimensional situation fusion method based on distributed perception of UAVs, which is characterized in that in a rectangular coordinate system constructed for the top view of the UAV, it is assumed that for the kth UAV u k , the location information of the drone at this time is Environmental information perceived by the drone at its current location As shown in formula (1), it is the two-dimensional grid information And the grid elevation information based on this

[0005]

[0006] For any grid coordinate There is always elevation information on this grid point Make the three-dimensional grid map complete, that is:

[0007]

[0008] The UAV is at the current spatial position at time t The acquired three-dimensional grid environment information for:

[0009]

[0010] The method specifically comprises the following steps:

[0011] Step 1: Independent data acquisition;

[0012] The perception control model of independent data acquisition of UAV is: k Get the current UAV's spatial position information And perceive environmental information, including two-dimensional grid information of the environment and 3D elevation information The drone u k The independently acquired environmental data is combined with its own location information to obtain the three-dimensional grid environmental information: and feeds this back into the drone’s flight control system;

[0013] The drone uses environmental perception sensors to obtain 3D terrain information and converts it into 2D grid information and 3D elevation information. The drone's status information is obtained through an onboard status information sensor, providing real-time feedback to the flight control system. Through the closed-loop feedback loop of 2D grid information, 3D elevation information, and real-time drone status feedback, the flight control system outputs control signals based on actual conditions to control the drone's flight.

[0014] Step 2: Shared data association;

[0015] The 3D information sensed by UAV 1 and UAV 2 is extracted. Based on this 3D feature information, the two distributed UAVs are matched using an L-window sliding method. The similarity of the 3D feature points is calculated, and the one with the highest similarity is selected for 3D situation fusion. This creates a global 3D terrain situation for the distributed UAVs to optimize subsequent missions.

[0016] Assume that UAV 1 and UAV 2 are labeled as u1 and u2 respectively, and the three-dimensional grid environment information obtained by the two UAVs at time t is and The feature points obtained in the three-dimensional grid information are and

[0017]

[0018] Where: (x q1 ,y q1 ),(x q2 ,y q2 ),(x q3 ,y q3 ),…,(x qn ,y qn ) and (x p1 ,y p1 ),(x p2,y p2 ),(x p3 ,y p3 ),…,(x qm ,y qm are the abscissa and ordinate of the three-dimensional feature points in the two-dimensional coordinates respectively;

[0019] Step 3: Distributed data matching;

[0020] Set the sliding window L u1 and L u2 are respectively and in the independent sliding windows. Through the feature information extraction process as shown in Equation (4), calculate the similarity value of the feature points. The specific method is as follows: Define the similarity and between the visual information as:

[0021]

[0022] Where: and respectively represent the calculation of the feature point range box selection for the feature point information u1 and based on the sliding window sizes of L and , is an arbitrary symbol, is the existence symbol, and represent the feature points that meet the above conditions, where 丨 means the former is based on the latter; and are the sliding window similarity calculation functions. When traversing the feature information and processed by the UAV perception, the similarity value obtained based on the sliding window sizes of L u1 and L u2 is as follows: The calculation method is as follows:

[0023]

[0024]

[0025] Where: i and j respectively represent the horizontal grid increment and vertical grid increment of the grid coordinates, and are the abscissa and ordinate of the centroid of the sliding window;

[0026] The sliding window L u1and L u2 Traverse each visual feature information separately, and calculate the minimum difference between the two to obtain the optimal fusion splicing area:

[0027]

[0028] Will The value is compared with the similarity threshold. If If the similarity is greater than the threshold, the visual information within the window is considered and resemblance;

[0029] Step 4: Cross-linked situation fusion;

[0030] In the sliding window L u1 and L u2 Based on the three-dimensional feature point similarity calculation, the perception information of the two drones is combined according to the feature point information. and Perform situational fusion, including and The surrounding environment information perceived by the two UAVs within the visual range is respectively obtained, and the fused three-dimensional situation information Ξ(u1,u2) is obtained:

[0031]

[0032] in: is the union function, which represents the information of the drone feature points and The union operation performed.

[0033] In step three of a specific embodiment of the present invention, the similarity threshold is 0.75.

[0034] In one embodiment of the present invention, it is assumed that there are N distributed UAVs in the environment. In step 4, the data fusion method of formula (11) is used to obtain the three-dimensional overall situation fusion information of the distributed UAVs for the environmental information:

[0035]

[0036] This method uses drone-mounted sensors to acquire environmental information and, using feature extraction and a sliding window matching algorithm, enables information sharing and situational fusion between drones. This method not only improves the comprehensiveness and accuracy of information acquisition but also optimizes drone flight control and mission execution, possessing significant application value and promising development prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the steps of the present invention;

[0038] Figure 2 Showing the construction of a three-dimensional situation grid map of the present invention;

[0039] Figure 3 The present invention shows a situation fusion method based on three-dimensional feature points;

[0040] Figure 4 A diagram showing a single drone perception control model of the present invention is shown;

[0041] Figure 5 A diagram showing a model of a distributed sensing cloud sharing device for drones according to the present invention is shown; DETAILED DESCRIPTION

[0042] The present invention will be described in detail below with reference to the accompanying drawings.

[0043] The drone obtains environmental information through onboard sensors, including two-dimensional grid information and three-dimensional elevation information, and combines the two to obtain a three-dimensional environmental map. Figure 2 The figure shows the detailed process of constructing a 3D situational grid map. Based on the 3D grid map and the real-time status of the drone, the drone inputs the information into the flight control system to control the drone's flight.

[0044] In the rectangular coordinate system constructed for the top view of the UAV, it is assumed that for the kth UAV u k , the location information of the drone at this time is Environmental information perceived by the drone at its current location As shown in formula (1), it is the two-dimensional grid information And the grid elevation information based on this

[0045]

[0046] For any grid coordinate There is always elevation information on this grid point Make the three-dimensional grid map complete, that is:

[0047]

[0048] The UAV is at the current spatial position at time t The acquired three-dimensional grid environment information for:

[0049]

[0050] The present invention provides a three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles (UAVs), which mainly involves four steps: independent data acquisition, shared data association, distributed data alignment, and cross-linked situation fusion. Based on the fused environmental information, the flight of distributed UAVs is planned through global information guidance.

[0051] Step 1: Independent data acquisition.

[0052] The perception control model of independent data acquisition of UAV is as follows: Figure 3 As shown, the kth drone u k The current spatial position information of the UAV can be obtained through the onboard inertial navigation, Beidou and other positioning systems And the environmental information sensed by airborne infrared, visible light, ultrasonic and other airborne sensors, mainly including the two-dimensional grid information of the environment and 3D elevation information The above method can be used to k The independently acquired environmental data is combined with its own location information to obtain the three-dimensional grid environmental information: And feed it back into the drone's flight control system.

[0053] The UAV obtains the three-dimensional terrain information in the environment through the environmental perception sensor and converts it into two-dimensional raster information and three-dimensional elevation information (such as Figure 2 (as shown); onboard state information sensors (such as inertial navigation and BeiDou navigation) obtain the drone's status information and provide real-time feedback to the flight control system. Through the closed-loop feedback loop of 2D grid information, 3D elevation information, and real-time drone status feedback, the flight control system outputs control signals based on actual conditions (how the flight control system outputs control signals based on actual conditions is well known to those skilled in the art and will not be further described here), controlling the drone's flight and achieving safe and controllable flight.

[0054] Step 2: Shared data association.

[0055] The three-dimensional information perceived by UAV 1 and UAV 2 is feature extracted. Based on the three-dimensional feature information, the two distributed UAVs are slidingly matched through an L window, and the similarity of the three-dimensional feature points is calculated. The highest similarity is selected for three-dimensional situation fusion, thereby forming a global three-dimensional terrain situation for the distributed UAVs to optimize the execution of subsequent tasks.

[0056] like Figure 4 As shown in the figure, assuming that UAV 1 and UAV 2 are marked as u1 and u2 respectively, the three-dimensional grid environment information obtained by the two UAVs at time t is and The feature points obtained in the three-dimensional grid information are and

[0057]

[0058] Where: (x q1 ,y q1 ),(x q2 ,y q2 ),(x q3 ,y q3 ),…,(x qn ,y qn ) and (x p1 ,y p1 ),(x p2 ,y p2 ),(x p3 ,y p3 ),…,(x qm ,y qm ) are the horizontal and vertical coordinates of the three-dimensional feature points in the two-dimensional coordinates.

[0059] Step 3: Distributed data matching.

[0060] According to the basic operating principle of UAV, the control model diagram of a single UAV perception model can be obtained (such as Figure 3 As shown), this technology is well known to those skilled in the art and will not be repeated here. On this basis, the information of each drone in the distributed drone is input into the drone distributed perception cloud sharing device constructed by the present invention (as shown in FIG. Figure 5 shown).

[0061] The proposed UAV distributed perception cloud-based sharing device is based on a perception control model for independent data acquisition by individual UAVs (this model is known to those skilled in the art). The perception information obtained by each UAV (primarily including three-dimensional grid map flight guidance information and real-time UAV status feedback information) is input into the sharing device for information processing. The process is as follows: The UAV distributed perception cloud-based sharing device performs coarse alignment of distributed perception information using a feature association algorithm, then performs fine alignment using a sliding window similarity matching algorithm, and finally fuses the three-dimensional situation information Ξ(u1,u2) of the UAV distributed perception, thereby more optimally guiding the flight of each UAV in a global information environment. The specific method is as follows:

[0062] The feature point information obtained by the two drones and In order to complete the 3D situation fusion of the two, the key is to find the most similar area between the two for analogy judgment. Set the sliding window L u1 and Lu2 respectively and in the independent sliding window, as Figure 4 shown, through the feature information extraction process as in Equation (4), so as to quantitatively calculate the similarity value of feature points. The specific method is as follows: Define the visual information and ( and respectively represent the calculation of feature point information u1 and based on the sliding window sizes of L and (where the vertical bar in the middle indicates that the former is based on the latter, which is well-known to those skilled in the art). The similarity is:

[0063]

[0064] Where: and respectively represent the calculation of the bounding box of the feature point range for the feature point information u1 and based on the sliding window sizes of L and ; is an arbitrary symbol, is the existence symbol, and represent the feature points that meet the above conditions, where the vertical bar indicates that the former is based on the latter; and are the sliding window similarity calculation functions constructed by the present invention. When traversing the feature information and processed by the UAV perception, the similarity value obtained based on the sliding window sizes of L u1 and L u2 is as follows:

[0065]

[0066] Where: represents an arbitrary grid coordinate, i and j respectively represent the horizontal grid increment and vertical grid increment of the grid coordinate, and are the abscissa and ordinate of the centroid of the sliding window.

[0067] The sliding windows L u1 and L u2By traversing each visual feature information separately and calculating the minimum difference between the two, the optimal fusion splicing area can be obtained:

[0068] The final calculation result is a smaller number between 0 and 1. The value is compared with the similarity threshold (set artificially, for example, 0.75). If it is greater than the threshold, it is considered that the visual information within the window and resemblance.

[0069] In the sliding window L u1 and L u2 Based on the three-dimensional feature point similarity calculation, the perception information of the two drones is combined according to the feature point information. and Perform situational fusion, including and are the surrounding environment information perceived by the two UAVs within the visual range, and the fused three-dimensional situation information Ξ(u1,u2) can be obtained:

[0070]

[0071] in: is the union function, which means the union operation is performed on the UAV visual information.

[0072] Step 4: Cross-linked situation fusion.

[0073] like Figure 5 As shown in Figure 1, assuming that there are N distributed UAVs in the environment, the data fusion method of formula (11) can be used to obtain the three-dimensional overall situation fusion information of the distributed UAVs for the environmental information:

[0074]

[0075] This method uses drone-mounted sensors to acquire environmental information and, using feature extraction and a sliding window matching algorithm, enables information sharing and situational fusion between drones. This method not only improves the comprehensiveness and accuracy of information acquisition but also optimizes drone flight control and mission execution, possessing significant application value and promising development prospects.

Claims

1. A three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles, characterized by: In the rectangular coordinate system constructed for the top view of the UAV, it is assumed that for the kth UAV u k , the location information of the drone at this time is Environmental information perceived by the drone at its current location As shown in formula (1), it is the two-dimensional grid information And the grid elevation information based on this For any grid coordinate There is always elevation information on this grid point Make the three-dimensional grid map complete, that is: make The UAV is at the current spatial position at time t The acquired three-dimensional grid environment information for: The method specifically comprises the following steps: Step 1: Independent data acquisition; The perception control model of independent data acquisition of UAV is: k Get the current UAV's spatial position information And perceive environmental information, including two-dimensional grid information of the environment and 3D elevation information The drone u k The independently acquired environmental data is combined with its own location information to obtain the three-dimensional grid environmental information: and feeds this back into the drone’s flight control system; The drone uses environmental perception sensors to obtain 3D terrain information and converts it into 2D grid information and 3D elevation information. The drone's status information is obtained through an onboard status information sensor, providing real-time feedback to the flight control system. Through the closed-loop feedback loop of 2D grid information, 3D elevation information, and real-time drone status feedback, the flight control system outputs control signals based on actual conditions to control the drone's flight. Step 2: Shared data association; The 3D information sensed by UAV 1 and UAV 2 is extracted. Based on this 3D feature information, the two distributed UAVs are matched using an L-window sliding method. The similarity of the 3D feature points is calculated, and the one with the highest similarity is selected for 3D situation fusion. This creates a global 3D terrain situation for the distributed UAVs to optimize their subsequent missions. Assume that UAV 1 and UAV 2 are labeled as u1 and u2 respectively, and the three-dimensional grid environment information obtained by the two UAVs at time t is and The feature points obtained in the three-dimensional grid information are and Where: (x q1 ,y q1 ),(x q2 ,y q2 ),(x q3 ,y q3 ),…,(x qn ,y qn ) and (x p1 ,y p1 ),(x p2 ,y p2 ),(x p3 ,y p3 ),…,(x qm ,y qm ) are the horizontal and vertical coordinates of the three-dimensional feature points in the two-dimensional coordinates; Step 3: Distributed data matching; Set the sliding window L u1 and L u2 They are and The independent sliding window in the window is subjected to the feature information extraction process as shown in formula (4), and the similarity value of the feature points is quantitatively calculated. The specific method is as follows: and The similarity between for: in: and Respectively expressed in L u1 and Based on the sliding window size, the feature point information and The feature point range selection calculation is performed. is an arbitrary symbol, is an existential symbol, and represents the feature points that meet the above conditions, where 丨 indicates that the former is based on the latter; and is a sliding window similarity calculation function, which is when traversing the feature information after the UAV's perception and processing and after that, on the basis of the sizes of the sliding windows L u1 and L u2 the similarity value obtained; the calculation method is as follows: Where: i and j represent the horizontal grid increment and vertical grid increment of the grid coordinate respectively. and is the horizontal coordinate and vertical coordinate of the centroid of the sliding window; The sliding window L u1 and L u2 Traverse each visual feature information separately, and calculate the minimum difference between the two to obtain the optimal fusion splicing area: Will The value is compared with the similarity threshold. If If the similarity is greater than the threshold, the visual information within the window is considered and resemblance; Step 4: Cross-linked situation fusion; In the sliding window L u1 and L u2 Based on the three-dimensional feature point similarity calculation, the perception information of the two drones is combined according to the feature point information. and Perform situational fusion, including and The surrounding environment information perceived by the two UAVs within the visual range is respectively obtained, and the fused three-dimensional situation information Ξ(u1,u2) is obtained: in: is the union function, which represents the information of the drone feature points and The union operation performed.

2. The three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles according to claim 1 is characterized in that: In step 3, the similarity threshold is 0.

75.

3. The three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles according to claim 1 is characterized in that: Assuming that there are N distributed UAVs in the environment, in step 4, the data fusion method of formula (11) is used to obtain the three-dimensional overall situation fusion information of the distributed UAVs for the environmental information:

Images