Three-dimensional situation fusion method based on unmanned aerial vehicle distributed sensing

Abstract

The invention discloses a three-dimensional situation fusion method based on distributed sensing of an unmanned aerial vehicle. The method comprises the following steps: obtaining independent data; carrying out shared data association; carrying out distributed data registration; and carrying out cross-linked situation fusion. According to the method, environment information is acquired by using an airborne sensor through an unmanned aerial vehicle, and a three-dimensional environment map is constructed in combination with two-dimensional grid information and three-dimensional elevation information; and the unmanned aerial vehicle realizes similarity calculation and situation fusion of three-dimensional feature points through feature extraction and a sliding window matching algorithm. According to the invention, the flight control of the unmanned aerial vehicle is optimized, and the flight safety and task execution efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of distributed information fusion, and in particular to a three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles. Background Art

[0002] With the development of drone technology, its application in military reconnaissance, environmental monitoring, disaster relief and other fields is becoming more and more extensive. However, the perception ability of a single drone is limited, and it is difficult to fully obtain detailed information in a complex environment. For this reason, a distributed drone perception system has emerged, which realizes comprehensive perception of the environment and information sharing through the collaborative operation of multiple drones.

[0003] Traditional drone perception systems mainly rely on a single sensor for data collection, which has problems such as incomplete information acquisition and low processing efficiency. In addition, when drones are performing tasks, they often need to update environmental information in real time to ensure flight safety and accuracy of mission execution. Existing technologies have certain limitations in data fusion and situational awareness, especially when multiple drones are working together. How to efficiently and accurately perform information fusion and situational awareness is a hot topic and difficulty in current research. Summary of the invention

[0004] The present invention proposes a three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles, characterized in that in a rectangular coordinate system constructed for a bird's-eye view of the unmanned aerial vehicle, assuming that for the kth unmanned aerial vehicle u k , the location information of the drone at this time is Environmental information perceived by the drone at its current location As shown in formula (1), it is the two-dimensional grid information And the grid elevation information based on this

[0005]

[0006] For any grid coordinate There is always elevation information on this grid point Make the three-dimensional grid map complete, that is:

[0007]

[0008] The drone is at the current spatial position at time t The acquired three-dimensional grid environment information for:

[0009]

[0010] The method specifically comprises the following steps:

[0011] Step 1: Independent data acquisition;

[0012] The perception control model of independent data acquisition of UAV is: k Get the spatial position information of the current drone And perceive environmental information, including two-dimensional grid information of the environment and 3D elevation information UAV k The independently acquired environmental data is combined with its own location information to obtain the three-dimensional grid environmental information: and feeds it back into the drone’s flight control system;

[0013] The drone obtains three-dimensional terrain information in the environment through environmental perception sensors, and converts it into two-dimensional grid information and three-dimensional elevation information; obtains the drone's status information through the onboard status information sensor, and provides real-time feedback to the flight control system; under the common closed-loop feedback of two-dimensional grid information, three-dimensional elevation information, and real-time status feedback of the drone, the flight control system outputs control signals according to the actual situation to control the flight of the drone;

[0014] Step 2: Shared data association;

[0015] The feature extraction is performed on the 3D information sensed by UAV 1 and UAV 2. Based on the 3D feature information, the two distributed UAVs are matched by sliding through an L window, the similarity of the 3D feature points is calculated, and the highest similarity is selected for 3D situation fusion to form a global 3D terrain situation for the distributed UAVs to optimize the subsequent tasks.

[0016] Assume that UAV 1 and UAV 2 are labeled as u1 and u2 respectively, and the three-dimensional grid environment information obtained by the two UAVs at time t is and The feature points obtained in the three-dimensional grid information are and

[0017]

[0018] Where: (x q1 ,y q1 ),(x q2 ,y q2 ),(x q3 ,y q3 ),…,(x qn ,y qn ) and (x p1 ,y p1 ),(x p2, y p2 ), (x p3 , y p3 ), …, (x qm , y qm ) are the abscissa and ordinate of the 3D feature points in the 2D coordinates respectively;

[0019] Step 3: Distributed data matching;

[0020] Set the sliding windows L u1 and L u2 are respectively and in the independent sliding windows. After the feature information extraction process as shown in Equation (4), the similarity value of the feature points is calculated quantitatively. The specific method is as follows: Define the similarity and between the visual information as:

[0021]

[0022] Where: and respectively represent the calculation of the feature point range bounding of the feature point information u1 and on the basis of the sliding window sizes of L and ,

[0023] is an arbitrary symbol, is the existence symbol, and represent the feature points that meet the above conditions, where 丨 means the former is based on the latter; and are the sliding window similarity calculation functions. After traversing the feature information and processed by the UAV perception, the similarity value obtained on the basis of the sliding window sizes of L u1 and L u2 is as follows: The calculation method is as follows:

[0024]

[0025]

[0026] Where: i and j respectively represent the horizontal grid increment and vertical grid increment of the grid coordinates, and are the abscissa of the centroid and the ordinate of the centroid of the sliding window;

[0027] The sliding window L u1 and L u2 Traverse each visual feature information separately, and obtain the optimal fusion splicing area by calculating the minimum difference between the two:

[0028]

[0029] Will The value is compared with the similarity threshold. is greater than the similarity threshold, then the visual information within the window is considered and resemblance;

[0030] Step 4: Cross-linked situation fusion;

[0031] In the sliding window L u1 and L u2 Based on the three-dimensional feature point similarity calculation, the perception information of the two drones is combined according to the feature point information. and Perform situation fusion, including and are the surrounding environment information perceived by the two UAVs within the visual range, and the fused three-dimensional situation information Ξ(u1,u2) is obtained:

[0032]

[0033] in: is the union function, which represents the information of the drone feature points and The union operation performed.

[0034] In step three of a specific embodiment of the present invention, the similarity threshold is 0.75.

[0035] In one embodiment of the present invention, it is assumed that there are N distributed UAVs in the environment. In step 4, the data fusion method of formula (11) is used to obtain the three-dimensional overall situation fusion information of the distributed UAVs for the environmental information:

[0036]

[0037] The present invention obtains environmental information through the UAV airborne sensors, and uses feature extraction and sliding window matching algorithms to achieve information sharing and situation fusion between UAVs. This method can not only improve the comprehensiveness and accuracy of information acquisition, but also optimize the flight control and mission execution of UAVs, and has important application value and broad development prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the steps of the present invention;

[0039] Figure 2 The three-dimensional situation grid map construction of the present invention is shown;

[0040] Figure 3 The situation fusion method based on three-dimensional feature points of the present invention is shown;

[0041] Figure 4 A diagram showing a single drone perception control model of the present invention is shown;

[0042] Figure 5 The model diagram of the UAV distributed sensing cloud sharing device of the present invention is shown; DETAILED DESCRIPTION

[0043] The present invention will be described in detail below with reference to the accompanying drawings.

[0044] The drone obtains environmental information through onboard sensors, including two-dimensional grid information and three-dimensional elevation information, and combines the two to obtain a three-dimensional environmental map. Figure 2 The figure shows the detailed process of constructing a 3D situation grid map. Based on the 3D grid map and the real-time status of the drone, the drone inputs the information into the flight control system to control the flight of the drone.

[0045] In the rectangular coordinate system constructed for the top view of the UAV, assume that for the kth UAV u k , the location information of the drone at this time is Environmental information perceived by the drone at its current location As shown in formula (1), it is the two-dimensional grid information And the grid elevation information based on this

[0046]

[0047] For any grid coordinate There is always elevation information on this grid point Make the three-dimensional grid map complete, that is:

[0048]

[0049] The drone is at the current spatial position at time t The acquired three-dimensional grid environment information for:

[0050]

[0051] The present invention provides a three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles, which mainly involves four steps: independent data acquisition, shared data association, distributed data alignment and cross-linked situation fusion. Based on the fused environmental information, the flight of distributed unmanned aerial vehicles is planned through global information guidance.

[0052] Step 1: Independent data acquisition.

[0053] The perception control model of independent data acquisition of UAV is as follows Figure 3 As shown, the kth drone u k The current spatial position information of the drone can be obtained through the onboard inertial navigation, Beidou and other positioning systems And the environmental information sensed by the airborne infrared, visible light, ultrasonic and other airborne sensors, mainly including the two-dimensional grid information of the environment and 3D elevation information Then the above method can be used to k The independently acquired environmental data is combined with its own location information to obtain the three-dimensional grid environmental information: And feed it back into the drone's flight control system.

[0054] The drone obtains the three-dimensional terrain information in the environment through the environmental perception sensor and converts it into two-dimensional grid information and three-dimensional elevation information (such as Figure 2 The flight control system obtains the status information of the UAV through the onboard status information sensor (such as inertial navigation, Beidou, etc.) and provides real-time feedback to the flight control system. Under the common closed-loop feedback of two-dimensional grid information, three-dimensional elevation information, and real-time status feedback of the UAV, the flight control system outputs a control signal according to the actual situation (the output of the control signal by the flight control system according to the actual situation is known to those skilled in the art and will not be repeated here), controls the flight of the UAV, and achieves the purpose of safe and controllable flight.

[0055] Step 2: Shared data association.

[0056] The feature extraction of the three-dimensional information perceived by UAV 1 and UAV 2 is performed. Based on the three-dimensional feature information, the two distributed UAVs are slidingly matched through the L window, the similarity of the three-dimensional feature points is calculated, and the highest similarity is selected for three-dimensional situation fusion, thereby forming a global three-dimensional terrain situation for the distributed UAVs to optimize the execution of subsequent tasks.

[0057] like Figure 4 As shown in the figure, assuming that UAV 1 and UAV 2 are marked as u1 and u2 respectively, the three-dimensional grid environment information obtained by the two UAVs at time t is and The feature points obtained in the three-dimensional grid information are and

[0058]

[0059] Where: (x q1 ,y q1 ),(x q2 ,y q2 ),(x q3 ,y q3 ),…,(x qn ,y qn ) and (x p1 ,y p1 ),(x p2 ,y p2 ),(x p3 ,y p3 ),…,(x qm ,y qm ) are the horizontal and vertical coordinates of the three-dimensional feature points in the two-dimensional coordinates.

[0060] Step 3: Distributed data matching.

[0061] According to the basic operating principle of UAV, the control model diagram of a single UAV perception model can be obtained (such as Figure 3 As shown in FIG. 1 , this technology is well known to those skilled in the art and will not be described again. On this basis, the information of each drone in the distributed drone is input into the drone distributed perception cloud sharing device constructed by the present invention (such as FIG. Figure 5 shown).

[0062] The UAV distributed perception cloud-based sharing device proposed in the present invention is based on the perception control model of independent data acquisition of a single UAV (this model is known to those skilled in the art), and the perception information obtained by a single UAV (mainly including three-dimensional grid map flight guidance information, UAV real-time status feedback information) is input into the sharing device for information processing. The process is as follows: The UAV distributed perception cloud-based sharing device performs coarse alignment of distributed perception information through a feature association algorithm, and then performs fine alignment through a sliding window similarity matching algorithm, and finally fuses the three-dimensional situation information Ξ(u1,u2) of the UAV distributed perception, so as to guide the flight of each UAV in a more optimized global information environment. The specific method is as follows:

[0063] The feature point information obtained by the two drones and In order to complete the 3D situation fusion of the two, the key is to find the most similar area between the two for analogy judgment. Set the sliding window L u1 and Lu2 respectively and in the independent sliding window, as Figure 4 shown, through the feature information extraction process as in Equation (4), so as to quantitatively calculate the similarity value of feature points. The specific method is as follows: Define the visual information and ( and respectively represent the calculation of feature point information u1 and on the basis of the sliding window sizes of L and (where the vertical bar in the middle indicates that the former is based on the latter, which is well-known to those skilled in the art). The similarity is:

[0064]

[0065] Where: and respectively represent the calculation of the bounding box of feature points for feature point information u1 and on the basis of the sliding window sizes of L and (

[0066] is an arbitrary symbol, is an existence symbol, and represent feature points that meet the above conditions, where the vertical bar indicates that the former is based on the latter; and are the sliding window similarity calculation functions constructed by the present invention. When traversing the feature information and after the drone perception and processing, the similarity value obtained on the basis of the sliding window sizes of L u1 and L u2 . Its calculation method is as follows:

[0067]

[0068] Where: represents any grid coordinate, and i and j respectively represent the horizontal grid increment and vertical grid increment of the grid coordinate, and are the abscissa and ordinate of the centroid of the sliding window.

[0069] The sliding windows L u1 and Lu2 By traversing each visual feature information separately and calculating the minimum difference between the two, the optimal fusion splicing area can be obtained:

[0070]

[0071] The final calculation result is a smaller number between 0 and 1. The value is compared with the similarity threshold (set manually, for example, 0.75). If it is greater than the threshold, the visual information within the window is considered and resemblance.

[0072] In the sliding window L u1 and L u2 Based on the three-dimensional feature point similarity calculation, the perception information of the two drones is combined according to the feature point information. and Perform situation fusion, including and are the surrounding environment information perceived by the two drones within the visual range, and the fused three-dimensional situation information Ξ(u1,u2) can be obtained:

[0073]

[0074] in: is the union function, which means the union operation is performed on the UAV visual information.

[0075] Step 4: Cross-linked situation fusion.

[0076] like Figure 5 As shown in Figure 1, assuming that there are N distributed UAVs in the environment, the data fusion method of formula (11) can be used to obtain the three-dimensional overall situation fusion information of the distributed UAVs for the environmental information:

[0077]

[0078] The present invention obtains environmental information through the drone's onboard sensors, and uses feature extraction and sliding window matching algorithms to achieve information sharing and situation fusion between drones. This method can not only improve the comprehensiveness and accuracy of information acquisition, but also optimize the flight control and mission execution of drones, and has important application value and broad development prospects.

Claims

1. A three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles, characterized in that: In the rectangular coordinate system constructed for the top view of the UAV, assume that for the kth UAV u k , the location information of the drone at this time is Environmental information perceived by the drone at its current location As shown in formula (1), it is the two-dimensional grid information And the grid elevation information based on this For any grid coordinate There is always elevation information on this grid point Make the three-dimensional grid map complete, that is: The drone is at the current spatial position at time t The acquired three-dimensional grid environment information for: The method specifically comprises the following steps: Step 1: Independent data acquisition; The perception control model of independent data acquisition of UAV is: k Get the spatial position information of the current drone And perceive environmental information, including two-dimensional grid information of the environment and 3D elevation information UAV k The independently acquired environmental data is combined with its own location information to obtain the three-dimensional grid environmental information: and feeds it back into the drone’s flight control system; The drone obtains three-dimensional terrain information in the environment through environmental perception sensors, and converts it into two-dimensional grid information and three-dimensional elevation information; obtains the drone's status information through the onboard status information sensor, and provides real-time feedback to the flight control system; under the common closed-loop feedback of two-dimensional grid information, three-dimensional elevation information, and real-time status feedback of the drone, the flight control system outputs control signals according to the actual situation to control the flight of the drone; Step 2: Shared data association; The feature extraction is performed on the 3D information sensed by UAV 1 and UAV 2. Based on the 3D feature information, the two distributed UAVs are matched by sliding through an L window, the similarity of the 3D feature points is calculated, and the highest similarity is selected for 3D situation fusion to form a global 3D terrain situation for the distributed UAVs to optimize the subsequent tasks. Assume that UAV 1 and UAV 2 are labeled as u1 and u2 respectively, and the three-dimensional grid environment information obtained by the two UAVs at time t is and The feature points obtained in the three-dimensional grid information are and Where: (x q1 ,y q1 ),(x q2 ,y q2 ),(x q3 ,y q3 ),…,(x qn ,y qn ) and (x p1 ,y p1 ),(x p2 ,y p2 ),(x p3 ,y p3 ),…,(x qm ,y qm ) are respectively the horizontal and vertical coordinates of the three-dimensional feature points in the two-dimensional coordinates; Step 3: Distributed data matching; Set the sliding window L u1 and L u2 They are and The independent sliding window in the window is subjected to the feature information extraction process as shown in formula (4), and the similarity value of the feature points is quantified and calculated. The specific method is as follows: and Similarity between for: Wherein: and respectively represent the calculation of the feature point range bounding on the basis of the sliding window sizes of L u1 and for the feature point information and where , , are arbitrary symbols, is the existence symbol, , and represent feature points that meet the above conditions, where 丨 means the former is based on the latter; are arbitrary symbols, is the existence symbol, and represent feature points that meet the above conditions, where 丨 means the former is based on the latter; and are sliding window similarity calculation functions, which are the similarity values obtained on the basis of the sliding window sizes of L and after traversing the feature information processed by the drone perception; the calculation method is as follows: u1 and L u2 size; the calculation method is as follows: Where: i and j represent the horizontal grid increment and the vertical grid increment of the grid coordinates respectively. and is the horizontal coordinate and vertical coordinate of the centroid of the sliding window; The sliding window L u1 and L u2 Traverse each visual feature information separately, and obtain the optimal fusion splicing area by calculating the minimum difference between the two: Will The value is compared with the similarity threshold. is greater than the similarity threshold, then the visual information within the window is considered and resemblance; Step 4: Cross-linked situation fusion; In the sliding window L u1 and L u2 Based on the three-dimensional feature point similarity calculation, the perception information of the two drones is combined according to the feature point information. and Perform situation fusion, including and are the surrounding environment information perceived by the two UAVs within the visual range, and the fused three-dimensional situation information Ξ(u1,u2) is obtained: in: is the union function, which represents the information of the drone feature points and The union operation performed.

2. The three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles according to claim 1 is characterized in that: In step 3, the similarity threshold is 0.

75.

3. The three-dimensional situation fusion method based on distributed perception of unmanned aerial vehicles according to claim 1 is characterized in that: Assuming that there are N distributed UAVs in the environment, in step 4, the data fusion method of formula (11) is used to obtain the three-dimensional overall situation fusion information of the distributed UAVs for the environmental information:

Images