Low-altitude unmanned aerial vehicle multi-source fusion positioning method and system

Abstract

The invention discloses a low-altitude unmanned aerial vehicle multi-source fusion positioning method and system, relates to the technical field of unmanned aerial vehicle positioning, and solves the technical problem that target positioning accuracy is affected due to the fact that it is difficult to recognize a target object in combination with various information data of the target object. By combining the data of the infrared image and the high-definition image and fully utilizing the information of the infrared image and the high-definition image, the accuracy of target detection and recognition is improved. Information about the surface temperature of an object is provided through infrared imaging, which is very important for identifying some types of target objects; through temperature distribution, the state or behavior of the target object can be better understood. The temperature information and the visual information are combined, so that objects with similar appearances but different thermal characteristics can be distinguished in a complex environment, and the classification effect is improved. By mapping the extracted temperature estimation data to the corresponding pixel points in the high-definition image, more accurate position positioning is realized, and a reliable basis is provided for subsequent tasks.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned aerial vehicle (UAV) positioning, and in particular relates to a multi-source fusion positioning method and system for low-altitude UAVs. Background Art

[0002] With the rapid development of drone technology, low-altitude drones are increasingly used in agricultural monitoring, environmental protection, urban management, disaster relief and other fields. However, the low-altitude flight environment is usually complex and dynamic, such as buildings, trees and other obstacles, which may pose challenges to the positioning accuracy of drones. Traditional single positioning methods often face problems such as weak signals, large drift or complete failure in these scenarios. Therefore, a method that can integrate data from multiple sensors is needed to improve the positioning accuracy and reliability of drones in complex environments. In recent years, multi-source data fusion technology has gradually become an important means to solve this problem. By fusing data from different sensors, such as GPS, IMU, lidar, visual sensors, etc., the limitations of a single sensor can be effectively overcome to achieve more accurate and stable positioning. This not only helps to improve the autonomous flight capabilities of drones, but also enhances their execution efficiency in specific tasks.

[0003] Most of the existing multi-source fusion positioning solutions for low-altitude UAVs locate targets by identifying them in high-definition images. It is difficult to identify the targets by combining multiple information data of the targets, which affects the accuracy of target positioning. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a low-altitude UAV multi-source fusion positioning method and system, which is used to solve the technical problem that it is difficult to identify the target object by combining multiple information data of the target object, affecting the accuracy of target positioning.

[0005] To solve the above problems, the first aspect of the present invention provides a low-altitude UAV multi-source fusion positioning method, comprising: The target location area is marked on the map, and the marked target location area is divided into blocks. At least one UAV is assigned to each search block for target search, and a specific flight trajectory is set for each UAV. The UAV flies low over a specific position of the search block according to the flight trajectory to collect image data of the search block; Collect the real-time position data, attitude data and 3D point cloud data in front of the drone, and collect infrared image data and high-definition image data from multiple directions of the drone; The temperature is estimated based on the infrared image data of the drone. The temperature estimation data extracted from the infrared image data of the drone is mapped to the corresponding pixel points of the high-definition image data. Combined with the contour information of the target object, the deviation coefficient between the detected similar contour information of the target object and the actual deviation coefficient of the target object is analyzed through the detected similar contour information and the temperature information of the corresponding pixel points. Filter similar profile information whose actual deviation coefficient is less than a preset threshold, and establish a target object tracking list according to the actual deviation coefficient of the filtered similar profile information; According to the target tracking list, move to the similar contour position one by one, establish the similar contour position tracking rules, track the target according to the similar contour position tracking rules, and calculate the target position information based on the position data, posture data of the drone that detects the target and the three-dimensional point cloud data in front of the drone.

[0006] As a further solution of the present invention: collecting the attitude data of the drone and the three-dimensional point cloud data in front of the drone, and at the same time, collecting the infrared image data and high-definition image data of the drone in multiple directions, including the following steps: The IMU inertial measurement unit is used to obtain the drone's attitude data in real time. The infrared thermal imaging camera is used to collect infrared image data in the four directions of front, bottom, left and right. The RGB camera is used to collect high-definition image data in the corresponding direction of the infrared thermal imaging camera. The front of the drone is scanned in real time through the laser radar to collect three-dimensional point cloud data in front of the drone.

[0007] As a further solution of the present invention: performing temperature estimation based on the infrared image data of the drone, and mapping the temperature estimation data extracted from the infrared image data of the drone to the corresponding pixel points of the high-definition image data, including the following steps: Align the infrared image and high-definition image data collected by the infrared thermal imaging camera and RGB camera in the same direction at the same time, and map several pixels in the corresponding high-definition image data to corresponding pixels in the infrared image; The temperature of each pixel of the infrared images acquired from four directions of the drone is estimated, and the obtained temperature estimate is associated with the pixel of the high-definition image mapped corresponding to the pixel of the infrared image.

[0008] As a further solution of the present invention, the temperature of each pixel point of the infrared images obtained from the drone in four directions is estimated, including the following steps: Collect a large number of infrared images of the same type of target objects under the same conditions in the target positioning area through an infrared camera, measure the temperature of the same type of target objects at different positions through an infrared thermometer, and record the corresponding temperature values; Map the acquisition position for temperature measurement to the pixel point of the infrared image of the target object, and use the recorded corresponding temperature value as the label of the corresponding pixel point; The convolutional neural network model is trained by adding labeled infrared images of similar substances to the target object; The infrared images collected in four directions by the drone in real time are input into the trained convolutional neural network model to estimate the temperature of each pixel in the infrared image.

[0009] As a further solution of the present invention: by using the detected similar contour information and the temperature information of the corresponding pixel point, analyzing the deviation coefficient between the detected similar contour of the target object and the actual deviation coefficient of the target object, the following steps are included: Collect temperature information of a large number of target objects of the same type under the same conditions in the target positioning area, set the target object temperature range according to the collected temperature information, and set the target object standard temperature value according to the average value of the collected temperature information; Collect a large amount of high-definition image data of similar target objects under the same conditions in the target positioning area, and mark the contours of similar target objects; The deep learning model is trained through labeled high-definition image data to identify the contour information of the target object; Input the high-definition image data collected in real time into the trained deep learning model to identify similar contours of the target object; The similar contour parts of the target object detected in the high-definition image data are marked, and the deviation coefficient between the detected similar contour of the target object and the actual target object is analyzed through the detected similar contour information and the temperature information of the corresponding pixel points.

[0010] As a further solution of the present invention: by using the detected similar contour information and the temperature information of the corresponding pixel point, analyzing the deviation coefficient between the detected similar contour of the target object and the actual deviation coefficient of the target object, the following steps are included: Grayscale processing is performed on the high-definition image of the outline of the target object of the same substance as the marked object, and the grayscale value range of the target object is set according to the maximum and minimum grayscale values ​​of each pixel point of the outline part; Extract the contour part of the high-definition image of the target object of the same substance after grayscale processing, scale it to the same size, and move all contours to the same position in the unified coordinate system for image fusion; Count the grayscale value of each pixel of the contour part image in the coordinates, divide it by the total number of contours, and get the average value as the standard grayscale value; Rescaling contour portions of similar contours of the target object retrieved from the high-definition image data to the same size; The grayscale value of the similar contour part of the retrieved target object is counted, and according to the temperature estimation value corresponding to each pixel point of the similar contour part of the target object, the detected similar contour information and the temperature information of the corresponding pixel point are used to analyze the deviation coefficient between the detected similar contour of the target object and the actual target object, which is performed by the following formula: Among them, μ is the actual deviation coefficient, G i is the gray value of the i-th pixel in the detected contour-like information, G0 is the standard gray value of the i-th pixel in the contour image in the coordinates, T i is the temperature estimation value of the i-th pixel point in the detected similar contour information, T0 is the standard temperature value of the target object, i∈(1, 2, …, n).

[0011] As a further solution of the present invention: according to the target tracking list, the objects are moved to the similar contour positions one by one, and similar contour position tracking rules are established, including the following steps: According to the target tracking list, the drone leaves the flight track and moves to similar contour positions one by one, and establishes the following similar contour position tracking rules: According to the drone attitude data and the 3D point cloud data in front of the drone, the distance between the similar contour position and the drone is detected. If the distance exceeds the preset threshold, the movement to the corresponding similar contour position is abandoned and deleted from the target tracking list. Otherwise, the movement to the similar contour position is performed. The drone moves toward the corresponding similar contour position and analyzes the actual deviation coefficient between the detected similar contour position and the target object in real time based on the collected data. If the actual deviation coefficient is less than the minimum similarity threshold, the drone stops moving toward the corresponding similar contour position and deletes it from the target tracking list. Otherwise, it continues moving toward the similar contour position. If the distance between the detected similar contour position and the drone is less than the preset detection distance, and the actual deviation coefficient is greater than the maximum similarity threshold, it is determined that the target has been locked. Otherwise, it abandons moving to the corresponding similar contour position and deletes it from the target tracking list. If it is determined that the target has been locked, other drones will be stopped immediately to stop the target search.

[0012] As a further solution of the present invention: tracking the target object according to the similar contour position tracking rule, and calculating the target object position information according to the position data, attitude data of the drone detecting the target object and the three-dimensional point cloud data in front of the drone, including the following steps: According to the similar contour position tracking rule, the target object is tracked, and the UAV moves to the similar contour position. At the same time, the three-dimensional point cloud data of the similar contour position direction in front of the UAV is detected; Obtain a high-definition image of a similar outline of a marked target object collected in front of the drone, map the similar outline of the marked target object to the three-dimensional point cloud data in front of the drone, and calculate the target object position information based on the position data and posture data of the drone that detects the target object and the mapping relationship of the similar outline in the three-dimensional point cloud data in front of the drone.

[0013] As a further solution of the present invention: according to the position data and attitude data of the drone detecting the target object, and the mapping relationship of the three-dimensional point cloud data of the similar contour in front of the drone, the target object position information is calculated, including the following steps: According to the GPS position and attitude data of the drone, the drone's own coordinate system is converted into the world coordinate system; Convert the acquired 3D point cloud data from the LiDAR coordinate system to the world coordinate system; According to the internal reference data of the RGB camera, the coordinates of the contour part of the high-definition image similar to the contour of the marked target object are mapped to the world coordinate system, and the depth value of each contour point of the contour part is calculated; According to the depth value of each contour point in the contour part, the mean distance from each contour point to each contour point position is calculated as the distance between the detected target and the UAV. According to the GPS position and attitude data of the UAV and the distance between the detected target and the UAV, the target position information is obtained.

[0014] As another aspect of the present invention: a low-altitude UAV multi-source fusion positioning system, the system adopts a low-altitude UAV multi-source fusion positioning method as mentioned above to achieve low-altitude UAV multi-source fusion positioning.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention combines the data of infrared images and high-definition images, making full use of the information of both, and improving the accuracy of target detection and recognition. Infrared imaging provides information about the surface temperature of objects, which is very important for identifying certain types of target objects; through temperature distribution, the state or behavior of the target object can be better understood. Combining temperature information with visual information helps to distinguish objects with similar appearance but different thermal characteristics in complex environments, thereby improving the classification effect. By mapping the extracted temperature estimation data to the corresponding pixel points in the high-definition image, more accurate position positioning can be achieved, providing a reliable basis for subsequent tasks.

[0016] The present invention moves to similar contour positions one by one according to the target object tracking list, establishes similar contour position tracking rules, tracks the target object according to the similar contour position tracking rules, and calculates the target object position information according to the position data, attitude data of the drone that detects the target object and the three-dimensional point cloud data in front of the drone; by combining the position information, attitude information and three-dimensional point cloud data of the drone, it is convenient to improve the estimation accuracy of the target object position, thereby achieving more accurate positioning, and by dynamically acquiring and processing multi-source data, the position estimation of the target object can be adjusted in real time to improve the response speed; by establishing similar contour position tracking rules, the drone can flexibly respond to environmental changes or target object movement, and achieve continuous and effective tracking. According to the tracking rules, the optimal flight route is planned so that the drone can efficiently cover all relevant areas, thereby saving energy and time. Concentrating resources on confirmed targets helps to improve overall mission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 This is a flow chart of a method for mapping temperature estimation data to corresponding pixels of high-definition image data according to the present invention. DETAILED DESCRIPTION

[0019] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] See also Figure 1-Figure 2 The first embodiment of the present invention provides a low-altitude UAV multi-source fusion positioning method, comprising: The target location area is marked on the map, and the marked target location area is divided into blocks. At least one UAV is assigned to each search block for target search, and a specific flight trajectory is set for each UAV. The UAV flies low over a specific position of the search block according to the flight trajectory to collect image data of the search block; Collect the real-time position data, attitude data and 3D point cloud data in front of the drone, and collect infrared image data and high-definition image data from multiple directions of the drone; The temperature is estimated based on the infrared image data of the drone. The temperature estimation data extracted from the infrared image data of the drone is mapped to the corresponding pixel points of the high-definition image data. Combined with the contour information of the target object, the detected similar contour information and the temperature information of the corresponding pixel points are used to analyze the deviation coefficient between the detected similar contour of the target object and the actual deviation coefficient of the target object. Filter similar profile information whose actual deviation coefficient is less than a preset threshold, and establish a target object tracking list according to the actual deviation coefficient of the filtered similar profile information; According to the target tracking list, move to the similar contour position one by one, establish the similar contour position tracking rules, track the target according to the similar contour position tracking rules, and calculate the target position information based on the position data, posture data of the drone that detects the target and the three-dimensional point cloud data in front of the drone.

[0021] Specifically, in this embodiment, GIS software, QGIS map tool or ArcGIS map tool is used to manually draw the target positioning area to be searched.

[0022] Mark the selected target area on the map, draw the bounding box or polygon of the area according to the actual situation, and save its coordinate information.

[0023] Divide the marked target positioning area into blocks: Grid division: Divide the entire target area into grids of a certain size, for example, into 100m x 100m grids. This function can be achieved using the numpy library in Python or GIS software.

[0024] Each drone is assigned to each block based on the number of blocks and the number of available drones.

[0025] Set the flight to start from any point on the edge of a target area and gradually move inward to form a spiral path. Set a specific flight trajectory for each drone to cover the search area it is responsible for.

[0026] In this embodiment, temperature estimation is performed based on the infrared image data of the drone, and the temperature estimation data extracted from the infrared image data of the drone is mapped to the corresponding pixel points of the high-definition image data. Combined with the contour information of the target object, the detected similar contour information and the temperature information of the corresponding pixel points are used to analyze the deviation coefficient between the detected similar contour of the target object and the actual deviation coefficient of the target object; By combining the data of infrared images and high-definition images, the information of both can be fully utilized to improve the accuracy of target detection and recognition. Infrared imaging provides information about the surface temperature of objects, which is very important for identifying certain types of target objects, such as mechanical equipment, animals, etc. The temperature distribution can better understand the state or behavior of the target object. Combining temperature information with visual information helps to distinguish objects with similar appearance but different thermal characteristics in complex environments, thereby improving classification results. By mapping the extracted temperature estimation data to the corresponding pixel points in the high-definition image, more accurate positioning can be achieved, providing a reliable basis for subsequent tasks.

[0027] Filter similar contour information with actual deviation coefficient less than the preset threshold, and establish a target tracking list based on the actual deviation coefficient of the filtered similar contour information; by setting the deviation coefficient threshold, it is convenient to effectively exclude contours that do not match the characteristics of the target object, thereby reducing the false alarm rate and improving the tracking accuracy. Filter out high-credibility targets for tracking, which is convenient for reducing the amount of calculation and storage required for subsequent processing and improving system efficiency. Establish a target tracking list based on a reliable data source, making subsequent analysis, decision-making and operation more intuitive and easy to manage.

[0028] According to the target tracking list, move to similar contour positions one by one, establish similar contour position tracking rules, track the target according to similar contour position tracking rules, and calculate the target position information based on the position data, attitude data of the drone that detects the target and the three-dimensional point cloud data in front of the drone; by combining the drone's position information, attitude information and three-dimensional point cloud data, it is easy to improve the estimation accuracy of the target object's position, thereby achieving more accurate positioning, and by dynamically acquiring and processing multi-source data, the position estimation of the target object can be adjusted in real time to improve the response speed; by establishing similar contour position tracking rules, the drone can flexibly respond to environmental changes or target object movement to achieve continuous and effective tracking. According to the tracking rules, the optimal flight route is planned so that the drone can efficiently cover all relevant areas, thereby saving energy and time. Focusing resources on confirmed targets helps improve overall mission efficiency.

[0029] The temperature is estimated based on the infrared image data of the drone. The temperature estimation data extracted from the infrared image data of the drone is mapped to the corresponding pixel points of the high-definition image data. Combined with the contour information of the target object, the deviation coefficient between the detected similar contour information of the target object and the actual deviation coefficient of the target object is analyzed through the detected similar contour information and the temperature information of the corresponding pixel points. Filter similar profile information whose actual deviation coefficient is less than a preset threshold, and establish a target object tracking list according to the actual deviation coefficient of the filtered similar profile information; According to the target tracking list, move to the similar contour position one by one, establish the similar contour position tracking rules, track the target according to the similar contour position tracking rules, and calculate the target position information based on the position data, posture data of the drone that detects the target and the three-dimensional point cloud data in front of the drone.

[0030] In one embodiment of the present invention, collecting the attitude data of the drone and the three-dimensional point cloud data in front of the drone, and collecting the infrared image data and high-definition image data of the drone in multiple directions, includes the following steps: The IMU inertial measurement unit is used to obtain the drone's attitude data in real time. The infrared thermal imaging camera is used to collect infrared image data in the four directions of front, bottom, left and right. The RGB camera is used to collect high-definition image data in the corresponding direction of the infrared thermal imaging camera. The front of the drone is scanned in real time through the laser radar to collect three-dimensional point cloud data in front of the drone.

[0031] In one embodiment of the present invention, temperature estimation is performed based on infrared image data of a drone, and temperature estimation data extracted from the infrared image data of the drone is mapped to corresponding pixel points of high-definition image data, including the following steps: Align the infrared image and high-definition image data collected by the infrared thermal imaging camera and RGB camera in the same direction at the same time, and map several pixels in the corresponding high-definition image data to corresponding pixels in the infrared image; The temperature of each pixel of the infrared images acquired from four directions of the drone is estimated, and the obtained temperature estimate is associated with the pixel of the high-definition image mapped corresponding to the pixel of the infrared image.

[0032] In one embodiment of the present invention, estimating the temperature of each pixel of infrared images acquired from four directions of the drone includes the following steps: Collect a large number of infrared images of the same type of target objects under the same conditions in the target positioning area through an infrared camera, measure the temperature of the same type of target objects at different positions through an infrared thermometer, and record the corresponding temperature values; Map the acquisition position for temperature measurement to the pixel point of the infrared image of the same material as the target object, and use the recorded corresponding temperature value as the label of the corresponding pixel point; The convolutional neural network model is trained by adding labeled infrared images of similar substances to the target object; The infrared images collected in four directions by the drone in real time are input into the trained convolutional neural network model to estimate the temperature of each pixel in the infrared image.

[0033] In one embodiment of the present invention, the deviation coefficient between the detected similar contour and the actual contour of the target object is analyzed by using the detected similar contour information and the temperature information of the corresponding pixel point, including the following steps: Collect temperature information of a large number of target objects of the same type under the same conditions in the target positioning area, set the target object temperature range according to the collected temperature information, and set the target object standard temperature value according to the average value of the collected temperature information; Collect a large amount of high-definition image data of similar target objects under the same conditions in the target positioning area, and mark the contours of similar target objects; The deep learning model is trained through labeled high-definition image data to identify the contour information of the target object; Input the high-definition image data collected in real time into the trained deep learning model to identify similar contours of the target object; The similar contour parts of the target object detected in the high-definition image data are marked, and the deviation coefficient between the detected similar contour of the target object and the actual target object is analyzed through the detected similar contour information and the temperature information of the corresponding pixel points.

[0034] In one embodiment of the present invention, the deviation coefficient between the detected similar contour and the actual contour of the target object is analyzed by using the detected similar contour information and the temperature information of the corresponding pixel point, including the following steps: Grayscale processing is performed on the high-definition image of the outline of the target object of the same substance as the marked object, and the grayscale value range of the target object is set according to the maximum and minimum grayscale values ​​of each pixel point of the outline part; Extract the contour part of the high-definition image of the target object of the same substance after grayscale processing, scale it to the same size, and move all contours to the same position in the unified coordinate system for image fusion; Count the grayscale value of each pixel of the contour part image in the coordinates, divide it by the total number of contours, and get the average value as the standard grayscale value; Rescaling contour portions of similar contours of the target object retrieved from the high-definition image data to the same size; The grayscale value of the similar contour part of the retrieved target object is counted, and according to the temperature estimation value corresponding to each pixel point of the similar contour part of the target object, the detected similar contour information and the temperature information of the corresponding pixel point are used to analyze the deviation coefficient between the detected similar contour of the target object and the actual target object, which is performed by the following formula: Among them, μ is the actual deviation coefficient, G i is the gray value of the i-th pixel in the detected contour-like information, G0 is the standard gray value of the i-th pixel in the contour part image in the coordinate, T iis the temperature estimation value of the i-th pixel in the detected similar contour information, T0 is the standard temperature value of the target object, i∈(1, 2, …, n).

[0035] In one embodiment of the present invention, according to the target tracking list, the target objects are moved to the similar contour positions one by one, and the similar contour position tracking rules are established, including the following steps: According to the target tracking list, the drone leaves the flight track and moves to similar contour positions one by one, and establishes the following similar contour position tracking rules: According to the drone attitude data and the 3D point cloud data in front of the drone, the distance between the similar contour position and the drone is detected. If the distance exceeds the preset threshold, the movement to the corresponding similar contour position is abandoned and deleted from the target tracking list. Otherwise, the movement to the similar contour position is performed. The drone moves toward the corresponding similar contour position and analyzes the actual deviation coefficient between the detected similar contour position and the target object in real time based on the collected data. If the actual deviation coefficient is less than the minimum similarity threshold, the drone stops moving toward the corresponding similar contour position and deletes it from the target tracking list. Otherwise, it continues moving toward the similar contour position. If the distance between the detected similar contour position and the drone is less than the preset detection distance, and the actual deviation coefficient is greater than the maximum similarity threshold, it is determined that the target has been locked. Otherwise, it abandons moving to the corresponding similar contour position and deletes it from the target tracking list. If it is determined that the target has been locked, other drones will be stopped immediately to stop the target search.

[0036] In one embodiment of the present invention, tracking a target object according to a similar contour position tracking rule, and calculating the position information of the target object according to the position data, attitude data of a drone detecting the target object and the three-dimensional point cloud data in front of the drone, comprises the following steps: According to the similar contour position tracking rule, the target object is tracked, and the UAV moves to the similar contour position. At the same time, the three-dimensional point cloud data of the similar contour position direction in front of the UAV is detected; Obtain a high-definition image of a similar outline of a marked target object collected in front of the drone, map the similar outline of the marked target object to the three-dimensional point cloud data in front of the drone, and calculate the target object position information based on the position data and posture data of the drone that detects the target object and the mapping relationship of the similar outline in the three-dimensional point cloud data in front of the drone.

[0037] In one embodiment of the present invention, the target object position information is calculated according to the position data and attitude data of the drone that detects the target object and the mapping relationship of the three-dimensional point cloud data of the similar contour in front of the drone, including the following steps: According to the GPS position and attitude data of the drone, the drone's own coordinate system is converted into the world coordinate system; Convert the acquired 3D point cloud data from the LiDAR coordinate system to the world coordinate system; According to the internal reference data of the RGB camera, the coordinates of the contour part of the high-definition image similar to the contour of the marked target object are mapped to the world coordinate system, and the depth value of each contour point of the contour part is calculated; According to the depth value of each contour point in the contour part, the mean distance from each contour point to each contour point position is calculated as the distance between the detected target and the UAV. According to the GPS position and attitude data of the UAV and the distance between the detected target and the UAV, the target position information is obtained.

[0038] Specifically, in this embodiment, a local coordinate system is established with the current position of the drone as the origin. In this coordinate system, the flight direction of the drone can be defined as the X-axis, the vertical downward is the Z-axis, and the Y-axis is perpendicular to the XZ plane.

[0039] According to the attitude data of the UAV, including: roll angle, pitch angle and yaw angle, the attitude transformation matrix is ​​calculated, and the matrix is ​​used to transform the position of the target object in the local coordinate system into the position in the global geographic coordinate system.

[0040] In the local coordinate system, the position of the target object is represented as a distance vector whose length is equal to the distance measured by the sensor and whose direction is determined by the orientation of the drone and the direction of the target object relative to the drone.

[0041] The distance vector is converted from the local coordinate system to the global geographic coordinate system using the attitude transformation matrix, and then added to the GPS position of the drone to obtain the position of the target object in the global geographic coordinate system.

[0042] In an embodiment of another aspect of the present invention, a low-altitude UAV multi-source fusion positioning system is provided, which adopts a low-altitude UAV multi-source fusion positioning method as described above to achieve low-altitude UAV multi-source fusion positioning.

[0043] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A low-altitude UAV multi-source fusion positioning method, characterized in that: include: The target location area is marked on the map, and the marked target location area is divided into blocks. At least one UAV is assigned to each search block for target search, and a specific flight trajectory is set for each UAV. The UAV flies low over a specific position of the search block according to the flight trajectory to collect image data of the search block; Collect the real-time position data, attitude data and 3D point cloud data in front of the drone, and collect infrared image data and high-definition image data from multiple directions of the drone; The temperature is estimated based on the infrared image data of the drone. The temperature estimation data extracted from the infrared image data of the drone is mapped to the corresponding pixel points of the high-definition image data. Combined with the contour information of the target object, the detected similar contour information and the temperature information of the corresponding pixel points are used to analyze the deviation coefficient between the detected similar contour of the target object and the actual deviation coefficient of the target object. Filter similar profile information whose actual deviation coefficient is less than a preset threshold, and establish a target object tracking list according to the actual deviation coefficient of the filtered similar profile information; According to the target tracking list, move to the similar contour position one by one, establish the similar contour position tracking rules, track the target according to the similar contour position tracking rules, and calculate the target position information based on the position data, posture data of the drone that detects the target and the three-dimensional point cloud data in front of the drone.

2. A low-altitude UAV multi-source fusion positioning method according to claim 1, characterized in that: Collect the attitude data of the drone and the three-dimensional point cloud data in front of the drone. At the same time, collect the infrared image data and high-definition image data of the drone in multiple directions, including the following steps: The IMU inertial measurement unit is used to obtain the drone's attitude data in real time. The infrared thermal imaging camera is used to collect infrared image data in the four directions of front, bottom, left and right. The RGB camera is used to collect high-definition image data in the corresponding direction of the infrared thermal imaging camera. The front of the drone is scanned in real time through the laser radar to collect three-dimensional point cloud data in front of the drone.

3. The low-altitude UAV multi-source fusion positioning method according to claim 1 is characterized in that: The temperature is estimated according to the infrared image data of the drone, and the temperature estimation data extracted from the infrared image data of the drone is mapped to the corresponding pixel points of the high-definition image data, including the following steps: Align the infrared image and high-definition image data collected by the infrared thermal imaging camera and RGB camera in the same direction at the same time, and map several pixels in the corresponding high-definition image data to corresponding pixels in the infrared image; The temperature of each pixel of the infrared images acquired from four directions of the drone is estimated, and the obtained temperature estimate is associated with the pixel of the high-definition image mapped corresponding to the pixel of the infrared image.

4. The low-altitude UAV multi-source fusion positioning method according to claim 2 is characterized in that: The temperature of each pixel point of the infrared images acquired from the drone in four directions is estimated, including the following steps: Collect a large number of infrared images of the same type of target objects under the same conditions in the target positioning area through an infrared camera, measure the temperature of the same type of target objects at different positions through an infrared thermometer, and record the corresponding temperature values; Map the acquisition position for temperature measurement to the pixel point of the infrared image of the target object, and use the recorded corresponding temperature value as the label of the corresponding pixel point; The convolutional neural network model is trained by adding labeled infrared images of similar substances to the target object; The infrared images collected in four directions by the drone in real time are input into the trained convolutional neural network model to estimate the temperature of each pixel in the infrared image.

5. The low-altitude UAV multi-source fusion positioning method according to claim 1 is characterized in that: Analyzing the deviation coefficient between the detected similar contour information and the actual deviation coefficient between the detected similar contour and the target object by using the detected similar contour information and the temperature information of the corresponding pixel point, including the following steps: Collect temperature information of a large number of target objects of the same type under the same conditions in the target positioning area, set the target object temperature range according to the collected temperature information, and set the target object standard temperature value according to the average value of the collected temperature information; Collect a large amount of high-definition image data of similar target objects under the same conditions in the target positioning area, and mark the contours of similar target objects; The deep learning model is trained through labeled high-definition image data to identify the contour information of the target object; Input the high-definition image data collected in real time into the trained deep learning model to identify similar contours of the target object; The similar contour parts of the target object detected in the high-definition image data are marked, and the deviation coefficient between the detected similar contour of the target object and the actual target object is analyzed through the detected similar contour information and the temperature information of the corresponding pixel points.

6. The multi-source fusion positioning method for low-altitude UAV according to claim 5 is characterized in that: Analyzing the deviation coefficient between the detected similar contour information and the actual deviation coefficient between the detected similar contour and the target object by using the detected similar contour information and the temperature information of the corresponding pixel point, including the following steps: Grayscale processing is performed on the high-definition image of the outline of the target object of the same substance as the marked object, and the grayscale value range of the target object is set according to the maximum and minimum grayscale values ​​of each pixel point of the outline part; Extract the contour part of the high-definition image of the target object of the same substance after grayscale processing, scale it to the same size, and move all contours to the same position in the unified coordinate system for image fusion; Count the grayscale value of each pixel of the contour part image in the coordinates, divide it by the total number of contours, and get the average value as the standard grayscale value; Rescaling contour portions of similar contours of the target object retrieved from the high-definition image data to the same size; The grayscale value of the similar contour part of the retrieved target object is counted, and according to the temperature estimation value corresponding to each pixel point of the similar contour part of the target object, the detected similar contour information and the temperature information of the corresponding pixel point are used to analyze the deviation coefficient between the detected similar contour of the target object and the actual target object, which is performed by the following formula: Among them, μ is the actual deviation coefficient, G i is the gray value of the i-th pixel in the detected contour-like information, G0 is the standard gray value of the i-th pixel in the contour part image in the coordinate, T i is the temperature estimation value of the i-th pixel in the detected similar contour information, T0 is the standard temperature value of the target object, i∈(1, 2, …, n).

7. The low-altitude UAV multi-source fusion positioning method according to claim 1 is characterized in that: According to the target tracking list, move to similar contour positions one by one, and establish similar contour position tracking rules, including the following steps: According to the target tracking list, the drone leaves the flight track and moves to similar contour positions one by one, and establishes the following similar contour position tracking rules: According to the drone attitude data and the 3D point cloud data in front of the drone, the distance between the similar contour position and the drone is detected. If the distance exceeds the preset threshold, the movement to the corresponding similar contour position is abandoned and deleted from the target tracking list. Otherwise, the movement to the similar contour position is performed. The drone moves toward the corresponding similar contour position and analyzes the actual deviation coefficient between the detected similar contour position and the target object in real time based on the collected data. If the actual deviation coefficient is less than the minimum similarity threshold, the drone stops moving toward the corresponding similar contour position and deletes it from the target tracking list. Otherwise, it continues moving toward the similar contour position. If the distance between the detected similar contour position and the drone is less than the preset detection distance, and the actual deviation coefficient is greater than the maximum similarity threshold, it is determined that the target has been locked. Otherwise, it abandons moving to the corresponding similar contour position and deletes it from the target tracking list. If it is determined that the target has been locked, other drones will be stopped immediately to stop the target search.

8. The low-altitude UAV multi-source fusion positioning method according to claim 1 is characterized in that: Tracking the target object according to the similar contour position tracking rule, and calculating the target object position information according to the position data, attitude data of the drone detecting the target object and the three-dimensional point cloud data in front of the drone, including the following steps: According to the similar contour position tracking rule, the target object is tracked, and the UAV moves to the similar contour position. At the same time, the three-dimensional point cloud data of the similar contour position direction in front of the UAV is detected; Obtain a high-definition image of a similar outline of a marked target object collected in front of the drone, map the similar outline of the marked target object to the three-dimensional point cloud data in front of the drone, and calculate the target object position information based on the position data and posture data of the drone that detects the target object and the mapping relationship of the similar outline in the three-dimensional point cloud data in front of the drone.

9. A low-altitude UAV multi-source fusion positioning method according to claim 8, characterized in that: According to the position data and attitude data of the drone that detects the target object, and the mapping relationship of the three-dimensional point cloud data of the similar contour in front of the drone, the target object position information is calculated, including the following steps: According to the GPS position and attitude data of the drone, the drone's own coordinate system is converted into the world coordinate system; Convert the acquired 3D point cloud data from the LiDAR coordinate system to the world coordinate system; According to the internal reference data of the RGB camera, the coordinates of the contour part of the high-definition image similar to the contour of the marked target object are mapped to the world coordinate system, and the depth value of each contour point of the contour part is calculated; According to the depth value of each contour point in the contour part, the mean distance from each contour point to each contour point position is calculated as the distance between the detected target and the UAV. According to the GPS position and attitude data of the UAV and the distance between the detected target and the UAV, the target position information is obtained.

10. A low-altitude UAV multi-source fusion positioning system, characterized in that: The system adopts a low-altitude UAV multi-source fusion positioning method as described in any one of claims 1-9 to achieve low-altitude UAV multi-source fusion positioning.

Images