Unmanned aerial vehicle positioning method and device, electronic equipment, storage medium and computer program product

Abstract

The invention relates to the technical field of unmanned aerial vehicles, and provides an unmanned aerial vehicle positioning method and device, electronic equipment, a storage medium and a computer program product, and the method comprises the steps: carrying out the pose evaluation based on a target odometer and an image shot by an unmanned aerial vehicle, and obtaining the pose information of the unmanned aerial vehicle; the target odometer is formed by fusing the measurement distance of the single-point laser radar on the basis of the visual inertia odometer; performing chart matching based on an image shot by the unmanned aerial vehicle to obtain position information of the unmanned aerial vehicle; and performing information fusion based on the pose information and the position information to obtain a positioning result of the unmanned aerial vehicle. The single-point laser radar and the visual inertial odometer are combined to form the target odometer, so that the problem of accumulative errors in long-distance movement can be effectively improved, and more accurate pose evaluation is realized; information fusion is carried out based on the pose information and the position information, and the advantage that the odometer is matched with the chart can be fully utilized, so that the accuracy of unmanned aerial vehicle positioning can be improved.

Description

Technical Field

[0001] The present application relates to the field of drone technology, and in particular to a drone positioning method, device, electronic device, storage medium and computer program product. Background Art

[0002] With the advancement of technology and the emergence of geopolitical challenges, the demand for reliable and safe positioning of UAV systems continues to increase. The Global Navigation Satellite System (GNSS) is an important basis for UAV flight navigation, and its signals are also the most susceptible to interference. Interference with the GNSS signal is one of the key reasons for the failure or even damage of UAV systems. In order to maintain a certain working and safety guarantee capability of the UAV after the GNSS signal is interfered, vision-based UAV positioning technology has emerged. Vision-based UAV positioning technology mainly solves the problem of failure of UAV positioning and navigation information in the case of GNSS signal denial or interference. It mainly uses vision and integrates sensors such as inertial navigation to solve and provide position information, so that the UAV can continue to work or return and land safely based on the position information. Visual-Inertial Odometry (VIO) is another commonly used UAV positioning solution. This method calculates the position of the UAV by tracking continuous image frames and integrating inertial measurement unit (IMU) information. However, with long-distance flight, the visual inertial odometer will have a large cumulative error. Simultaneous Localization and Mapping (SLAM) can partially solve the problem of cumulative errors through loop detection, but it requires the task to pass through the places it has passed before. Both VIO and SLAM can only perform relative positioning and cannot output geographic positioning results.

[0003] As a result, the accuracy of drone positioning based on current existing technologies is low. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a method, device, electronic device, storage medium and computer program product for positioning a drone to improve the accuracy of positioning a drone.

[0005] According to the first aspect of the present application, the drone positioning method includes: The pose information of the UAV is obtained by performing pose evaluation based on the target odometer and the image taken by the UAV; the target odometer is formed by integrating the measurement distance of the single-point laser radar on the basis of the visual inertial odometer; Performing chart matching based on the image taken by the drone to obtain the location information of the drone; Based on the posture information and the position information, information fusion is performed to obtain the positioning result of the UAV.

[0006] According to an embodiment of the present application, performing chart matching based on the image taken by the drone to obtain the location information of the drone includes: Based on the image captured by the drone, determining at least one candidate slice from map slices of a preset offline map; Based on the image taken by the drone, feature points are matched with each candidate slice to obtain a matching result; Based on the matching result, determining a target slice from each candidate slice; Based on the image taken by the drone and the target slice, the position information of the drone is determined.

[0007] According to an embodiment of the present application, determining at least one candidate slice from map slices of a preset offline map based on the image taken by the drone includes: If there is a historical positioning result, determining an initial slice set from each map slice of a preset offline map according to the historical positioning result; determining a first global descriptor of the image taken by the drone; and determining at least one candidate slice from the initial slice set based on the first global descriptor and a second global descriptor corresponding to each map slice in the initial slice set; If there is no historical positioning result, determine the first global descriptor of the image taken by the drone; based on the first global descriptor and the second global descriptor corresponding to each map slice in the preset offline map, determine at least one candidate slice from each map slice of the preset offline map.

[0008] According to an embodiment of the present application, determining at least one candidate slice from the initial slice set based on the first global descriptor and the second global descriptor corresponding to each map slice in the initial slice set respectively includes: Determine the distance between the first global descriptor and the second global descriptor corresponding to each map slice in the initial slice set; determining at least one target distance from the distances less than a preset distance threshold; A map slice corresponding to the target distance in the initial slice set is determined as a candidate slice.

[0009] According to an embodiment of the present application, determining the location information of the drone based on the image taken by the drone and the target slice includes: Determine the homography transformation between the image taken by the drone and the target slice to obtain a homography matrix; According to the homography matrix, projecting the center point of the image taken by the drone to the target slice; The geographic coordinates of the projection point are used as the location information of the UAV.

[0010] According to one embodiment of the present application, the expression of the cost function of the target odometer is as follows: ; in, represents the cost function of visual inertial odometry; represents the laser ranging residual; represents the weight of the laser ranging residual; F represents the set of all measured distances in the sliding window.

[0011] According to the second aspect of the present application, the drone positioning device includes: An evaluation module is used to perform posture evaluation based on a target odometer and an image taken by a UAV to obtain the posture information of the UAV; the target odometer is formed by integrating the distance measurement of a single-point laser radar on the basis of a visual inertial odometer; A matching module, used to perform chart matching based on the image taken by the drone to obtain the location information of the drone; A fusion module is used to perform information fusion based on the posture information and the position information to obtain a positioning result of the drone.

[0012] According to an electronic device of an embodiment of the third aspect of the present application, the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the drone positioning method described above is implemented.

[0013] According to the storage medium of the fourth aspect embodiment of the present application, the storage medium is a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements any of the above-mentioned drone positioning methods.

[0014] A computer program product according to an embodiment of the fifth aspect of the present application includes a computer program, which, when executed by a processor, implements any of the above-mentioned drone positioning methods.

[0015] The above one or more technical solutions in the embodiments of the present application have at least the following technical effects: By combining a single-point lidar with a visual inertial odometer to form a target odometer, the cumulative error problem in long-distance movement can be effectively improved, and a more accurate posture assessment can be achieved. Therefore, by performing posture assessment based on the target odometer and the images taken by the drone, accurate posture information of the drone can be obtained; and by performing chart matching based on the images taken by the drone, the location information of the drone can be obtained; and then by performing information fusion based on the posture information and the location information, the advantages of the odometer and chart matching can be fully utilized to obtain accurate positioning results of the drone, thereby improving the accuracy of drone positioning.

[0016] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the UAV positioning method provided in the embodiment of the present application.

[0019] Figure 2 It is a schematic diagram of the geometric relationship of the ranging model described in the embodiment of the present application.

[0020] Figure 3 It is a flow chart of information fusion in the UAV positioning method provided in the embodiment of the present application.

[0021] Figure 4 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0022] The following is a further detailed description of the implementation of the present application in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but cannot be used to limit the scope of the present application.

[0023] In the description of the embodiments of the present application, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0024] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0025] In the embodiments of the present application, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0026] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0027] The present application proposes a drone positioning method, device, electronic device, storage medium and computer program product.

[0028] Figure 1 is a flow chart of the drone positioning method provided in the embodiment of the present application, such as Figure 1 As shown, the UAV positioning method includes: Step 110, perform posture evaluation based on the target odometer and the image taken by the UAV to obtain the posture information of the UAV; the target odometer is formed by integrating the measurement distance of the single-point laser radar on the basis of the visual inertial odometer.

[0029] Step 120: performing map matching based on the image taken by the drone to obtain the location information of the drone.

[0030] Step 130, performing information fusion based on the posture information and the position information to obtain the positioning result of the UAV.

[0031] It should be noted that the executor of the drone positioning method provided in the embodiment of the present application may be a drone and a computer device that can be connected to the drone, etc. The computer device may be, for example, a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc.

[0032] The drone or computer device of the present application may be provided with or connected with a drone positioning device, thereby controlling the drone positioning device to execute the drone positioning method of the present application. The drone positioning device of the present application may include an evaluation module, a matching module and a fusion module, wherein the evaluation module may specifically be an odometer, the matching module may specifically be a chart matching module, and the fusion module may specifically be a fusion framework.

[0033] The role of the odometer in this application is to track continuous image frames, estimate the six-degree-of-freedom position of the drone, and achieve a higher frequency of positioning result output.

[0034] Specifically, this application integrates the distance measurement of a single-point laser radar (Laser Range Finder, LRF) on the basis of a conventional visual inertial odometry to obtain a range visual inertial odometry (Range-Visual-Inertial-Odometry, RVIO) and uses it as a target odometry to improve the operating effect of the odometry.

[0035] It should be noted that the visual inertial odometer is based on the monocular visual inertial navigation system (Visual-Inertial Navigation System-Mono, VINS-Mono) and adopts a nonlinear optimization method based on sliding window keyframes. When constructing the optimization problem, the odometer uses the state of the drone as the optimization quantity, the visual reprojection error and IMU pre-integration as constraints, and solves it through the Gauss-Newton (GN) or Levenberg-Marquardt (LM) method to obtain the position and posture of the camera at each moment.

[0036] The state of the drone is defined as ,in Represents the inverse depth of the map point initially observed by the camera, Represents the state of m image frames. The kth image frame state contains the position of the drone ,speed ,direction And IMU zero bias ba and bg.

[0037] For the images taken by the drone, feature points are extracted and detected and tracked between consecutive frames using the optical flow method. For the continuously tracked feature points, they are triangulated to obtain map points. Reprojection to keyframes And compared with the original visual measurement value (pixel coordinates of feature points), the reprojection error is obtained, that is, the visual residual At the same time, for the acquired IMU data, the IMU measurement values ​​between the two frames are pre-integrated to obtain the pre-integration constraint, that is, the inertial residual .

[0038] Specifically, the cost function of the visual inertial odometry is as follows: ; Among them, C represents the set of feature points that are observed at least twice in the sliding window, I represents the set of all IMU measurement values ​​in the sliding window, and W represents the weight corresponding to the residual term.

[0039] After completing the construction of the optimization problem, the six-degree-of-freedom position and posture of the UAV can be obtained by minimizing the above cost function using the Gauss-Newton or Levenberg-Marquardt method.

[0040] However, for high-altitude drones, they often move in a uniform straight line, and the acceleration is constant, which makes VIO unable to estimate the scale. In order to solve this problem, this application adds a single-point laser radar to assist high-altitude VIO on the basis of VIO, realizes distance visual inertial odometer, and provides a robust, efficient and accurate VIO algorithm.

[0041] In this application, the distance visual inertial odometry adds laser ranging residuals compared to the visual inertial odometry. This residual uses a ranging model based on laser measurement and visual features. Whenever a new image frame is added to the sliding window, the distance measurement value corresponding to the timestamp is Associated with the image frame. Assuming that the optical center of the camera coincides with the origin of the LRF, Figure 2 Schematic diagram of the geometric relationship of the distance measurement model described in the embodiment of the present application. Figure 2 As shown, is the unit vector along the LRF optical axis at time i, It is the intersection of the optical axis and the ground. , and is in image space around The map points corresponding to the feature points that form the triangle. n is the number of points that contain , , and The normal vector of the plane.

[0042] In the ranging model, { } is the camera coordinate system at time i, Represents the position vector of the object in the world coordinate system {w}. In order to build the measurement model in the real world, the feature points are Delaunay triangulated in the image space, and the triangle where the LRF beam intersects with the scene is selected.

[0043] Furthermore, the distance It can be expressed as a nonlinear function of the drone state as follows: ; ; Among them, Compared with the actual measured value By comparison, we can get the laser ranging residual: .

[0044] All measurement information is fused in the sliding window to obtain the cost function of the distance visual inertial odometry: ; in, represents the cost function of visual inertial odometry; represents the laser ranging residual; represents the weight of the laser ranging residual; F represents the set of all measured distances in the sliding window.

[0045] Therefore, the present application can use the distance visual inertial odometry as the target odometry, and input the continuous images taken by the UAV into the distance visual inertial odometry, and track the continuous images taken by the UAV through the target odometry to perform six-degree-of-freedom posture evaluation of the UAV, and obtain the posture information of the UAV after completing the evaluation.

[0046] The present application can also match the image currently taken by the drone through the overhead camera with a pre-stored map with geographic coordinates, and after the matching is completed, output the longitude and latitude coordinates of the drone as the location information of the drone.

[0047] Furthermore, the present application can fuse the relative pose estimation result of the odometer and the global position estimation result of the chart matching through a fusion framework, so that a higher frequency geolocation result can be output. The fusion framework can be implemented by optimization or filtering, and the present application can specifically perform fusion through a graph optimization method.

[0048] Figure 3 is a schematic diagram of the information fusion process in the UAV positioning method provided in the embodiment of the present application, such as Figure 3 As shown, in the fusion framework, the transformation matrix from the coordinate system of the target odometer to the global coordinate system is maintained , and initialized to the unit matrix. When constructing the optimization problem, the local pose information estimated by the odometer is converted into the global pose through this transformation matrix and added to the graph optimization as a vertex (i.e., optimization variable), corresponding to Figure 3 The relative pose information between images estimated by the target odometer is added as an edge to the graph optimization to constrain the relative pose relationship between image frames, corresponding to Figure 3 The image taken by the drone obtains the corresponding geographical location as the location information through the map matching process, and then adds it to the graph optimization as an edge to constrain the global position of the image frame, corresponding to Figure 3 The global position constraint is then used to solve the graph optimization problem using the Levenberg-Marquardt algorithm to obtain the optimized pose of each image frame in the global coordinate system. Based on the pose of the latest image frame in the global coordinate system and the pose of the local coordinate system, the latest transformation matrix from the odometer coordinate system to the global coordinate system can be obtained and updated. By using this transformation matrix, the odometer output is transformed into the global coordinate system, and a high-frequency positioning result equivalent to the odometer output frequency can be output.

[0049] According to the UAV positioning method of the embodiment of the present application, by combining a single-point laser radar with a visual inertial odometer to form a target odometer, the cumulative error problem in long-distance movement can be effectively improved, and a more accurate posture evaluation can be achieved. Therefore, by performing posture evaluation based on the target odometer and the images taken by the UAV, accurate posture information of the UAV can be obtained; and by performing chart matching based on the images taken by the UAV, the location information of the UAV can be obtained; and then by performing information fusion based on the posture information and the location information, the advantages of the odometer and the chart matching can be fully utilized to obtain an accurate positioning result of the UAV, thereby improving the accuracy of UAV positioning.

[0050] Based on the above embodiment, the image captured by the drone is matched with the map to obtain the location information of the drone, including: Based on the image captured by the drone, determining at least one candidate slice from map slices of a preset offline map; The image taken by the drone is matched with each candidate slice for feature points to obtain the matching result; Based on the matching results, determine the target slice from each candidate slice; Based on the images and target slices taken by the drone, the location information of the drone is determined.

[0051] Further, based on the image captured by the drone, determining at least one candidate slice from the map slices of the preset offline map includes: If there is a historical positioning result, determine an initial slice set from each map slice of the preset offline map according to the historical positioning result; determine a first global descriptor of the image taken by the drone; and determine at least one candidate slice from the initial slice set based on the first global descriptor and a second global descriptor corresponding to each map slice in the initial slice set; If there is no historical positioning result, determine the first global descriptor of the image taken by the drone; based on the first global descriptor and the second global descriptor corresponding to each map slice in the preset offline map, determine at least one candidate slice from each map slice of the preset offline map.

[0052] Specifically, based on the first global descriptor and the second global descriptor corresponding to each map slice in the initial slice set, determining at least one candidate slice from the initial slice set includes: Determine the distance between the first global descriptor and the second global descriptor corresponding to each map slice in the initial slice set; determining at least one target distance from the distances less than a preset distance threshold; The map slices corresponding to the target distance in the initial slice set are determined as candidate slices.

[0053] Furthermore, based on the image and the target slice taken by the drone, the location information of the drone is determined, including: Determine the homography transformation between the image taken by the drone and the target slice, and obtain the homography matrix; According to the homography matrix, the center point of the image taken by the drone is projected to the target slice; The geographic coordinates of the projection point are used as the location information of the drone.

[0054] Specifically, the implementation of chart matching in this application mainly involves two parts: offline map production and online chart matching.

[0055] Offline map production first obtains orthophoto satellite images, which can be directly obtained through legitimate channels or obtained by two-dimensional reconstruction through data collection. The satellite images are further sliced ​​according to the flight altitude of the drone and the shooting range of the drone's camera, and the longitude and latitude coordinates of the upper left corner and lower right corner of each map slice are calculated. In order to improve the matching success rate, there needs to be a certain overlap between map slices. Considering that online chart matching needs to run in real time, the global descriptors of the map slices and the local descriptors of the feature points are pre-calculated to construct a map database. Among them, the global descriptor of each map slice in the offline map can be defined as the second global descriptor.

[0056] The global descriptor can be calculated using the NetVald algorithm, and the feature point and its descriptor can be calculated using the Superpoint algorithm. The longitude and latitude coordinates of the upper left corner and lower right corner of the map slice, the global descriptor, the feature point and its local descriptor constitute the map database, and the original map slice is no longer stored in the database.

[0057] Among them, NetVLAD is a feature descriptor extraction algorithm for image retrieval and scene recognition. Its core idea is to aggregate local features into a global descriptor for efficient matching and retrieval.

[0058] SuperPoint is a feature point detection and description method based on deep learning. It detects and describes feature points simultaneously and can extract high-quality features in complex scenes.

[0059] Online map matching first obtains the image taken by the drone through the overhead camera. The image is further preprocessed, such as scaling and interception. If the drone has a large inclination angle, the image can be orthorectified according to the drone's posture.

[0060] A rough match can be further performed. In this step, the global descriptor of the image is calculated by the NetVlad algorithm, and then the distance between the global descriptor and the global descriptor of the map slice in the map database is calculated to find the closest preset N pictures.

[0061] It should be noted that if there are historical positioning results in this positioning process, that is, the positioning results of the drone at the corresponding time have been obtained before the current time, then based on the historical positioning results, such as the positioning results at the moment closest to the current time, the map slices in each map slice of the preset offline map whose positions are within a certain range of the positioning results can be formed into an initial slice set.

[0062] Specifically, the number of feature points matched by the positioning result used in the precise matching of the chart matching can be determined, and the number can be compared with a preset number threshold. If the number is greater than the preset threshold, it is determined that the confidence of the positioning result is high, and the positioning result can be used as the center to search in a smaller range pre-set according to the actual situation; if the number is less than or equal to the preset threshold, it is determined that the confidence of the positioning result is low, and the positioning result can be used as the center to search in a larger range pre-set according to the actual situation. The map slices obtained by the search form an initial slice set.

[0063] Furthermore, a first global descriptor of the image taken by the drone is determined.

[0064] Furthermore, the distance between the first global descriptor and the second global descriptor corresponding to each map slice in the initial slice set is calculated. The process of calculating the above distance is not specifically limited in the present application.

[0065] The similarity calculation of the present application requires a threshold test, and only those exceeding a predefined threshold are considered candidates. Therefore, at least one target distance can be determined from distances less than a preset distance threshold.

[0066] Further, the map slices corresponding to the target distance in the initial slice set are determined as candidate slices.

[0067] If there is no historical positioning result in the current positioning process, that is, the positioning of the drone has not been completed in the current positioning process before the current moment, the first global descriptor of the image taken by the drone can be determined at this time.

[0068] Based on the first global descriptor and the second global descriptor corresponding to each map slice in the preset offline map, at least one candidate slice is determined from each map slice of the preset offline map. The specific determination process refers to the above content.

[0069] This allows a rough match to be completed.

[0070] Furthermore, after the rough matching, fine matching is performed. This step uses the Superpoint algorithm to extract the feature points of the image and calculate its descriptor. Then the LightGlue algorithm is used to match the feature points of the drone image with the most similar N map slices found in the rough matching. The slice with the most feature points is successfully matched as the best candidate (i.e., the target slice) and is used in the subsequent registration process. Among them, LightGlue is an algorithm for feature point matching, which is mainly aimed at tasks such as image registration and 3D reconstruction in computer vision.

[0071] Furthermore, assuming that the ground is a plane, the homography transformation between the drone image and the target slice is calculated based on the feature point matching relationship. According to the homography matrix, the center point of the drone image is projected onto the target slice, and the longitude and latitude coordinates corresponding to the projection point are calculated as the location information of the drone. Geographic coordinates of The calculation is as follows:

[0072] ; in and Represents the geographic coordinates of the upper left and lower right corners of the map tile.

[0073] The present application can improve the accuracy of the obtained UAV position information through a two-step chart matching method from coarse to fine, which helps to improve the accuracy of the UAV positioning results obtained based on information fusion of posture information and position information, thereby improving the accuracy of UAV positioning.

[0074] This application uses the distance visual inertial odometer to perform short-term data association, tracks the continuous images taken by the drone, and outputs the drone's position. Chart matching performs long-term data association, matches the images taken by the drone with pre-downloaded maps with geographic coordinates, and outputs the location of the drone. The fusion framework subscribes to the odometer and chart matching outputs, and fuses them using optimization or filtering methods to output high-frequency positioning results. Chart matching subscribes to high-frequency geolocation results as predictions to reduce the search range of chart matching, reduce the amount of calculation, and improve the success rate.

[0075] The drone positioning device provided in the present application is described below. The drone positioning device described below and the drone positioning method described above can be referenced to each other.

[0076] Furthermore, the present application also provides a drone positioning device.

[0077] The UAV positioning device comprises: An evaluation module is used to perform posture evaluation based on a target odometer and an image taken by a UAV to obtain the posture information of the UAV; the target odometer is formed by integrating the distance measurement of a single-point laser radar on the basis of a visual inertial odometer; A matching module, used to perform chart matching based on the image taken by the drone to obtain the location information of the drone; A fusion module is used to perform information fusion based on the posture information and the position information to obtain a positioning result of the drone.

[0078] The UAV positioning device of the present application, by combining a single-point laser radar with a visual inertial odometer to form a target odometer, can effectively improve the cumulative error problem in long-distance movement and achieve more accurate posture evaluation. Therefore, by performing posture evaluation based on the target odometer and the images taken by the UAV, accurate posture information of the UAV can be obtained; and by performing chart matching based on the images taken by the UAV, the position information of the UAV can be obtained; and then by performing information fusion based on the posture information and the position information, the advantages of the odometer and the chart matching can be fully utilized to obtain an accurate positioning result of the UAV, thereby improving the accuracy of UAV positioning.

[0079] In one embodiment, the matching module is specifically used for: Based on the image captured by the drone, determining at least one candidate slice from map slices of a preset offline map; Based on the image taken by the drone, feature points are matched with each candidate slice to obtain a matching result; Based on the matching result, determining a target slice from each candidate slice; Based on the image taken by the drone and the target slice, the position information of the drone is determined.

[0080] In one embodiment, the matching module includes a first determining unit, and the first determining unit is configured to: If there is a historical positioning result, determining an initial slice set from each map slice of a preset offline map according to the historical positioning result; determining a first global descriptor of the image taken by the drone; and determining at least one candidate slice from the initial slice set based on the first global descriptor and a second global descriptor corresponding to each map slice in the initial slice set; If there is no historical positioning result, determine the first global descriptor of the image taken by the drone; based on the first global descriptor and the second global descriptor corresponding to each map slice in the preset offline map, determine at least one candidate slice from each map slice of the preset offline map.

[0081] In one embodiment, the matching module includes a second determining unit, and the second determining unit is used to: Determine the distance between the first global descriptor and the second global descriptor corresponding to each map slice in the initial slice set; determining at least one target distance from the distances less than a preset distance threshold; A map slice corresponding to the target distance in the initial slice set is determined as a candidate slice.

[0082] In one embodiment, the matching module includes a third determining unit, and the third determining unit is used to: Determine the homography transformation between the image taken by the drone and the target slice to obtain a homography matrix; According to the homography matrix, projecting the center point of the image taken by the drone to the target slice; The geographic coordinates of the projection point are used as the location information of the UAV.

[0083] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the following method: perform posture evaluation based on the target odometer and the image taken by the drone to obtain the posture information of the drone; the target odometer is formed by integrating the measurement distance of the single-point laser radar on the basis of the visual inertial odometer; Performing chart matching based on the image taken by the drone to obtain the location information of the drone; Based on the posture information and the position information, information fusion is performed to obtain the positioning result of the UAV.

[0084] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a drone, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0085] On the other hand, the embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the method provided in each of the above embodiments, for example, including: performing posture evaluation based on the target odometer and the image taken by the drone to obtain the posture information of the drone; the target odometer is formed by integrating the measurement distance of the single-point laser radar on the basis of the visual inertial odometer; Performing chart matching based on the image taken by the drone to obtain the location information of the drone; Based on the posture information and the position information, information fusion is performed to obtain the positioning result of the UAV.

[0086] On the other hand, the embodiments of the present application further provide a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, the method provided in each of the above embodiments is implemented, for example, including: performing posture evaluation based on a target odometer and an image taken by a drone to obtain posture information of the drone; the target odometer is formed by integrating the measurement distance of a single-point laser radar on the basis of a visual inertial odometer; Performing chart matching based on the image taken by the drone to obtain the location information of the drone; Based on the posture information and the position information, information fusion is performed to obtain the positioning result of the UAV.

[0087] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0088] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on this understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the relevant technology. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a drone, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0089] Finally, it should be noted that the above implementation modes are only used to illustrate the present application, rather than to limit the present application. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that various combinations, modifications or equivalent substitutions of the technical solutions of the present application do not depart from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for positioning a drone, characterized in that: include: Performing posture evaluation based on the target odometer and the image taken by the UAV to obtain the posture information of the UAV; The target odometer is formed by integrating the measurement distance of a single-point laser radar on the basis of the visual inertial odometer; Performing chart matching based on the image taken by the drone to obtain the location information of the drone; Based on the posture information and the position information, information fusion is performed to obtain the positioning result of the UAV.

2. The method for positioning a UAV according to claim 1, characterized in that: The performing of chart matching based on the image taken by the drone to obtain the location information of the drone includes: Based on the image captured by the drone, determining at least one candidate slice from map slices of a preset offline map; Based on the image taken by the drone, feature points are matched with each candidate slice to obtain a matching result; Based on the matching result, determining a target slice from each candidate slice; Based on the image taken by the drone and the target slice, the position information of the drone is determined.

3. The method for positioning a UAV according to claim 2, characterized in that: The determining at least one candidate slice from map slices of a preset offline map based on the image taken by the drone includes: If there is a historical positioning result, determining an initial slice set from each map slice of a preset offline map according to the historical positioning result; determining a first global descriptor of the image taken by the drone; and determining at least one candidate slice from the initial slice set based on the first global descriptor and a second global descriptor corresponding to each map slice in the initial slice set; If there is no historical positioning result, determine the first global descriptor of the image taken by the drone; based on the first global descriptor and the second global descriptor corresponding to each map slice in the preset offline map, determine at least one candidate slice from each map slice of the preset offline map.

4. The method for positioning a UAV according to claim 3, characterized in that: The determining at least one candidate slice from the initial slice set based on the first global descriptor and the second global descriptor corresponding to each map slice in the initial slice set respectively includes: Determine the distance between the first global descriptor and the second global descriptor corresponding to each map slice in the initial slice set; determining at least one target distance from the distances less than a preset distance threshold; A map slice corresponding to the target distance in the initial slice set is determined as a candidate slice.

5. The method for positioning a UAV according to claim 2, characterized in that: The determining the location information of the drone based on the image taken by the drone and the target slice includes: Determine the homography transformation between the image taken by the drone and the target slice to obtain a homography matrix; According to the homography matrix, projecting the center point of the image taken by the drone to the target slice; The geographic coordinates of the projection point are used as the location information of the UAV.

6. The method for positioning a UAV according to claim 1, characterized in that: The expression of the cost function of the target odometer is as follows: ； in, represents the cost function of visual inertial odometry; represents the laser ranging residual; represents the weight of the laser ranging residual; F represents the set of all measured distances in the sliding window.

7. A drone positioning device, characterized in that: include: An evaluation module, used to perform posture evaluation based on the target odometer and the image taken by the UAV to obtain the posture information of the UAV; The target odometer is formed by integrating the distance measurement of the single-point laser radar on the basis of the visual inertial odometer; A matching module, used to perform chart matching based on the image taken by the drone to obtain the location information of the drone; A fusion module is used to perform information fusion based on the posture information and the position information to obtain a positioning result of the drone.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the drone positioning method according to any one of claims 1 to 6 is implemented.

9. A storage medium, the storage medium being a non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that: When the computer program is executed by a processor, the drone positioning method as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the drone positioning method according to any one of claims 1 to 6 is implemented.

Images