Joint positioning method and system of vehicle-mounted platform and unmanned aerial vehicle

Abstract

The application relates to a combined positioning method of a vehicle-mounted platform and a UAV, which comprises the following steps: acquiring LiDAR point cloud and vehicle coordinates from the vehicle-mounted platform, acquiring UAV coordinates and a first relative pose between the vehicle and the UAV from the UAV, generating a semantic map according to the LiDAR point cloud based on a parameterized curve fitting algorithm, determining accurate coordinates of the UAV according to the vehicle coordinates and the first relative pose, sending the semantic map to the UAV to instruct the UAV to shoot image data according to lane lines in the semantic map, obtaining a second relative pose between the UAV and the vehicle-mounted platform according to the image data and the semantic map, and determining accurate coordinates of the vehicle based on the second relative pose and the accurate coordinates of the UAV. Through the application, the problem that the vehicle and the UAV are not accurately positioned is solved, the bidirectional correction reduces the cumulative error, and the lightweight semantic map reduces the transmission bandwidth occupation while maintaining a high recognition rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the positioning technical field, in particular to a vehicle platform and unmanned aerial vehicle joint positioning method and system. BACKGROUND

[0002] With the rapid development of intelligent transportation systems and unmanned systems, the application demand of high-precision positioning technology in complex environments is increasing.

[0003] Current positioning of vehicles and unmanned aerial vehicles is mainly divided into single-platform independent positioning and vehicle-unmanned aerial vehicle cooperative positioning. Single-platform independent positioning of the vehicle-mounted platform relies on GPS / IMU+LiDAR point cloud matching, requires an a priori map, and has large calculation delay; single-platform independent positioning of the unmanned aerial vehicle needs to be based on visual odometry (VIO) or pre-set markers, and has large drift error in weak light or rain and fog. Current vehicle-unmanned aerial vehicle cooperative positioning includes hierarchical positioning and mechanical auxiliary positioning. Hierarchical positioning relies on a fixed switching threshold, and cannot achieve positioning when UWB is interfered by multipath or visual calibration objects are blocked. Mechanical auxiliary positioning relies on physical contact devices, cannot correct air pose deviation, and is only suitable for static scenes. SUMMARY

[0004] Embodiments of the present application provide a vehicle platform and unmanned aerial vehicle joint positioning method, system, electronic device and storage medium to at least solve the problem of inaccurate positioning of vehicles and unmanned aerial vehicles in related technologies.

[0005] In a first aspect, embodiments of the present application provide a vehicle platform and unmanned aerial vehicle joint positioning method, which comprises:

[0006] obtaining LiDAR point cloud and vehicle coordinates from a vehicle platform, obtaining unmanned aerial vehicle coordinates and a first relative pose between the vehicle and the unmanned aerial vehicle from the unmanned aerial vehicle, the first relative pose including relative distance and direction angle;

[0007] generating a semantic map according to the LiDAR point cloud based on a parameterized curve fitting algorithm;

[0008] determining accurate coordinates of the unmanned aerial vehicle according to the vehicle coordinates and the first relative pose;

[0009] sending the semantic map to the unmanned aerial vehicle to instruct the unmanned aerial vehicle to take an image according to lane lines in the semantic map to obtain image data, obtaining a second relative pose between the unmanned aerial vehicle and the vehicle platform according to the image data and the semantic map, and determining accurate coordinates of the vehicle based on the second relative pose and the accurate coordinates of the unmanned aerial vehicle.

[0010] In some embodiments, the generating a semantic map according to the LiDAR point cloud based on a parameterized curve fitting algorithm comprises:

[0011] projecting the LiDAR point cloud to a 2D grid map and obtaining a lane line point set by randomly sampling consistency and eliminating outliers;

[0012] parameterizing modeling the lane line point set based on a cubic Bezier curve fitting algorithm to obtain the semantic map.

[0013] In some embodiments, the vehicle coordinates are RTK coordinates obtained by a vehicle-mounted real-time dynamic differential technique, and the UAV coordinates are GNSS coordinates obtained by a satellite positioning system; and determining the accurate coordinates of the UAV according to the vehicle coordinates and the first relative pose includes:

[0014] correcting the GNSS coordinates of the UAV according to the RTK coordinates and the first relative pose to obtain the accurate coordinates of the UAV.

[0015] In some embodiments, after obtaining the UAV coordinates and the first relative pose between the vehicle and the UAV, the method further includes:

[0016] obtaining time sequence characteristic parameters of the UAV, the time sequence characteristic parameters including time sequences of an IMU temperature, a UWB signal-to-noise ratio and a visual feature tracking number;

[0017] analyzing the time sequence characteristic parameters to obtain a correction coefficient, and correcting the UAV coordinates and the first relative pose based on the correction coefficient.

[0018] In some embodiments, the correction coefficient is a Kalman filter gain coefficient, and the analyzing the time sequence characteristic parameters to obtain a correction coefficient and correcting the UAV coordinates and the first relative pose based on the correction coefficient include:

[0019] determining an adjustment amount of the Kalman filter gain coefficient according to the time sequence characteristic parameters based on a pre-constructed LSTM network;

[0020] obtaining the Kalman filter gain coefficient according to the adjustment amount, and correcting the UAV coordinates and the first relative pose based on the Kalman filter gain coefficient.

[0021] In some embodiments, the analyzing the time sequence characteristic parameters to obtain a correction coefficient and correcting the UAV coordinates and the first relative pose based on the correction coefficient include:

[0022] obtaining a positioning error according to the time sequence characteristic parameters by a DQN-based reinforcement learning controller;

[0023] correct the unmanned aerial vehicle coordinates and the first relative pose based on the positioning error.

[0024] In some embodiments, the method further comprises: establishing a low-latency data channel between the vehicle-mounted platform and the unmanned aerial vehicle through 5G communication and vehicle-to-everything technology.

[0025] In a second aspect, the embodiments of the present application provide a joint positioning system of a vehicle-mounted platform and an unmanned aerial vehicle, the system comprising:

[0026] a data acquisition module configured to acquire LiDAR point cloud and vehicle coordinates from the vehicle-mounted platform, and acquire unmanned aerial vehicle coordinates and a first relative pose between the vehicle and the unmanned aerial vehicle, the first relative pose comprising a relative distance and a direction angle;

[0027] a map generation module configured to generate a semantic map according to the LiDAR point cloud based on a parameterized curve fitting algorithm;

[0028] a first correction module configured to determine accurate coordinates of the unmanned aerial vehicle according to the vehicle coordinates and the first relative pose;

[0029] a second correction module configured to send the semantic map to the unmanned aerial vehicle to instruct the unmanned aerial vehicle to capture image data according to lane lines in the semantic map, obtain a second relative pose between the unmanned aerial vehicle and the vehicle-mounted platform according to the image data and the semantic map, and determine accurate coordinates of the vehicle based on the second relative pose and the accurate coordinates of the unmanned aerial vehicle.

[0030] In a third aspect, the embodiments of the present application provide a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the joint positioning method of the vehicle-mounted platform and the unmanned aerial vehicle according to the first aspect.

[0031] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium having a computer program stored thereon, and the program is executable on a processor to implement the joint positioning method of the vehicle-mounted platform and the unmanned aerial vehicle according to the first aspect.

[0032] Compared with the related art, the joint positioning method of the vehicle-mounted platform and the unmanned aerial vehicle provided by the embodiments of the present application generates a lightweight semantic map through a parameterized curve fitting algorithm, reduces the data transmission cost between the vehicle-mounted platform and the unmanned aerial vehicle while improving the transmission success rate, and performs bidirectional error compensation according to the semantic map and the relative pose to optimize the unmanned aerial vehicle coordinates and the vehicle coordinates, thereby solving the problem of inaccurate positioning of the vehicle and the unmanned aerial vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0034] Figure 1 is a flow chart of a joint positioning method of a vehicle-mounted platform and a UAV according to an embodiment of the application;

[0035] Figure 2 is a structural block diagram of a joint positioning system of a vehicle-mounted platform and a UAV according to an embodiment of the application;

[0036] Figure 3 is a schematic diagram of an internal structure of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0037] In order to make the objects, technical solutions and advantages of the application clearer, the application is described and explained below in connection with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and should not be used to limit the application. Based on the embodiments provided in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the application.

[0038] Obviously, the drawings in the following description are only some examples or embodiments of the application, and for those of ordinary skill in the art, the application can be applied to other similar scenarios without creative labor on the basis of these drawings. In addition, it can be understood that although the efforts made in this development process can be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the application, some design, manufacture or production changes based on the technical content disclosed in the application are only routine technical means and should not be understood as insufficient disclosure of the content disclosed in the application.

[0039] In the present application, the term "embodiment" means that the specific features, structures or characteristics described in connection with the embodiment can be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in the application can be combined with other embodiments without conflict.

[0040] Unless otherwise defined, technical terms and scientific terms used in the present application shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terms "a", "an", "one", "this", and the like, as used in the present application, do not denote a limitation of quantity, and can be construed to mean either the singular or the plural. The terms "include", "comprise", "have", and any variations thereof, as used in the present application, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a list of steps or modules (units) is not limited to the listed steps or units, but can further include other steps or units not listed, or can further include other steps or units inherent to such process, method, product, or device. The terms "connect", "connected", "coupling", and the like, as used in the present application, are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The term "multiple" refers to two or more. The term "and / or" describes an association relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects. The terms "first", "second", "third", and the like, as used in the present application, are merely to distinguish similar objects, and do not represent a specific order for the objects.

[0041] The embodiment provides a joint positioning method of a vehicle-mounted platform and a UAV. Figure 1 The flowchart of the joint positioning method of the vehicle-mounted platform and the UAV according to the embodiment of the present application is shown in Figure 1 The flowchart includes the following steps:

[0042] In step S101, LiDAR point cloud and vehicle coordinates are obtained from the vehicle-mounted platform, UAV coordinates and a first relative pose between the vehicle and the UAV are obtained from the UAV, and the first relative pose includes a relative distance and a direction angle.

[0043] The LiDAR point cloud and the vehicle coordinates are both obtained from the vehicle-mounted platform. The vehicle coordinates can be obtained by vehicle-mounted RTK positioning (centimeter-level positioning technology). The vehicle-mounted RTK positioning is a satellite positioning method based on real-time kinematic carrier phase difference, which can improve the global positioning accuracy of the vehicle-mounted platform from meter level to centimeter level by receiving error correction signals from the ground reference station.

[0044] The first relative pose can be obtained by UWB ranging, preferably, the carrier frequency of the UWB ranging is 6.5 GHz and the bandwidth is 500 MHz. The UWB ranging refers to a Time of Flight device using Ultra-Wideband radio pulse signals, which realizes precise distance measurement against multipath interference by transmitting nanosecond-level pulses between the vehicle-mounted platform and the unmanned aerial vehicle.

[0045] The first relative pose can also be obtained by frequency-modulated continuous wave (FMCW) millimeter wave radar (such as TI AWR1843) ranging. The FMCW millimeter wave radar transmits a 77 GHz linear frequency modulation wave, receives the reflected signal of the unmanned aerial vehicle and calculates the frequency difference Δf, and deduces the relative distance d. The deduction formula of the relative distance is as follows:

[0046]

[0047] Wherein, c is the speed of light, and k is the frequency modulation slope.

[0048] Meanwhile, the Doppler shift Δf d Solving the relative speed v, the calculation formula of the relative speed v is as follows:

[0049]

[0050] Wherein, λ is the wavelength.

[0051] In this embodiment, a low-latency (such as a latency less than 10 ms) data channel between the vehicle-mounted platform and the unmanned aerial vehicle can be established by 5G communication and vehicle-to-everything technology (5G / V2X). The data channel can synchronously transmit the centimeter-level global coordinates of the vehicle-mounted RTK-GPS, the 3D point cloud scanned by the LiDAR in real time, and the 6DOF pose of the visual inertial odometer (VIO) and the millimeter wave radar point set of the unmanned aerial vehicle end.

[0052] The semantic map and the control instruction are transmitted in parallel through the 5G and vehicle wireless communication technology (V2X) double channels.

[0053] The dynamic channel selection algorithm switches the main link based on the packet loss rate η and the latency τ:

[0054]

[0055] In step S102, a semantic map is generated according to the LiDAR point cloud based on a parameterized curve fitting algorithm.

[0056] The parameterized curve fitting algorithm includes but is not limited to a cubic Bezier curve fitting algorithm and a quadratic uniform B-spline curve.

[0057] The semantic map in the embodiment is a compressed environment model in which three-dimensional point cloud data scanned by a laser radar is processed by a random sample consensus algorithm to remove noise points, and key environmental features such as lane lines and road edges are extracted to obtain geometric parameters (such as curvature and inclination), and the parameters are stored in a parametric curve equation.

[0058] In some embodiments, step S102 specifically includes:

[0059] In step S1021, the LiDAR point cloud is projected onto a 2D grid map, and outliers are removed by random sample consensus to obtain a lane line point set.

[0060] In step S1022, a cubic Bezier curve fitting algorithm is used to parameterize the lane line point set to obtain a semantic map.

[0061] The embodiment is based on a vehicle-mounted LiDAR point cloud, and an improved RANSAC-cubic Bezier curve fitting algorithm is used to extract lane line features, specifically including:

[0062] The original point cloud (LiDAR point cloud) is projected onto a 2D grid map, and outliers are removed by random sample consensus (RANSAC) to obtain a lane line point set. The lane line point set is parameterized and modeled, and the curvature change is described by a cubic Bezier curve equation as follows:

[0063]

[0064] where P0, P1, P2, and P3 are curve control point coordinate vectors, and t is a curve parameter.

[0065] In the embodiment, the control point selection criterion can be: when the curvature change rate of adjacent points (P0, P1) is greater than a predetermined threshold, a control point is added to ensure that the fitting error of a sharp curve is less than 0.1 m. and

[0066] The RANSAC-cubic Bezier curve fitting algorithm compresses a single frame of point cloud to below a predetermined threshold (such as 1 KB) to obtain a lightweight semantic map, and the semantic map is broadcast to the unmanned aerial vehicle through the controller.

[0067] Alternatively, the above cubic Bezier curve fitting algorithm is replaced by a quadratic uniform B-spline curve. Given a control point sequence {Q0, Q1,..., Q m}, m is the number of control points, and the quadratic uniform B-spline curve equation is represented as:

[0068]

[0069] where N​​​i,2 (t) is a quadratic basis function, and the node vector takes a uniform distribution t i =i / (m+1).

[0070] B-spline only needs to store control points Q i (no need for end tangent constraints), so that the single-frame map data is further reduced.

[0071] In a rain and fog environment, the millimeter wave radar has strong penetration, and the point cloud fitted by the RANSAC curve still maintains the accuracy of the curvature k, ensuring the recognition rate of the unmanned aerial vehicle vision within a certain range (visibility 50m).

[0072] In step S103, the precise coordinates of the unmanned aerial vehicle are determined according to the vehicle coordinates and the first relative pose.

[0073] In some embodiments, the vehicle coordinates are RTK coordinates obtained by a vehicle-mounted real-time dynamic differential technique, and the unmanned aerial vehicle coordinates are GNSS coordinates obtained by a satellite positioning system; and the precise coordinates of the unmanned aerial vehicle are determined according to the vehicle coordinates and the first relative pose, including:

[0074] The GNSS coordinates of the unmanned aerial vehicle are corrected according to the RTK coordinates and the first relative pose to obtain the precise coordinates of the unmanned aerial vehicle.

[0075] In this embodiment, the positioning coordinates of the unmanned aerial vehicle and the vehicle are corrected by bidirectional correction (forward compensation and backward compensation). The forward compensation corrects the coordinates of the unmanned aerial vehicle by vehicle-mounted data, specifically including:

[0076] When the first relative pose is obtained by UWB ranging, the RTK position (x c ,y c ,z c ) of the vehicle, and the relative distance d and the azimuth angle ф measured by UWB are used to correct the GNSS coordinates (x u ,y u ,z u ) of the unmanned aerial vehicle, and the forward compensation correction formula is:

[0077]

[0078] where R(θ c ) is a rotation matrix constructed with the vehicle-mounted heading angle θ c , λ is an IMU error attenuation factor (for example, 0.25), and ▽e IMU is an IMU angular velocity integral drift vector, (x u corr ,y u corr) is the corrected UAV coordinates (the accurate coordinates of the UAV). This embodiment mainly corrects the planar (x and y) coordinates of the UAV.

[0079] In the case that the first relative pose is obtained by FMCW millimeter wave radar ranging, since the radar beam angle θ beam causes the azimuth deviation, and the forward compensation correction formula is:

[0080]

[0081] In step S104, the semantic map is sent to the UAV to instruct the UAV to capture image data according to the lane lines in the semantic map, to obtain a second relative pose of the UAV and the vehicle platform according to the image data and the semantic map, and to determine the accurate coordinates of the vehicle based on the second relative pose and the accurate coordinates of the UAV.

[0082] The backward compensation corrects the vehicle coordinates by the UAV data. The UAV identifies the lane lines in the semantic map by the onboard camera, and calculates the relative pose (the second relative pose) of the vehicle platform by the EPnP algorithm. The backward compensation correction formula is:

[0083]

[0084] wherein X i represents the 3D coordinates of the i-th lane line control point in the semantic map, x i is the image corresponding point coordinates, π is the camera projection model, and the output E is the rotation matrix R and the translation vector t to compensate the cumulative error of the vehicle SLAM (Simultaneous localization and mapping).

[0085] Optionally, the above PnP algorithm can be replaced by the iterative closest point (ICP) algorithm. The UAV matches the millimeter wave radar point cloud P radar with the vehicle semantic map point set M lidar , and minimizes the objective function:

[0086]

[0087] wherein P i represents the coordinates of the i-th point in the millimeter wave radar point cloud, and q i represents the coordinates of the i-th point in the semantic map. This method is more robust in weak texture scenes (such as tunnel walls).

[0088] Through the above steps, the bidirectional correction (forward compensation of the UAV GNSS multipath error and backward compensation of the vehicle SLAM position error) reduces the cross-platform cumulative error; and the lightweight semantic map reduces the transmission bandwidth occupation while maintaining high recognition rate.

[0089] The bidirectional compensation mechanism allows the single UWB anchor point to achieve omnidirectional positioning. The traditional scheme requires 4 anchor points to solve the 3D position (cost C UWB ×4), and the embodiment combines the visual azimuth angle f and the UWB distance d to achieve vehicle positioning, and the geometric relationship is:

[0090]

[0091] wherein β is the pitch angle. The single unmanned aerial vehicle anchor point achieves omnidirectional positioning, reducing the hardware cost.

[0092] In some embodiments, after obtaining the unmanned aerial vehicle coordinates and the first relative pose between the vehicle and the unmanned aerial vehicle, the method further comprises:

[0093] Step S201, obtaining the time sequence characteristic parameters of the unmanned aerial vehicle, the time sequence characteristic parameters including the time sequence of the IMU temperature, the UWB signal-to-noise ratio and the visual feature tracking number.

[0094] Step S202, analyzing the time sequence characteristic parameters to obtain a correction coefficient, and correcting the unmanned aerial vehicle coordinates and the first relative pose based on the correction coefficient.

[0095] The embodiment corrects the relative pose between the vehicle and the unmanned aerial vehicle through the real-time environmental parameters of the unmanned aerial vehicle, constructs a vehicle-unmanned aerial vehicle virtual cooperative body, realizes dynamic gain control, and reduces the multipath interference error of the unmanned aerial vehicle.

[0096] In some embodiments, the correction coefficient is a Kalman filter gain coefficient, and step S202 specifically comprises:

[0097] Step S2021, based on the pre-constructed LSTM network, determining the adjustment amount of the Kalman filter gain coefficient according to the time sequence characteristic parameters.

[0098] Step S2022, obtaining the Kalman filter gain coefficient according to the adjustment amount, and correcting the unmanned aerial vehicle coordinates and the first relative pose based on the Kalman filter gain coefficient.

[0099] Input data: time sequence of IMU temperature T, UWB signal-to-noise ratio SNR, and visual feature tracking number N (preferably, sampling period 100 ms). feat

[0100] LSTM network structure: input layer 3 nodes→hidden layer 128 nodes (Tanh activation)→output layer 1 node (linear activation).

[0101] Output data: adjustment amount Δα of Kalman filter gain coefficient α. k

[0102] The adjustment amount Δα calculation formula is: ​​

[0103]

[0104] where W h is the hidden layer weight matrix, b h is the bias vector.

[0105] Training configuration: sliding time window T W (covering 1 second of data), loss function is MAE+regularization term.

[0106] The dynamically adjusted gain coefficient α k new =α k +Δα is sent to the filter of the UAV platform in real time to suppress temperature drift or multipath interference.

[0107] The LSTM network predicts the Kalman gain adjustment amount Δα according to real-time environmental parameters (temperature, signal-to-noise ratio, and visual feature tracking number). When the UWB is disturbed by multipath, SNR<10dB, the LSTM output Δα<0 reduces the gain weight to suppress noise amplification. The actual measurement shows that this mechanism reduces the UWB ranging error by 40% in the interference environment.

[0108] In some embodiments, step S202 specifically comprises:

[0109] Step S2023, through the DQN-based reinforcement learning controller, the positioning error is obtained according to the time sequence feature parameters.

[0110] Step S2024, based on the positioning error, the UAV coordinates and the first relative pose are corrected.

[0111] The above LSTM network can be replaced by a DQN (Deep Q-Network) based reinforcement learning controller.

[0112] Define the state space s=[T,SNR,N feat ], the action space α=Δα, and the reward function:

[0113]

[0114] where ε pos is the positioning error, and ε energy is the calculation energy consumption.

[0115] The Q network iteration update strategy is:

[0116]

[0117] where η is the learning rate and γ is the discount factor. Through the DQN reinforcement learning controller, the temperature drift can be suppressed in the long term.

[0118] The bidirectional constraint is a precision cornerstone in the above method, the lightweight map and the dynamic parameter adjustment (dynamic gain control) support real-time robustness, and the single anchor point and multi-link design guarantee engineering feasibility.

[0119] Preferably, the positioning algorithm is performed on a vehicle platform.

[0120] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0121] The embodiment also provides a joint positioning system of a vehicle platform and a UAV, which is used to implement the above embodiment and preferred embodiment, and will not be described again. As used below, the terms "module", "unit", "sub-unit" and the like can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiment is preferably implemented in software, hardware, or a combination of software and hardware is also possible and contemplated.

[0122] Figure 2 is a structural block diagram of a joint positioning system of a vehicle platform and a UAV according to the embodiment of the application, as Figure 2 shown, the system comprises:

[0123] A data acquisition module 31 is configured to acquire LiDAR point cloud and vehicle coordinates from the vehicle platform, acquire UAV coordinates and a first relative pose between the vehicle and the UAV from the UAV, and the first relative pose comprises a relative distance and a direction angle.

[0124] A map generation module 32 is configured to generate a semantic map according to the LiDAR point cloud based on a parameterized curve fitting algorithm.

[0125] A first correction module 33 is configured to determine accurate coordinates of the UAV according to the vehicle coordinates and the first relative pose.

[0126] A second correction module 34 is configured to send the semantic map to the UAV to instruct the UAV to obtain image data by photographing according to lane lines in the semantic map, obtain a second relative pose between the UAV and the vehicle platform according to the image data and the semantic map, and determine accurate coordinates of the vehicle based on the second relative pose and the accurate coordinates of the UAV.

[0127] In some embodiments, the map generation module 32 comprises:

[0128] A preprocessing module is configured to project the LiDAR point cloud to a 2D grid map, and remove outliers by random sample consensus to obtain a lane line point set.

[0129] a map construction module configured to perform parameterized modeling on the lane line point set based on a cubic Bezier curve fitting algorithm to obtain a semantic map.

[0130] In some embodiments, the vehicle coordinates are RTK coordinates obtained by a vehicle-mounted real-time dynamic differential technique, and the UAV coordinates are GNSS coordinates obtained by a satellite positioning system; the first correction module 33 is configured to correct the GNSS coordinates of the UAV according to the RTK coordinates and the first relative pose to obtain accurate coordinates of the UAV.

[0131] In some embodiments, the system further comprises:

[0132] The parameter acquisition module is configured to acquire time sequence characteristic parameters of the UAV, the time sequence characteristic parameters including time sequences of an IMU temperature, a UWB signal-to-noise ratio, and a number of visual feature tracking.

[0133] The data correction module is configured to analyze the time sequence characteristic parameters to obtain correction coefficients, and correct the UAV coordinates and the first relative pose based on the correction coefficients.

[0134] In some embodiments, the correction coefficients are Kalman filter gain coefficients, and the data correction module comprises:

[0135] The LSTM network module is configured to determine an adjustment amount of the Kalman filter gain coefficients based on the time sequence characteristic parameters according to a pre-constructed LSTM network.

[0136] The first data correction module is configured to obtain the Kalman filter gain coefficients according to the adjustment amount, and correct the UAV coordinates and the first relative pose based on the Kalman filter gain coefficients.

[0137] In some embodiments, the LSTM network comprises:

[0138] The DQN module is configured to obtain a positioning error based on the time sequence characteristic parameters by a DQN-based reinforcement learning controller.

[0139] The second data correction module is configured to correct the UAV coordinates and the first relative pose based on the positioning error.

[0140] In some embodiments, the system further comprises a data channel construction module configured to establish a low-latency data channel between the vehicle-mounted platform and the UAV by 5G communication and vehicle-to-everything technology.

[0141] Through the above system, bidirectional correction (forward compensation of UAV GNSS multipath error and backward compensation of vehicle-mounted SLAM position error) reduces the cross-platform cumulative error; the lightweight semantic map reduces the transmission bandwidth occupation while maintaining a high recognition rate.

[0142] It should be noted that each of the above modules can be a functional module or a program module, which can be implemented by software or hardware. For the module implemented by hardware, each of the above modules can be located in the same processor; or each of the above modules can also be located in different processors in any combination.

[0143] The embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program, and the processor being configured to execute the computer program to perform the steps in any of the above method embodiments.

[0144] Optionally, the electronic device can further include a transmission device and an input and output device, wherein the transmission device is connected with the processor, and the input and output device is connected with the processor.

[0145] Optionally, in the embodiment, the processor can be configured to execute the following steps through the computer program:

[0146] S1, obtaining LiDAR point cloud and vehicle coordinates from a vehicle-mounted platform, obtaining UAV coordinates and a first relative pose between the vehicle and the UAV from a UAV, the first relative pose including a relative distance and a direction angle.

[0147] S2, generating a semantic map according to the LiDAR point cloud based on a parameterized curve fitting algorithm.

[0148] S3, determining accurate coordinates of the UAV according to the vehicle coordinates and the first relative pose.

[0149] S4, sending the semantic map to the UAV to instruct the UAV to obtain image data by photographing according to lane lines in the semantic map, obtaining a second relative pose between the UAV and the vehicle-mounted platform according to the image data and the semantic map, and determining accurate coordinates of the vehicle based on the second relative pose and the accurate coordinates of the UAV.

[0150] It should be noted that the specific examples in the embodiment can refer to the examples described in the above embodiments and optional implementation manners, which will not be described here again.

[0151] In one embodiment, Figure 3 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, as Figure 3 shown, an electronic device is provided, which can be a server, and the internal structure diagram of the electronic device can be as Figure 3As shown. The electronic device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement a joint positioning method of a vehicle-mounted platform and a UAV.

[0152] Those skilled in the art can understand that Figure 3 The skilled in the art can understand that the structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or less components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM) and the like.

[0154] Those skilled in the art should understand that each technical feature of the above-mentioned embodiments can be combined arbitrarily, and for the sake of brevity, not all possible combinations of the technical features in the above-mentioned embodiments are described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0155] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a more specific and detailed manner, but should not be construed as limiting the scope of the patent. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for joint positioning of a vehicle-mounted platform and a UAV, characterized in that, The method comprises: obtaining LiDAR point cloud and vehicle coordinates from a vehicle-mounted platform, obtaining UAV coordinates and a first relative pose between the vehicle and the UAV from a UAV, the first relative pose comprising a relative distance and a direction angle; generating a semantic map from the LiDAR point cloud based on a parameterized curve fitting algorithm; determining accurate coordinates of the UAV according to the vehicle coordinates and the first relative pose; sending the semantic map to the UAV to instruct the UAV to capture image data according to lane lines in the semantic map, obtaining a second relative pose between the UAV and the vehicle-mounted platform according to the image data and the semantic map, and determining accurate coordinates of the vehicle based on the second relative pose and the accurate coordinates of the UAV.

2. The method of claim 1, wherein, The generating a semantic map from the LiDAR point cloud based on a parameterized curve fitting algorithm comprises: projecting the LiDAR point cloud to a 2D grid map and removing outliers by random sample consensus to obtain a lane line point set; parameterizing modeling the lane line point set based on a cubic Bezier curve fitting algorithm to obtain the semantic map.

3. The method of claim 1, wherein, The vehicle coordinates are RTK coordinates obtained by a vehicle-mounted real-time kinematic differential technique, and the UAV coordinates are GNSS coordinates obtained by a satellite positioning system; the determining accurate coordinates of the UAV according to the vehicle coordinates and the first relative pose comprises: correcting the GNSS coordinates of the UAV according to the RTK coordinates and the first relative pose to obtain accurate coordinates of the UAV.

4. The method of claim 1, wherein, After the obtaining UAV coordinates and a first relative pose between the vehicle and the UAV from a UAV, the method further comprises: obtaining time sequence characteristic parameters of the UAV, the time sequence characteristic parameters comprising time sequences of IMU temperature, UWB signal-to-noise ratio, and visual feature tracking number; analyzing the time sequence characteristic parameters to obtain a correction coefficient, and correcting the UAV coordinates and the first relative pose based on the correction coefficient.

5. The method of claim 4, wherein, The correction coefficient is a Kalman filter gain coefficient, and the analyzing the time sequence characteristic parameters to obtain a correction coefficient and correcting the UAV coordinates and the first relative pose based on the correction coefficient comprise: determining an adjustment amount of the Kalman filter gain coefficient based on a pre-constructed LSTM network according to the time sequence characteristic parameters; obtaining the Kalman filter gain coefficient according to the adjustment amount, and correcting the UAV coordinates and the first relative pose based on the Kalman filter gain coefficient.

6. The method of claim 4, wherein, The analyzing the time sequence characteristic parameters to obtain a correction coefficient and correcting the UAV coordinates and the first relative pose based on the correction coefficient comprise: obtaining a positioning error based on a DQN-based reinforcement learning controller according to the time sequence characteristic parameters; correcting the UAV coordinates and the first relative pose based on the positioning error.

7. The method of claim 1, wherein, The method further comprises establishing a low-latency data channel between the vehicle-mounted platform and the UAV through 5G communication and vehicle-to-everything technology.

8. A joint positioning system of a vehicle-mounted platform and a UAV, characterized in that, The system comprises: The data acquisition module is configured to acquire LiDAR point cloud and vehicle coordinates from the vehicle-mounted platform, acquire UAV coordinates and a first relative pose between the vehicle and the UAV from the UAV, the first relative pose including a relative distance and a direction angle; The map generation module is configured to generate a semantic map according to the LiDAR point cloud based on a parameterized curve fitting algorithm; The first correction module is configured to determine accurate coordinates of the UAV according to the vehicle coordinates and the first relative pose; The second correction module is configured to send the semantic map to the UAV to instruct the UAV to capture image data according to lane lines in the semantic map, obtain a second relative pose between the UAV and the vehicle-mounted platform according to the image data and the semantic map, and determine accurate coordinates of the vehicle based on the second relative pose and the accurate coordinates of the UAV.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the joint positioning method of the vehicle-mounted platform and the UAV according to any one of claims 1 to 7.

10. A storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the joint positioning method of the vehicle-mounted platform and the UAV according to any one of claims 1 to 7.

Images