Combined positioning method and system of vehicle-mounted platform and unmanned aerial vehicle

Abstract

The invention relates to a vehicle-mounted platform and unmanned aerial vehicle combined positioning method, and the method comprises the steps: obtaining a LiDAR point cloud and a vehicle coordinate from a vehicle-mounted platform, obtaining an unmanned aerial vehicle coordinate and a first relative pose between a vehicle and an unmanned aerial vehicle from an unmanned aerial vehicle, generating a semantic map according to the LiDAR point cloud based on a parameterized curve fitting algorithm, determining accurate coordinates of the unmanned aerial vehicle according to the vehicle coordinates and the first relative pose, sending a semantic map to the unmanned aerial vehicle to instruct the unmanned aerial vehicle to shoot according to lane lines in the semantic map to obtain image data, obtaining a second relative pose of the unmanned aerial vehicle and the vehicle-mounted platform according to the image data and the semantic map, and sending the second relative pose to the vehicle-mounted platform. And determining accurate coordinates of the vehicle based on the second relative pose and the accurate coordinates of the unmanned aerial vehicle. Through the method and the device, the problem of inaccurate positioning of the vehicle and the unmanned aerial vehicle is solved, accumulated errors are reduced through bidirectional correction, and the transmission bandwidth occupation is reduced while the lightweight semantic map keeps a high recognition rate.

Description

Technical Field

[0001] The present application relates to the field of positioning technology, and in particular to a joint positioning method and system for a vehicle-mounted platform and an unmanned aerial vehicle. Background Art

[0002] With the rapid development of intelligent transportation systems and unmanned systems, the demand for the application of high-precision positioning technology in complex environments is increasing.

[0003] Current vehicle and drone positioning methods are primarily categorized as single-platform independent positioning and vehicle-drone collaborative positioning. Vehicle-mounted single-platform independent positioning relies on GPS / IMU+LiDAR point cloud matching, requires a priori maps, and suffers from significant computational latency. UAV single-platform independent positioning relies on visual odometry (VIO) or preset markers, and suffers from significant drift errors in low light or rain and fog. Currently, vehicle-drone collaborative positioning includes layered positioning and mechanically assisted positioning. Layered positioning relies on a fixed switching threshold and cannot achieve positioning when the UWB is subject to multipath interference or obstructed by visual calibration objects. Mechanically assisted positioning relies on physical contact devices, cannot correct for mid-air posture deviations, and is only applicable to static scenarios. Summary of the Invention

[0004] The embodiments of the present application provide a method, system, electronic device, and storage medium for jointly positioning a vehicle-mounted platform and a drone, so as to at least solve the problem of inaccurate positioning of vehicles and drones in related technologies.

[0005] In a first aspect, an embodiment of the present application provides a method for joint positioning of a vehicle-mounted platform and a drone, the method comprising: Acquire a LiDAR point cloud and vehicle coordinates from the vehicle-mounted platform, and acquire drone coordinates and a first relative pose between the vehicle and the drone from the drone, wherein the first relative pose includes a relative distance and a direction angle; generating a semantic map based on the LiDAR point cloud based on a parameterized curve fitting algorithm; Determining the precise coordinates of the drone based on the vehicle coordinates and the first relative pose; The semantic map is sent to the drone to instruct the drone to obtain image data based on the lane lines in the semantic map. A second relative position of the drone and the vehicle-mounted platform is obtained based on the image data and the semantic map. The precise coordinates of the vehicle are determined based on the second relative position and the precise coordinates of the drone.

[0006] In some embodiments, generating a semantic map from the LiDAR point cloud based on a parameterized curve fitting algorithm includes: Projecting the LiDAR point cloud onto a 2D grid map and removing outliers through random sampling consistency to obtain a lane line point set; Based on a cubic Bezier curve fitting algorithm, the lane line point set is parameterized and modeled to obtain the semantic map.

[0007] In some embodiments, the vehicle coordinates are RTK coordinates obtained by on-board real-time kinematic differential technology, and the UAV coordinates are GNSS coordinates obtained by a satellite positioning system; and determining the precise coordinates of the UAV based on the vehicle coordinates and the first relative pose includes: According to the RTK coordinates and the first relative pose, the GNSS coordinates of the UAV are corrected to obtain the precise coordinates of the UAV.

[0008] In some embodiments, after obtaining the drone coordinates and the first relative pose between the vehicle and the drone from the drone, the method further includes: Acquire time series characteristic parameters of the UAV, wherein the time series characteristic parameters include IMU temperature, UWB signal-to-noise ratio, and time series of visual feature tracking number; The time series characteristic parameters are analyzed to obtain correction coefficients, and the UAV coordinates and the first relative posture are corrected based on the correction coefficients.

[0009] In some embodiments, the correction coefficient is a Kalman filter gain coefficient, and analyzing the time series characteristic parameters to obtain the correction coefficient, and correcting the drone coordinates and the first relative pose based on the correction coefficient includes: Based on the pre-built LSTM network, the adjustment amount of the Kalman filter gain coefficient is determined according to the time series characteristic parameters; The Kalman filter gain coefficient is obtained according to the adjustment amount, and the UAV coordinates and the first relative posture are corrected based on the Kalman filter gain coefficient.

[0010] In some embodiments, analyzing the time series characteristic parameters to obtain a correction coefficient, and correcting the drone coordinates and the first relative posture based on the correction coefficient includes: A positioning error is obtained according to the timing characteristic parameters through a DQN-based reinforcement learning controller; The UAV coordinates and the first relative pose are corrected based on the positioning error.

[0011] In some embodiments, the method further includes: establishing a low-latency data channel between the vehicle platform and the drone through 5G communication and vehicle-to-everything technology.

[0012] In a second aspect, an embodiment of the present application provides a joint positioning system for a vehicle-mounted platform and a drone, the system comprising: a data acquisition module, configured to acquire a LiDAR point cloud and vehicle coordinates from the vehicle-mounted platform, and acquire drone coordinates and a first relative position between the vehicle and the drone from the drone, wherein the first relative position includes a relative distance and a direction angle; A map generation module, configured to generate a semantic map based on the LiDAR point cloud based on a parameterized curve fitting algorithm; a first correction module, configured to determine the precise coordinates of the UAV based on the vehicle coordinates and the first relative pose; The second correction module is used to send the semantic map to the drone to instruct the drone to capture image data according to the lane lines in the semantic map, obtain a second relative position between the drone and the vehicle-mounted platform based on the image data and the semantic map, and determine the precise coordinates of the vehicle based on the second relative position and the precise coordinates of the drone.

[0013] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for joint positioning of a vehicle-mounted platform and a drone as described in the first aspect above is implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the joint positioning method of the vehicle-mounted platform and the drone as described in the first aspect above.

[0015] Compared with related technologies, the joint positioning method of the vehicle-mounted platform and the drone provided in the embodiment of the present application generates a lightweight semantic map through a parameterized curve fitting algorithm, reduces the data transmission cost between the vehicle-mounted platform and the drone while improving the transmission success rate, performs two-way error compensation based on the semantic map and relative posture, optimizes the drone coordinates and vehicle coordinates, and solves the problem of inaccurate positioning of vehicles and drones. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flow chart of a joint positioning method of a vehicle-mounted platform and a drone according to an embodiment of the present application; Figure 2 is a structural block diagram of a joint positioning system of a vehicle-mounted platform and a UAV according to an embodiment of the present application; Figure 3 Schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0018] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0019] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0020] Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meaning as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "an," "the," and similar expressions used herein do not denote quantitative limitations and may refer to either the singular or the plural. The terms "comprise," "include," "have," and any variations thereof, used herein, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules (units) is not limited to the listed steps or units but may also include steps or units not listed, or may include other steps or units inherent to the process, method, product, or apparatus. The terms "connected," "connected," "coupled," and similar expressions used herein are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used herein, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" may mean: A exists alone; A and B exist simultaneously; or B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0021] This embodiment provides a joint positioning method for a vehicle-mounted platform and a drone. Figure 1 Flowchart of the joint positioning method of the vehicle-mounted platform and the UAV according to an embodiment of the present application. Figure 1 As shown, the process includes the following steps: Step S101: Obtain a LiDAR point cloud and vehicle coordinates from a vehicle-mounted platform, obtain drone coordinates and a first relative position between the vehicle and the drone from a drone, where the first relative position includes a relative distance and a direction angle.

[0022] Both the LiDAR (Light Detection and Ranging) point cloud and vehicle coordinates are acquired from the vehicle-mounted platform. Vehicle coordinates can be obtained using vehicle-mounted RTK positioning (centimeter-level positioning technology). Vehicle-mounted RTK positioning is based on a satellite positioning method called Real-Time Kinematic (RTK). By receiving error correction signals from ground reference stations, it improves the vehicle-mounted platform's global positioning accuracy from meters to centimeters.

[0023] The first relative position can be obtained through UWB ranging. Preferably, the UWB ranging carrier frequency is 6.5 GHz and the bandwidth is 500 MHz. UWB ranging refers to a time-of-flight ranging device that uses ultra-wideband (UWB) radio pulse signals. Nanosecond-level pulses are transmitted between the vehicle-mounted platform and the drone to achieve accurate distance measurement that is resistant to multipath interference.

[0024] The first relative pose can also be obtained by ranging using a frequency modulated continuous wave (FMCW) millimeter wave radar (such as the TI AWR1843). The FMCW millimeter wave radar transmits a 77GHz linear frequency modulated wave, receives the reflected signal from the drone, and calculates the frequency difference Δf to derive the relative distance d. The relative distance derivation formula is as follows:

[0025] Where c is the speed of light and k is the frequency modulation slope.

[0026] At the same time, the Doppler frequency shift Δf d Calculate the relative speed v. The calculation formula for the relative speed v is as follows:

[0027] Where λ is the wavelength.

[0028] In this embodiment, 5G communications and vehicle-to-everything (5G / V2X) technology can be used to establish a low-latency (e.g., less than 10ms) data channel between the vehicle platform and the drone. This data channel can simultaneously transmit centimeter-level global coordinates from the vehicle's RTK-GPS, 3D point clouds from real-time LiDAR scanning, and 6DOF poses and millimeter-wave radar point sets from the drone's visual-inertial odometry (VIO).

[0029] Semantic maps and control commands are transmitted in parallel through dual channels of 5G and vehicle-to-everything (V2X) wireless communication technology.

[0030] The dynamic channel selection algorithm switches the primary link based on the packet loss rate η and the delay τ:

[0031] Step S102 : generating a semantic map based on the LiDAR point cloud based on a parameterized curve fitting algorithm.

[0032] The parameterized curve fitting algorithm includes but is not limited to a cubic Bezier curve fitting algorithm and a quadratic uniform B-spline curve.

[0033] The semantic map in this embodiment is a compressed environmental model that extracts the geometric parameters (such as curvature and inclination) of key environmental features such as lane lines and curbs after removing noise from the three-dimensional point cloud data scanned by the lidar through a random sampling consistency algorithm, and stores the data as a parameterized curve equation.

[0034] In some embodiments, step S102 specifically includes: In step S1021, the LiDAR point cloud is projected onto a 2D grid map, and outliers are eliminated through random sampling consistency to obtain a lane line point set.

[0035] Step S1022: Based on a cubic Bezier curve fitting algorithm, the lane line point set is parameterized and modeled to obtain a semantic map.

[0036] This embodiment uses an improved RANSAC-cubic Bezier curve fitting algorithm to extract lane line features based on the vehicle-mounted LiDAR point cloud, specifically including: The original point cloud (LiDAR point cloud) is projected onto a 2D grid map. Outliers are removed through Random Sampling Consensus (RANSAC) to obtain a lane line point set. The lane line point set is parametrically modeled, and the curvature change is described by the cubic Bezier curve equation. The cubic Bezier curve equation is as follows:

[0037] Among them, P0, P1, P2, and P3 are the coordinate vectors of the curve control points, and t is the curve parameter.

[0038] In this embodiment, the control point selection criterion can be: when the adjacent points ( and ) Curvature change rate Greater than Control points are added to ensure that the fitting error of the sharp curve is less than 0.1 m.

[0039] The RANSAC-cubic Bezier curve fitting algorithm compresses a single-frame point cloud to below a preset threshold (e.g., 1KB) to obtain a lightweight semantic map, which is then broadcast to the drone through the controller.

[0040] Optionally, the above cubic Bezier curve fitting algorithm is replaced by a quadratic uniform B-spline curve. Given a control point sequence {Q0, Q1, ..., Q m}, m is the number of control points, and the quadratic uniform B-spline curve equation is expressed as:

[0041] Among them, N i,2 (t) is a quadratic basis function, and the node vector is uniformly distributed t i =i / (m+1).

[0042] B-spline only needs to store control points Q i (No endpoint tangent constraint is required), which further reduces the single-frame map data.

[0043] In rainy and foggy environments, millimeter-wave radar has strong penetration, and the curve fitted by RANSAC on its point cloud still maintains the accuracy of the curvature k, ensuring the recognition rate of the drone vision within a certain range (visibility 50m).

[0044] Step S103: determining the precise coordinates of the UAV based on the vehicle coordinates and the first relative posture.

[0045] In some embodiments, the vehicle coordinates are RTK coordinates obtained by on-board real-time kinematic differential technology, and the UAV coordinates are GNSS coordinates obtained by a satellite positioning system; determining the precise coordinates of the UAV based on the vehicle coordinates and the first relative pose includes: According to the RTK coordinates and the first relative pose, the GNSS coordinates of the UAV are corrected to obtain the precise coordinates of the UAV.

[0046] This embodiment corrects the positioning coordinates of the drone and vehicle through bidirectional correction (forward compensation and backward compensation). Forward compensation corrects the drone coordinates using vehicle-borne data, specifically including: When the first relative pose is obtained by UWB ranging, the vehicle RTK position (x c ,y c ,z c ), and the relative distance d and azimuth ф measured by UWB, correct the UAV GNSS coordinates (x u ,y u ,z u ), the forward compensation correction formula is:

[0047] Among them, R(θ c ) is the vehicle heading angle θ c The constructed rotation matrix, λ is the IMU error attenuation factor (e.g., 0.25), ▽e IMU is the IMU angular velocity integral drift vector, (x u corr ,y u corr ) are the corrected coordinates of the drone (the precise coordinates of the drone). This embodiment mainly corrects the plane (x and y) coordinates of the drone.

[0048] When the first relative posture is obtained by FMCW millimeter wave radar ranging, due to the radar beam angle θ beam The azimuth deviation caused by the forward compensation correction formula is:

[0049] In step S104, the semantic map is sent to the UAV to instruct the UAV to obtain image data based on the lane lines in the semantic map. Based on the image data and the semantic map, a second relative position of the UAV and the vehicle platform is obtained. Based on the second relative position and the precise coordinates of the UAV, the precise coordinates of the vehicle are determined.

[0050] Backward compensation uses drone data to correct vehicle coordinates. The drone uses its onboard camera to identify lane lines in the semantic map and uses the EPnP algorithm to calculate the relative pose of the vehicle platform (the second relative pose). The backward compensation correction formula is:

[0051] Among them, X i represents the 3D coordinate of the i-th lane control point in the semantic map, x i is the coordinate of the corresponding point in the image, π is the camera projection model, and the output E is the rotation matrix R and the translation vector t to compensate for the cumulative error of the vehicle-mounted SLAM (Simultaneous localization and mapping).

[0052] Optionally, the above PnP algorithm can be replaced by the Iterative Closest Point (ICP) algorithm. radar and the vehicle-borne semantic map point set M lidar Perform matching and minimize the objective function:

[0053] Among them, P i represents the coordinates of the i-th point in the millimeter-wave radar point cloud, q i Represents the coordinates of the i-th point in the semantic map. This method is more robust in weakly textured scenes (such as tunnel walls).

[0054] Through the above steps, two-way correction (forward compensation of drone GNSS multipath error, backward compensation of vehicle-mounted SLAM position error) is performed to reduce cross-platform cumulative error; lightweight semantic maps reduce transmission bandwidth while maintaining high recognition rate.

[0055] The two-way compensation mechanism allows a single UWB anchor point to achieve omnidirectional positioning. The traditional solution requires 4 anchor points to solve the 3D position (cost C UWB ×4), and this embodiment combines the visual azimuth angle ф and the UWB distance d to realize vehicle positioning, and the geometric relationship is:

[0056] Where β is the pitch angle. A single drone anchor point achieves omnidirectional positioning, reducing hardware costs.

[0057] In some embodiments, after obtaining the drone coordinates and the first relative pose between the vehicle and the drone from the drone, the method further includes: Step S201: Acquire the time series characteristic parameters of the UAV, which include the time series of IMU temperature, UWB signal-to-noise ratio, and visual feature tracking number.

[0058] Step S202: Analyze the time series characteristic parameters to obtain correction coefficients, and correct the coordinates of the UAV and the first relative posture based on the correction coefficients.

[0059] This embodiment corrects the relative position of the vehicle and the UAV through the real-time environmental parameters of the UAV, constructs a vehicle-UAV virtual collaborative body, realizes dynamic gain control, and reduces the multipath interference error of the UAV.

[0060] In some embodiments, the correction coefficient is a Kalman filter gain coefficient, and step S202 specifically includes: Step S2021: Based on the pre-built LSTM network and the time series characteristic parameters, determine the adjustment amount of the Kalman filter gain coefficient.

[0061] Step S2022: Obtain a Kalman filter gain coefficient according to the adjustment amount, and correct the UAV coordinates and the first relative posture based on the Kalman filter gain coefficient.

[0062] Input data: IMU temperature T, UWB signal-to-noise ratio SNR, number of visual feature tracks N feat time series (preferably, the sampling period is 100ms).

[0063] LSTM network structure: input layer 3 nodes → hidden layer 128 nodes (Tanh activation) → output layer 1 node (linear activation).

[0064] Output data: Kalman filter gain coefficient α k The adjustment amount Δα.

[0065] The calculation formula of the adjustment amount Δα is:

[0066] Among them, W h is the hidden layer weight matrix, b h is the bias vector.

[0067] Training configuration: using sliding time window T W (covering 1 second of data), the loss function is MAE + regularization term.

[0068] The dynamically adjusted gain coefficient α k new =α k +Δα is sent to the UAV platform in real time to suppress temperature drift or multipath interference.

[0069] The LSTM network predicts the Kalman gain adjustment Δα based on real-time environmental parameters (temperature, signal-to-noise ratio, and the number of visual feature tracks). When the UWB is subject to multipath interference and the SNR is less than 10dB, the LSTM output Δα < 0, reducing the gain weight and suppressing noise amplification. Field measurements have shown that this mechanism reduces UWB ranging error by 40% in interference environments.

[0070] In some embodiments, step S202 specifically includes: In step S2023, a positioning error is obtained according to the timing characteristic parameters through a DQN-based reinforcement learning controller.

[0071] Step S2024: Correct the coordinates of the UAV and the first relative posture based on the positioning error.

[0072] The above LSTM network can be replaced by a reinforcement learning controller based on DQN (Deep Q-Network).

[0073] Define the state space s=[T,SNR,N feat ], action space α=Δα, reward function:

[0074] Among them, ε pos is the positioning error, ε energy To calculate energy consumption.

[0075] The Q network iterative update strategy is:

[0076] Where η is the learning rate and γ is the discount factor. The DQN reinforcement learning controller can suppress temperature drift in the long term.

[0077] In the above method, bidirectional constraints are the cornerstone of accuracy, lightweight maps and dynamic parameter adjustment (dynamic gain control) support real-time robustness, and single anchor point and multi-link design ensure engineering feasibility.

[0078] Preferably, the above positioning algorithm is performed on a vehicle-mounted platform.

[0079] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0080] This embodiment also provides a joint positioning system for a vehicle-mounted platform and a drone, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be repeated here. As used below, the terms "module," "unit," "subunit," etc. may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

[0081] Figure 2 FIG is a structural block diagram of a joint positioning system of a vehicle-mounted platform and a UAV according to an embodiment of the present application. Figure 2 As shown, the system includes: The data acquisition module 31 is used to obtain the LiDAR point cloud and vehicle coordinates from the vehicle-mounted platform, and obtain the drone coordinates and the first relative position between the vehicle and the drone from the drone. The first relative position includes a relative distance and a direction angle.

[0082] The map generation module 32 is used to generate a semantic map according to the LiDAR point cloud based on a parameterized curve fitting algorithm.

[0083] The first correction module 33 is used to determine the precise coordinates of the UAV based on the vehicle coordinates and the first relative posture.

[0084] The second correction module 34 is used to send the semantic map to the drone to instruct the drone to capture image data based on the lane lines in the semantic map, obtain a second relative position between the drone and the vehicle platform based on the image data and the semantic map, and determine the precise coordinates of the vehicle based on the second relative position and the precise coordinates of the drone.

[0085] In some embodiments, the map generation module 32 includes: The preprocessing module is used to project the LiDAR point cloud onto a 2D grid map and remove outliers through random sampling consistency to obtain a lane line point set.

[0086] The map construction module is used to perform parameterized modeling of lane line point sets based on the cubic Bezier curve fitting algorithm to obtain a semantic map.

[0087] In some embodiments, the vehicle coordinates are RTK coordinates obtained through vehicle-mounted real-time dynamic differential technology, and the UAV coordinates are GNSS coordinates obtained through a satellite positioning system; the first correction module 33 is used to correct the GNSS coordinates of the UAV based on the RTK coordinates and the first relative posture to obtain the precise coordinates of the UAV.

[0088] In some embodiments, the system further comprises: The parameter acquisition module is used to obtain the time series characteristic parameters of the drone, which include the time series of IMU temperature, UWB signal-to-noise ratio and visual feature tracking number.

[0089] The data correction module is used to analyze the time series characteristic parameters to obtain a correction coefficient, and correct the drone coordinates and the first relative posture based on the correction coefficient.

[0090] In some embodiments, the correction coefficient is a Kalman filter gain coefficient, and the data correction module includes: The LSTM network module is used to determine the adjustment amount of the Kalman filter gain coefficient based on the time series characteristic parameters based on the pre-built LSTM network.

[0091] The first data correction module is used to obtain a Kalman filter gain coefficient according to the adjustment amount, and correct the UAV coordinates and the first relative posture based on the Kalman filter gain coefficient.

[0092] In some embodiments, the LSTM network includes: The DQN module is used to obtain the positioning error according to the timing feature parameters through a DQN-based reinforcement learning controller.

[0093] The second data correction module is used to correct the coordinates of the UAV and the first relative posture based on the positioning error.

[0094] In some embodiments, the system further includes: a data channel construction module for establishing a low-latency data channel between the vehicle platform and the drone through 5G communication and vehicle-to-everything technology.

[0095] Through the above system, two-way correction (forward compensation of drone GNSS multipath error, backward compensation of vehicle-mounted SLAM position error) is carried out to reduce the cumulative error across platforms; lightweight semantic maps reduce transmission bandwidth while maintaining high recognition rate.

[0096] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0097] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0098] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0099] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program: S1, obtains the LiDAR point cloud and vehicle coordinates from the vehicle platform, obtains the drone coordinates and the first relative pose between the vehicle and the drone from the drone, and the first relative pose includes a relative distance and a direction angle.

[0100] S2, based on the parameterized curve fitting algorithm, generates a semantic map from the LiDAR point cloud.

[0101] S3, determining the precise coordinates of the UAV according to the vehicle coordinates and the first relative pose.

[0102] S4, sending the semantic map to the UAV to instruct the UAV to capture image data based on the lane lines in the semantic map, and obtaining a second relative position between the UAV and the vehicle-mounted platform based on the image data and the semantic map. Based on the second relative position and the precise coordinates of the UAV, the precise coordinates of the vehicle are determined.

[0103] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0104] In one embodiment, Figure 3 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 3 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 3 As shown. The electronic device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a joint positioning method of a vehicle-mounted platform and a drone is implemented.

[0105] Those skilled in the art will understand that Figure 3The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0106] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0107] Those skilled in the art should understand that the various technical features of the above-described embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0108] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A joint positioning method for a vehicle-mounted platform and an unmanned aerial vehicle, characterized in that: The method comprises: Acquire a LiDAR point cloud and vehicle coordinates from the vehicle-mounted platform, and acquire drone coordinates and a first relative pose between the vehicle and the drone from the drone, wherein the first relative pose includes a relative distance and a direction angle; generating a semantic map based on the LiDAR point cloud based on a parameterized curve fitting algorithm; Determining the precise coordinates of the drone based on the vehicle coordinates and the first relative pose; The semantic map is sent to the drone to instruct the drone to obtain image data based on the lane lines in the semantic map. A second relative position of the drone and the vehicle-mounted platform is obtained based on the image data and the semantic map. The precise coordinates of the vehicle are determined based on the second relative position and the precise coordinates of the drone.

2. The method according to claim 1, characterized in that Generating a semantic map according to the LiDAR point cloud based on a parameterized curve fitting algorithm includes: Project the LiDAR point cloud onto a 2D grid map and remove outliers through random sampling consistency to obtain a lane line point set; Based on a cubic Bezier curve fitting algorithm, the lane line point set is parameterized and modeled to obtain the semantic map.

3. The method according to claim 1, characterized in that The vehicle coordinates are RTK coordinates obtained by on-board real-time dynamic differential technology, and the UAV coordinates are GNSS coordinates obtained by a satellite positioning system; determining the precise coordinates of the UAV based on the vehicle coordinates and the first relative pose includes: According to the RTK coordinates and the first relative pose, the GNSS coordinates of the UAV are corrected to obtain the precise coordinates of the UAV.

4. The method according to claim 1, wherein After acquiring the drone coordinates and the first relative pose between the vehicle and the drone from the drone, the method further includes: Acquire time series characteristic parameters of the UAV, wherein the time series characteristic parameters include IMU temperature, UWB signal-to-noise ratio, and time series of visual feature tracking number; The time series characteristic parameters are analyzed to obtain correction coefficients, and the UAV coordinates and the first relative posture are corrected based on the correction coefficients.

5. The method according to claim 4, characterized in that The correction coefficient is a Kalman filter gain coefficient, and the analysis of the time series characteristic parameters to obtain the correction coefficient, and the correction of the UAV coordinates and the first relative posture based on the correction coefficient includes: Based on the pre-built LSTM network, the adjustment amount of the Kalman filter gain coefficient is determined according to the time series characteristic parameters; The Kalman filter gain coefficient is obtained according to the adjustment amount, and the UAV coordinates and the first relative posture are corrected based on the Kalman filter gain coefficient.

6. The method according to claim 4, characterized in that Analyzing the time series characteristic parameters to obtain a correction coefficient, and correcting the UAV coordinates and the first relative posture based on the correction coefficient includes: A positioning error is obtained according to the timing characteristic parameters through a DQN-based reinforcement learning controller; The UAV coordinates and the first relative pose are corrected based on the positioning error.

7. The method according to claim 1, characterized in that The method also includes: establishing a low-latency data channel between the vehicle platform and the drone through 5G communication and vehicle-to-everything technology.

8. A joint positioning system of a vehicle-mounted platform and an unmanned aerial vehicle, characterized in that: The system comprises: a data acquisition module, configured to acquire a LiDAR point cloud and vehicle coordinates from the vehicle-mounted platform, and acquire drone coordinates and a first relative position between the vehicle and the drone from the drone, wherein the first relative position includes a relative distance and a direction angle; A map generation module, configured to generate a semantic map based on the LiDAR point cloud based on a parameterized curve fitting algorithm; a first correction module, configured to determine the precise coordinates of the UAV based on the vehicle coordinates and the first relative pose; The second correction module is used to send the semantic map to the drone to instruct the drone to capture image data according to the lane lines in the semantic map, obtain a second relative position between the drone and the vehicle-mounted platform based on the image data and the semantic map, and determine the precise coordinates of the vehicle based on the second relative position and the precise coordinates of the drone.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the joint positioning method of the vehicle-mounted platform and the unmanned aerial vehicle according to any one of claims 1 to 7 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the joint positioning method of the vehicle-mounted platform and the unmanned aerial vehicle according to any one of claims 1 to 7 is implemented.

Images