Positioning method, apparatus and device based on multi-sensor data and storage medium

Abstract

The application relates to a positioning method and device based on multi-sensor data, equipment and a storage medium, which comprises the following steps: acquiring a plurality of satellite positioning data sent by a global satellite navigation system of a movable target object; screening first positioning data and second positioning data from the plurality of satellite positioning data, wherein the first positioning data is used as position data of a virtual reference station, and the second positioning data is used as position data of the movable target object; acquiring sensor data of a plurality of sensors of the movable target object; determining an absolute position of the virtual reference station according to the sensor data of the plurality of sensors; determining a relative position between the movable target object and the virtual reference station according to the first positioning data and the second positioning data; and calculating an absolute position of the movable target object according to the absolute position of the virtual reference station and the relative position. The application can realize high-precision positioning.

Description

Technical Field

[0001] This application relates to the field of object positioning technology, and in particular to a positioning method, apparatus, device and storage medium based on multi-sensor data. Background Technology

[0002] Most current multi-sensor fusion positioning solutions for autonomous driving adopt network RTK (Real-time kinematic) technology. Network RTK is a GNSS (Global Navigation Satellite System) real-time dynamic differential positioning technology. The rover receives raw observation data from a virtual base station through the network. After a double-difference process of carrier phase, receiver errors, satellite clock errors, ionospheric errors, tropospheric errors, etc., can be eliminated, thereby achieving dynamic centimeter-level positioning accuracy. Due to its high accuracy and fast convergence, it is widely used in the field of autonomous driving positioning.

[0003] Existing multi-sensor fusion positioning technologies, such as Figure 1 As shown, the common practice is to obtain positioning data from a global navigation satellite system (GNSS), then use network-based real-time kinematics (RTK) to calculate a fixed solution representing the vehicle's position. This fixed solution is then combined with multi-sensor data and an error-state Kalman filter algorithm to estimate the vehicle's high-precision position. However, existing technologies have the following drawbacks: when the vehicle's network is poor, it cannot receive differential data, and RTK will not be able to obtain a fixed solution, thus failing to provide a centimeter-level absolute position. In such cases, multi-sensor fusion methods are typically used, such as vision-based lane matching and radar-based point cloud matching, to obtain the vehicle's position. However, both vision-based lane matching and radar-based point cloud matching rely on high-precision maps. When these high-precision maps change and cannot be updated in time, errors in vision- and radar-based matching and positioning will inevitably occur, posing a significant risk to positioning and representing a current pain point. Therefore, how to locate a vehicle when it lacks a network and cannot obtain a high-precision absolute position through network-based RTK technology is a pressing issue that needs to be addressed. Summary of the Invention

[0004] Based on this, this application provides a positioning method, apparatus, device, and storage medium based on multi-sensor data to solve the problems existing in the prior art.

[0005] Firstly, a positioning method based on multi-sensor data is provided, the method comprising:

[0006] Acquire multiple satellite positioning data transmitted by the global navigation satellite system for a movable target object;

[0007] First positioning data and second positioning data are selected from the plurality of satellite positioning data. The first positioning data is used as the location data of the virtual base station, and the second positioning data is used as the location data of the movable target object.

[0008] Acquire sensor data from multiple sensors of a movable target object, and determine the absolute position of the virtual reference station based on the sensor data from the multiple sensors;

[0009] The relative position of the movable target object and the virtual base station is determined based on the first positioning data and the second positioning data;

[0010] The absolute position of the movable target object is calculated based on the absolute position of the virtual base station and the relative position.

[0011] According to one achievable method in an embodiment of this application, determining the relative position of the movable target object and the virtual base station based on the first positioning data and the second positioning data includes:

[0012] The first positioning data is optimized based on the sensor data from the multiple sensors, and the optimized data is used to obtain the high-precision positioning data of the virtual reference station.

[0013] The second positioning data is optimized based on the sensor data from the multiple sensors to obtain high-precision positioning data for the rover.

[0014] The phase position is determined based on the high-precision positioning data of the virtual reference station and the high-precision positioning data of the rover.

[0015] According to one achievable method in an embodiment of this application, optimizing the first positioning data based on sensor data from the plurality of sensors includes:

[0016] The sensor data from the multiple sensors and the first positioning data are processed according to the optimal estimation algorithm to optimize the first positioning data;

[0017] And / or,

[0018] The optimization of the second positioning data based on the sensor data from the plurality of sensors includes:

[0019] The sensor data from the multiple sensors and the second positioning data are processed according to the optimal estimation algorithm to optimize the second positioning data;

[0020] According to one possible implementation method in an embodiment of this application, the method further includes:

[0021] Configure a sliding time window with a preset duration;

[0022] According to the sliding time window, multiple satellite positioning data sent by the global satellite navigation system are received and stored, and each satellite positioning data has a corresponding timestamp;

[0023] The step of filtering the first positioning data and the second positioning data from the plurality of satellite positioning data includes:

[0024] Based on the timestamps corresponding to each satellite positioning data, the first positioning data and the second positioning data are selected from the plurality of satellite positioning data.

[0025] According to one achievable method in an embodiment of this application, the step of filtering out the first positioning data and the second positioning data from the plurality of satellite positioning data based on the timestamps corresponding to each satellite positioning data includes:

[0026] Based on the timestamps corresponding to each satellite positioning data, the positioning data at the start time of the sliding time window is selected from the plurality of satellite positioning data as the first positioning data;

[0027] Based on the timestamps corresponding to each satellite positioning data, any positioning data at a time other than the start time of the sliding time window is selected from the plurality of satellite positioning data as the second positioning data.

[0028] According to one achievable method in an embodiment of this application, determining the absolute position of the virtual base station based on sensor data from the plurality of sensors includes:

[0029] The visual semantic features and / or lidar point cloud features of the high-precision map are determined based on the sensor data from the multiple sensors.

[0030] The absolute position of the virtual reference station is determined based on the visual semantic features and / or the lidar point cloud features.

[0031] According to one possible implementation method in an embodiment of this application, the method further includes:

[0032] Based on the sensor data from the multiple sensors, the location data of the movable target object, determined by the visual semantic features of a high-precision map and / or the point cloud features of a lidar, is obtained.

[0033] Compare the absolute position of the movable target object with the deviation of the position data;

[0034] If the deviation exceeds a preset threshold, the movable target object is determined to be in the high-precision map change area.

[0035] Secondly, a positioning device based on multi-sensor data is provided, the device comprising:

[0036] Data acquisition module: used to acquire multiple satellite positioning data sent by the global satellite navigation system for movable target objects;

[0037] Data determination module: used to filter out first positioning data and second positioning data from the plurality of satellite positioning data, wherein the first positioning data is used as the location data of the virtual reference station and the second positioning data is used as the location data of the movable target object;

[0038] Location determination module: used to acquire sensor data from multiple sensors of a movable target object, and determine the absolute position of the virtual reference station based on the sensor data from the multiple sensors;

[0039] Location calculation module: determines the relative position of the movable target object and the virtual reference station based on the first positioning data and the second positioning data; calculates the absolute position of the movable target object based on the absolute position of the virtual reference station and the relative position.

[0040] Thirdly, a computer device is provided, comprising:

[0041] At least one processor; and

[0042] A memory communicatively connected to the at least one processor; wherein,

[0043] The memory stores computer instructions that can be executed by the at least one processor to enable the at least one processor to perform the method involved in the first aspect above.

[0044] Fourthly, a computer-readable storage medium is provided, having stored thereon computer instructions, wherein the computer instructions are used to cause a computer to perform the methods involved in the first aspect above.

[0045] According to the technical content provided in the embodiments of this application, this application acquires satellite positioning data sent by the Global Navigation Satellite System (GNSS) of a movable target object, and uses the first positioning data in the satellite positioning data as the position data of a virtual reference station to establish a virtual base station. It uses the second positioning data in the satellite positioning data as the position data of the movable target object, which can be used as a rover. The relative position between the rover and the virtual base station is determined based on the position data of the virtual base station and the rover. Furthermore, the absolute position of the virtual base station is determined using data from multiple sensors. The high-precision absolute position of the rover is calculated using the relative position between the rover and the virtual base station and the absolute position of the virtual base station. By establishing a virtual base station as a virtual reference station, this application can calculate the high-precision position of the movable target object when network RTK is unavailable, thereby achieving accurate positioning. Attached Figure Description

[0046] Figure 1 The flowchart is from the prior art;

[0047] Figure 2 This is a flowchart illustrating a positioning method based on multi-sensor data in one embodiment;

[0048] Figure 3 This is a flowchart illustrating a positioning method based on multi-sensor data in another embodiment;

[0049] Figure 4 This is a structural block diagram of a positioning device based on multi-sensor data in one embodiment;

[0050] Figure 5 This is a schematic structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the scope of the present application.

[0052] Figure 2 A flowchart of a positioning method based on multi-sensor data provided in this application embodiment is shown below. Figure 2 As shown, the method may include the following steps:

[0053] Step 101: Obtain multiple satellite positioning data sent by the global satellite navigation system of the movable target object.

[0054] Specifically, movable target objects can include vehicles, aircraft, drones, and other movable objects. Below, we take an autonomous vehicle as an example. Obtaining multiple satellite positioning data transmitted by the Global Navigation Satellite System (GNSS) of a movable target object can be understood as obtaining multiple satellite positioning data of the autonomous vehicle at different locations transmitted by its GNSS system. These multiple satellite positioning data represent the satellite positioning data of the movable target object at different locations at different points in time.

[0055] Step 102: Select the first positioning data and the second positioning data from multiple satellite positioning data. The first positioning data is used as the location data of the virtual base station, and the second positioning data is used as the location data of the movable target object.

[0056] Specifically, first positioning data and second positioning data are selected from multiple satellite positioning data sets. The first and second positioning data represent the positioning data of a movable target object at different points in time, with the time point of the first positioning data preceding the time point of the second positioning data. The first positioning data serves as the location data of a virtual reference station; it is equivalent to a base station because this base station is hypothetical and does not actually exist, hence the name virtual reference station. The second positioning data serves as the current location data of the movable target object, equivalent to a rover. Preferably, the first positioning data can be the location data at the initial moment when the vehicle is in a network-free state.

[0057] Step 103: Obtain sensor data from multiple sensors of the movable target object, and determine the absolute position of the virtual base station based on the sensor data from multiple sensors.

[0058] Specifically, when the movable target is an autonomous vehicle, the multiple sensors mainly include: wheel speed, Global Navigation Satellite System (GNSS), Inertial Measurement Unit (IMU), 3D LiDAR, and camera. Acquiring sensor data from these multiple sensors on the movable target includes: acquiring wheel speed data, inertial navigation data from the IMU, LiDAR data, camera data, and high-precision map data. Based on the inertial navigation data from the IMU, point cloud matching or feature matching data from the LiDAR, visual semantic feature matching data from the high-precision map and camera data, and the wheel speed data, combined with the initial positioning data obtained by the virtual base station through the GNSS, the high-precision absolute position of the virtual base station can be determined using optimal estimation algorithms, such as the error state Kalman filter algorithm.

[0059] Preferably, in one embodiment of this application, step 103, determining the absolute position of the virtual base station based on sensor data from multiple sensors, includes: determining the visual semantic features of the high-precision map and / or the lidar point cloud features based on the sensor data from multiple sensors; and determining the absolute position of the virtual base station based on the visual semantic features and / or the lidar point cloud features.

[0060] Step 104: Determine the relative position of the movable target object and the virtual base station based on the first positioning data and the second positioning data.

[0061] Specifically, the first positioning data represents the data of the virtual base station, which is equivalent to building a virtual base station as a base station in the absence of a network; the second positioning data represents the current positioning data of the mobile target object, which is equivalent to the data of the rover. In this way, based on the position data of the virtual base station and the rover, differential calculation can be performed to obtain the relative position of the rover relative to the virtual base station, that is, to determine the relative position of the mobile target object and the virtual base station.

[0062] Step 105: Calculate the absolute position of the movable target object based on the absolute and relative positions of the virtual base station.

[0063] Specifically, based on the absolute position of the virtual base station determined in step 103 and the relative position of the movable target object and the virtual base station determined in step 104, the high-precision absolute position of the movable target object can be calculated.

[0064] As can be seen, this application embodiment acquires satellite positioning data sent by the Global Navigation Satellite System (GNSS) of the movable target object, and establishes a virtual base station using the first positioning data as a virtual reference station, and uses the second position data, i.e., the position data of the movable target object, as a rover station. Furthermore, the absolute position of the virtual reference station is determined using multiple sensor data of the movable target object, thereby determining the relative position of the movable target object and the virtual reference station based on the virtual reference station data and the rover station data, and further calculating the high-precision absolute position of the movable target object. This application, by establishing a virtual base station as a virtual reference station, can calculate the high-precision position of the movable target object even when network RTK is unavailable, thereby achieving accurate positioning.

[0065] Currently, in existing technologies, when network unavailability renders RTK (Real-Time Kinematics) unavailable, positioning relies solely on visual semantic feature matching and LiDAR point cloud matching or feature matching. The positioning accuracy depends on the high-precision map. However, when road construction occurs or the actual physical environment changes significantly, the prior high-precision map cannot accurately reflect the situation, inevitably leading to errors in high-precision map matching and thus incorrect target object positioning. In the field of autonomous driving, positional errors typically result in deviations exceeding 30cm, posing a significant threat to positioning accuracy. This application addresses this by establishing a virtual base station as a virtual reference station. This allows for the calculation of the high-precision position of movable target objects even when network RTK is unavailable, resolving the positioning deviations caused by high-precision map matching when the physical environment changes significantly. This achieves high-precision positioning with high accuracy and high precision.

[0066] In one embodiment of this application, the step 104 above, which determines the relative position of the movable target object and the virtual base station based on the first positioning data and the second positioning data, includes: optimizing the first positioning data based on sensor data from multiple sensors to obtain high-precision positioning data of the virtual base station; optimizing the second positioning data based on sensor data from multiple sensors to obtain high-precision positioning data of the rover; and determining the phase position based on the high-precision positioning data of the virtual base station and the high-precision positioning data of the rover.

[0067] Specifically, the positioning method provided in this embodiment may include the following steps: First, satellite positioning data of multiple location points are acquired based on GNSS; further, first positioning data and second positioning data are selected from the multiple satellite positioning data, the first positioning data is used as the location data of the virtual reference station, and the second positioning data is used as the location data of the movable target object.

[0068] Furthermore, sensor data from multiple sensors is acquired, and the first positioning data is optimized based on this data to obtain high-precision positioning data for the virtual base station. Similarly, the second positioning data is optimized based on the same sensor data to obtain high-precision positioning data for the rover. Finally, the phase position is determined using both the high-precision positioning data from the virtual base station and the rover.

[0069] In existing technologies, such as Figure 1As shown, the vehicle directly receives RTK results in a network state. The RTK results here are RTK differential data calculated using RTK technology. That is, the vehicle directly receives RTK differential data. This RTK differential data is then combined with data from multiple sensors within the vehicle for optimization to obtain the vehicle's high-precision absolute position. In this embodiment, the rover is used when the vehicle performs RTK differential data calculation. Because the vehicle has multiple sensors, its position information can be obtained through these sensors. The sensor data from these multiple sensors contains the vehicle's position information. By using the sensor data from these multiple sensors to optimize the first and second positioning data, high-precision positioning data is obtained. This allows for precise adjustment of the virtual base station's position data and the rover's position data before performing RTK differential data calculation, resulting in RTK differential data that more accurately represents the phase positions of the virtual base station and the rover. Existing technologies cannot perform precision adjustments on the virtual base station's position data and the rover's position data before performing RTK calculation. In this embodiment, before calculating the RTK differential data, the vehicle's own sensor data is used to adjust the accuracy of the virtual base station's position data and the rover's position data, thus further improving the accuracy of the relative position. Preferably, in one embodiment of this application, optimizing the first positioning data based on sensor data from multiple sensors includes: processing the sensor data from multiple sensors and the first positioning data according to an optimal estimation algorithm to optimize the first positioning data; and / or, optimizing the second positioning data based on sensor data from multiple sensors includes: processing the sensor data from multiple sensors and the second positioning data according to an optimal estimation algorithm to optimize the second positioning data.

[0070] like Figure 3 As shown, the multiple sensors here mainly include: wheel speed, Global Navigation Satellite System (GNSS), Inertial Measurement Unit (IMU), 3D LiDAR, and camera. The sensor data from these multiple sensors and the initial positioning data are processed according to an optimal estimation algorithm. The initial positioning data is optimized by acquiring the wheel speed from the autonomous vehicle, the inertial navigation data from the IMU, the LiDAR data, and the camera and high-precision map data. Based on the inertial navigation data from the IMU, the point cloud matching or feature matching data from the LiDAR, the visual semantic feature matching data from the high-precision map and the camera, and the wheel speed data, combined with the initial positioning data obtained from the GNSS, the data is input into an optimal estimation algorithm, such as an error-state Kalman filter optimal estimation algorithm, to optimize the initial positioning data. The optimization of the second positioning data follows the same principle.

[0071] Finally, dynamic differential calculations are performed based on the high-precision positioning data of the virtual base station and the rover station to determine the phase position between the rover station and the virtual base station. Based on the high-precision absolute position of the virtual base station, combined with the phase position between the rover station and the virtual base station, the high-precision absolute position of the rover station is determined.

[0072] In one embodiment of this application, the positioning method based on multi-sensor data provided in this application further includes: configuring a sliding time window of a preset duration; receiving and storing multiple satellite positioning data sent by a global satellite navigation system according to the sliding time window, each satellite positioning data having a corresponding timestamp; and filtering out first positioning data and second positioning data from the multiple satellite positioning data in step 102, including: filtering out first positioning data and second positioning data according to the timestamps corresponding to each satellite positioning data.

[0073] Specifically, such as Figure 3 As shown, a preset sliding time window is configured to store raw GNSS observation data, including carrier phase, pseudorange, and Doppler data, for a certain duration. This means receiving and storing multiple satellite positioning data transmitted by the Global Navigation Satellite System (GNSS) according to the sliding time window. Each satellite positioning data point has a corresponding timestamp, representing the satellite positioning data at different points in time. Step 102, selecting the first and second positioning data from the multiple satellite positioning data, includes: selecting the first and second positioning data based on the timestamps corresponding to each satellite positioning data point.

[0074] Preferably, in one embodiment of this application, filtering out first positioning data and second positioning data from multiple positioning data includes: filtering out positioning data at the start time of a sliding time window as first positioning data from multiple positioning data; and filtering out any positioning data at a time other than the start time of the sliding time window as second positioning data from multiple positioning data.

[0075] Specifically, such as Figure 3 As shown, positioning data at the start of the sliding time window is selected from multiple positioning data sets as the first positioning data, i.e., the data for the virtual base station (virtual base station). Any positioning data outside the start of the sliding time window is selected from the multiple positioning data sets as the second positioning data, i.e., the data for the rover. RTK calculation is performed based on the virtual station data and the rover data to obtain the position of the rover relative to the virtual station. Since areas with high-precision map changes during vehicle movement typically last for more than 15 seconds (taking a high-speed driving speed of 80 km / h as an example), the preset sliding time can be 60s-120s. This solves the problem of high-precision vehicle positioning in areas with high-precision map changes.

[0076] In one embodiment, the positioning method provided in this application further includes: acquiring position data of a movable target object based on visual semantic features of a high-precision map and / or lidar point cloud features, according to sensor data from multiple sensors; comparing the absolute position of the movable target object with the deviation of the position data; and determining that the movable target object is located in a change area of ​​the high-precision map if the deviation exceeds a preset threshold.

[0077] Specifically, the location data of a movable target object determined based on the visual semantic features and / or lidar point cloud features of a high-precision map refers to the location of the movable target object at the time point corresponding to the second positioning data, determined based on the visual semantic features and / or lidar point cloud features of the high-precision map from the sensors. This is the absolute location of the target object determined directly by multiple sensors in the absence of network RTK in existing technologies. The absolute location of the movable target object obtained in step 105 of the above embodiments is compared with the positioning data determined by the visual semantic features and / or lidar point cloud features of the movable target object based on the high-precision map. The difference can be calculated. If the difference exceeds a preset threshold, the positioning data determined based on the visual semantic features and / or lidar point cloud features of the high-precision map is inaccurate, thus determining that the movable target object is currently in a map change area. Here, a map change area refers to an area where the high-precision map changes due to changes in the physical environment. Changes in the high-precision map will lead to inaccurate positioning data determined based on the visual semantic features and / or lidar point cloud features of the high-precision map. Preferably, after identifying that the movable target object is currently in a map change area, the visual semantic data and / or lidar data of the high-precision map are removed, and the absolute position of the movable target object obtained in step 105 above is updated as an observation, such as... Figure 3 As shown, this allows for more accurate positioning. This embodiment can identify areas of map change and further eliminate gross errors in visual matching and LiDAR point cloud matching, making multi-sensor fusion positioning more robust and stable.

[0078] The implementation of this application can solve the problem of high-precision positioning when network RTK is unavailable, and can also be used to identify areas of map change and eliminate gross errors in visual matching and lidar point cloud matching, making multi-sensor fusion positioning more robust and stable.

[0079] As can be seen, the embodiments of this application have the following advantages: RTK relative positioning is achieved by storing and filtering data through a sliding window as a virtual base station and rover. The virtual base station data of the sliding window can use the raw GNSS observation data at the start time of the sliding window as the virtual base station. The absolute position of the virtual base station is obtained by visual matching or lidar point cloud matching based on a high-precision map. In areas where the high-precision map has not changed, the position accuracy can be less than 10cm. That is to say, the data of the virtual base station uses the multi-sensor fusion positioning results outside the map change area. When the autonomous vehicle spends 50 seconds in the high-precision map change area, the positioning accuracy provided by the positioning technology used in this application is much higher than that of IMU + wheel velocity dead reckoning and visual / lidar calculation. This solves the high-precision positioning problem on the one hand, and can be used to identify and eliminate gross errors in visual matching and lidar point cloud matching on the other hand, making multi-sensor fusion positioning more robust and stable.

[0080] It should be understood that, although Figure 2 , Figure 3 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this application, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Furthermore, Figure 2 , Figure 3 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0081] Figure 4 A schematic diagram of a positioning device based on multi-sensor data is provided for an embodiment of this application, wherein the main functions of each component module are as follows:

[0082] Data acquisition module 401: used to acquire multiple satellite positioning data sent by the global satellite navigation system of the movable target object;

[0083] Data determination module 402: used to filter out first positioning data and second positioning data from multiple satellite positioning data, the first positioning data being used as the position data of the virtual reference station, and the second positioning data being used as the position data of the movable target object;

[0084] Position determination module 403: used to acquire sensor data from multiple sensors of a movable target object, and determine the absolute position of the virtual base station based on the sensor data from multiple sensors;

[0085] Position calculation module 404: used to determine the relative position of the movable target object and the virtual reference station based on the first positioning data and the second positioning data; and to calculate the absolute position of the movable target object based on the absolute position and the relative position of the virtual reference station.

[0086] In one embodiment of this application, the location calculation module 404 is further configured to: optimize the first positioning data based on sensor data from multiple sensors, and obtain high-precision positioning data of the virtual reference station after optimization;

[0087] The second positioning data is optimized based on sensor data from multiple sensors to obtain high-precision positioning data for the rover; the phase position is determined based on the high-precision positioning data of the virtual reference station and the high-precision positioning data of the rover.

[0088] In one embodiment of this application, the location calculation module 404 is further configured to: process sensor data from multiple sensors and first positioning data according to an optimal estimation algorithm, and optimize the first positioning data; and / or optimize second positioning data based on sensor data from multiple sensors, including: processing sensor data from multiple sensors and second positioning data according to an optimal estimation algorithm, and optimizing the second positioning data.

[0089] In one embodiment of this application, a sliding window module 405 is further included: configured to set a sliding time window of a preset duration; receive and store multiple satellite positioning data sent by the global satellite navigation system according to the sliding time window, each satellite positioning data having a corresponding timestamp; and a data determination module 402 is further configured to: filter out the first positioning data and the second positioning data according to the timestamps corresponding to each satellite positioning data.

[0090] In one embodiment of this application, the data determination module 402 is further configured to: select the positioning data at the start time of the sliding time window from multiple satellite positioning data as the first positioning data; and select any positioning data at a time other than the start time of the sliding time window from multiple satellite positioning data as the second positioning data.

[0091] In one embodiment of this application, the location determination module 403 is further configured to: determine the visual semantic features and / or lidar point cloud features of a high-precision map based on sensor data from multiple sensors; and determine the absolute position of a virtual base station based on the visual semantic features and / or lidar point cloud features.

[0092] In one embodiment of this application, it further includes an identification module 406: configured to acquire positioning data of a movable target object based on visual semantic features of a high-precision map and / or lidar point cloud features, according to sensor data from multiple sensors; compare the absolute position of the movable target object with the deviation of the positioning data; and determine that the movable target object is in a map change area if the deviation exceeds a preset threshold.

[0093] According to the specific embodiments provided in this application, the technical solution provided in this application can have the following advantages:

[0094] RTK relative positioning is achieved by storing and filtering data through a sliding window to serve as a virtual base station and rover. The virtual base station data in the sliding window can use raw GNSS observation data at the start of the sliding window as the virtual base station. The absolute position of the virtual base station is obtained through visual matching based on a high-precision map or LiDAR point cloud matching. In areas where the high-precision map has not changed, the positional accuracy can be less than 10cm. In other words, the virtual base station data uses multi-sensor fusion positioning results outside the map-changed area. When the autonomous vehicle spends 50 seconds in the high-precision map-changed area, the positioning accuracy provided by the positioning technology used in this application is far higher than that calculated by IMU+wheel velocity measurement dead reckoning and visual / LiDAR. This solves the high-precision positioning problem on the one hand, and can be used to identify and eliminate gross errors in visual matching and LiDAR point cloud matching on the other, making multi-sensor fusion positioning more robust and stable.

[0095] The same or similar parts among the above embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments.

[0096] It should be noted that the embodiments of this application may involve the use of user data. In practical applications, user-specific personal data may be used in the scheme described herein within the scope permitted by applicable laws and regulations, provided that it complies with the applicable laws and regulations of the country (e.g., explicit consent from the user, actual notification to the user, explicit authorization from the user, etc.).

[0097] According to embodiments of this application, this application also provides a computer device and a computer-readable storage medium. This application further provides a computer device including at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores computer instructions executable by the at least one processor, the computer instructions being executed by the at least one processor to enable the at least one processor to perform the positioning method based on multi-sensor data described in any of the above embodiments.

[0098] like Figure 5 The diagram shown is a block diagram of a computer device according to an embodiment of this application. The term "computer device" is intended to represent various forms of digital computers or mobile devices. The digital computer may include a desktop computer, a portable computer, a workbench, a personal digital assistant, a server, a mainframe computer, and other suitable computers. The mobile device may include a tablet computer, a smartphone, a wearable device, etc.

[0099] like Figure 5 As shown, the computer device 500 includes a computing unit 501, a ROM 502, a RAM 503, a bus 504, and an input / output (I / O) interface 505. The computing unit 501, ROM 502, and RAM 503 are interconnected via the bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0100] The computing unit 501 can execute various processes in the method embodiments of this application according to computer instructions stored in the read-only memory (ROM) 502 or computer instructions loaded from the storage unit 508 into the random access memory (RAM) 503. The computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. The computing unit 501 can include, but is not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. In some embodiments, the methods provided in the embodiments of this application can be implemented as computer software programs, which are tangibly contained in a computer-readable storage medium, such as the storage unit 508.

[0101] RAM 503 can also store various programs and data required for the operation of device 500. Part or all of the computer program can be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509.

[0102] The input unit 506, output unit 507, storage unit 508, and communication unit 509 in computer device 500 can be connected to I / O interface 505. The input unit 506 can be, for example, a keyboard, mouse, touchscreen, or microphone; the output unit 507 can be, for example, a monitor, speaker, or indicator light. Device 500 can exchange information and data with other devices through the communication unit 509.

[0103] It should be noted that the device may also include other components necessary for normal operation. It may also include only the components necessary for implementing the solution of this application, without necessarily including all the components shown in the figures.

[0104] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), payload programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof.

[0105] The computer instructions used to implement the methods of this application may be written in any combination of one or more programming languages. These computer instructions may be provided to the computing unit 501 such that when executed by the computing unit 501, such as a processor, the computer instructions cause the execution of the steps involved in the embodiments of the methods of this application.

[0106] This application also provides a computer-readable storage medium storing computer instructions thereon, the computer instructions being used to cause a computer to execute the positioning method based on multi-sensor data described in any of the above embodiments.

[0107] The computer-readable storage medium provided in this application can be a tangible medium that can contain or store computer instructions for performing the steps involved in the method embodiments of this application. The computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, and other forms of storage media.

[0108] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A positioning method based on multi-sensor data, characterized in that, The method includes: Acquire multiple satellite positioning data transmitted by the global navigation satellite system for a movable target object; First positioning data and second positioning data are selected from the plurality of satellite positioning data. The first positioning data is used as the location data of the virtual base station, and the second positioning data is used as the location data of the movable target object. Acquire sensor data from multiple sensors of a movable target object, and determine the visual semantic features and / or lidar point cloud features of a high-precision map based on the sensor data from the multiple sensors. The absolute position of the virtual reference station is determined based on the visual semantic features and / or the lidar point cloud features; The first positioning data is optimized based on the sensor data from the multiple sensors, and the optimized data is used to obtain the high-precision positioning data of the virtual reference station. The second positioning data is optimized based on the sensor data from the multiple sensors to obtain high-precision positioning data for the rover. The relative position is determined based on the high-precision positioning data of the virtual reference station and the high-precision positioning data of the rover station; The absolute position of the movable target object is calculated based on the absolute position of the virtual base station and the relative position.

2. The positioning method based on multi-sensor data according to claim 1, characterized in that, The optimization of the first positioning data based on the sensor data from the plurality of sensors includes: The sensor data from the multiple sensors and the first positioning data are processed according to the optimal estimation algorithm to optimize the first positioning data; And / or, The optimization of the second positioning data based on the sensor data from the plurality of sensors includes: The sensor data from the multiple sensors and the second positioning data are processed according to the optimal estimation algorithm to optimize the second positioning data.

3. The positioning method based on multi-sensor data according to claim 1, characterized in that, The method further includes: Configure a sliding time window with a preset duration; According to the sliding time window, multiple satellite positioning data sent by the global satellite navigation system are received and stored, and each satellite positioning data has a corresponding timestamp; The step of filtering the first positioning data and the second positioning data from the plurality of satellite positioning data includes: Based on the timestamps corresponding to each satellite positioning data, the first positioning data and the second positioning data are selected from the plurality of satellite positioning data.

4. The positioning method based on multi-sensor data according to claim 3, characterized in that, The step of filtering the first positioning data and the second positioning data from the plurality of satellite positioning data according to the timestamps corresponding to each satellite positioning data includes: Based on the timestamps corresponding to each satellite positioning data, the positioning data at the start time of the sliding time window is selected from the plurality of satellite positioning data as the first positioning data; Based on the timestamps corresponding to each satellite positioning data, any positioning data at a time other than the start time of the sliding time window is selected from the plurality of satellite positioning data as the second positioning data.

5. The positioning method based on multi-sensor data according to claim 1, characterized in that, The method further includes: Based on the sensor data from the multiple sensors, the location data of the movable target object, determined by visual semantic features based on a high-precision map and / or lidar point cloud features, is obtained. Compare the absolute position of the movable target object with the deviation of the position data; If the deviation exceeds a preset threshold, the movable target object is determined to be in the high-precision map change area.

6. A positioning device based on multi-sensor data, characterized in that, The device includes: Data acquisition module: used to acquire multiple satellite positioning data sent by the global satellite navigation system for movable target objects; Data determination module: used to filter out first positioning data and second positioning data from the plurality of satellite positioning data, wherein the first positioning data is used as the location data of the virtual reference station and the second positioning data is used as the location data of the movable target object; Location determination module: used to acquire sensor data from multiple sensors of a movable target object, determine the visual semantic features and / or lidar point cloud features of a high-precision map based on the sensor data from the multiple sensors, and determine the absolute position of the virtual base station based on the visual semantic features and / or the lidar point cloud features; The position calculation module is used to optimize the first positioning data based on the sensor data from the multiple sensors to obtain high-precision positioning data for the virtual base station; optimize the second positioning data based on the sensor data from the multiple sensors to obtain high-precision positioning data for the rover; determine the relative position based on the high-precision positioning data of the virtual base station and the high-precision positioning data of the rover; and calculate the absolute position of the movable target object based on the absolute position of the virtual base station and the relative position.

7. A computer device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores computer instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-5.

8. A computer-readable storage medium storing computer instructions thereon, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

Images