Positioning method based on multi-feature information and electronic device

Abstract

The application provides a positioning method based on multi-feature information and an electronic device. In the process of carrier movement, point cloud data obtained by a laser radar is acquired, and feature points in the point cloud data are extracted. Residuals of the feature points to a map are constructed, and iterative optimization is performed based on the residuals to calculate a laser odometry. Dead reckoning information is calculated based on speed information obtained by a millimeter wave radar and pre-integration information obtained by an IMU. Whether the laser odometry is degraded is determined according to the dead reckoning information and the laser odometry. If the laser odometry is degraded, the pose of the carrier is updated based on the dead reckoning information to realize the positioning of the carrier. In the scheme, the laser odometry is optimized in the case of laser odometry degradation, the laser odometry is combined with the millimeter wave and IMU information fusion odometry, the accuracy of the carrier pose is improved, and the safety and accuracy of the driving control are ensured.

Description

Technical Field

[0001] This invention relates to the field of navigation and control technology, and more specifically, to a positioning method and electronic device based on multi-feature information. Background Technology

[0002] LiDAR technology obtains three-dimensional information about a target object by emitting a laser beam and receiving the reflected signal. This technology has been widely used in fields such as autonomous driving, robot navigation, and geographic information systems.

[0003] Currently, the best-performing and most widely used Simultaneous Localization and Mapping (SLAM) solutions are based on a series of Lidar Odometry and Mapping in Real-time (LOAM) schemes. LOAM is primarily designed for localization and mapping using multi-line Lidar systems. In LOAM schemes, the front end typically extracts features from the current Lidar frame, and the back end combines this with information from other sensors to match the current frame to the map, thereby achieving localization.

[0004] In existing implementations, in some point cloud degradation scenarios, such as subway tunnels and highways, the geometric features of point cloud data are almost identical or nonexistent. This makes it easy for incorrect matching to occur when matching and optimizing point cloud data with maps, thus making it impossible to accurately estimate the pose based on laser point clouds, which in turn affects the safety and accuracy of driving control. Summary of the Invention

[0005] The purpose of this invention is to provide a positioning method and electronic device based on multi-feature information, so as to improve the accuracy of the carrier's pose and ensure the safety and accuracy of driving control.

[0006] In a first aspect, the present invention provides a localization method based on multi-feature information, the method comprising:

[0007] During the movement of the carrier, point cloud data obtained by lidar is acquired, and feature points in the point cloud data are extracted;

[0008] Construct the residual from the feature points to the pre-established map, and perform iterative optimization based on the residual to calculate the laser odometry.

[0009] Dead reckoning information is calculated based on velocity information obtained from millimeter-wave radar and pre-integration information obtained from IMU.

[0010] Based on the dead reckoning information and the laser odometry, it is determined whether the laser odometry has degraded. If degraded, the pose of the carrier is updated based on the dead reckoning information to achieve the positioning of the carrier.

[0011] In an optional implementation, the feature points include reflectance feature points, which are extracted from the point cloud data in the following manner:

[0012] The point cloud data is divided into multiple segments;

[0013] For each of the aforementioned sections, the reflection intensity at each point within that section is obtained;

[0014] Points that meet the reflectivity requirements are extracted based on the reflection intensity of each point and used as reflectivity feature points.

[0015] In an optional implementation, the step of extracting points that meet the reflectivity requirements based on the reflection intensity of each point includes:

[0016] The points whose reflection intensity ranks first in a preset position and whose reflection intensity is greater than a first preset intensity are extracted, or the points whose reflectivity change rate ranks first in a preset position and whose reflectivity change rate is greater than a preset change rate and whose reflection intensity is greater than a second preset intensity are extracted, wherein the reflectivity change rate is calculated based on the emission intensity of each point and its neighboring points.

[0017] In an optional implementation, the feature points include line feature points, surface feature points, and reflectivity feature points;

[0018] The step of constructing the residual from the feature points to the pre-established map includes:

[0019] Construct the residuals of the line feature points, the area feature points, and the reflectance feature points to the pre-established line feature map, area feature map, and reflectance feature map, respectively.

[0020] In an optional implementation, the step of constructing the residual between the reflectance feature points and the reflectance feature map in a pre-established map includes:

[0021] The reflectance feature points are converted to the map coordinate system of the reflectance feature map in the pre-established map;

[0022] For each converted reflectance feature point, search for neighboring reflectance feature points in the reflectance feature map;

[0023] A reference plane is constructed based on the neighboring reflectivity feature points;

[0024] The distance between the reflectivity feature point and the reference plane is obtained as the residual to be optimized.

[0025] In an optional implementation, the step of determining whether the laser odometry has degraded based on the dead reckoning information and the laser odometry includes:

[0026] The first displacement increment between two adjacent poses determined based on the point cloud data is calculated using the laser odometry.

[0027] The second displacement increment between two adjacent poses is calculated based on the dead reckoning information, which is determined by the velocity information and the pre-integration information.

[0028] The difference between the first displacement increment and the second displacement increment is detected to be greater than a preset threshold. If it is greater than the preset threshold, the laser odometer is determined to have degraded.

[0029] In an optional implementation, the method further includes:

[0030] Construct a track vector map based on the track positions in a pre-built map;

[0031] Based on the orbital vector map and the current pose of the carrier, determine whether the current pose needs to be optimized;

[0032] If it is determined that the current pose needs to be optimized, then the optimization of the current pose is achieved based on the track information in the track vector map.

[0033] In an optional implementation, the step of determining whether the current pose needs optimization based on the orbital vector map and the current pose of the carrier includes:

[0034] Obtain the spatial extent of the track and the centerline of the track in the track vector map;

[0035] Detect whether the position information in the current pose of the carrier exceeds the spatial range of the track, or detect whether the angle difference between the heading angle in the current pose and the deflection angle determined by the centerline of the track exceeds a preset angle difference;

[0036] If the position information exceeds the spatial range, or the angle difference exceeds the preset angle difference, then the current pose needs to be optimized.

[0037] In an optional implementation, the step of optimizing the current pose based on the orbit information in the orbit vector map includes:

[0038] Determine the intersection point of a circle drawn with the current pose position as the origin and the displacement increment as the radius with the center line of the track in the track vector map, and use the position information of the intersection point as the optimized position information;

[0039] Determine two points on the centerline of the track that are adjacent to the position in the current pose. Calculate the deflection angle of the centerline on the horizontal plane based on the line connecting the two points, and use the deflection angle as the optimized heading angle.

[0040] In a second aspect, the present invention provides an electronic device including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor executing the machine-executable instructions to implement the method described in any of the foregoing embodiments.

[0041] This invention provides a positioning method and electronic device based on multi-feature information. During vehicle movement, point cloud data obtained from a lidar radar is acquired, and feature points are extracted from the point cloud data. A residual from the feature points to the map is constructed, and iterative optimization is performed based on the residual to calculate the lidar odometry. Dead reckoning information is calculated based on velocity information obtained from millimeter-wave radar and pre-integration information obtained from an IMU. Based on the dead reckoning information and the lidar odometry, it is determined whether the lidar odometry has degraded. If degradation occurs, the vehicle's pose is updated based on the dead reckoning information to achieve vehicle positioning. In this scheme, even in the case of lidar odometry degradation, the odometry is optimized by combining millimeter-wave and IMU information fusion, improving the accuracy of the vehicle's pose and ensuring the safety and accuracy of driving control. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A structural block diagram of an electronic device provided in an embodiment of the present invention;

[0044] Figure 2 A flowchart of a localization method based on multi-feature information provided in an embodiment of the present invention;

[0045] Figure 3 This refers to image information in a tunnel scene as described in this embodiment of the invention.

[0046] Figure 4 This refers to point cloud information in a tunnel scenario as described in this embodiment of the invention.

[0047] Figure 5 This is a flowchart of the optimization method in the localization method based on multi-feature information of an embodiment of the present invention;

[0048] Figure 6 This is a functional block diagram of a positioning device based on multi-feature information provided in an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0050] Please see Figure 1 The present invention provides an electronic device, and the positioning method based on multi-feature information provided in the present invention can be applied to the electronic device. The electronic device includes a memory, a processor, and a communication module. The memory, processor, and communication module are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0051] The memory is used to store programs or data. The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read Only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), etc.

[0052] The processor is used to read / write data or programs stored in memory and to perform corresponding functions.

[0053] The communication module is used to establish communication connections between electronic devices and other communication terminals via a network, and to send and receive data via the network.

[0054] It should be understood that, Figure 1 The structure shown is only a schematic diagram of an electronic device; the electronic device may also include components that are larger than those shown. Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0055] In some embodiments, the electronic device can be mounted on a carrier equipped with an inertial measurement unit (IMU), millimeter-wave radar, and lidar. This device can serve as the control system or data processing system for the carrier, executing the multi-feature information-based positioning method provided in this embodiment of the invention to update the carrier's pose in real time, ensuring the accuracy of the carrier's positioning for better subsequent applications. The carrier can be, but is not limited to, subways, robots, automobiles, aircraft, etc.

[0056] The following, combined with Figure 2 The localization method based on multi-feature information provided in the embodiments of the present invention will be described below. Figure 2 This is a flowchart of a localization method based on multiple feature information provided in an embodiment of the present invention. The localization method based on multiple feature information includes the following steps:

[0057] S11: During the movement of the carrier, acquire point cloud data obtained by the lidar and extract feature points from the point cloud data.

[0058] S12: Construct the residuals from feature points to a pre-established map, and perform iterative optimization based on the residuals to calculate the laser odometry.

[0059] S13, dead reckoning information is calculated based on the speed information obtained by millimeter-wave radar and the pre-integration information obtained by IMU.

[0060] S14. Based on dead reckoning information and laser odometry, determine whether the laser odometry has degraded. If it has degraded, proceed to step S15. If it has not degraded, proceed to step S16.

[0061] S15 updates the carrier's pose based on dead reckoning information to achieve carrier positioning.

[0062] S16, the carrier is positioned based on the pose determined by the laser odometry.

[0063] In this embodiment, the carrier is equipped with devices such as lidar, millimeter-wave radar, and IMU. During the carrier's movement, each device performs real-time detection to obtain relevant data. Specifically, the lidar measures the distance to surrounding objects by emitting laser beams and receiving reflected signals. The reflected signal from each laser beam forms a three-dimensional coordinate point, thus constituting point cloud data of the surrounding environment.

[0064] In addition, a map is pre-built, and the method of building the map is similar to the real-time positioning method when the carrier is moving. That is, the real-time positioning process when the carrier is moving requires obtaining point cloud data of the surrounding environment, and this point cloud data is also the data required for map construction.

[0065] The point cloud data obtained during the vehicle's movement is matched with a pre-built map, extracting feature points from the point cloud data during the matching process. Feature points with distinct characteristics can be extracted, such as those on a plane, along a line, or in highlighted areas. Using these distinctive feature points to match with the pre-built map improves matching efficiency and accuracy.

[0066] When matching feature points with the map, a residual from the feature points to the map is constructed. The laser odometry is determined through iterative optimization of the residuals from the feature points to the map in consecutive adjacent LiDAR frames. This iterative optimization can employ a combination of the scan2map and Levenberg-Marquardt (LM) algorithms commonly used in the SLAM field. The LM algorithm, a combination of Gauss-Newton's method and gradient descent, is a commonly used iterative optimization algorithm in this field and will not be elaborated upon in this embodiment. This laser odometry can be understood as the estimation of the carrier's pose information based on continuous point cloud data.

[0067] In scenarios such as subway tunnels and highways, the geometric features of the point cloud data of the surrounding environment change very little. Therefore, the point cloud features are degraded, and the pose information of the carrier estimated based on the point cloud data may also be degraded, that is, the pose information is not accurate enough.

[0068] Furthermore, millimeter-wave radar mounted on the carrier can acquire velocity information, which can be the relative velocity of the carrier with respect to its surrounding environment during movement. An IMU mounted on the carrier can detect and acquire acceleration and angular velocity information during the carrier's motion. Acceleration and angular velocity information from consecutive frames are integrated, and the integrated results are accumulated to obtain pre-integrated information.

[0069] By combining velocity information provided by millimeter-wave radar and pre-integration information provided by IMU, dead reckoning information can be calculated. Dead reckoning is a method that, based on a known initial position, calculates the pose at the next moment using velocity information, relative displacement, and time intervals. Here, dead reckoning information can be understood as the pose information obtained based on this method.

[0070] As mentioned above, the pose information estimated from point cloud data may be inaccurate. In this embodiment, the obtained dead reckoning information can be used to determine whether the laser odometry has degraded, that is, whether the pose information estimated based on point cloud data has degraded. For example, the difference between the dead reckoning information and the laser odometry can be compared, and the magnitude of the difference can be used to determine whether the laser odometry has degraded.

[0071] If the laser odometry is found to be degraded, the carrier's pose can be updated based on the obtained dead reckoning information. Based on the updated pose, the carrier's positioning information can be determined.

[0072] In the positioning method based on multi-feature information provided in this embodiment, when the laser odometry degrades, the odometry is optimized by combining millimeter wave and IMU information fusion, thereby improving the accuracy of the vehicle's pose and ensuring the safety and accuracy of driving control.

[0073] Furthermore, when this positioning method is applied to tunnel scenarios, since tunnels are enclosed spaces, the carrier positioning is limited to the track area within the tunnel. Therefore, this embodiment also introduces a track vector map constructed based on point cloud data, that is, a vector map mainly containing track information. Based on this track vector map, it is used to check whether the pose information determined by the aforementioned method satisfies the actual situation of the track in the track vector map. If not, the pose can be further optimized based on the track information in the track vector map. This further improves the rationality and accuracy of the positioning results.

[0074] The specific implementation methods of each of the above steps will be explained in detail below.

[0075] In existing methods, when extracting feature points from point cloud data and matching them with a map, the matching is generally performed by extracting line feature points and area feature points from the point cloud data.

[0076] Line feature points are points in a point cloud that reflect the edges, contours, or straight-line structures of an object. They are identified by calculating the curvature of the points in the point cloud and selecting points with significant changes in curvature. Surface feature points are points in a point cloud that reflect the surface planes of an object; these points typically reside within the same planar region. They are identified by calculating the covariance matrix of the points in the point cloud, allowing for the identification of sets of points with significantly different eigenvalue distributions; these sets of points are the surface feature points.

[0077] However, in scenarios such as tunnels, line feature points and polygon feature points cannot reflect the uniqueness of features. Therefore, map matching based on line feature points and polygon feature points may lead to incorrect matching, which in turn affects the accuracy of mapping and localization.

[0078] Tunnels typically contain infrastructure such as signs, which generally have highly reflective surfaces. Therefore, this type of infrastructure is relatively unique within the scene, which is beneficial for map matching. For example... Figure 3 and Figure 4 The images and point cloud data of the tunnel scene are shown in the figure, with the area defined by the box being the region with high reflectivity.

[0079] Based on this, this embodiment introduces reflectivity feature points, which are feature points corresponding to this type of highly reflective infrastructure. Map matching is achieved by combining reflectivity feature points, line feature points, and area feature points.

[0080] In this embodiment, in the step of extracting feature points from point cloud data, the reflectance feature points are extracted from the point cloud data in the following manner:

[0081] The point cloud data is divided into multiple segments; for each segment, the reflection intensity of each point within the segment is obtained; based on the reflection intensity of each point, points that meet the reflectivity requirements are extracted as reflectivity feature points.

[0082] The laser beam emitted by a lidar consists of multiple beams that span a certain angular range within the horizontal field of view. To ensure that the extracted reflectivity feature points are uniformly distributed within the field of view, the point cloud data can be divided into multiple segments.

[0083] The points can be divided based on the line bundle and its horizontal field of view. For example, if the line bundle contains 256 lines and the horizontal field of view is 120 degrees, it can be divided into 256*6 segments. Then, reflectivity feature points are extracted in each segment. In this way, the extracted reflectivity feature points will be evenly distributed in the point cloud.

[0084] When extracting reflectivity feature points in each segment, the reflection intensity of each point within that segment can be obtained. The reflection intensity can be determined based on the power of the echo signal after the lidar emits a laser beam and receives the reflected echo signal.

[0085] Reflection intensity can reflect the reflectivity and other properties of an object. Therefore, points that meet the reflectivity requirements can be selected based on the reflection intensity of each point and used as reflectivity feature points.

[0086] As one possible approach, points with the highest reflection intensity and greater than a first preset intensity can be extracted as reflectivity feature points.

[0087] Specifically, the reflection intensity of each point within the segment can be sorted in descending order, and points with a reflection intensity greater than a first preset intensity can be identified. The first preset intensity could be, for example, 128, or any other value. Based on the identified points, the points ranked in the top preset positions are selected, such as, for example, the top 10, or any other value. The points selected in this way can be identified as reflectivity characteristic points.

[0088] In addition, as another possible implementation, points with the highest reflectance change rate, a reflectance change rate greater than a preset rate, and a reflection intensity greater than a second preset intensity can be extracted as reflectance feature points. The reflectance change rate is calculated based on the reflection intensity of each point and its neighboring points.

[0089] Specifically, the points in the obtained point cloud data are generally numbered according to a prescribed numbering method, for example, numbered 1-n, where n is the number of points in the point cloud. For point i in segment a, the reflection intensity intensity_i of point i can be obtained, as well as the reflection intensity of points in the neighborhood of point i. The points in the neighborhood of point i can refer to points within a certain range centered on point i, for example, they can include points i-5 to i-1, and points i+1 to i+5.

[0090] Based on the reflection intensity of point i and its neighboring points, the rate of change of reflectance at point i is calculated as follows:

[0091]

[0092] The reflectance change rate of each point within the segment is calculated using the above method. These reflectance change rates can then be arranged in descending order. Points with a reflectance intensity greater than a second preset intensity (e.g., 32, etc.) are identified. Based on this, points with a reflectance change rate greater than a preset change rate (e.g., 10000, etc.) are selected. Finally, the points with the highest reflectance change rates (ranked in the top preset positions) are extracted as reflectance feature points. These top preset positions (e.g., the top 10, etc.) are not limited.

[0093] Since the number of points in the point cloud obtained by lidar is huge, in order to avoid excessive workload, when extracting reflectivity feature points using any of the above methods, if a reflectivity feature point is determined within a segment, the points in the area surrounding the reflectivity feature point can be masked. The masked points will not participate in the subsequent extraction of reflectivity feature points.

[0094] In this way, the number of points that need to be detected can be reduced. Furthermore, if reflectance feature points have been identified within a certain area, map matching can be performed based on the identified reflectance feature points without having to combine them with other reflectance feature points in the surrounding area. Therefore, this masking operation does not reduce the accuracy of the matching.

[0095] Based on the extraction of line feature points, area feature points, and reflectance feature points from point cloud data, map matching can be performed by combining these features. In this embodiment, the step of constructing the residual between feature points and the pre-established map can be implemented in the following way:

[0096] The residuals of the line feature points, area feature points, and reflectance feature points to the pre-built line feature map, area feature map, and reflectance feature map are constructed.

[0097] In this embodiment, the pre-built map can be divided into a line feature map, an area feature map, and a reflectance feature map. Specifically, the line feature map is constructed based on line feature points, the area feature map is constructed based on area feature points, and the reflectance feature map is constructed based on reflectance feature points.

[0098] During map matching, the system constructs line feature residuals from line feature points to the line feature map, area feature residuals from area feature points to the area feature map, and reflectivity feature residuals from reflectivity feature points to the reflectivity feature map. Finally, the three residuals are combined for iterative optimization to calculate the current laser odometry.

[0099] The residual from the reflectance feature points to the reflectance feature map can be constructed in the following way:

[0100] Transform the reflectivity feature points into the map coordinate system of the pre-established reflectivity feature map; for each transformed reflectivity feature point, search for neighboring reflectivity feature points in the reflectivity feature map; construct a reference plane based on the neighboring reflectivity feature points; obtain the distance between the reflectivity feature points and the reference plane as the residual to be optimized.

[0101] In this embodiment, the reflectivity feature points are first transformed into a map coordinate system. Then, the nearest multiple reflectivity feature points to each feature point are searched in the reflectivity feature map and designated as neighboring reflectivity feature points. For example, three neighboring reflectivity feature points can be identified. A plane is then constructed based on these three identified neighboring reflectivity feature points, serving as a reference plane. The method for constructing the plane based on multiple points can employ existing techniques, which will not be elaborated upon in this embodiment.

[0102] Based on the location of the reflectivity feature point and the plane equation of the reference plane, the distance from the reflectivity feature point to the reference plane is determined, and this distance is the residual to be optimized.

[0103] Furthermore, the methods for obtaining the residuals from line feature points to line feature maps and the residuals from area feature points to area feature maps are similar to those for obtaining the residuals from reflectance feature points to reflectance feature maps, and will not be elaborated upon in this embodiment.

[0104] After obtaining the residuals from reflectivity feature points to reflectivity feature maps, from line feature points to line feature maps, and from area feature points to area feature maps, the scan2map method in SLAM can be used for multiple iterative optimizations to determine the pose of the carrier, i.e., the current laser odometry.

[0105] In addition, dead reckoning information is calculated by combining velocity information detected by millimeter-wave radar and pre-integration information obtained by IMU.

[0106] The majority of the signals returned by millimeter-wave radar during the vehicle's movement are environmental information, such as signals from tunnel walls, the ground, and surrounding static infrastructure. In addition, it also includes a small amount of signals from dynamic targets, such as people waiting at stations and vehicles on public roads next to open-air depots. Therefore, the velocity information detected by millimeter-wave radar is the relative velocity information of the vehicle.

[0107] In this embodiment, the relative velocity information may not be consistent for different static and dynamic targets. By constructing a histogram of the obtained relative velocity information, the relative velocity information within the statistical interval with the most relative velocity information is used as the current relative velocity information of the carrier.

[0108] IMU pre-integration information provides relative displacement and attitude changes. Euler angles from the IMU are used to convert the vehicle's relative velocity into relative velocity in the map coordinate system. The relative velocity and displacement information are accumulated to the initial position to calculate the current position. Based on this, combined with the Euler angles from the IMU's pre-integration information, dead reckoning information, including attitude and position, can be obtained.

[0109] The following is an example of pseudocode for obtaining dead reckoning information:

[0110]

[0111] Based on this, the steps described above for determining whether the laser odometry has degraded, using dead reckoning information and laser odometry data, can be implemented in the following ways:

[0112] The first displacement increment between two adjacent poses determined based on point cloud data is calculated using laser odometry; the second displacement increment between two adjacent poses determined based on velocity information and pre-integration information is calculated using dead reckoning information; the difference between the first displacement increment and the second displacement increment is detected to be greater than a preset threshold. If it is greater than the preset threshold, the laser odometry is determined to have degraded.

[0113] In this embodiment, the displacement increment determined based on point cloud data and the displacement increment determined based on velocity information and pre-integration information are calculated separately. When the difference between the two displacement increments exceeds a preset threshold, it indicates that the pose determined based on the point cloud data may degrade, that is, the laser odometry degrades. The preset threshold can be set according to requirements, for example, it can be 0.2.

[0114] In cases of laser odometry degradation, the current pose can be updated based on the calculated dead reckoning information. Following this, the scan2map LM algorithm can be used for iterative optimization based on the updated pose. The number of iterations can be set relatively low, for example, only one. This is because performing LM optimization after updating the pose based on dead reckoning information prevents inconsistencies between the current point cloud and the map after the dead reckoning update. Iterative optimization ensures consistency between the current point cloud and the map. Setting a low number of iterations prevents over-optimization due to incorrect feature association in degraded scenarios such as tunnels.

[0115] The following example shows pseudocode for determining whether a laser odometry has degraded:

[0116]

[0117] lastLaserPose=currentLaserPose;

[0118] As described above, this embodiment also introduces an orbital vector map for further optimization of the current pose. For details, please refer to... Figure 5 The localization method based on multi-feature information provided in this embodiment may further include the following steps:

[0119] S17, Construct a track vector map based on the track positions in a pre-established map.

[0120] S18. Based on the orbital vector map and the current pose of the carrier, determine whether the current pose needs to be optimized. If the current pose needs to be optimized, execute the following step S19; otherwise, exit the optimization process.

[0121] S19 optimizes the current pose based on the orbit information in the orbit vector map.

[0122] In a tunnel scenario, vehicles can only travel on the tracks; therefore, a track vector map is constructed based on the track positions in the map, such as... Figure 3 and Figure 4 The image shows a point cloud map and a track vector map in a tunnel scene, where the solid line represents the constructed track vector map.

[0123] When determining whether the current pose needs optimization based on the orbit vector map, it is mainly achieved by judging whether the position of the vehicle exceeds the orbit range in the orbit vector map, and whether the direction of the vehicle's movement deviates too much from the direction of the orbit.

[0124] Specifically, the step of determining whether the current pose needs optimization based on the orbital vector map and the current pose of the carrier can be achieved in the following way:

[0125] Obtain the spatial range of the track and the centerline of the track in the track vector map; detect whether the position information in the current pose of the vehicle exceeds the spatial range of the track, or detect whether the angle difference between the heading angle in the current pose and the deflection angle determined by the centerline of the track exceeds the preset angle difference; if the position information exceeds the spatial range, or the angle difference exceeds the preset angle difference, it is determined that the current pose needs to be optimized.

[0126] In the orbit vector map, the spatial range of the orbit refers to the area between the two edge lines of the orbit, and the center line of the orbit is the line at the midpoint between the two edge lines. The current pose of the vehicle includes position information and attitude information, where the attitude information includes the heading angle.

[0127] Based on the vehicle's position information and the spatial range defined by the track, it can be determined whether the vehicle's position exceeds the track's spatial range. Since the vehicle can only travel on the track, if the vehicle's position information exceeds the track's spatial range, it indicates that the currently determined position information may be incorrect. Furthermore, the vehicle's heading angle indicates its direction of travel, and the yaw angle determined by the track's centerline characterizes the track's direction. By comparing the vehicle's heading angle with the yaw angle determined by the centerline, if the angle difference exceeds a preset angle difference, it indicates that the vehicle's direction of travel deviates too much from the direction pointed to by the centerline. In this case, it indicates that the heading angle in the vehicle's current pose may be incorrect.

[0128] When determining the deflection angle based on the centerline, two points on the centerline that are a certain distance apart can be located, and the deflection angle is determined based on the angle between the line connecting the two points and the coordinate axis on the horizontal plane.

[0129] If the carrier's position information or the deflection angle is found to be deviated, it is determined that the current pose needs to be optimized.

[0130] In this embodiment, the step of optimizing the current pose based on the orbit information in the orbit vector map can be implemented in the following way:

[0131] Determine the intersection point of a circle drawn with the current pose position as the origin and the displacement increment as the radius with the center line of the track in the track vector map, and use the position information of the intersection point as the optimized position information; determine two points on the center line of the track that are adjacent to the current pose position, calculate the deflection angle of the center line on the horizontal plane based on the line connecting the two points, and use the deflection angle as the optimized heading angle.

[0132] In this embodiment, when optimizing the position information of the carrier, a circle can be drawn with the current position as the origin and the displacement increment as the radius. This circle intersects with the center line of the track in the track vector map, and the position information of this intersection point can be used as the optimized position information.

[0133] In addition, when optimizing the heading angle of the carrier, the two points closest to the current position can be found on the center line of the track based on the position information in the current pose. The angle between the two points and the coordinate axis on the horizontal plane is the deflection angle, which can be used as the optimized heading angle.

[0134] Finally, the carrier can be positioned based on the optimized position information and heading angle.

[0135] The following example illustrates pseudocode for pose optimization based on an orbital vector map:

[0136]

[0137]

[0138] The localization method based on multi-feature information provided in this embodiment aims to solve the mapping and localization problems in point cloud feature degradation scenarios. First, reflectivity feature points are extracted from the point cloud. Based on the relative uniqueness of reflectivity feature points, the accuracy of matching feature points with the map is improved, thereby improving the accuracy of localization.

[0139] Furthermore, by combining millimeter-wave and IMU information fusion to optimize the degraded laser odometry, the problem of pose deviations determined by the laser odometry due to point cloud feature degradation is avoided.

[0140] Furthermore, considering that the carrier generally only travels on a track, a track vector map is introduced. Based on the track vector map, the carrier's pose is further optimized, thereby further improving the accuracy and robustness of the carrier's positioning.

[0141] To execute the corresponding steps in the above-described embodiments and possible methods of the positioning method based on multi-feature information, an implementation of a positioning device based on multi-feature information is provided below. Optionally, this positioning device based on multi-feature information can employ the methods described above. Figure 1 The device structure of the electronic device shown.

[0142] Further, please refer to Figure 6 , Figure 6 This is a functional block diagram of a positioning device based on multi-feature information provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the positioning device based on multi-feature information provided in this embodiment are the same as those in the corresponding method embodiments described above. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the aforementioned method embodiments. The positioning device based on multi-feature information includes:

[0143] The extraction module is used to acquire point cloud data obtained by the lidar during the movement of the carrier and extract feature points from the point cloud data.

[0144] The construction module is used to construct the residuals from feature points to a pre-built map, and to perform iterative optimization based on the residuals to calculate the laser odometry.

[0145] The calculation module is used to calculate dead reckoning information based on the velocity information obtained by millimeter-wave radar and the pre-integration information obtained by IMU;

[0146] The judgment module is used to determine whether the laser odometry has degraded based on dead reckoning information and laser odometry data.

[0147] The update module is used to update the pose of the carrier based on dead reckoning information when the judgment module determines that degradation has occurred, so as to achieve the positioning of the carrier.

[0148] As one possible implementation, the feature points include reflectance feature points, and the extraction module described above is used to extract reflectance feature points in the following way:

[0149] Divide multiple points in the point cloud data into multiple segments;

[0150] For each section, obtain the reflection intensity at each point within the section;

[0151] Points that meet the reflectivity requirements are extracted based on the reflection intensity of each point and used as reflectivity feature points.

[0152] As one possible implementation, the extraction module described above is used to extract points that meet the reflectivity requirements in the following way:

[0153] Extract points whose reflection intensity ranks first in a preset position and whose reflection intensity is greater than a first preset intensity, or extract points whose reflectivity change rate ranks first in a preset position and whose reflectivity change rate is greater than a preset change rate and whose reflection intensity is greater than a second preset intensity. The reflectivity change rate is calculated based on the emission intensity of each point and its neighboring points.

[0154] As one possible implementation, feature points include line feature points, area feature points, and reflectance feature points. The aforementioned construction module is used to build the feature point-to-map residual in the following way:

[0155] The residuals of the line feature points, area feature points, and reflectance feature points to the pre-built line feature map, area feature map, and reflectance feature map are constructed.

[0156] As one possible implementation, the above-mentioned construction module is used to construct the residual between the reflectance feature points and the reflectance feature map in the following way:

[0157] Transform the reflectance feature points into the map coordinate system of the pre-established reflectance feature map;

[0158] For each converted reflectance feature point, search for neighboring reflectance feature points in the reflectance feature map;

[0159] A reference plane is constructed based on neighboring reflectivity feature points;

[0160] The distance between the reflectivity feature point and the reference plane is obtained as the residual to be optimized.

[0161] As one possible implementation, the above-mentioned judgment module is used to determine whether the laser odometry has degraded in the following way:

[0162] The first displacement increment between two adjacent poses determined based on point cloud data is calculated using laser odometry.

[0163] The second displacement increment between two adjacent poses is calculated based on dead reckoning information, determined by velocity information and pre-integration information.

[0164] The difference between the first displacement increment and the second displacement increment is detected to be greater than a preset threshold. If it is greater than the preset threshold, the laser odometer is determined to have degraded.

[0165] As one possible implementation, the positioning device based on multi-feature information may also include an optimization module, which can be used for:

[0166] Construct a track vector map based on the track positions in a pre-built map;

[0167] Based on the orbital vector map and the current pose of the vehicle, determine whether the current pose needs to be optimized;

[0168] If it is determined that the current pose needs to be optimized, then the optimization of the current pose is achieved based on the orbit information in the orbit vector map.

[0169] As one possible implementation, the optimization module determines whether the current pose needs optimization in the following way:

[0170] Obtain the spatial extent of the track and the centerline of the track in the track vector map;

[0171] The detection carrier's current pose position information exceeds the spatial range of the track, or the detection carrier's current pose heading angle and the deviation angle determined by the centerline of the track exceed the preset angle difference.

[0172] If the position information exceeds the spatial range, or the angle difference exceeds the preset angle difference, then the current pose needs to be optimized.

[0173] As one possible implementation, the optimization module described above is used to optimize the current pose in the following way:

[0174] Determine the intersection point of a circle drawn with the current pose position as the origin and the displacement increment as the radius with the center line of the orbit in the orbit vector map, and use the position information of the intersection point as the optimized position information;

[0175] Determine two points on the centerline of the orbit that are adjacent to the current pose. Calculate the deflection angle of the centerline on the horizontal plane based on the line connecting the two points, and use the deflection angle as the optimized heading angle.

[0176] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory shown is either stored in or embedded in the operating system (OS) of the electronic device, and can be... Figure 1 The processor executes the commands. Meanwhile, the data and program code required to execute these modules can be stored in memory.

[0177] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0178] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0179] If the functionality is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0180] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A localization method based on multi-feature information, characterized in that, The method includes: During the movement of the carrier, point cloud data obtained by lidar is acquired, and feature points in the point cloud data are extracted; Construct the residual from the feature points to the pre-established map, and perform iterative optimization based on the residual to calculate the laser odometry. Dead reckoning information is calculated based on velocity information obtained from millimeter-wave radar and pre-integration information obtained from IMU. Based on the dead reckoning information and the laser odometry, it is determined whether the laser odometry has degraded. If it has degraded, the pose of the vehicle is updated based on the dead reckoning information to achieve the positioning of the vehicle. The feature points include line feature points, area feature points, and reflectance feature points. The step of constructing the residual from the feature points to the pre-established map includes: Construct the residuals of the line feature points, the area feature points, and the reflectance feature points to the pre-established line feature map, area feature map, and reflectance feature map, respectively; The step of constructing the residual between the reflectance feature points and the reflectance feature map in the pre-established map includes: The reflectivity feature points are transformed into the map coordinate system of the reflectivity feature map in a pre-established map; for each transformed reflectivity feature point, the neighboring reflectivity feature points of the reflectivity feature point are searched in the reflectivity feature map; a reference plane is constructed based on the neighboring reflectivity feature points; the distance between the reflectivity feature point and the reference plane is obtained as the residual to be optimized. The method further includes: A track vector map is constructed based on the track positions in a pre-established map; based on the track vector map and the current pose of the vehicle, it is determined whether the current pose needs to be optimized; If it is determined that the current pose needs to be optimized, then the intersection point of a circle drawn with the position in the current pose as the origin and the displacement increment as the radius and the center line of the track in the track vector map is determined, and the position information of the intersection point is used as the optimized position information. Determine two points on the centerline of the track that are adjacent to the position in the current pose. Calculate the deflection angle of the centerline on the horizontal plane based on the line connecting the two points, and use the deflection angle as the optimized heading angle.

2. The localization method based on multi-feature information according to claim 1, characterized in that, The feature points include reflectance feature points, which are extracted from the point cloud data in the following manner: The point cloud data is divided into multiple segments; For each of the aforementioned sections, the reflection intensity at each point within that section is obtained; Points that meet the reflectivity requirements are extracted based on the reflection intensity of each point and used as reflectivity feature points.

3. The localization method based on multi-feature information according to claim 2, characterized in that, The step of extracting points that meet the reflectivity requirements based on the reflection intensity of each point includes: The points whose reflection intensity ranks first in a preset position and whose reflection intensity is greater than a first preset intensity are extracted, or the points whose reflectivity change rate ranks first in a preset position and whose reflectivity change rate is greater than a preset change rate and whose reflection intensity is greater than a second preset intensity are extracted, wherein the reflectivity change rate is calculated based on the emission intensity of each point and its neighboring points.

4. The localization method based on multi-feature information according to claim 1, characterized in that, The step of determining whether the laser odometry has degraded based on the dead reckoning information and the laser odometry includes: The first displacement increment between two adjacent poses determined based on the point cloud data is calculated using the laser odometry. The second displacement increment between two adjacent poses is calculated based on the dead reckoning information, which is determined by the velocity information and the pre-integration information. The difference between the first displacement increment and the second displacement increment is detected to be greater than a preset threshold. If it is greater than the preset threshold, the laser odometer is determined to have degraded.

5. The localization method based on multi-feature information according to claim 1, characterized in that, The step of determining whether the current pose needs optimization based on the orbital vector map and the current pose of the carrier includes: Obtain the spatial extent of the track and the centerline of the track in the track vector map; Detect whether the position information in the current pose of the carrier exceeds the spatial range of the track, or detect whether the angle difference between the heading angle in the current pose and the deflection angle determined by the centerline of the track exceeds a preset angle difference; If the position information exceeds the spatial range, or the angle difference exceeds the preset angle difference, then the current pose needs to be optimized.

6. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor to implement the method of any one of claims 1-5.

Images