Terminal positioning method based on preset map and intelligent automobile

Abstract

The application discloses a terminal positioning method based on a preset map and an intelligent automobile. The terminal positioning method comprises the following steps: acquiring a current geographical position, and loading a preset map according to the current geographical position; acquiring point cloud data obtained by scanning of a radar at the current geographical position, and performing clustering processing on the point cloud data to obtain a clustering target; acquiring a camera image obtained by shooting of a camera at the current geographical position, and performing target detection on the camera image to extract a target element; matching the clustering target and the target element with each other to obtain depth information of the target element; mapping the depth information of the target element from a camera coordinate system to a terminal coordinate system to calculate position information of the target element; mapping the position information from the terminal coordinate system to an inertial coordinate system of the terminal to generate a corresponding planar bird's-eye view, and matching the planar bird's-eye view and the preset map to calculate a terminal pose. In the foregoing manner, the application can improve the problem that the positioning of the intelligent automobile in the prior art is not accurate enough.

Description

Technical Field

[0001] This application relates to the field of terminal positioning methods, and in particular to terminal positioning methods based on pre-set maps and intelligent vehicles. Background Technology

[0002] With the development of intelligent vehicles, people hope that vehicles can achieve stable, reliable and high-precision positioning during driving.

[0003] However, most current positioning methods rely on direct GNSS (Global Navigation Satellite System) positioning. But in actual driving scenarios, vehicles may not be able to receive GNSS signals in real time while driving, which may lead to inaccurate positioning of intelligent vehicles. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a terminal positioning method and intelligent vehicle based on a pre-set map, which can improve the problem that the positioning of intelligent vehicles may not be accurate enough in the prior art.

[0005] To address the aforementioned technical problems, this application provides a terminal positioning method based on a pre-set map, comprising: acquiring the current geographical location and loading a pre-set map based on the current geographical location; acquiring point cloud data obtained by radar scanning at the current geographical location and performing clustering processing on the point cloud data to obtain cluster targets; acquiring camera images captured by a camera at the current geographical location and performing target detection on the camera images to extract target elements; matching the cluster targets and target elements with each other to obtain the depth information of the target elements; mapping the depth information of the target elements from the camera coordinate system to the terminal coordinate system to calculate the position information of the target elements; mapping the position information from the terminal coordinate system to the terminal's inertial coordinate system to generate a corresponding planar bird's-eye view; and matching the planar bird's-eye view with the pre-set map to calculate the terminal pose.

[0006] To address the aforementioned technical problems, another technical solution adopted in this application is: providing an intelligent vehicle, comprising: a positioning system, a lidar, a camera, a memory, and a processor; the positioning system is used to calculate the current geographical location of the intelligent vehicle; the lidar is used for scene scanning to acquire point cloud data; the camera is used for scene shooting to acquire camera images; the memory is used to store program instructions; the processor is coupled to the positioning system, the lidar, the camera, and the memory, and the processor receives the point cloud data, the camera images, and the current geographical location, and executes program instructions to implement a terminal positioning method based on a preset map.

[0007] The beneficial effects of this application are as follows: Unlike related technologies, during terminal movement, a pre-set map is loaded based on the current geographical location, and scene information near the terminal is acquired through the terminal's sensors (such as LiDAR and cameras). Matching this scene information with the pre-set map improves vehicle positioning accuracy. Furthermore, a planar bird's-eye view is generated based on the scene information acquired by the terminal's sensors. In this case, the pre-set map can be a commercial 2D map, and matching this planar bird's-eye view with the commercial 2D map can also achieve high-precision positioning. Additionally, since the pre-set map can be a commercial 2D map, terminal suppliers do not need to expend additional effort to build high-precision maps, reducing costs. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of a module in an embodiment of a pre-intelligent vehicle of this application;

[0009] Figure 2 This is a flowchart illustrating an embodiment of the terminal positioning method based on a pre-set map in this application. Detailed Implementation

[0010] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0011] Intelligent vehicles are common vehicles in daily life; they can also be called automotive terminals or intelligent mobile robots. (See also...) Figure 1 Generally speaking, an intelligent vehicle 10 includes at least: the vehicle body 11 and a positioning system 12.

[0012] The main body of a car (11) generally includes the power system, transmission system, braking system, electronic control system, steering system, body, and wheels. The body typically includes the body shell, doors, windows, interior and exterior trim, and seats, primarily providing space for the driver and passengers and ensuring their safety. Additionally, the driver can control the power system via the electronic control system to start the power system, which in turn drives the wheels through the transmission system; control the braking system to decelerate or stop the car; and control the steering system to steer the wheels, thus controlling the car's direction of movement.

[0013] The positioning system 12 can be installed on the vehicle body 11 to calculate the current geographical location of the intelligent vehicle 10. The positioning system 12 may include a signal receiver. Generally, the signal receiver can receive GNSS (Global Navigation Satellite System) signals. The positioning system 12 can obtain the vehicle's location information based on the received GNSS signals.

[0014] In other examples, the positioning system 12 may also include an inertial measurement unit (IMU) or a wheel speed odometer. The inertial measurement unit (IMU) or wheel speed odometer can be used to measure the relative motion of the intelligent vehicle 10, and then calculate the movement path of the intelligent vehicle 10.

[0015] The following embodiments of the intelligent vehicle described in this application illustrate an exemplary structure of an intelligent vehicle 10.

[0016] Through long-term research, the inventors of this application have discovered that during the operation of the intelligent vehicle 10, the intelligent vehicle 10 may not be able to receive GNSS signals in real time. If the intelligent vehicle 10 cannot receive GNSS signals, or the received GNSS signals are weak, the positioning of the intelligent vehicle 10 may be inaccurate. To solve the above problems, this application proposes the following embodiments.

[0017] The intelligent vehicle 10 may include: a vehicle body 11, a positioning system 12, a lidar sensor 13, a camera 14, a memory 15, and a processor 16. Descriptions of the vehicle body 11 and the positioning system 12 can be found above and will not be repeated here.

[0018] The lidar 13 can be mounted on the vehicle body 11 for scene scanning to acquire point cloud data. Specifically, the lidar 13 can emit detection signals towards targets in the scene and obtain point cloud data based on the reflected signals. The point cloud data can be used to analyze information such as the target's distance, orientation, speed, or shape. Furthermore, the lidar coordinate system of the lidar 13 generally has a known positional relationship with the vehicle body coordinate system of the vehicle body 11. This positional relationship allows the point cloud data to be substituted into the vehicle body coordinate system to calculate the relative positional relationship between each target in the scene and the vehicle body 11.

[0019] Camera 14 can be mounted on the vehicle body 11 for scene capture to obtain camera images. The working principle of camera 14 is not detailed here. Generally, camera 14 can acquire two-dimensional camera images. Furthermore, the camera coordinate system of camera 14 and the vehicle body coordinate system of the vehicle body 11 have a known positional relationship. This positional relationship allows coordinate transformation of various targets in the camera image and their input into the vehicle body coordinate system. Similarly, the point cloud data acquired by lidar 13 and the camera images can be matched using the relative positional relationship between the camera coordinate system and the radar coordinate system.

[0020] The memory 15 can be located in the vehicle body 11 and is used to store program instructions. The processor 16 can be located in the vehicle body 11 and coupled to the positioning system 12, the lidar 13 and the memory 15. The processor 16 can receive point cloud data, camera images and current geographical location, and execute program instructions to implement a terminal positioning method based on a preset map.

[0021] Specifically, the processor 16 can load a pre-set map based on the terminal's current geographical location, and then obtain scene information (such as point cloud data and camera images) near the terminal through the terminal's sensors (such as LiDAR 13 and camera 14), and match the scene information with the pre-set map to achieve the positioning of the intelligent vehicle 10.

[0022] The preset map can be either a 3D map or a 2D map. Since 3D maps may involve confidential information, most commercially available maps are generally 2D. Of course, the preset map can also be a 3D map pre-built by the car manufacturer. Correspondingly, if the preset map is a 3D map, the processor 16 needs to match it with the 3D scene information. If the preset map is a 2D map, the processor 16 needs to match it with the 2D scene information.

[0023] Alternatively, processor 16 can also be referred to as CPU (Central Processing Unit). Processor 16 can be an integrated circuit chip with signal processing capabilities. Processor 16 can also be a general-purpose processor 16, a digital signal processor 16 (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a graphics processing unit (GPU), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. General-purpose processor 16 can be a microprocessor 16 (MCU), or it can be any conventional processor 16, etc.

[0024] Based on the above description of the intelligent vehicle embodiments of this application, the following embodiments of the terminal positioning method based on a pre-set map exemplarily describe the terminal positioning method based on a pre-set map of this application.

[0025] See Figure 2 The terminal positioning method based on a pre-set map may include: Step S100: Obtaining the current geographical location and loading a pre-set map according to the current geographical location; Step S200: Obtaining point cloud data scanned by radar at the current geographical location and performing clustering processing on the point cloud data to obtain cluster targets; Step S300: Obtaining camera images captured by camera 14 at the current geographical location and performing target detection on the camera images to extract target elements; Step S400: Matching cluster targets and target elements with each other to obtain depth information of target elements; Step S500: Mapping the depth information of target elements from the camera coordinate system to the terminal coordinate system to calculate the position information of target elements; Step S600: Mapping the position information from the terminal coordinate system to the terminal's inertial coordinate system to generate a corresponding planar bird's-eye view, and matching the planar bird's-eye view with the pre-set map to calculate the terminal pose; Step S700: Verifying the terminal pose to determine whether the terminal pose is usable.

[0026] Unlike other technologies, during the movement of the terminal (e.g., the intelligent vehicle 10), a pre-built map is loaded based on the current geographical location. The terminal's sensors (e.g., radar and camera 14) acquire scene information (e.g., point cloud data and camera images) about the vicinity of the terminal. Matching this scene information with the pre-built map improves vehicle positioning accuracy. Furthermore, a planar bird's-eye view generated from the scene information acquired by the terminal's sensors can also be used. In this case, the pre-built map can be a commercial 2D map. Matching this planar bird's-eye view with the commercial 2D map can also achieve high-precision positioning. Additionally, since the pre-built map can be a commercial 2D map, the terminal supplier does not need to expend additional effort to build a high-precision map, reducing costs.

[0027] It should be noted that the terminal in this application can also be a smartphone, drone, or other smart device. In other words, the terminal positioning method based on a pre-set map in this application can be used in the smart car 10, and of course, it can also be applied to smartphones, drones, and other smart devices, achieving the same technical effect.

[0028] The following sections will further describe each step in the terminal positioning method based on a pre-set map.

[0029] Step S100: Obtain the current geographical location and load a preset map based on the current geographical location.

[0030] The pre-loaded map based on the current geographical location can be used to match elements in the nearby scene perceived by the terminal, thereby assisting in positioning.

[0031] In some examples, the terminal (e.g., the intelligent vehicle 10) is able to receive GNSS signals. In this case, the current geographical location can be obtained relatively easily using real-time kinematic (RTK) technology.

[0032] In other examples, the terminal may not be able to obtain a GNSS signal of sufficient strength, or the real-time dynamic positioning technology may fail. In such cases, the terminal can be located using a recursive trajectory (DR) method.

[0033] Specifically, step S100: obtaining the current geographical location and loading a preset map based on the current geographical location may include: obtaining the initial location of the terminal and the movement path of the terminal, and calculating the current geographical location based on the initial location and the movement path.

[0034] The initial position of the terminal can be determined by acquiring satellite signals received by the signal receiver. The movement path can be calculated by recursively calculating the trajectory using measurement data from the inertial measurement unit and / or wheel speedometer.

[0035] In other words, when an accurate current geographical location cannot be obtained directly from GNSS signals, an initial position can be calculated while GNSS signals are available. After GNSS signals fail, the terminal's movement path after the loss of GNSS signals can be obtained. The current geographical location can be determined using the initial position and the movement path.

[0036] Step S200: Obtain point cloud data scanned by radar at the current geographical location, and perform clustering processing on the point cloud data to obtain clustered targets.

[0037] Specifically, the radar can be a lidar 13. Considering that the more sensing elements acquired for matching with a pre-set map, the more accurate and reliable the positioning may be, the more point cloud data the radar acquires, the higher the positioning accuracy and the better the reliability of the positioning.

[0038] In some examples, step S200: acquiring point cloud data scanned by radar at the current geographical location and performing clustering processing on the point cloud data to obtain clustering targets may include: acquiring point cloud data scanned by radar within a preset range or within a preset time before the terminal moves to the current geographical location to generate a point cloud map. In other words, point cloud data can be obtained by radar scanning the scene at the current geographical location and within a certain range before the current geographical location.

[0039] Since the radar's position may move during the data acquisition process, the acquired point cloud data needs to be processed to remove motion distortion. Specifically, step S200 may include: acquiring point cloud frame data from multiple moments obtained by radar scanning within a preset range or time period before the terminal moves to the current geographical location, and processing the point cloud frame data to remove motion distortion in order to construct a point cloud map.

[0040] The point cloud data includes point cloud frame data from multiple time points. That is, each point cloud frame corresponds to a sampling time and the current geographical location. Based on the temporal relationship of the point cloud frame data and the positional relationship of the radar at each time point, motion distortion correction processing can be performed on the point cloud data.

[0041] Specifically, for any point in the point cloud frame data acquired by the radar, such as point p1, the corresponding current geographical location result can be found based on the sampling time of point p1. Then, combined with the radar's extrinsic parameters, the first relative position of point p1 at the sampling time can be calculated.

[0042] Specifically, the first relative position of the point can be calculated using the following formula:

[0043]

[0044] in, This represents the original relative position of the point before motion distortion. This is the first relative position of the point. and The current geographical location corresponding to the sampling time. and This is a radar extrinsic parameter.

[0045] After obtaining the first relative position of point p at the sampling time, the point can be transformed into the current point cloud frame data obtained by radar scanning when the terminal moves to the current geographical location (that is, the point cloud frame data at the current moment) to obtain the second relative position.

[0046] Similarly, each point in the point cloud frame data from multiple time points can be transformed to the current time point to obtain a point cloud map, and the impact of motion distortion can be reduced.

[0047] In some examples, step S200: acquiring point cloud data obtained by radar scanning at the current geographical location and performing clustering processing on the point cloud data to obtain clustered targets may include: generating a point cloud map based on the point cloud data, performing point cloud segmentation on the point cloud map to obtain ground point cloud, clustering the ground point cloud to obtain ground targets; and clustering point cloud data that does not belong to the ground point cloud to obtain non-ground targets.

[0048] Ground targets can refer to objects such as lane lines, the ground, and curbs in the scene. Non-ground targets can refer to objects such as road signs and lampposts. In some examples, the clustered targets obtained after clustering the point cloud map can include ground targets and / or non-ground targets. Generally, a scene will have objects such as lane lines, the ground, curbs, road signs, and lampposts. Correspondingly, clustering the point cloud map can yield ground targets and non-ground targets. In other examples, such as when there is standing water on the ground or when there are no road signs or lampposts as reference points along the road, clustering the point cloud map may result in either non-ground targets or ground targets.

[0049] Specifically, the point cloud map can first be segmented into ground points to obtain the ground point cloud. Optionally, the segmentation method can be SVD (Singular Value Decomposition), SLOPE, RANSAC (Random Sample Consensus), etc., without specific limitations here.

[0050] Taking the RANSAC segmentation method as an example, within a set number of iterations, three point cloud data are randomly selected each time to determine a plane equation Ax+By+Cz+D=0. Then, all point cloud data are substituted into this plane equation in turn. Based on the set distance threshold, it is determined whether each point in all point cloud data is an interior point. The number of interior points is counted. The plane equation with the most interior points in the set number of iterations is the ground equation. The interior points under this equation are the ground point cloud.

[0051] For non-terrestrial point clouds in point cloud maps, target clustering can be performed. Optionally, clustering methods can be KD-Tree (K-dimensional tree), kmeans (k-means clustering algorithm), DBSCAN (Density-Based Spatial Clustering of Applications with Noise), etc., without specific limitations here.

[0052] Taking the DBSCAN clustering method as an example, we can first initialize all point cloud data in the point cloud map and mark them all as "unvisited". Then, we randomly select a point from the point cloud data. Based on pre-set distance and quantity thresholds, if the number of point cloud data within the neighboring area of ​​a point cloud data exceeds the quantity threshold, the point cloud data is determined to be a core object. If two point cloud data marked as core objects are both within each other's neighborhood, these two point cloud data are grouped into a cluster. We repeat the above steps, traversing other "unvisited" objects, until all objects have been visited. Each cluster represents a non-ground target.

[0053] In some examples, outlier filtering can be performed on non-ground targets obtained from clustering. Specifically, the ground clearance can be set based on the height of the road sign, and the normal direction can be set based on the direction of the light pole. Based on the set thresholds for ground clearance and normal direction, non-ground targets whose ground clearance and normal direction do not meet the thresholds can be removed.

[0054] Step S300: Acquire camera images taken by camera 14 at the current geographical location, and perform target detection on the camera images to extract target elements.

[0055] The target element can include ground elements and / or non-ground elements. If the scene contains objects such as lane lines, ground, curbs, road signs, and lampposts, the target element can include both ground and non-ground elements. If there is standing water on the road, the target element can include non-ground elements. Similarly, in a relatively open scene, the target element can also include ground elements.

[0056] Correspondingly, in step S300, target detection can be performed on the camera image to extract target elements corresponding to lane lines, curbs, ground, road signs, or lampposts, respectively. Optionally, the target detection method can be edge detection, YOLO (You Only Look Once) algorithm, or SCNN lane line detection algorithm, without specific limitations here.

[0057] Step S400: Match the clustering target and the target element with each other to obtain the depth information of the target element.

[0058] In this context, depth information can refer to the distance from the object corresponding to the target element to the camera or to the ground. Furthermore, the distance from the object to the ground can be calculated from the distance from the object to the camera. Similarly, the distance from the object to the camera can also be calculated from the distance from the object to the ground. In this embodiment, depth information refers to the distance between the target element and the camera.

[0059] Since the positional information of objects in the scene is three-dimensional, while the target element in the camera 14 can only reflect two-dimensional information, the depth information of the target element can be calculated by performing depth recovery on the target element. This can be used to calculate the three-dimensional information of the object corresponding to the target element.

[0060] Furthermore, the cluster targets are obtained in the radar coordinate system, while the target elements are obtained in the camera coordinate system. Therefore, in the process of matching target elements and cluster targets, at least one of the target elements and cluster targets needs to be transformed.

[0061] In some examples, in step S400, matching clustering targets and target elements to each other to obtain depth information of the target elements may include: using ground targets to calculate ground equations, obtaining pixel coordinates of ground elements, and substituting the pixel coordinates into the ground equations to calculate depth information of ground elements.

[0062] Furthermore, at least three point cloud data points from the ground target are used to calculate the ground equation, obtain the pixel coordinates of the ground elements, transform the pixel coordinates to the radar coordinate system of the radar and substitute them into the ground equation to calculate the depth information of the ground elements.

[0063] Specifically, the radar and camera 14 have a known positional relationship. By calibrating the radar and camera 14, extrinsic parameters between them can be obtained. These extrinsic parameters can then be used to substitute pixel coordinates into the radar coordinate system.

[0064] For example, for any target element O(u,v), the depth information of target element O can be calculated using the following formula:

[0065]

[0066] Where (u,v) are the pixel coordinates of the target element O, A, B, C, and D are the coefficients in the ground equation Ax+By+Cz+D=0, K is the intrinsic parameter matrix of camera 14, and λ is the depth information of the target element.

[0067] In addition, the three-dimensional information of the target element O (ground element) in the camera coordinate system can be calculated using the following formula:

[0068]

[0069] Where x, y, z are the three-dimensional information of the target element O in the camera coordinate system, K is the intrinsic parameter matrix of camera 14, λ is the depth information of the target element, and (u,v) are the pixel coordinates of the target element O.

[0070] In some examples, in step S400, matching clustering targets and target elements to obtain depth information of the target elements may include: obtaining the homogeneous coordinates of non-ground elements in the camera coordinate system, substituting the homogeneous coordinates into the point cloud map and matching them with the non-ground targets to obtain depth information of the non-ground elements. This depth information can also be used to calculate the 3D information of the target elements (non-ground elements) in the camera coordinate system, which will not be elaborated here.

[0071] Furthermore, the homogeneous coordinates of non-ground elements in the camera coordinate system are obtained. These homogeneous coordinates are then substituted into the point cloud map to generate target points of different scales. The scale of the target points that hit the non-ground targets is then used as the depth information of the non-ground elements. Here, scale can refer to either length or depth information.

[0072] Since homogeneous coordinates can represent the direction from a non-target element to camera 14, in the radar coordinate system, a ray emitted in the direction indicated by the homogeneous coordinates can hit the non-ground target corresponding to that non-ground element. The distance at which the ray hits the non-ground target is the depth information of that non-ground element.

[0073] In other words, by using the homogeneous coordinates of non-ground elements in the camera coordinate system and then traversing the scale within a certain range, multiple target points of different scales can be generated. For a target point that can hit a non-ground target, its scale is the depth information of the non-ground element.

[0074] Step S500: Map the depth information of the target element from the camera coordinate system to the terminal coordinate system to calculate the position information of the target element.

[0075] Furthermore, the depth information of the target element can be used to calculate its 3D information in the camera coordinate system. Projecting this 3D information onto the terminal coordinate system allows us to calculate the target element's 3D information in the terminal coordinate system, which is also its position information.

[0076] Specifically, the 3D information in the camera coordinate system can be transformed using the intrinsic and extrinsic parameters of the camera and terminal coordinate systems, and then projected into the terminal coordinate system. The coordinate transformation method can be any of the transformation methods commonly used in intelligent vehicles, and no restrictions are imposed here.

[0077] Step S600: Map the position information from the terminal coordinate system to the terminal's inertial coordinate system to generate a corresponding planar bird's-eye view. Match the planar bird's-eye view with a preset map to calculate the terminal pose. The inertial coordinate system can refer to a horizontal coordinate system.

[0078] Generally, pose information can include displacement components and rotation angles. In a planar coordinate system, pose information typically includes X-axis components, Y-axis components, and rotation angles.

[0079] In some examples, mapping position information from the terminal coordinate system to the terminal's inertial coordinate system to generate a corresponding planar bird's-eye view may include: performing coordinate transformation on the position information of the target element in the terminal coordinate system according to the relative positional relationship between the terminal coordinate system and the inertial coordinate system to obtain the three-dimensional coordinate information of the target element in the inertial coordinate system, and generating a planar bird's-eye view based on the horizontal two-dimensional coordinates in the three-dimensional coordinate information.

[0080] Furthermore, mapping the position information from the terminal coordinate system to the terminal's inertial coordinate system to generate a corresponding planar bird's-eye view may include: obtaining the terminal's pitch and roll angles to calculate the relative positional relationship between the terminal coordinate system and the inertial coordinate system, and using the pitch and roll angles to perform coordinate transformation on the target element's position information in the terminal coordinate system. In this case, the influence of the terminal's own pitch angle and other information can be reduced to some extent during the terminal's movement, thereby improving positioning accuracy.

[0081] Specifically, the position information can be mapped from the terminal coordinate system to the terminal's inertial coordinate system using the following formula:

[0082]

[0083] Where (x′,y′,z′) represents the three-dimensional coordinate information of the target element in the inertial coordinate system, and (x,y,z) represents the three-dimensional coordinate information of the target element in the terminal coordinate system. γ is the pitch angle, and γ is the roll angle.

[0084] In addition, the two-dimensional coordinate information (x′, y′) in the three-dimensional coordinate information (x′, y′, z′) of the target element in the inertial coordinate system can be used to generate a bird's-eye view.

[0085] In some examples, in step S600, the pose estimation method used to calculate the terminal pose can be matching optimization or particle filtering, etc., and no specific restrictions are imposed here.

[0086] Taking particle filtering as a pose estimation method as an example, particle filtering generally includes an initialization step: obtaining the approximate current position of the terminal and randomly scattering a preset number of particles near the current approximate position; a prediction step: for each particle, adding control input (such as movement speed or angular velocity) to predict the next position of all particles; an update step: updating the particles based on the terminal's observation information (such as the distances to various objects and the 3D information of objects in the scene) to obtain particle weights; and a weighting step: using the calculated weights to weight all particles to obtain the final pose (i.e., the terminal pose).

[0087] Step S700: Verify the terminal pose to determine whether the terminal pose is usable.

[0088] Specifically, in some examples, the comparison pose can be obtained through GNSS signals; the terminal pose and the comparison pose are compared to calculate the position difference between the terminal pose and the comparison pose; if the position difference exceeds the error threshold, the terminal pose is determined to be unusable; otherwise, it is determined to be usable.

[0089] In other examples, the terminal pose can be used to project the bird's-eye view into an inertial coordinate system and calculate the reprojection residual; if the reprojection residual exceeds a preset residual value, the terminal pose is determined to be unusable; otherwise, it is determined to be usable.

[0090] In other examples, if particle filtering is used as the pose estimation method, the weighted covariance of the particles can be calculated; if the calculated weighted covariance exceeds a set value, the terminal pose is determined to be unusable; otherwise, it is determined to be usable.

[0091] In some examples, the calculation results of steps S600 and S700 can be output together as the output result. Specifically, the output result can be the terminal pose and whether the terminal pose is available. Of course, in other examples, if the accuracy of calculating the terminal pose is sufficient, the terminal localization method based on the preset map may not include step S700.

[0092] In summary, by loading a pre-built map based on the current geographical location and acquiring scene information near the terminal using the terminal's sensors (such as radar and camera 14), and then matching the scene information with the pre-built map, the accuracy of vehicle positioning can be improved. Furthermore, a planar bird's-eye view generated based on the scene information acquired by the terminal's sensors can also be used. In this case, the pre-built map can be a commercial 2D map, and matching the planar bird's-eye view with the commercial 2D map can also achieve high-precision positioning. Additionally, since the pre-built map can be a commercial 2D map, the terminal supplier does not need to expend additional effort to build a high-precision map, thus reducing costs.

[0093] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A terminal positioning method based on a pre-set map, characterized in that, include: Obtain the current geographical location and load a preset map based on the current geographical location. The preset map is a commercial two-dimensional map. The radar acquires point cloud data obtained by scanning at the current geographical location, and performs clustering processing on the point cloud data to obtain clustered targets, including ground targets and non-ground targets; Acquire a camera image captured by the camera at the current geographical location, and perform target detection on the camera image to extract target elements, the target elements including ground elements and non-ground elements; The ground equation is calculated using the ground target to obtain the pixel coordinates of the ground element. The pixel coordinates are then substituted into the ground equation to calculate the depth information of the ground element. Obtain the homogeneous coordinates of the non-ground element in the camera coordinate system, substitute the homogeneous coordinates into the point cloud map and match them with the non-ground target to obtain the depth information of the non-ground element. The depth information of the target element is mapped from the camera coordinate system to the terminal coordinate system to calculate the position information of the target element; Based on the relative positional relationship between the terminal coordinate system and the inertial coordinate system, the position information of the target element in the terminal coordinate system is transformed to obtain the three-dimensional coordinate information of the target element in the inertial coordinate system. A planar bird's-eye view is generated based on the horizontal two-dimensional coordinates in the three-dimensional coordinate information. The planar bird's-eye view is matched with the preset map to calculate the terminal pose.

2. The terminal positioning method based on a pre-set map according to claim 1, characterized in that, The step of performing coordinate transformation on the position information of the target element in the terminal coordinate system based on the relative positional relationship between the terminal coordinate system and the inertial coordinate system includes: The pitch and roll angles of the terminal are obtained to calculate the relative positional relationship between the terminal coordinate system and the inertial coordinate system, and the positional information of the target element in the terminal coordinate system is transformed using the pitch and roll angles.

3. The terminal positioning method based on a pre-set map according to claim 1, characterized in that, Obtaining the depth information of the target element includes: Using at least three point cloud data from the ground target, a ground equation is calculated to obtain the pixel coordinates of the ground element. The pixel coordinates are then transformed into the radar coordinate system of the radar and substituted into the ground equation to calculate the depth information of the ground element. And / or, Obtain the homogeneous coordinates of the non-ground element in the camera coordinate system, substitute the homogeneous coordinates into the point cloud map, generate target points of different scales, and obtain the scale of the target point that hits the non-ground target as the depth information of the non-ground element.

4. The terminal positioning method based on a pre-set map according to claim 3, characterized in that, The process of acquiring point cloud data obtained by radar scanning at the current geographical location and performing clustering processing on the point cloud data to obtain clustered targets includes: A point cloud map is generated based on the point cloud data. The point cloud map is segmented to obtain a ground point cloud. The ground point cloud is clustered to obtain the ground target. Point cloud data that do not belong to the ground point cloud are clustered to obtain the non-ground target.

5. The terminal positioning method based on a pre-set map according to claim 4, characterized in that, The process of acquiring point cloud data obtained by radar scanning at the current geographical location and performing clustering processing on the point cloud data to obtain clustered targets includes: The point cloud data scanned by the radar is obtained within a preset range or within a preset time before the terminal moves to the current geographical location to generate the point cloud map.

6. The terminal positioning method based on a pre-set map according to claim 5, characterized in that, The process of acquiring point cloud data obtained by radar scanning at the current geographical location and performing clustering processing on the point cloud data to obtain clustered targets includes: The point cloud data is obtained from radar scanning at multiple times within a preset range or within a preset time before the terminal moves to the current geographical location. The point cloud data includes the point cloud frame data at the multiple times. Motion distortion removal processing is performed on the point cloud frame data to construct the point cloud map.

7. The terminal positioning method based on a pre-set map according to claim 1, characterized in that, The step of obtaining the current geographical location and loading a preset map based on the current geographical location includes: The initial location and the movement path of the terminal are obtained, and the current geographical location is calculated based on the initial location and the movement path.

8. The terminal positioning method based on a pre-set map according to claim 7, characterized in that, The step of obtaining the current geographical location and loading a preset map based on the current geographical location includes: The system acquires satellite signals received by the signal receiver to determine the initial position of the terminal; it acquires measurement data from the inertial measurement unit and / or wheel speedometer to perform trajectory recursion to calculate the movement path.

9. The terminal positioning method based on a pre-set map according to claim 1, characterized in that, It also includes verifying the terminal pose to determine whether the terminal pose is usable.

10. An intelligent vehicle, characterized in that, include: A positioning system is used to calculate the current geographical location of the intelligent vehicle. LiDAR is used for scene scanning to obtain point cloud data. A camera used for scene photography to obtain camera images. Memory, used to store program instructions; A processor, coupled to the positioning system, the lidar, the camera, and the memory, receives the point cloud data, the camera image, and the current geographical location, and executes the program instructions to implement the terminal positioning method based on a preset map as described in any one of claims 1-9.

Images