High-precision vehicle positioning, vectorized map construction and positioning model training method

Abstract

The present disclosure provides a method for high-precision vehicle positioning, vectorized map construction and positioning model training, which relates to the field of artificial intelligence technology, especially to the technical fields of autonomous driving, deep learning, computer vision, etc. The implementation scheme is: obtaining the initial posture of the vehicle, the point cloud collected by the radar deployed on the vehicle, and multiple map elements used to locate the vehicle; encoding the point cloud to obtain point cloud features; encoding multiple map elements to obtain map features; based on the point cloud features and map features, determining the target posture offset used to correct the initial posture; and superimposing the initial posture with the target posture offset to obtain the corrected posture of the vehicle. The present disclosure can improve the positioning accuracy of autonomous driving vehicles.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to technical fields such as autonomous driving, deep learning, and computer vision, and specifically to a high-precision vehicle positioning method and device, a vectorized map construction method and device, a positioning model training method and device, an electronic device, a computer-readable storage medium, a computer program product, and an autonomous driving vehicle. Background Art

[0002] Autonomous driving technology involves multiple aspects such as environmental perception, behavioral decision-making, trajectory planning, and motion control. Relying on the coordinated cooperation of sensors, visual computing systems, and positioning systems, vehicles with autonomous driving functions can operate automatically without the need for driver operation or with only a small amount of driver operation. Accurate positioning of autonomous driving vehicles is an important prerequisite for ensuring the safe and smooth operation of autonomous driving vehicles.

[0003] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0004] The present disclosure provides a high-precision vehicle positioning method and device, a vectorized map construction method and device, a positioning model training method and device, an electronic device, a computer-readable storage medium, a computer program product, and an autonomous driving vehicle.

[0005] According to one aspect of the present disclosure, a vehicle positioning method is provided, comprising: acquiring an initial posture of the vehicle, a point cloud collected by a radar deployed on the vehicle, and a plurality of map elements for positioning the vehicle; encoding the point cloud to obtain point cloud features; encoding the plurality of map elements to obtain map features; determining a target posture offset for correcting the initial posture based on the point cloud features and the map features; and superimposing the initial posture with the target posture offset to obtain a corrected posture of the vehicle.

[0006] According to one aspect of the present disclosure, a vectorized map construction method is provided, comprising: acquiring a point cloud in a point cloud map; dividing a projection plane of the point cloud map into a plurality of two-dimensional grids of a first unit size; for any two-dimensional grid among the plurality of two-dimensional grids: extracting a plane in the two-dimensional grid based on a point cloud in a three-dimensional space corresponding to the two-dimensional grid; and storing the plane as a surface element in the vectorized map.

[0007] According to one aspect of the present disclosure, a training method for a positioning model is provided, wherein the positioning model includes a point cloud encoder, a map encoder and a pose solver, and the method includes: obtaining an initial pose of a sample vehicle, a pose truth value corresponding to the initial pose, a point cloud collected by a radar deployed on the sample vehicle, and a plurality of map elements used to locate the sample vehicle; inputting the point cloud into the point cloud encoder to obtain a point cloud feature map in a target three-dimensional space; inputting category information of the plurality of map elements into the map encoder, so that the map encoder: determines a semantic code corresponding to the category information of each map element in the plurality of map elements based on a correspondence between a plurality of category information and a plurality of semantic codes, wherein the plurality of semantic codes are parameters of the positioning model; and determines a coding vector of the map element based on the semantic code, wherein the map feature includes the coding vectors of the plurality of map elements respectively; and projects a target map element belonging to a target category among the plurality of map elements into the target three-dimensional space. , to obtain a truth map of semantic segmentation in the target three-dimensional space, wherein the value of a first pixel in the truth map indicates whether the first pixel is occupied by the target map element; based on the point cloud feature map, determine a prediction map of semantic segmentation, wherein the value of a second pixel in the prediction map indicates the similarity between a corresponding feature vector and a semantic encoding of the target category, and the corresponding feature vector is a feature vector of a pixel in the point cloud feature map corresponding to the position of the second pixel; input the point cloud feature map, the map feature and the initial pose into the pose solver to obtain a predicted pose offset output by the pose solver; determine a first loss based on the predicted pose offset and a true value of the pose offset, wherein the true value of the pose offset is the difference between the true value of the pose and the initial pose; determine a second loss based on the truth map and the prediction map; determine an overall loss of the positioning model based at least on the first loss and the second loss; and adjust the parameters of the positioning model based on the overall loss.

[0008] According to one aspect of the present disclosure, a vehicle positioning device is provided, comprising: an acquisition module configured to acquire an initial posture of the vehicle, a point cloud collected by a radar deployed on the vehicle, and a plurality of map elements for positioning the vehicle; a point cloud encoding module configured to encode the point cloud to obtain point cloud features; a map encoding module configured to encode the plurality of map elements to obtain map features; a determination module configured to determine a target posture offset for correcting the initial posture based on the point cloud features and the map features; and a superposition module configured to superimpose the initial posture with the target posture offset to obtain a corrected posture of the vehicle.

[0009] According to one aspect of the present disclosure, a vectorized map construction device is provided, including: an acquisition module, configured to acquire a point cloud in a point cloud map; a division module, configured to divide the projection plane of the point cloud map into a plurality of two-dimensional grids of a first unit size; an extraction module, configured to extract, for any two-dimensional grid of the plurality of two-dimensional grids, a plane in the two-dimensional grid based on a point cloud in a three-dimensional space corresponding to the two-dimensional grid; and a storage module, configured to store the plane as a surface element in the vectorized map.

[0010] According to one aspect of the present disclosure, a training device for a positioning model is provided, the positioning model comprising a point cloud encoder, a map encoder and a pose solver, the device comprising: an acquisition module, configured to acquire an initial pose of a sample vehicle, a pose truth value corresponding to the initial pose, a point cloud collected by a radar deployed on the sample vehicle, and a plurality of map elements for positioning the sample vehicle; a first input module, configured to input the point cloud into the point cloud encoder to obtain a point cloud feature map in a target three-dimensional space; a second input module, configured to input category information of the plurality of map elements into the map encoder, so that the map encoder: determines a semantic code corresponding to the category information of each map element in the plurality of map elements based on a correspondence between a plurality of category information and a plurality of semantic codes, wherein the plurality of semantic codes are parameters of the positioning model; and determines a coding vector of the map element based on the semantic code, wherein the map feature comprises the coding vectors of the plurality of map elements respectively; a semantic projection module, configured to project a target map element belonging to a target category among the plurality of map elements into the target three-dimensional space to obtain a target three-dimensional space; A truth map of semantic segmentation in space, wherein the value of a first pixel in the truth map indicates whether the first pixel is occupied by the target map element; a semantic prediction module, configured to determine a prediction map of semantic segmentation based on the point cloud feature map, wherein the value of a second pixel in the prediction map indicates the similarity between a corresponding feature vector and a semantic encoding of the target category, and the corresponding feature vector is a feature vector of a pixel in the point cloud feature map corresponding to the position of the second pixel; a third input module, configured to input the point cloud feature map, the map features and the initial pose into The pose solver is used to obtain the predicted pose offset output by the pose solver; a first determination module is configured to determine a first loss based on the predicted pose offset and a true value of the pose offset, wherein the true value of the pose offset is the difference between the true value of the pose and the initial pose; a second determination module is configured to determine a second loss based on the true value map and the predicted map; a determination module is configured to determine the overall loss of the positioning model based on at least the first loss and the second loss; and an adjustment module is configured to adjust the parameters of the positioning model based on the overall loss.

[0011] According to one aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the methods described above.

[0012] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute any of the above methods.

[0013] According to one aspect of the present disclosure, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions implement any of the above methods when executed by a processor.

[0014] According to one aspect of the present disclosure, an autonomous driving vehicle is provided, comprising the above-mentioned electronic device.

[0015] According to one or more embodiments of the present disclosure, the positioning accuracy of an autonomous driving vehicle can be improved.

[0016] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0018] Figure 1 A schematic diagram showing an exemplary system in which various methods described herein may be implemented according to some embodiments of the present disclosure;

[0019] Figure 2 A flow chart of a vehicle positioning method according to some embodiments of the present disclosure is shown;

[0020] Figure 3 A schematic diagram of a vehicle positioning process based on a trained positioning model according to some embodiments of the present disclosure is shown;

[0021] Figure 4 A flowchart of a method for constructing a vectorized map according to some embodiments of the present disclosure is shown;

[0022] Figure 5 A flowchart of a method for training a positioning model according to some embodiments of the present disclosure is shown;

[0023] Figure 6 A structural block diagram of a vehicle positioning device according to some embodiments of the present disclosure is shown;

[0024] Figure 7A structural block diagram of a vectorized map construction device according to some embodiments of the present disclosure is shown;

[0025] Figure 8 A structural block diagram of a training device for a positioning model according to some embodiments of the present disclosure is shown; and

[0026] Fig. 9 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0027] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0028] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0029] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0030] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0031] In related technologies, a combined positioning system is usually used to locate autonomous vehicles. The combined positioning system usually includes a global navigation satellite system (GNSS) and an inertial navigation system (INS). Global positioning is achieved by receiving satellite signals through GNSS, and the calibration of positioning information is achieved through IMU. However, in complex road environments, such as tunnels, overpasses, and urban roads with high-rise buildings, satellite signals are often lost or errors are large, resulting in low positioning accuracy of the combined positioning system, which cannot provide continuous and reliable positioning services.

[0032] In response to the above problems, the present disclosure provides a vehicle positioning method that can improve the positioning accuracy of an autonomous driving vehicle.

[0033] The present disclosure also provides a method for constructing a vectorized map and a method for training a positioning model. The constructed vectorized map and the trained positioning model can be used for vehicle positioning to improve the positioning accuracy of the autonomous driving vehicle.

[0034] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0035] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes a motor vehicle 110 , a server 120 , and one or more communication networks 130 coupling the motor vehicle 110 to the server 120 .

[0036] In an embodiment of the present disclosure, the motor vehicle 110 may include an electronic device according to an embodiment of the present disclosure and / or be configured to perform a method according to an embodiment of the present disclosure.

[0037] The server 120 may run one or more services or software applications that enable the execution of the vectorized map construction method or the positioning model training method of the disclosed embodiment. In some embodiments, the server 120 may also provide other services or software applications, which may include non-virtual environments and virtual environments. Figure 1In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. A user of the motor vehicle 110 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0038] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0039] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0040] In some embodiments, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from motor vehicle 110. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of motor vehicle 110.

[0041] The network 130 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including, but not limited to, TCP / IP, SNA, IPX, etc. By way of example only, one or more networks 130 may be a satellite communications network, a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (including, for example, Bluetooth, Wi-Fi), and / or any combination of these and other networks.

[0042] The system 100 may also include one or more databases 150. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 150 may be used to store information such as audio files and video files. The data repository 150 may reside in various locations. For example, the data repository used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The data repository 150 may be of different types. In some embodiments, the data repository used by the server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to a command.

[0043] In some embodiments, one or more of the databases 150 may also be used by an application to store application data. The databases used by the application may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0044] The motor vehicle 110 may include a sensor 111 for sensing the surrounding environment. The sensor 111 may include one or more of the following sensors: a visual camera, an infrared camera, an ultrasonic sensor, a millimeter wave radar, and a laser radar (LiDAR). Different sensors may provide different detection accuracy and range. The camera may be installed at the front, rear, or other location of the vehicle. The visual camera may capture the situation inside and outside the vehicle in real time and present it to the driver and / or passengers. In addition, by analyzing the images captured by the visual camera, information such as traffic light indications, intersection conditions, and the operating status of other vehicles may be obtained. The infrared camera may capture objects under night vision conditions. Ultrasonic sensors may be installed around the vehicle to measure the distance of objects outside the vehicle from the vehicle using the characteristics of strong ultrasonic directionality. Millimeter wave radars may be installed at the front, rear, or other location of the vehicle to measure the distance of objects outside the vehicle from the vehicle using the characteristics of electromagnetic waves. Laser radars may be installed at the front, rear, or other location of the vehicle to detect object edges and shape information, thereby identifying and tracking objects. Due to the Doppler effect, the radar device may also measure the speed change of the vehicle and the moving object.

[0045] The motor vehicle 110 may also include a communication device 112. The communication device 112 may include a satellite positioning module capable of receiving satellite positioning signals (e.g., Beidou, GPS, GLONASS, and GALILEO) from satellites 141 and generating coordinates based on these signals. The communication device 112 may also include a module for communicating with a mobile communication base station 142, and the mobile communication network may implement any suitable communication technology, such as GSM / GPRS, CDMA, LTE, and other current or evolving wireless communication technologies (e.g., 5G technology). The communication device 112 may also have a vehicle-to-vehicle or vehicle-to-everything (V2X) module, which is configured to implement, for example, vehicle-to-vehicle (V2V) communication with other vehicles 143 and vehicle-to-infrastructure (V2I) communication with infrastructure 144. In addition, the communication device 112 may also have a module configured to communicate with a user terminal 145 (including but not limited to a smartphone, a tablet computer, or a wearable device such as a watch) by, for example, using a wireless LAN or Bluetooth using the IEEE 802.11 standard. Using the communication device 112, the motor vehicle 110 may also access the server 120 via the network 130.

[0046] The motor vehicle 110 may further include an inertial navigation module. The inertial navigation module and the satellite positioning module may be combined into a combined positioning system to achieve initial positioning of the motor vehicle 110.

[0047] The motor vehicle 110 may also include a control device 113. The control device 113 may include a processor that communicates with various types of computer-readable storage devices or media, such as a central processing unit (CPU) or a graphics processing unit (GPU), or other dedicated processors. The control device 113 may include an automatic driving system for automatically controlling various actuators in the vehicle, and accordingly, the motor vehicle 110 is an automatic driving vehicle. The automatic driving system is configured to control the powertrain, steering system, and braking system of the motor vehicle 110 (not shown) via multiple actuators in response to inputs from multiple sensors 111 or other input devices to control acceleration, steering, and braking, respectively, without human intervention or limited human intervention. Some processing functions of the control device 113 can be implemented through cloud computing. For example, some processing can be performed using an on-board processor, while other processing can be performed using computing resources in the cloud. The control device 113 can be configured to perform a method according to the present disclosure. In addition, the control device 113 can be implemented as an example of an electronic device on the motor vehicle side (client) according to the present disclosure.

[0048] Figure 1 The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0049] According to some embodiments, the server 120 may construct a vectorized map by executing the vectorized map construction method of the embodiment of the present disclosure, and train the positioning model by executing the positioning model training method of the embodiment of the present disclosure. The constructed vectorized map and the trained positioning model may be transmitted to the motor vehicle 110. The motor vehicle 110 may use the vectorized map and the positioning model to execute the vehicle positioning method of the embodiment of the present disclosure to achieve accurate positioning of itself.

[0050] According to other embodiments, the vectorized map construction method and the positioning model training method may also be executed by the motor vehicle 110. This generally requires the motor vehicle 110 to have a higher hardware configuration and a stronger computing capability.

[0051] According to some embodiments, the vehicle positioning method may also be executed by the server 120. In this case, the motor vehicle 110 uploads relevant data (including initial position and multimodal sensor data) to the server 120. Accordingly, the server 120 obtains the data uploaded by the motor vehicle 110, and processes the data by executing the vehicle positioning method, thereby accurately positioning the motor vehicle 110.

[0052] The high-precision positioning information obtained by executing the vehicle positioning method of the embodiment of the present disclosure can be applied to tasks such as trajectory planning, behavior decision-making, and motion control of the motor vehicle 110.

[0053] Figure 2 FIG. 2 is a flow chart of a vehicle positioning method 200 according to an embodiment of the present disclosure. As described above, the execution subject of the method 200 may be an autonomous driving vehicle (eg, Figure 1 ), or a server (e.g., a motor vehicle 110 shown in FIG. Figure 1 120 shown in FIG. Figure 2 As shown, the method 200 includes steps S210 - S250 .

[0054] In step S210, the initial position of the vehicle, the point cloud collected by the radar deployed on the vehicle, and a plurality of map elements used to locate the vehicle are obtained.

[0055] In step S220, the point cloud is encoded to obtain point cloud features.

[0056] In step S230, a plurality of map elements are encoded to obtain map features.

[0057] In step S240, based on the point cloud features and the map features, a target pose offset for correcting the initial pose is determined.

[0058] In step S250, the initial posture and the target posture offset are superimposed to obtain a corrected posture of the vehicle.

[0059] According to the embodiments of the present disclosure, the point cloud contains accurate depth information and can therefore be used to achieve high-precision positioning. Determining the target pose offset based on point cloud features and map features, and correcting the initial pose based on the target pose offset can improve the positioning accuracy of the vehicle and enable accurate positioning of the vehicle even in complex environments.

[0060] The following is a detailed description of each step of method 200.

[0061] In step S210, the initial position of the vehicle, the point cloud collected by the radar deployed on the vehicle, and a plurality of map elements used to locate the vehicle are obtained.

[0062] The vehicle in step S210 may be a vehicle with an autonomous driving function, that is, an autonomous driving vehicle.

[0063] In the embodiments of the present disclosure, the initial posture refers to an uncorrected posture.

[0064] According to some embodiments, the initial position of the vehicle may be a position output by a combined positioning system of the vehicle. The combined positioning system generally includes a satellite positioning system and an inertial navigation system.

[0065] According to some embodiments, the vehicle may be positioned at a preset frequency (eg, 1 Hz). The initial position at the current moment may be the corrected position at the previous moment.

[0066] The vehicle's posture (including the uncorrected initial posture and the corrected posture) is used to indicate the position and posture of the vehicle. The position of the vehicle can be represented by three-dimensional coordinates such as (x, y, z). The posture of the vehicle can be represented by posture angles, for example. The posture angles further include roll angle (roll or φ), pitch angle (pitch or θ) and yaw angle (yaw or ψ).

[0067] Since the vehicle usually does not leave the ground during driving and does not roll or pitch, in practice, the accuracy of the z coordinate, roll angle, and pitch angle is usually not concerned. Accordingly, in an embodiment of the present disclosure, only the x coordinate, y coordinate, and yaw angle yaw in the initial posture can be corrected, without correcting the z coordinate, roll angle roll, and pitch angle pitch. In other words, the z coordinate, roll angle roll, and pitch angle pitch of the corrected posture are the same as the initial posture, but the x coordinate, y coordinate, and yaw angle yaw may be different from the initial posture.

[0068] Vehicles are usually equipped with radar equipment for environmental perception, such as laser radar, millimeter wave radar, etc. Radar can collect point cloud data. It can be understood that point clouds usually include the position coordinates and reflection intensity values ​​of multiple three-dimensional space points. Since point clouds are a collection of position coordinates and reflection intensity values ​​of a series of three-dimensional space points, point clouds can reflect the depth information from objects in the surrounding environment to the vehicle, and have the potential to achieve high-precision positioning.

[0069] According to some embodiments, a vectorized map may be stored locally in the vehicle or on a server.

[0070] Vector map is a data set that represents geographic elements by using information such as identification, name, location, attributes, and topological relationships between them. Vector map includes multiple geographic elements, each of which is stored in a vector data structure. Vector data structure is a data organization method that uses points, lines, surfaces, and their combinations in geometry to represent the spatial distribution of geographic elements. It expresses the location of elements by recording the coordinates of the elements and their spatial relationships.

[0071] According to some embodiments, the geographic elements in the vectorized map include road elements and geometric elements. Road elements refer to elements with specific semantics in the road, including lane lines, curbs, stop lines, crosswalks, traffic signs, poles, etc. Among them, poles further include tree trunks, traffic sign posts, and street light poles. Geometric elements refer to elements with a certain shape, including surface elements (surfel), line elements, etc. Surface elements represent planes in the physical world, such as the outer surface of a building, the surface of a traffic light, a traffic sign, etc. It should be noted that surface elements can have a certain overlap with road elements. For example, some surface elements are also road elements.

[0072] Road elements are usually sparse. Some sections have few or even no road elements. On sections with few or no road elements, it is difficult to accurately locate the vehicle through road elements. According to the above embodiment, the vectorized map also includes geometric elements such as surface elements. As a supplement to road elements, geometric elements can improve the richness and density of geographical elements in the vectorized map, thereby enabling accurate positioning of the vehicle.

[0073] According to the embodiments of the present disclosure, a vectorized map is used for vehicle positioning. The vectorized map is small in size and easy to update, which reduces storage costs, thereby improving the computational efficiency and usability of the vehicle positioning method and meeting mass production requirements.

[0074] According to some embodiments, in a vectorized map, lane lines, curbs, and stop lines are represented as line segments, and the endpoints of the line segments are two-dimensional xy coordinates in a global coordinate system, such as a UTM (Universal Transverse Mercator) coordinate system. A crosswalk is represented as a polygon, and the vertices of the polygon are represented by two-dimensional xy coordinates in a UTM coordinate system. Traffic signs are represented as rectangles perpendicular to the xy plane, and the vertices are three-dimensional UTM coordinates, where the z coordinate is represented by the height relative to the ground. The pole is represented by the two-dimensional xy coordinates in the UTM coordinate system and the height of the pole.

[0075] According to some embodiments, in a vectorized map, a surface element P is represented as in, represents the field of real numbers, is the horizontal and vertical coordinates of a point on the surface element (such as the center point), is the projection of the normal vector of the face element on the xy plane. The method of extracting face elements will be described in detail in the vectorized map construction method 400 below.

[0076] According to some embodiments, the multiple map elements used for vehicle positioning in step S210 can be obtained by screening multiple geographic elements in the vectorized map based on the initial posture. According to some embodiments, geographic elements located near the initial posture (i.e., the distance to the initial posture is less than a threshold) can be used as map elements for vehicle positioning. For example, geographic elements within a range of 100 meters near the initial posture (i.e., the distance to the initial posture is less than 100 meters) are used as map elements for vehicle positioning.

[0077] According to some embodiments, in order to balance the computational efficiency and the reliability of the positioning result, a preset number of geographic elements whose distance to the initial posture is less than a threshold value can be used as map elements for vehicle positioning. The preset number can be set as needed. For example, the preset number can be set to 100, 500, 1000, etc. If the number of geographic elements near the initial posture is more than the preset number, the nearby geographic elements can be sampled to obtain the preset number of map elements. Further, for road elements, the sampling can be carried out in order from near to far from the initial posture. For surface elements, random sampling can be carried out. Surface elements may correspond to different types of entities in the physical world, such as the outer surface of a building, traffic signs, etc. Different types of surface elements can impose positioning constraints on the vehicle in different directions. For example, the outer surface of a building (parallel to the lane line) can constrain the positioning of the left and right directions of the vehicle, and the traffic signs can constrain the positioning of the forward direction of the vehicle. Random sampling of surface elements can make the sampling results cover various types of surface elements more evenly, thereby ensuring the accuracy of vehicle positioning. If the number of geographic elements near the initial posture is less than the preset number, the geographic elements can be expanded to the preset number by copying the existing geographic elements.

[0078] According to some embodiments, the plurality of map elements acquired in step S210 may be preprocessed to improve the computational efficiency and accuracy of subsequent vehicle positioning.

[0079] As described above, the plurality of map elements obtained from the vectorized map include lane lines, curbs, stop lines, crosswalks, traffic signs, poles, and surface elements. According to some embodiments, lane lines, curbs, and stop lines may be broken into line segments of the same length, each of which is represented by a four-dimensional vector The four values ​​in the vector are the xy coordinates of the starting and ending points of the line segment. Traffic signs and poles are represented as The first two values ​​in the vector are the xy coordinates of the center of the traffic sign or the pole, and the last value in the vector represents the height of the center of the traffic sign or the pole relative to the ground. For the surface elements, no preprocessing is required. That is, the representation of the surface elements can be the same as that in the vectorized map.

[0080] According to the above embodiment, based on the initial position of the vehicle, multiple vectorized map elements located near the vehicle can be extracted from the vectorized map to form a map element set M = {(m i ,s i )|i=1,2,…K}. K is the number of map elements. For example, K=500. is the location information of the i-th map element, where Is the endpoint of the map element. It is the additional geometric information of the map element, which has different meanings according to the category of the map element: for lane lines, curbs, crosswalks, and stop lines, (a i ,b i ) indicates the other end point; for poles and cards, (a i ,b i ) indicates the height; for surface elements, (a i ,b i ) represents the projection of the normal vector on the xy plane. i ∈{1,2,…,N s} represents the category information of the i-th map element, i.e., semantic information. s is the number of category information. As mentioned above, map elements include lane lines, curbs, stop lines, crosswalks, traffic signs, poles, and surface elements. Therefore, N s =7.

[0081] In step S220, the point cloud is encoded to obtain point cloud features.

[0082] According to some embodiments, the encoding of the point cloud can be completed in the target three-dimensional space. The target three-dimensional space can be, for example, the bird's eye view (BEV) space of the vehicle. The bird's eye view refers to a top-down view. The bird's eye view space refers to the space in a right-handed rectangular Cartesian coordinate system with the position of the vehicle (i.e., the initial posture) as the origin. In some embodiments, the bird's eye view space can take the position of the vehicle as the origin, the right direction of the vehicle as the positive direction of the x-axis, the forward direction of the vehicle as the positive direction of the y-axis, and the vertical top of the vehicle as the positive direction of the z-axis. In other embodiments, the bird's eye view space can also take the position of the vehicle as the origin, the forward direction of the vehicle as the positive direction of the x-axis, the left direction of the vehicle as the positive direction of the y-axis, and the vertical top of the vehicle as the positive direction of the z-axis.

[0083] According to some embodiments, step S220 includes steps S221 - S223 .

[0084] In step S221, based on the initial pose, the spatial range for extracting point cloud features is determined.

[0085] In step S222, the above-mentioned spatial range is divided into a plurality of grids of unit size.

[0086] In step S223, for any grid among the above-mentioned multiple grids, a feature vector of the point cloud in the grid is extracted to obtain a point cloud feature. The point cloud feature includes a point cloud feature map composed of the feature vectors of the multiple grids, multiple pixels in the point cloud feature map correspond to the multiple grids respectively (that is, each pixel corresponds to one grid), and the number of channels of any pixel among the multiple pixels is equal to the dimension of the feature vector.

[0087] According to the above embodiment, the point cloud is screened and encoded based on the initial posture of the vehicle, which can improve the encoding efficiency and positioning accuracy.

[0088] According to some embodiments, in step S221, the local space with the initial posture as the origin may be determined as the spatial range for extracting point cloud features, and the point cloud within the spatial range may be used to generate point cloud features. The spatial range may be, for example, the spatial range with the initial posture as the origin, the vehicle forward direction (x-axis positive direction) [-80m, 80m], the vehicle left direction (y-axis positive direction) [-80m, 80m], and the vehicle top (z-axis positive direction) [-10m, 10m].

[0089] In step S222, the above-mentioned spatial range is divided into multiple grids of unit size. For example, the above-mentioned spatial range can be divided into multiple cylindrical spaces with a cross-section (parallel to the xy plane) of a square (for example, 0.5m*0.5m). Each cylindrical space is a grid. After the above division, the point cloud near the vehicle will fall into the corresponding grid. Each grid corresponds to a pixel in the point cloud feature map. The point cloud feature map can be a feature map in the BEV space. The resolution of the point cloud feature map (that is, the resolution of the BEV space) refers to the length in the physical world corresponding to a single pixel (that is, a grid in the BEV space), that is, the side length of the cross-section of the above-mentioned cylindrical space, for example 0.5m per pixel.

[0090] It is understood that different grids include different numbers of points. According to some embodiments, in order to improve computational efficiency, the point cloud of each grid may be preprocessed so that each grid includes the same number of points. The number of points included in the grid is recorded as N. If the number of points in a grid is greater than N, N points are selected from it by random sampling; if the number of points in a grid is less than N, it is filled to N points by zero padding.

[0091] According to some embodiments, in step S223, for each grid, a feature vector of the point cloud in the grid is extracted.

[0092] According to some embodiments, each point can be initialized as a D-dimensional vector. For example, D=9, and the initialization vector of each point is (x, y, z, r, xc, yc, zc, xp, yp), where x, y, z, r represent the three-dimensional coordinates and reflection intensity of the point respectively; xc, yc, zc represent the distance from the point to the arithmetic mean point of all points in the grid where it is located, and xp, yp represent the offset value of the point to the x, y center of the grid where it is located. According to the above embodiment, the point cloud can be encoded as a dense tensor of dimension (D, P, N). Where D is the dimension of the initialization vector of a single point, P is the number of grids, and N is the number of points included in each grid.

[0093] According to some embodiments, each D-dimensional vector can be linearly mapped using a trained initial encoding network to obtain a C-dimensional vector (e.g., C=128), thereby mapping the tensor (D, P, N) to the tensor (C, P, N). Further, a pooling operation is performed on (C, P, N) to obtain the tensor (C, P).

[0094] Furthermore, the tensor (C, P) can be mapped to a point cloud feature map of size H*W*C Each grid p corresponds to a pixel (h, w) in the point cloud feature map. Among them, H, W, and C are the height, width, and number of channels of the point cloud feature map, respectively. Specifically, H is the quotient of the point cloud range of the x-axis and the resolution of the point cloud feature map; W is the quotient of the point cloud range of the y-axis and the resolution of the point cloud feature map; C is the dimension of the feature vector corresponding to each pixel. For example, in the above embodiment, the point cloud range of the x-axis and the y-axis are both 160m (ie, [-80m, 80m]), the resolution of the point cloud feature map is 0.5m per pixel, C=128, and accordingly, the point cloud feature map H=W=160 / 0.5=320, and the size of the point cloud feature map is 320*320*128.

[0095] According to some embodiments, the point cloud feature map F L As point cloud features.

[0096] According to some other embodiments, the point cloud feature map F L Input the trained residual network to train the point cloud feature map F L Further encoding is performed to obtain the point cloud feature map The point cloud feature map F B As point cloud features.

[0097] According to some embodiments, the above-mentioned initial encoding network and residual network can form a point cloud encoder. Accordingly, in step S223, the point cloud of each grid is input into the trained point cloud encoder, and the point cloud features output by the point cloud encoder can be obtained. The point cloud encoder can be a module in the positioning model. The training method of the positioning model will be described in detail in method 500 below.

[0098] In step S230, a plurality of map elements are encoded to obtain map features.

[0099] As described above, the plurality of map elements are obtained by screening the plurality of geographic elements in the vectorized map based on the initial pose. The geographic elements in the vectorized map include road elements and geometric elements, and accordingly, the plurality of map elements obtained by screening also include at least one road element and at least one geometric element. The at least one road element includes any one of a lane line, a curb, a crosswalk, a stop line, a traffic sign or a pole, and the at least one geometric element includes a surface element.

[0100] According to some embodiments, the surface elements are obtained by extracting planes from the point cloud map. The extraction method of the surface elements will be described in detail in the vectorized map construction method 400 below.

[0101] According to some embodiments, step S230 may include steps S231 - S234 .

[0102] For any map element among the multiple map elements, steps S231 - S233 are executed.

[0103] In step S231, the position information of the map element is encoded to obtain a position code.

[0104] In step S232, the category information of the map element is encoded to obtain a semantic code.

[0105] In step S233, the position code and the semantic code are fused to obtain a code vector of the map element.

[0106] In step S234, a map feature is generated based on the encoding vectors of each of the plurality of map elements.

[0107] According to the above embodiment, the position information and category information of the map elements are encoded respectively, and the encoding results are merged, so that the expression ability of the map elements can be improved.

[0108] According to some embodiments, as described above, the map elements include road elements and surface elements. The position information m of the i-th map element i is represented as a four-dimensional vector. Since the location information m of the map element i It is usually expressed in a global coordinate system (such as the UTM coordinate system), so it needs to be normalized according to the following formula to express it in a local coordinate system with the vehicle as the origin:

[0109]

[0110] In formula (1), is the normalized position information, O xy =[x o y o ]T is the xy coordinate of the initial pose in the global coordinate system, R xy =[x r y r ] T The xy range of the point cloud.

[0111] According to some embodiments, in step S231, the normalized position information may be coded using a trained position encoder. Encode. The position encoder can be implemented as a multi-layer perceptron (MLP), for example. The encoding process of the position encoder can be expressed as the following formula (2):

[0112]

[0113] in, is the position code of the ith map element, h() is the trained position coder, and C is the dimension of the position code. For example, C=128.

[0114] According to some embodiments, in step S232, the semantic coding of the map element may be determined based on the correspondence between the multiple types of category information and the multiple semantic codings, wherein the multiple semantic codings are parameters of the positioning model and are obtained by training the positioning model.

[0115] According to the above embodiment, the semantic coding is trainable, thereby improving the ability of the semantic coding to express map element category information, thereby improving positioning accuracy. The training method of the semantic coding will be described in detail in the positioning model training method 500 of the embodiment below.

[0116] The set of semantic codes can be expressed as Among them, N s is the number of semantic codes, that is, the number of category information. As mentioned above, map elements include lane lines, curbs, crosswalks, stop lines, traffic signs, poles, and surface elements, a total of seven types. Accordingly, N s =7, the category information numbers 1 to 7 correspond to the above seven map elements respectively.

[0117] The category information of the i-th map element is recorded as s i , then the semantic encoding of the i-th map element is C is the dimension of semantic encoding (the same as the dimension of position encoding), for example, C=128.

[0118] After the position code and semantic code of the map element are obtained through steps S231 and S232, in step S233, the code vector of the map element can be obtained by fusing the position code and the semantic code.

[0119] According to some embodiments, the sum of the position code and the semantic code may be used as the code vector of the map element. That is:

[0120]

[0121] Among them, e i is the encoding vector of the i-th map element.

[0122] According to some other embodiments, the weighted sum of the position code and the semantic code may also be used as the code vector of the map element.

[0123] After the coding vector of each map element is obtained in step S233, in step S234, a map feature may be generated based on the coding vectors of each of the plurality of map elements.

[0124] According to some embodiments, the encoding vectors of the respective map elements may be combined to generate a map feature, that is, the map feature is a collection of the encoding vectors of the respective map elements:

[0125]

[0126] According to some other embodiments, for any map element among the multiple map elements, the coding vector of the map element can be updated using the point cloud feature based on the attention mechanism, and the updated coding vectors of each map element are combined into a map feature. That is, the map feature is a collection of the updated coding vectors of each map element.

[0127] The attention mechanism can capture the correlation between features. Using the attention mechanism to update the encoding vector of a map element can improve the expressiveness and accuracy of the encoding vector for the map element.

[0128] According to some embodiments, the encoding vector of each map element can be used as a query vector (Query), and based on the attention mechanism, the correlation between the map element and each feature vector in the point cloud feature map (i.e., attention weight) can be obtained. Then, based on the correlation between the map element and each feature vector in the point cloud feature map, the encoding vector of the map element is fused with each feature vector to obtain an updated encoding vector of the map element.

[0129] According to some embodiments, the updating of the coding vector of the map element can be implemented by using a trained Transformer Decoder. Specifically, the coding vector of the point cloud feature and each map element can be input into the trained Transformer Decoder to obtain the updated coding vector of each map element output by the Transformer Decoder. The Transformer Decoder can be used as a module in the positioning model and is trained according to the method 500 described below.

[0130] After obtaining the point cloud features and the map features through steps S220 and S230 respectively, in step S240, a posture offset for correcting the initial posture is determined based on the point cloud features and the map features.

[0131] According to some embodiments, the target pose offset may be determined by matching the point cloud features with the map features. According to this embodiment, the target pose offset is estimated by matching the online point cloud features with the offline map features, which has strong interpretability and can improve positioning accuracy.

[0132] According to some embodiments, step S240 may include steps S241 - S243 .

[0133] In step S241, sampling is performed within a preset offset sampling range to obtain a plurality of candidate posture offsets.

[0134] In step S242, for any candidate pose offset among the multiple candidate pose offsets, the matching degree between the point cloud feature and the map feature under the candidate pose offset is determined.

[0135] In step S243, based on the matching degrees corresponding to the multiple candidate pose offsets, the multiple candidate pose offsets are fused to obtain the target pose offset.

[0136] According to the above embodiment, the offsets of each candidate pose are fused according to the matching degree between the point cloud features and the map features, which has strong interpretability, makes it easy to analyze the cause of positioning failure and explore the direction that may further improve the positioning accuracy.

[0137] According to some embodiments, for step S241, uniform sampling may be performed within the offset sampling range at a certain sampling interval to obtain a plurality of candidate posture offsets.

[0138] For example, the initial pose can be used as the center, and uniform sampling can be performed along the x, y and yaw directions according to the sampling interval d to obtain multiple candidate pose offsets, which are recorded as Where N H is the number of values ​​sampled in each direction. For example, N H =7, accordingly, 343 candidate pose offsets can be sampled. For the x and y directions, d can be, for example, 0.5 m; for the yaw direction, d can be, for example, 0.5°.

[0139] According to some embodiments, as described above, the point cloud feature includes a point cloud feature map of the target three-dimensional space, and the map feature includes encoding vectors of respective map elements. Accordingly, step S242 may include steps S2421-S2424.

[0140] In step S2421, the initial pose and the candidate pose offset are superimposed to obtain the candidate pose.

[0141] For any map element among the multiple map elements, steps S2422 and S2423 are executed.

[0142] In step S2422, based on the candidate poses, the map elements are projected into the target three-dimensional space (BEV space) to obtain feature vectors in the point cloud feature map corresponding to the map elements.

[0143] In step S2423, the similarity between the coding vector of the map element and the corresponding feature vector is calculated.

[0144] In step S2424, based on the similarities corresponding to the plurality of map elements, the matching degree between the point cloud feature and the map feature under the candidate pose offset is determined.

[0145] For step S2421, for example, the initial posture is in represents a special Euclidean group. The candidate pose offset is ΔT pqr , then the candidate pose T pqr for in, represents the generalized addition operation between positions, i.e., the special Euclidean group and the field of real numbers The addition operation between The x, y and yaw position offsets in space are added to In space.

[0146] According to some embodiments, in step S2422, the candidate pose T pqr , project the coordinates of the ith map element into the BEV space to obtain the projection coordinates of the ith map element in the BEV space Furthermore, the point cloud feature map is interpolated by an interpolation algorithm (such as a bilinear interpolation algorithm). By interpolation, we can get the point cloud feature map in the projection coordinates The eigenvector at

[0147] Specifically, for the endpoint (x i ,y i ), first through the endpoint (x i,y i ) and the perpendicular line of the BEV space to find the z coordinate of the endpoint of the map element in the BEV space. i ,y i ) and the BEV space at p i =(x i ,y i ,z i ), then p i Satisfies the following equation:

[0148] (p i -t pqr )⊙n=0 (5)

[0149] Among them, ⊙ represents the dot product, R pqr and t pqr They are the candidate poses T pqr Based on the rotation matrix and translation vector in equation (5), the intersection point p can be calculated i The z coordinates are as follows:

[0150]

[0151] Then, based on the candidate poses, the intersection point p i Projecting the global coordinate system into the radar coordinate system, we get the endpoint (x i ,y i ) in the radar coordinate system

[0152]

[0153] Afterwards, Normalize to get the endpoint (x i ,y i ) in BEV space:

[0154]

[0155] Among them, x min ,y min are the minimum x-coordinate and y-coordinate values ​​of the point cloud involved in the encoding (i.e., the minimum boundary of the BEV space in the x and y directions), and R is the resolution of the BEV space.

[0156] In getting the endpoint Projected coordinates in BEV space Then, through the point cloud feature map in BEV space Perform bilinear interpolation on the map to obtain the BEV feature vector corresponding to the i-th map element

[0157] According to some embodiments, in step S2423, the similarity between the encoding vector of the map element and the corresponding feature vector can be calculated based on the dot product of the two. i With the corresponding eigenvector The similarity S i (T pqr ) can be calculated according to the following formula (9):

[0158]

[0159] Here, ⊙ represents the dot product and g() represents a learnable multi-layer perceptron (MLP).

[0160] According to some embodiments, in step S2424, the sum or average of the similarities of the map elements may be determined as the matching degree between the point cloud feature and the map feature at the candidate pose offset.

[0161] For example, in the candidate pose offset ΔT pqr Next (i.e., in the candidate pose T pqr The matching degree between the point cloud features and the map features can be calculated according to the following formula (10).

[0162]

[0163] Where K is the number of map elements.

[0164] According to the above step S242, the matching degree between the point cloud feature and the map feature under each candidate pose offset can be obtained. Then, in step S243, multiple candidate pose offsets can be fused based on the matching degrees corresponding to each of the multiple candidate pose offsets to obtain the target pose offset.

[0165] According to some embodiments, step S243 may include step S2431 and step S2432.

[0166] In step S2431, for any candidate pose offset among multiple candidate pose offsets, the probability of the candidate pose offset is determined based on the ratio of the matching degree of the candidate pose offset to the sum of the matching degrees of multiple candidate pose offsets.

[0167] In step S2432, the expectation of multiple candidate pose offsets is determined as the target pose offset.

[0168] According to the above embodiment, the probability (posterior probability) of each candidate pose offset is calculated based on the matching degree, and the candidate pose offsets are fused according to the posterior probability. This has strong interpretability and makes it easy to analyze the reasons for positioning failure and explore directions that may further improve positioning accuracy.

[0169] According to some embodiments, the probability P(T pqr |X) can be calculated according to the following formula (11):

[0170]

[0171] Where X={M,F B ,E}.

[0172] Accordingly, the target posture offset ΔT est and its covariance Σ est Calculate according to the following formulas (12) and (13):

[0173]

[0174]

[0175] In the above formula, E and Var represent the expectation and variance respectively.

[0176] According to some embodiments, step S240 may be implemented by a trained pose solver. Specifically, the point cloud features, map features, and initial pose may be input into the trained pose solver to obtain the target pose offset output by the pose solver.

[0177] After the target posture offset is obtained in step S240, in step S250, the target posture offset ΔT may be calculated based on the target posture offset ΔT. est For the initial pose T init Correction is performed to obtain the corrected posture T est :

[0178]

[0179] The vehicle positioning method 200 of the embodiment of the present disclosure may be implemented using a trained positioning model. Figure 3 A schematic diagram of a vehicle positioning process based on a trained positioning model 300 according to some embodiments of the present disclosure is shown.

[0180] like Figure 3 As shown, the positioning model 300 includes a point cloud encoder 310, a map encoder 320, and a pose solver 330. The positioning model 300 can be trained by, for example, the positioning model training method 500 described below.

[0181] exist Figure 3In the vehicle positioning process shown, first, the point cloud 341 collected by the radar, the six-degree-of-freedom initial position 342 of the vehicle (including three-dimensional coordinates and three attitude angles) and the vector map 343 for positioning the vehicle are obtained. The initial position 342 can be the position output by the combined positioning system at the current moment, or it can be the corrected position at the previous moment.

[0182] The point cloud 341 is filtered based on the initial posture 342, and only the point cloud near the initial posture 342 is retained. For example, only the point cloud within the range of the initial posture 342 as the origin, the vehicle forward direction (x-axis positive direction) [-80m, 80m], the vehicle left direction (y-axis positive direction) [-80m, 80m], and the vehicle top (z-axis positive direction) [-10m, 10m] can be retained. Furthermore, the point cloud can be voxelized, that is, the space is divided into multiple non-intersecting blocks, and a maximum of N points are retained in each block. For example, N = 32.

[0183] Map elements located near the initial position 342 are filtered out from the vectorized map 343, and the position information and semantic information (ie, category information) of the map elements are obtained.

[0184] The filtered point cloud 341 is input into the point cloud encoder 310 to obtain a point cloud feature 351 output by the point cloud encoder 310. The point cloud feature 351 may be a point cloud feature map in the BEV space.

[0185] The position information and semantic information of each map element filtered out from the vectorized map 343 based on the initial pose 342 are input into the map encoder 320 to obtain the map feature 352 output by the map encoder 320 .

[0186] The map encoder 320 may further include a position encoder and a semantic encoder. The position encoder is used to encode the position information of each map element to obtain a position code. The semantic encoder is used to encode the semantic information of each map element to obtain a semantic code. By adding the position code and the semantic code, the encoding vector of each map element can be obtained, thereby obtaining the map feature 352.

[0187] The pose solver 330 determines a target pose offset 361 for correcting the initial pose 342 based on the point cloud features 351 and the map features 352 .

[0188] By superimposing the initial posture 342 and the target posture offset 361, the corrected posture of the vehicle can be obtained.

[0189] According to an embodiment of the present disclosure, a vectorized map construction method is also provided. The vectorized map constructed according to the method can be applied to the above-mentioned vehicle positioning method 200.

[0190] Figure 4 FIG. 4 is a flowchart of a vectorized map construction method 400 according to an embodiment of the present disclosure. The execution subject of the method 400 is usually a server (eg Figure 1 In some cases, the execution subject of method 400 may also be an autonomous driving vehicle (e.g. Figure 1 110 shown in FIG. Figure 4 As shown, method 400 includes steps S410-S440.

[0191] In step S410, a point cloud in a point cloud map is obtained.

[0192] In step S420, the projection plane of the point cloud map is divided into a plurality of two-dimensional grids of a first unit size.

[0193] For any two-dimensional grid among the multiple two-dimensional grids, steps S430 and S440 are performed.

[0194] In step S430, a plane in the two-dimensional grid is extracted based on a point cloud in the three-dimensional space corresponding to the two-dimensional grid.

[0195] In step S440, the plane is stored as a surface element in the vectorized map.

[0196] According to an embodiment of the present disclosure, a plane is extracted from a point cloud map, and the extracted plane is stored as a surface element in a vectorized map, thereby improving the richness and density of the geographic elements in the vectorized map, thereby improving the positioning accuracy of the vehicle.

[0197] Since vectorized maps only contain sparse vector information, the amount of data is greatly reduced compared to point cloud maps containing dense point clouds, and they are easy to update. Storing vectorized maps (rather than point cloud maps) on the vehicle side can greatly reduce the storage cost of the vehicle side, improve the availability of vehicle positioning methods, and meet the needs of large-scale positioning under limited storage resources. Experimental verification shows that the volume of vectorized maps is about 0.35M / km. Compared with point cloud maps, the volume is reduced by 97.5%.

[0198] The various steps of method 400 are described in detail below.

[0199] In step S410, a point cloud in a point cloud map is obtained.

[0200] Point cloud maps use dense point clouds to represent geographic elements, while vector maps use information such as identification, name, location, attributes, and topological relationships between them to represent geographic elements.

[0201] In step S420, the projection plane of the point cloud map is divided into a plurality of two-dimensional grids of a first unit size.

[0202] The projection plane of the point cloud map refers to the xy plane. The first unit size can be set as needed. For example, the first unit size can be set to 1m*1m, 2m*2m, etc.

[0203] In step S430, a plane in the two-dimensional grid is extracted based on the point cloud in the three-dimensional space corresponding to the two-dimensional grid. The three-dimensional space corresponding to the two-dimensional grid refers to a cylindrical space with the two-dimensional grid as a cross section.

[0204] According to some embodiments, step S430 may include steps S431 - S434 .

[0205] In step S431, the three-dimensional space is divided into a plurality of three-dimensional grids of a second unit size along the height direction. The second unit size can be set as required. For example, the second unit size can be set to 1m*1m*1m, 2m*2m*2m, etc.

[0206] For any three-dimensional grid among the multiple three-dimensional grids, steps S432 and S433 are performed.

[0207] In step S432, based on the point cloud in the three-dimensional grid, the confidence level of the plane included in the three-dimensional grid is calculated.

[0208] In step S433, in response to the confidence being greater than the threshold, a plane in the three-dimensional grid is extracted. The threshold can be set as needed. For example, the threshold can be set to 10, 15, etc.

[0209] In step S434, the plane with the highest confidence in the multiple three-dimensional grids is determined as the plane corresponding to the two-dimensional grid.

[0210] According to some embodiments, for step S432, the confidence that the plane is included in the three-dimensional grid can be calculated according to the following steps: singular value decomposition is performed on the covariance matrix of the point cloud in the three-dimensional grid to obtain a first singular value λ1, a second singular value λ2, and a third singular value λ3, wherein the first singular value is less than or equal to the second singular value, and the second singular value is less than or equal to the third singular value, that is, λ1≤λ2≤λ3. The ratio λ2 / λ1 of the second singular value to the first singular value is determined as the confidence s, that is, s=λ2 / λ1.

[0211] According to the above embodiment, if λ2 / λ1 is large, it can be considered that the change (variance) of the point cloud data along the eigenvector direction corresponding to λ1 is very small relative to other directions and can be ignored, so the point cloud can be approximated as a plane. λ2 / λ1 can indicate the possibility of including a plane in the three-dimensional grid, so it can be used as the confidence that the plane is included in the three-dimensional grid.

[0212] In step S440, the plane is stored as a surface element in the vectorized map. According to some embodiments, the identifier of the surface element corresponding to the plane may be determined, and the coordinates of a point on the plane and the unit normal vector of the plane may be associated with the identifier and stored.

[0213] According to some embodiments, the identifier of the surface element may be generated according to a preset rule. It is understandable that the identifiers of the surface elements in the vectorized map are different.

[0214] According to some embodiments, the centroid of the point cloud in the three-dimensional grid to which the plane belongs may be used as the center point of the plane, and its coordinates may be stored. The unit normal vector of the plane is obtained by normalizing the eigenvector corresponding to the first singular value λ1.

[0215] In a vector map, a surface element P can be represented as in, represents the field of real numbers, is the horizontal and vertical coordinates of a point on the surface element (such as the center point), It is the projection of the normal vector of the face element on the xy plane.

[0216] According to some embodiments, in addition to the surface elements, other geographic elements are stored in the vectorized map in the form of vectors, including road elements, such as lane lines, curbs, crosswalks, stop lines, traffic signs and poles.

[0217] In a vectorized map, lanes, curbs, and stop signs are represented as line segments, with endpoints in the 2D xy coordinates of the UTM coordinate system. A crosswalk is represented as a polygon, with the vertices in the 2D xy coordinates of the UTM coordinate system. Traffic signs are represented as rectangles perpendicular to the xy plane, with vertices in the 3D UTM coordinates, where the z coordinate is expressed as height relative to the ground. Poles are represented by the 2D xy coordinates in the UTM coordinate system and the height of the pole.

[0218] According to an embodiment of the present disclosure, a method for training a positioning model is also provided. The positioning model trained according to the method can be applied to the above-mentioned vehicle positioning method 200.

[0219] Figure 5 FIG. 5 is a flow chart of a method 500 for training a positioning model according to an embodiment of the present disclosure. The execution subject of the method 500 is usually a server (eg, Figure 1 In some cases, the execution subject of method 500 may also be an autonomous driving vehicle (e.g. Figure 1 The positioning model of the embodiment of the present disclosure includes a point cloud encoder, a map encoder and a pose solver. An exemplary structure of the positioning model can be found in Figure 3 .

[0220] like Figure 5 As shown, method 500 includes steps S501-S510.

[0221] In step S501, the initial posture of the sample vehicle, the true posture value corresponding to the initial posture, the point cloud collected by the radar deployed on the sample vehicle, and multiple map elements used to locate the sample vehicle are obtained.

[0222] In step S502, the point cloud is input into a point cloud encoder to obtain a point cloud feature map in the target three-dimensional space.

[0223] In step S503, the category information of each of the plurality of map elements is input into the map encoder so that the map encoder:

[0224] Based on the correspondence between the multiple types of category information and the multiple semantic codes, determining the semantic code corresponding to the category information of each of the multiple map elements, wherein the multiple semantic codes are parameters of the positioning model;

[0225] Determining a coding vector of the map element based on the semantic coding; and

[0226] A map feature is generated based on respective encoding vectors of a plurality of map elements.

[0227] In step S504, the target map elements belonging to the target category among the multiple map elements are projected into the target three-dimensional space to obtain a truth map of semantic segmentation in the target three-dimensional space, wherein the value of the first pixel in the truth map indicates whether the first pixel is occupied by the target map element.

[0228] In step S505, a prediction map for semantic segmentation is determined based on the point cloud feature map, wherein the value of the second pixel in the prediction map indicates the similarity between the corresponding feature vector and the semantic encoding of the target category, and the corresponding feature vector is the feature vector of the pixel corresponding to the position of the second pixel in the point cloud feature map.

[0229] In step S506, the point cloud feature map, map features and initial pose are input into a pose solver to obtain a predicted pose offset output by the pose solver.

[0230] In step S507, a first loss is determined based on the predicted pose offset and the true value of the pose offset, wherein the true value of the pose offset is the difference between the true value of the pose and the initial pose.

[0231] In step S508, a second loss is determined based on the true value map and the predicted map of the semantic segmentation.

[0232] In step S509 , the overall loss of the positioning model is determined based on at least the first loss and the second loss.

[0233] In step S510 , the parameters of the positioning model are adjusted based on the overall loss.

[0234] According to the embodiments of the present disclosure, the first loss can guide the positioning model to output a more accurate predicted pose offset. The second loss can enhance the ability of semantic coding to express map element category information. By determining the overall loss of the positioning model based on the first loss and the second loss and adjusting the parameters of the positioning model accordingly, the positioning accuracy of the positioning model can be improved.

[0235] In addition, comprehensive consideration of the first loss and the second loss helps the localization model converge faster.

[0236] According to some embodiments, the initial position and posture may be the position and posture output by the combined positioning system of the sample vehicle at the current moment, or may be the corrected position and posture at the previous moment.

[0237] According to some embodiments, the multiple map elements used to locate the sample vehicle may be geographic elements located near the initial position selected from the vectorized map. The multiple map elements may include, for example, road elements (lane lines, curbs, crosswalks, stop lines, traffic signs, poles, etc.) and surface elements.

[0238] Steps S502 and S503 correspond to steps S220 and S230 described above. The point cloud encoder and map encoder in steps S502 and S503 are used to execute steps S220 and S230 respectively. The internal processing logic of the point cloud encoder and the map encoder refers to the relevant description of steps S220 and S230 above, which will not be repeated here.

[0239] According to some embodiments, for step S504 , the target category may be any category.

[0240] For example, for target category j, the target map elements belonging to category j are projected into the BEV space to obtain the true value map of the semantic segmentation of category j in the point cloud feature map in, Indicates that the first pixel (h, w) in the truth map is occupied by a target map element belonging to category j, Indicates that the first pixel (h, w) in the truth map is not occupied by a target map element belonging to category j.

[0241] The training goal of semantic coding is to make the semantic coding of the jth category and the truth map of BEV semantic segmentation BEV feature vector at As close as possible.

[0242] According to some embodiments, the predicted map S of the semantic segmentation of category j in the point cloud feature map j Constructed according to the following formula:

[0243]

[0244] Among them, S j (h,w) represents the prediction graph S of category j j The value of the second pixel with coordinates (h, w) in F B (h,w) is the point cloud feature map F B The feature vector corresponding to the pixel with coordinates (h,w) in is is the semantic encoding of category j, ⊙ represents the dot product. sigmoid() is a mathematical function that maps the value of the real number domain to the range (0,1). B (h,w) and The higher the similarity, the j The larger the value of (h,w), the closer it is to 1. One training goal of the localization model is to minimize the true value map of the semantic segmentation of each category j. and prediction graph S j The difference between the two allows the localization model to learn the semantic encoding set E sem And point cloud feature map F B The corresponding relationship between them.

[0245] The pose solver in step S506 is used to execute step S240 described above. The internal processing logic of the pose solver is described in the above description of step S240, which will not be repeated here.

[0246] The first loss in step S507 is the mean square error loss of the pose offset. According to some embodiments, the first loss L rmse It can be calculated according to the following formula:

[0247]

[0248] Among them, the matrix U can be decomposed into covariance Σ by SVD est =USU T get, Is a diagonal matrix, the diagonal element values ​​of the matrix are obtained by -1 Normalizing the diagonal elements of , ΔT est To predict the pose offset, ΔT gt is the true value of the pose offset, that is, the difference between the true value of the pose and the initial pose.

[0249] It should be noted that if ΔT is directly est With ΔT gtAs the first loss, the influence of each direction on positioning is the same. However, in actual situations, different directions have different effects on positioning. For example, in a lateral degradation scenario (such as a tunnel, lack of lateral constraints on the x-axis), the lateral positioning error will be relatively large, and it is difficult to improve its accuracy through optimization. Therefore, in this case, it is hoped to reduce the lateral weight, thereby reducing the impact of lateral uncertainty on positioning accuracy. The weight of the direction is determined by the covariance. According to formula (16), the larger the covariance of a direction, the greater the uncertainty, and the weight of the direction. The smaller the setting, the smaller the impact on the first loss.

[0250] The second loss in step S508 is a semantic supervision loss. According to some embodiments, the semantic supervision loss It can be calculated according to the following formula:

[0251]

[0252]

[0253]

[0254]

[0255] α=0.8,γ=2.0 (21)

[0256] Among them, N S is the number of category information.

[0257] According to the second loss Semantic coding is trainable, which can improve the ability of semantic coding to express map element category information, thereby improving positioning accuracy.

[0258] According to some embodiments, in step S509, the first loss L rmse With the second loss The weighted sum of is taken as the overall loss of the positioning model.

[0259] According to some embodiments, step S506 further includes: inputting the point cloud feature map, map features and initial pose into the pose solver, so that the pose solver: performs sampling within a preset offset sampling range to obtain multiple candidate pose offsets; for any candidate pose offset among the multiple candidate pose offsets, determines the matching degree between the point cloud feature map and the map features under the candidate pose offset; and based on the matching degrees corresponding to each of the multiple candidate pose offsets, fuses the multiple candidate pose offsets to obtain and output a predicted pose offset. The above processing logic of the pose solver can refer to the description of steps S241-S243 above, which will not be repeated here.

[0260] Accordingly, the method 500 further includes: determining a third loss based on the matching degrees corresponding to each of the plurality of candidate pose offsets, wherein the third loss indicates the difference between the predicted probability distribution of the pose truth value and the true probability distribution of the pose truth value.

[0261] The third loss is the KL (Kullback-Leibler) divergence loss of the pose, which comes from the KL divergence D KL (t(T)||P(T|X)), where t(T) represents the true probability distribution of the pose. The goal of this loss function is to guide the distribution of the posterior probability P(T|X) to be close to the true probability distribution t(T), thereby avoiding multi-peak distribution. After dropping the constant term of the KL divergence, the KL divergence loss function is defined as follows:

[0262]

[0263] Among them, P(X|T) is the likelihood function of the posture, which satisfies P(X|T)∝P(T|X) and can be calculated by formula (11). Further define t(T)=δ(TT gt ), where δ(·) represents the Dirac delta function, so equation (22) is simplified to:

[0264]

[0265] Using Monte Carlo integration, we can get the final expression of the KL divergence loss of the pose:

[0266]

[0267] Among them, S(T gt ) represents the matching degree between the point cloud feature map and the map feature at the true value of the pose, which can be calculated by referring to the above formula (10).

[0268] The third loss It can indicate the difference between the predicted probability distribution of the true pose value and the true probability distribution of the true pose value. The predicted probability distribution of the true pose value is the probability distribution of multiple candidate pose offsets, that is, the probability distribution calculated by the above formula (11). The true probability distribution of the true pose value is a Dirac distribution (peaked distribution) with a probability of 1 at the true pose value.

[0269] According to some embodiments, the overall loss of the positioning model may be a first loss L rmse Second loss and the third loss The weighted sum of . That is:

[0270]

[0271] Among them, α1~α3 are the weights of the first loss to the third loss respectively.

[0272] After determining the overall loss of the localization model, the error back propagation method is used to adjust the parameters of the localization model according to the overall loss. The parameters of the localization model include semantic encoding, weights in the multi-layer perceptron, weights in the convolution kernel, etc.

[0273] It is understandable that steps S501-S510 may be executed repeatedly for multiple times until a preset termination condition is reached, such as when the overall loss is less than a loss threshold, when the number of cycles reaches a threshold, or when the overall loss converges.

[0274] The positioning model according to the embodiment of the present disclosure is an end-to-end model with a low number of parameters, easy to train and deploy, no need to modify the hyperparameters of the algorithm for different environments, and strong scalability. The positioning model can be applied to mass production of autonomous driving solutions to achieve high-precision vehicle positioning that only relies on lightweight vectorized maps.

[0275] The vehicle positioning method, vectorized map and positioning model of the disclosed embodiments can provide autonomous positioning functions for autonomous driving vehicles. Specific application areas include autonomous parking, assisted driving, cleaning and sanitation, automatic distribution, autonomous driving taxis, autonomous driving trucks, autonomous driving buses, etc.

[0276] According to an embodiment of the present disclosure, a vehicle positioning device is also provided. Figure 6 FIG. 6 shows a structural block diagram of a vehicle positioning device 600 according to an embodiment of the present disclosure. Figure 6 As shown, the device 600 includes an acquisition module 610 , a point cloud encoding module 620 , a map encoding module 630 , a determination module 640 and an overlay module 650 .

[0277] The acquisition module 610 is configured to acquire the initial position of the vehicle, a point cloud collected by a radar deployed on the vehicle, and a plurality of map elements for locating the vehicle.

[0278] The point cloud encoding module 620 is configured to encode the point cloud to obtain point cloud features.

[0279] The map encoding module 630 is configured to encode the plurality of map elements to obtain map features.

[0280] The determination module 640 is configured to determine a target pose offset for correcting the initial pose based on the point cloud features and the map features.

[0281] The superposition module 650 is configured to superimpose the initial posture with the target posture offset to obtain a corrected posture of the vehicle.

[0282] According to the embodiments of the present disclosure, the point cloud contains accurate depth information and can therefore be used to achieve high-precision positioning. Determining the target pose offset based on point cloud features and map features, and correcting the initial pose based on the target pose offset can improve the positioning accuracy of the vehicle and enable accurate positioning of the vehicle even in complex environments.

[0283] According to some embodiments, the point cloud encoding module includes: a first determination unit, configured to determine the spatial range for extracting the point cloud features based on the initial pose; a division unit, configured to divide the spatial range into multiple grids of unit size; an extraction unit, configured to extract the feature vector of the point cloud in any one of the multiple grids, wherein the point cloud features include a point cloud feature map composed of the feature vectors of each of the multiple grids, multiple pixels in the point cloud feature map correspond to the multiple grids respectively, and the number of channels of any one of the multiple pixels is equal to the dimension of the feature vector.

[0284] According to some embodiments, the map encoding module includes: an encoding unit, configured to: for any map element among the multiple map elements: encode the position information of the map element to obtain a position code; encode the category information of the map element to obtain a semantic code; and fuse the position code and the semantic code to obtain a coding vector of the map element; and a generation unit, configured to generate the map feature based on the respective coding vectors of the multiple map elements.

[0285] According to some embodiments, the encoding unit is further configured to: determine the semantic encoding of the map element based on the correspondence between multiple category information and multiple semantic encodings, wherein the multiple semantic encodings are parameters of a positioning model and are obtained by training the positioning model.

[0286] According to some embodiments, the generating unit is further configured to: for any map element among the multiple map elements, based on an attention mechanism, update the encoding vector of the map element using the point cloud feature, wherein the map feature includes the updated encoding vectors of each of the multiple map elements.

[0287] According to some embodiments, the determination module is further configured to determine the target pose offset by matching the point cloud features with the map features.

[0288] According to some embodiments, the determination module includes: a sampling unit, configured to perform sampling within a preset offset sampling range to obtain a plurality of candidate pose offsets; a second determination unit, configured to determine, for any one of the plurality of candidate pose offsets, a degree of matching between the point cloud feature and the map feature under the candidate pose offset; and a fusion unit, configured to fuse the plurality of candidate pose offsets based on the respective corresponding degrees of matching of the plurality of candidate pose offsets to obtain the target pose offset.

[0289] According to some embodiments, the point cloud feature includes a point cloud feature map of a target three-dimensional space, the map feature includes encoding vectors of each of the multiple map elements, and wherein the second determination unit is further configured to: superimpose the initial pose with the candidate pose offset to obtain a candidate pose; for any one of the multiple map elements: based on the candidate pose, project the map element into the target three-dimensional space to obtain a feature vector in the point cloud feature map corresponding to the map element; and calculate the similarity between the encoding vector of the map element and the feature vector; and based on the corresponding similarities of each of the multiple map elements, determine the matching degree between the point cloud feature and the map feature at the candidate pose offset.

[0290] According to some embodiments, the target three-dimensional space is a bird's-eye view space of the vehicle.

[0291] According to some embodiments, the fusion unit is further configured to: for any candidate pose offset among the multiple candidate pose offsets, determine the probability of the candidate pose offset based on the ratio of the matching degree of the candidate pose offset to the sum of the matching degrees of the multiple candidate pose offsets; and determine the expectation of the multiple candidate pose offsets as the target pose offset.

[0292] According to some embodiments, the determination module is further configured to: input the point cloud features, the map features and the initial pose into a trained pose solver to obtain the target pose offset output by the pose solver.

[0293] According to some embodiments, the plurality of map elements are obtained by screening a plurality of geographic elements in the vectorized map based on the initial pose.

[0294] According to some embodiments, the multiple map elements include at least one road element and at least one geometric element; the at least one road element includes at least one of the following: a lane line, a curb, a crosswalk, a stop line, a traffic sign or a pole; the at least one geometric element includes a surface element.

[0295] According to some embodiments, the surface element is obtained by extracting a plane in a point cloud map.

[0296] According to some embodiments, the initial posture is a posture output by a combined positioning system of the vehicle.

[0297] It should be understood that Figure 6 The various modules or units of the apparatus 600 shown in FIG. Figure 2 The steps in the described method 200 correspond to each other. Therefore, the operations, features and advantages described in the above method 200 are also applicable to the device 600 and the modules and units included therein. For the sake of brevity, some operations, features and advantages are not repeated here.

[0298] According to an embodiment of the present disclosure, a vectorized map construction device is also provided. Figure 7 FIG. 7 shows a structural block diagram of a vectorized map construction device 700 according to an embodiment of the present disclosure. Figure 7 As shown, the apparatus 700 includes an acquisition module 710 , a division module 720 , an extraction module 730 and a storage module 740 .

[0299] The acquisition module 710 is configured to acquire a point cloud in a point cloud map.

[0300] The division module 720 is configured to divide the projection plane of the point cloud map into a plurality of two-dimensional grids of a first unit size.

[0301] The extraction module 730 is configured to extract, for any two-dimensional grid among the multiple two-dimensional grids, a plane in the two-dimensional grid based on a point cloud in a three-dimensional space corresponding to the two-dimensional grid.

[0302] The storage module 740 is configured to store the plane as a surface element in the vectorized map.

[0303] According to an embodiment of the present disclosure, a plane is extracted from a point cloud map, and the extracted plane is stored as a surface element in a vectorized map, thereby improving the richness and density of the geographic elements in the vectorized map, thereby improving the positioning accuracy of the vehicle.

[0304] Compared with point cloud maps, vectorized maps are much smaller and easier to update. Storing vectorized maps (rather than point cloud maps) on the vehicle side can greatly reduce the storage cost of the vehicle side, improve the availability of vehicle positioning methods, and meet mass production needs. According to experimental verification, the volume of vectorized maps is about 0.35M / km. Compared with point cloud maps, the volume is reduced by 97.5%.

[0305] According to some embodiments, the extraction module includes: a division unit, configured to divide the three-dimensional space into multiple three-dimensional grids of a second unit size along the height direction; an extraction unit, configured to, for any three-dimensional grid among the multiple three-dimensional grids: based on the point cloud in the three-dimensional grid, calculate the confidence that the plane is included in the three-dimensional grid; and in response to the confidence being greater than a threshold, extract the plane in the three-dimensional grid; and a first determination unit, configured to determine the plane with the highest confidence among the multiple three-dimensional grids as the plane corresponding to the two-dimensional grid.

[0306] According to some embodiments, the extraction unit includes: a decomposition subunit, configured to perform singular value decomposition on the covariance matrix of the point cloud in the three-dimensional grid to obtain a first singular value, a second singular value and a third singular value, wherein the first singular value is less than or equal to the second singular value, and the second singular value is less than or equal to the third singular value; and a determination subunit, configured to determine the ratio of the second singular value to the first singular value as the confidence level.

[0307] According to some embodiments, the storage module includes: a second determination unit configured to determine the identifier of the surface element corresponding to the plane; and a storage unit configured to store the coordinates of a point on the plane and the unit normal vector of the plane in association with the identifier.

[0308] According to some embodiments, the vectorized map further includes a plurality of road elements, any of which is: a lane line, a curb, a crosswalk, a stop line, a traffic sign, or a pole.

[0309] It should be understood that Figure 7 The various modules or units of the apparatus 700 shown in FIG. Figure 4 The various steps in the described method 400 correspond to each other. Therefore, the operations, features and advantages described in the above method 400 are also applicable to the device 700 and the various modules and units included therein. For the sake of brevity, some operations, features and advantages are not repeated here.

[0310] According to an embodiment of the present disclosure, a training device for a positioning model is also provided. Figure 8 The structural block diagram of the training device 800 of the positioning model according to the embodiment of the present disclosure is shown. The positioning model includes a point cloud encoder, a map encoder and a pose solver.

[0311] like Figure 8As shown, the device 800 includes an acquisition module 801, a first input module 802, a second input module 803, a semantic projection module 804, a semantic prediction module 805, a third input module 806, a first determination module 807, a second determination module 808, a determination module 809 and an adjustment module 810.

[0312] The acquisition module 801 is configured to acquire the initial posture of the sample vehicle, the true posture value corresponding to the initial posture, the point cloud collected by the radar deployed on the sample vehicle, and multiple map elements used to locate the sample vehicle.

[0313] The first input module 802 is configured to input the point cloud into the point cloud encoder to obtain a point cloud feature map in the target three-dimensional space.

[0314] The second input module 803 is configured to input the category information of the multiple map elements into the map encoder so that the map encoder: determines the semantic coding corresponding to the category information of each map element in the multiple map elements based on the correspondence between the multiple category information and the multiple semantic codings, wherein the multiple semantic codings are parameters of the positioning model; determines the coding vector of the map element based on the semantic coding; and generates map features based on the respective coding vectors of the multiple map elements.

[0315] The semantic projection module 804 is configured to project the target map elements belonging to the target category among the multiple map elements into the target three-dimensional space to obtain a truth map of the semantic segmentation in the target three-dimensional space, wherein the value of the first pixel in the truth map indicates whether the first pixel is occupied by the target map element.

[0316] The semantic prediction module 805 is configured to determine a prediction map for semantic segmentation based on the point cloud feature map, wherein the value of a second pixel in the prediction map indicates the similarity between a corresponding feature vector and the semantic encoding of the target category, and the corresponding feature vector is a feature vector of a pixel in the point cloud feature map corresponding to the position of the second pixel.

[0317] The third input module 806 is configured to input the point cloud feature map, the map features and the initial pose into the pose solver to obtain the predicted pose offset output by the pose solver.

[0318] The first determination module 807 is configured to determine a first loss based on the predicted pose offset and a true value of the pose offset, wherein the true value of the pose offset is a difference between the true value of the pose and the initial pose.

[0319] The second determination module 808 is configured to determine a second loss based on the true value map and the prediction map.

[0320] The determination module 809 is configured to determine an overall loss of the positioning model based at least on the first loss and the second loss.

[0321] The adjustment module 810 is configured to adjust the parameters of the positioning model based on the overall loss.

[0322] According to the embodiments of the present disclosure, the first loss can guide the positioning model to output a more accurate predicted pose offset. The second loss can enhance the ability of semantic coding to express map element category information. By determining the overall loss of the positioning model based on the first loss and the second loss and adjusting the parameters of the positioning model accordingly, the positioning accuracy of the positioning model can be improved.

[0323] According to some embodiments, the third input module is further configured to: input the point cloud feature map, the map feature and the initial pose into the pose solver, so that the pose solver: performs sampling within a preset offset sampling range to obtain multiple candidate pose offsets; for any candidate pose offset among the multiple candidate pose offsets, determines the matching degree between the point cloud feature map and the map feature under the candidate pose offset; and based on the matching degrees corresponding to each of the multiple candidate pose offsets, fuses the multiple candidate pose offsets to obtain and output the predicted pose offset.

[0324] The device also includes: a third determination module, configured to determine a third loss based on the matching degrees corresponding to each of the multiple candidate pose offsets, wherein the third loss indicates the difference between the predicted probability distribution of the pose true value and the true probability distribution of the pose true value.

[0325] The determination module is further configured to determine the overall loss based on the first loss, the second loss, and the third loss.

[0326] It should be understood that Figure 8 The various modules or units of the apparatus 800 shown in FIG. Figure 5 The various steps in the described method 500 correspond to each other. Therefore, the operations, features and advantages described in the above method 500 are also applicable to the device 800 and the various modules and units included therein. For the sake of brevity, some operations, features and advantages are not repeated here.

[0327] Although specific functionality is discussed above with reference to specific modules, it should be noted that the functionality of the various modules discussed herein may be separated into multiple modules, and / or at least some functionality of multiple modules may be combined into a single module.

[0328] It should also be understood that various techniques may be described herein in the general context of software hardware elements or program modules. Figure 6-Figure 8 The various modules described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program codes / instructions, which are configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuits. For example, in some embodiments, one or more of modules 610-810 can be implemented together in a system on chip (System on Chip, SoC). SoC may include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or one or more components in other circuits), and may optionally execute the received program code and / or include embedded firmware to perform functions.

[0329] According to an embodiment of the present disclosure, there is also provided an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the vehicle positioning method, the vectorized map construction method, and the positioning model training method of the embodiment of the present disclosure.

[0330] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to enable a computer to execute any one of the vehicle positioning method, vectorized map construction method, and positioning model training method according to the embodiment of the present disclosure.

[0331] According to an embodiment of the present disclosure, a computer program product is also provided, comprising computer program instructions, wherein the computer program instructions, when executed by a processor, implement any one of the vehicle positioning method, vectorized map construction method, and positioning model training method according to the embodiment of the present disclosure.

[0332] According to an embodiment of the present disclosure, there is also provided an autonomous driving vehicle, comprising the above-mentioned electronic device.

[0333] refer to Fig. 9, a block diagram of an electronic device 900 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0334] like Fig. 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0335] Multiple components in the electronic device 900 are connected to the I / O interface 905, including: an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. The input unit 906 can be any type of device that can input information to the electronic device 900. The input unit 906 can receive input digital or character information and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 907 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 908 can include but is not limited to a disk, an optical disk. The communication unit 909 allows the electronic device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, an 802.11 device, a Wi-Fi device, a WiMAX device, a cellular communication device, and / or the like.

[0336] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as methods 200, 400, and 500. For example, in some embodiments, methods 200, 400, and 500 may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method 200 described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the methods 200 , 400 , and 500 in any other appropriate manner (eg, by means of firmware).

[0337] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0338] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0339] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0340] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0341] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0342] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0343] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0344] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present disclosure is not limited by these embodiments or examples, but is only limited by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples may be omitted or replaced by their equivalent elements. In addition, each step may be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples may be combined in various ways. It is important that with the evolution of technology, many of the elements described herein may be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A vehicle positioning method, comprising: Acquire an initial position of the vehicle, a point cloud collected by a radar deployed on the vehicle, and a plurality of map elements for locating the vehicle; Encoding the point cloud to obtain point cloud features; Encoding the plurality of map elements to obtain map features; Determining a target posture offset for correcting the initial posture based on the point cloud features and the map features includes: Sampling is performed within a preset offset sampling range to obtain multiple candidate pose offsets; For any candidate pose offset among the plurality of candidate pose offsets, determining a degree of matching between the point cloud feature and the map feature at the candidate pose offset; and Based on the matching degrees corresponding to the multiple candidate pose offsets, the multiple candidate pose offsets are fused to obtain the target pose offset; as well as The initial posture is superimposed with the target posture offset to obtain a corrected posture of the vehicle.

2. The method according to claim 1, wherein: The step of encoding the point cloud to obtain point cloud features includes: Based on the initial pose, determining a spatial range for extracting the point cloud features; Dividing the spatial range into a plurality of grids of unit size; For any grid among the plurality of grids, extracting a feature vector of a point cloud in the grid, Among them, the point cloud feature includes a point cloud feature map composed of feature vectors of each of the multiple grids, multiple pixels in the point cloud feature map correspond to the multiple grids respectively, and the number of channels of any pixel in the multiple pixels is equal to the dimension of the feature vector.

3. The method according to claim 1 or 2, wherein: The encoding of the plurality of map elements to obtain map features comprises: For any map element among the multiple map elements: Encoding the position information of the map element to obtain a position code; Encoding the category information of the map element to obtain a semantic code; and fusing the position code and the semantic code to obtain a code vector of the map element; as well as The map feature is generated based on the encoding vectors of each of the plurality of map elements.

4. The method according to claim 3, wherein: The encoding of the category information of the map element to obtain a semantic code includes: The semantic coding of the map element is determined based on the correspondence between multiple types of category information and multiple semantic codings, wherein the multiple semantic codings are parameters of a positioning model and are obtained by training the positioning model.

5. The method according to claim 3, wherein: The generating the map feature based on the respective encoding vectors of the plurality of map elements comprises: For any map element among the plurality of map elements, based on an attention mechanism, the encoding vector of the map element is updated by using the point cloud feature, The map feature includes updated encoding vectors of each of the plurality of map elements.

6. The method according to claim 1, wherein: The point cloud feature includes a point cloud feature map of the target three-dimensional space, the map feature includes encoding vectors of each of the plurality of map elements, and wherein determining the degree of matching between the point cloud feature and the map feature at the candidate pose offset includes: Superimposing the initial pose and the candidate pose offset to obtain a candidate pose; For any map element among the multiple map elements: Based on the candidate pose, projecting the map element into the target three-dimensional space to obtain a feature vector in the point cloud feature map corresponding to the map element; and Calculating the similarity between the encoding vector of the map element and the feature vector; as well as Based on the similarities corresponding to the multiple map elements, the matching degree between the point cloud feature and the map feature at the candidate pose offset is determined.

7. The method according to claim 6, wherein: The target three-dimensional space is the bird's-eye view space of the vehicle.

8. The method according to claim 1, wherein: The fusing the multiple candidate pose offsets based on the respective corresponding matching degrees of the multiple candidate pose offsets to obtain the target pose offset comprises: For any candidate pose offset among the plurality of candidate pose offsets, determining a probability of the candidate pose offset based on a ratio of a matching degree of the candidate pose offset to a sum of matching degrees of the plurality of candidate pose offsets; and The desired value of the plurality of candidate pose offsets is determined as the target pose offset.

9. The method according to claim 1, wherein: The determining, based on the point cloud features and the map features, of a target posture offset for correcting the initial posture comprises: The point cloud features, the map features and the initial pose are input into a trained pose solver to obtain the target pose offset output by the pose solver.

10. The method according to claim 1, wherein: The multiple map elements are obtained by screening multiple geographic elements in the vectorized map based on the initial pose.

11. The method according to claim 1, wherein: The plurality of map elements include at least one road element and at least one geometric element; The at least one road element comprises at least one of the following: a lane line, a curb, a crosswalk, a stop line, a traffic sign or a pole; The at least one geometric element comprises a surface element.

12. The method according to claim 11, wherein: The surface element is obtained by extracting a plane from a point cloud map.

13. The method according to claim 1, wherein: The initial position and posture is the position and posture output by the combined positioning system of the vehicle.

14. The method according to claim 10, wherein: The vectorized map includes surface elements, which are obtained by the following operations: Get the point cloud in the point cloud map; Dividing the projection plane of the point cloud map into a plurality of two-dimensional grids of a first unit size; For any two-dimensional grid in the plurality of two-dimensional grids: Extracting a plane in the two-dimensional grid based on a point cloud in a three-dimensional space corresponding to the two-dimensional grid; and The plane is stored as a surface element in the vectorized map.

15. The method according to claim 14, wherein: The extracting the plane in the two-dimensional grid based on the point cloud in the three-dimensional space corresponding to the two-dimensional grid comprises: Dividing the three-dimensional space into a plurality of three-dimensional grids of a second unit size along a height direction; For any three-dimensional grid of the plurality of three-dimensional grids: Calculating the confidence that the three-dimensional grid includes a plane based on the point cloud in the three-dimensional grid; and In response to the confidence being greater than a threshold, extracting a plane in the three-dimensional grid; as well as A plane with the highest confidence in the multiple three-dimensional grids is determined as the plane corresponding to the two-dimensional grid.

16. The method according to claim 15, wherein: The calculating the confidence of the plane included in the three-dimensional grid based on the point cloud in the three-dimensional grid comprises: Performing singular value decomposition on the covariance matrix of the point cloud in the three-dimensional grid to obtain a first singular value, a second singular value, and a third singular value, wherein the first singular value is less than or equal to the second singular value, and the second singular value is less than or equal to the third singular value; and A ratio of the second singular value to the first singular value is determined as the confidence level.

17. The method according to any one of claims 14 to 16, wherein: The storing the plane as a surface element in the vectorized map comprises: Determining the identity of the surface element corresponding to the plane; and The coordinates of a point on the plane and the unit normal vector of the plane are stored in association with the identifier.

18. The method according to claim 14, wherein: The vectorized map further includes a plurality of road elements, any of which is: a lane line, a curb, a crosswalk, a stop line, a traffic sign or a pole.

19. A method for training a positioning model, wherein: The positioning model is used to implement the vehicle positioning method according to any one of claims 1 to 18, the positioning model includes a point cloud encoder, a map encoder and a pose solver, and the training method includes: Acquire an initial posture of a sample vehicle, a true posture value corresponding to the initial posture, a point cloud collected by a radar deployed on the sample vehicle, and a plurality of map elements used to locate the sample vehicle; Inputting the point cloud into the point cloud encoder to obtain a point cloud feature map in the target three-dimensional space; Inputting the category information of each of the plurality of map elements into the map encoder so that the map encoder: Determining the semantic code corresponding to the category information of each of the plurality of map elements based on the correspondence between the plurality of category information and the plurality of semantic codes, wherein the plurality of semantic codes are parameters of the positioning model; Determining a coding vector of the map element based on the semantic coding; and generating a map feature based on the encoding vectors of each of the plurality of map elements; Projecting a target map element belonging to a target category among the multiple map elements into the target three-dimensional space to obtain a truth map of semantic segmentation in the target three-dimensional space, wherein a value of a first pixel in the truth map indicates whether the first pixel is occupied by the target map element; Based on the point cloud feature map, determining a prediction map for semantic segmentation, wherein a value of a second pixel in the prediction map indicates a similarity between a corresponding feature vector and a semantic encoding of the target category, and the corresponding feature vector is a feature vector of a pixel in the point cloud feature map corresponding to a position of the second pixel; Inputting the point cloud feature map, the map features and the initial pose into the pose solver to obtain a predicted pose offset output by the pose solver; Determine a first loss based on the predicted pose offset and a true value of the pose offset, wherein the true value of the pose offset is a difference between the true value of the pose and the initial pose; Determining a second loss based on the true value map and the predicted map; determining an overall loss of the positioning model based at least on the first loss and the second loss; and Based on the overall loss, parameters of the localization model are adjusted.

20. The method according to claim 19, wherein: The step of inputting the point cloud feature map, the map feature and the initial pose into the pose solver to obtain a predicted pose offset output by the pose solver comprises: The point cloud feature map, the map features and the initial pose are input into the pose solver so that the pose solver: Sampling is performed within a preset offset sampling range to obtain multiple candidate pose offsets; For any candidate pose offset among the plurality of candidate pose offsets, determining a degree of matching between the point cloud feature map and the map feature at the candidate pose offset; and Based on the matching degrees corresponding to the multiple candidate pose offsets, the multiple candidate pose offsets are fused to obtain and output the predicted pose offset; The method further comprises: Determine a third loss based on the matching degrees corresponding to each of the plurality of candidate pose offsets, wherein the third loss indicates a difference between a predicted probability distribution of the true pose value and a true probability distribution of the true pose value; The determining of the overall loss of the positioning model based at least on the first loss and the second loss comprises: The overall loss is determined based on the first loss, the second loss, and the third loss.

21. A vehicle positioning device, comprising: A first acquisition module is configured to acquire an initial position of the vehicle, a point cloud collected by a radar deployed on the vehicle, and a plurality of map elements for locating the vehicle; A point cloud encoding module is configured to encode the point cloud to obtain point cloud features; A map encoding module, configured to encode the plurality of map elements to obtain map features; A determination module is configured to determine a target posture offset for correcting the initial posture based on the point cloud features and the map features, the determination module comprising: A sampling unit is configured to perform sampling within a preset offset sampling range to obtain a plurality of candidate pose offsets; A second determining unit is configured to determine, for any candidate pose offset among the plurality of candidate pose offsets, a degree of matching between the point cloud feature and the map feature under the candidate pose offset; and a fusion unit configured to fuse the plurality of candidate pose offsets based on the respective matching degrees corresponding to the plurality of candidate pose offsets to obtain the target pose offset; and The superposition module is configured to superimpose the initial posture with the target posture offset to obtain a corrected posture of the vehicle.

22. The device according to claim 21, wherein The point cloud encoding module includes: A first determining unit is configured to determine a spatial range for extracting the point cloud features based on the initial pose; A first division unit is configured to divide the spatial range into a plurality of grids of unit size; A first extraction unit is configured to extract a feature vector of a point cloud in any grid among the plurality of grids, Among them, the point cloud feature includes a point cloud feature map composed of feature vectors of each of the multiple grids, multiple pixels in the point cloud feature map correspond to the multiple grids respectively, and the number of channels of any pixel in the multiple pixels is equal to the dimension of the feature vector.

23. The device according to claim 21 or 22, wherein: The map encoding module includes: The encoding unit is configured to: for any map element among the multiple map elements: Encoding the position information of the map element to obtain a position code; Encoding the category information of the map element to obtain a semantic code; and fusing the position code and the semantic code to obtain a code vector of the map element; as well as The generating unit is configured to generate the map feature based on the encoding vectors of the respective map elements.

24. The device according to claim 23, wherein: The encoding unit is further configured to: The semantic coding of the map element is determined based on the correspondence between multiple types of category information and multiple semantic codings, wherein the multiple semantic codings are parameters of a positioning model and are obtained by training the positioning model.

25. The device according to claim 23, wherein: The generating unit is further configured to: For any map element among the plurality of map elements, based on an attention mechanism, the encoding vector of the map element is updated by using the point cloud feature, The map feature includes updated encoding vectors of each of the plurality of map elements.

26. The device according to claim 21, wherein The point cloud feature includes a point cloud feature map of the target three-dimensional space, the map feature includes encoding vectors of each of the plurality of map elements, and wherein the second determining unit is further configured to: Superimposing the initial pose and the candidate pose offset to obtain a candidate pose; For any map element among the multiple map elements: Based on the candidate pose, projecting the map element into the target three-dimensional space to obtain a feature vector in the point cloud feature map corresponding to the map element; and Calculating the similarity between the encoding vector of the map element and the feature vector; as well as Based on the similarities corresponding to the multiple map elements, the matching degree between the point cloud feature and the map feature at the candidate pose offset is determined.

27. The device according to claim 26, wherein: The target three-dimensional space is the bird's-eye view space of the vehicle.

28. The device according to claim 21, wherein The fusion unit is further configured to: For any candidate pose offset among the multiple candidate pose offsets, determining a probability of the candidate pose offset based on a ratio of a matching degree of the candidate pose offset to a sum of matching degrees of the multiple candidate pose offsets; as well as The desired value of the plurality of candidate pose offsets is determined as the target pose offset.

29. The device according to claim 21, wherein The determination module is further configured to: The point cloud features, the map features and the initial pose are input into a trained pose solver to obtain the target pose offset output by the pose solver.

30. The device according to claim 21, wherein The multiple map elements are obtained by screening multiple geographic elements in the vectorized map based on the initial pose.

31. The device according to claim 21, wherein The plurality of map elements include at least one road element and at least one geometric element; The at least one road element comprises at least one of the following: a lane line, a curb, a crosswalk, a stop line, a traffic sign or a pole; The at least one geometric element comprises a surface element.

32. The device according to claim 31, wherein The surface element is obtained by extracting a plane from a point cloud map.

33. The apparatus of claim 21, wherein: The initial position and posture is the position and posture output by the combined positioning system of the vehicle.

34. The device according to claim 30, wherein: The vectorized map includes surface elements, which are obtained through the following modules: A second acquisition module is configured to acquire a point cloud in a point cloud map; A division module, configured to divide the projection plane of the point cloud map into a plurality of two-dimensional grids of a first unit size; An extraction module is configured to extract, for any two-dimensional grid among the plurality of two-dimensional grids, a plane in the two-dimensional grid based on a point cloud in a three-dimensional space corresponding to the two-dimensional grid; as well as The storage module is configured to store the plane as a surface element in the vectorized map.

35. The device according to claim 34, wherein The extraction module comprises: A second dividing unit is configured to divide the three-dimensional space into a plurality of three-dimensional grids of a second unit size along a height direction; The second extraction unit is configured to: for any three-dimensional grid among the plurality of three-dimensional grids: Calculating the confidence that the three-dimensional grid includes a plane based on the point cloud in the three-dimensional grid; and In response to the confidence being greater than a threshold, extracting a plane in the three-dimensional grid; as well as The third determining unit is configured to determine a plane with the highest confidence in the multiple three-dimensional grids as a plane corresponding to the two-dimensional grid.

36. The device according to claim 35, wherein The second extraction unit comprises: a decomposition subunit configured to perform singular value decomposition on a covariance matrix of the point cloud in the three-dimensional grid to obtain a first singular value, a second singular value, and a third singular value, wherein the first singular value is less than or equal to the second singular value, and the second singular value is less than or equal to the third singular value; and A determination subunit is configured to determine a ratio of the second singular value to the first singular value as the confidence level.

37. The device according to any one of claims 34 to 36, wherein: The storage module comprises: a fourth determining unit, configured to determine an identifier of a surface element corresponding to the plane; and The storage unit is configured to store the coordinates of a point on the plane and the unit normal vector of the plane in association with the identifier.

38. The device according to claim 34, wherein: The vectorized map further includes a plurality of road elements, any of which is: a lane line, a curb, a crosswalk, a stop line, a traffic sign or a pole.

39. A training device for a positioning model, wherein: The positioning model is used to implement the vehicle positioning device according to any one of claims 21 to 38, the positioning model includes a point cloud encoder, a map encoder and a pose solver, and the training device includes: An acquisition module is configured to acquire an initial posture of a sample vehicle, a true posture value corresponding to the initial posture, a point cloud collected by a radar deployed on the sample vehicle, and a plurality of map elements used to locate the sample vehicle; A first input module is configured to input the point cloud into the point cloud encoder to obtain a point cloud feature map in a target three-dimensional space; The second input module is configured to input the category information of the plurality of map elements into the map encoder so that the map encoder: Determining the semantic code corresponding to the category information of each of the plurality of map elements based on the correspondence between the plurality of category information and the plurality of semantic codes, wherein the plurality of semantic codes are parameters of the positioning model; Determining a coding vector of the map element based on the semantic coding; and generating a map feature based on the encoding vectors of each of the plurality of map elements; a semantic projection module configured to project a target map element belonging to a target category among the plurality of map elements into the target three-dimensional space to obtain a truth map of semantic segmentation in the target three-dimensional space, wherein a value of a first pixel in the truth map indicates whether the first pixel is occupied by the target map element; A semantic prediction module is configured to determine a prediction map for semantic segmentation based on the point cloud feature map, wherein a value of a second pixel in the prediction map indicates a similarity between a corresponding feature vector and a semantic encoding of the target category, and the corresponding feature vector is a feature vector of a pixel in the point cloud feature map corresponding to a position of the second pixel; A third input module is configured to input the point cloud feature map, the map feature and the initial pose into the pose solver to obtain a predicted pose offset output by the pose solver; a first determination module is configured to determine a first loss based on the predicted pose offset and a true value of the pose offset, wherein the true value of the pose offset is a difference between the true value of the pose and the initial pose; a second determination module configured to determine a second loss based on the truth map and the prediction map; a determination module configured to determine an overall loss of the positioning model based at least on the first loss and the second loss; and The adjustment module is configured to adjust the parameters of the positioning model based on the overall loss.

40. The device according to claim 39, wherein The third input module is further configured to: The point cloud feature map, the map features and the initial pose are input into the pose solver so that the pose solver: Sampling is performed within a preset offset sampling range to obtain multiple candidate pose offsets; For any candidate pose offset among the multiple candidate pose offsets, determining a matching degree between the point cloud feature map and the map feature under the candidate pose offset; as well as Based on the matching degrees corresponding to the multiple candidate pose offsets, the multiple candidate pose offsets are fused to obtain and output the predicted pose offset; The device also includes: A third determination module is configured to determine a third loss based on the matching degrees corresponding to each of the plurality of candidate pose offsets, wherein the third loss indicates a difference between a predicted probability distribution of the pose truth value and a true probability distribution of the pose truth value; Wherein, the determining module is further configured to: The overall loss is determined based on the first loss, the second loss, and the third loss.

41. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 20.

42. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-20.

43. A computer program product comprising computer program instructions, wherein: The computer program instructions implement the method of any one of claims 1-20 when executed by a processor.

44. An autonomous driving vehicle comprising an electronic device according to claim 41.

Images