Vehicle positioning method, medium, product, electronic equipment and vehicle

Abstract

The invention relates to a vehicle positioning method, a medium, a product, electronic equipment and a vehicle, and the method comprises the steps: determining the pose information of the vehicle according to the driving information of the vehicle, the driving information being the information outputted by a vehicle driving module; and obtaining positioning of the vehicle according to the pose information and the scene information of the vehicle. According to the method, the predicted value of the pose information is obtained through the driving information of the vehicle, and the scene information is processed by using the predicted value of the pose information, so that the vehicle positioning precision and the system reliability are improved, the method can adapt to various complex environments such as tunnels and underground parking lots, and even under the condition of weak GPS signals or low-speed driving, the vehicle positioning accuracy is improved. And reliable positioning can also be provided.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a vehicle positioning method, medium, product, electronic device, and vehicle. Background Technology

[0002] With the rapid development of autonomous driving, robot navigation, and smart mobile devices, the demand for vehicle positioning technology is becoming increasingly urgent. Common vehicle positioning relies on wheel speed information; however, in some low-speed driving or slipping scenarios, the resolution of wheel speed information is insufficient or the error is large, resulting in poor positioning performance. Summary of the Invention

[0003] This application provides a vehicle positioning method, medium, product, electronic device, and vehicle to solve the above-mentioned problems.

[0004] To achieve the above objectives, according to a first aspect of this application, a vehicle positioning method is provided, the method comprising:

[0005] The vehicle's pose information is determined based on the vehicle's driving information, which is the information output by the vehicle's driving module.

[0006] The vehicle's location is obtained based on the pose information and the scene information where the vehicle is located.

[0007] Optionally, obtaining the vehicle's location based on the pose information and the scene information in which the vehicle is located includes:

[0008] Motion compensation is performed on the scene information based on the pose information to obtain the compensated scene information, thereby obtaining the positioning of the vehicle.

[0009] Optionally, the scene information includes data points from multiple scanning times.

[0010] The step of performing motion compensation on the scene information based on the pose information to obtain the compensated scene information includes:

[0011] Based on the pose information within a preset time period, the data points at the multiple scanning times are compensated to obtain the compensated scene information.

[0012] Optionally, the preset time period is the scanning duration for acquiring the scene information.

[0013] Optionally, the step of compensating the data points at the multiple scanning times based on the pose information within a preset time period to obtain the compensated scene information includes:

[0014] Based on the pose information within the preset time period, the data points of each scanning moment are projected onto the target moment to obtain the compensated scene information.

[0015] Optionally, the target time is the last scan time within the preset time period.

[0016] Optionally, the step of projecting the data points of each scanning moment onto the target moment based on the pose information within the preset time period to obtain the compensated scene information includes:

[0017] Based on the pose information within the preset time period, determine the projection transformation matrix that projects the data points at each scanning moment to the target moment;

[0018] Based on the projection transformation matrix corresponding to each scanning time, the data points of each scanning time are projected onto the target time to obtain the compensated scene information.

[0019] Optionally, the projection transformation matrix is ​​a matrix in the vehicle coordinate system.

[0020] The step of projecting the data points of each scanning time onto the target time according to the projection transformation matrix corresponding to each scanning time to obtain the compensated scene information includes:

[0021] Based on the data points at each scanning time in the vehicle coordinate system and the projection transformation matrix corresponding to each scanning time, data points at multiple target times in the vehicle coordinate system are obtained to obtain the compensated scene information.

[0022] Optionally, the method further includes:

[0023] The data points at each scanning time in the original coordinate system are projected to the vehicle coordinate system to obtain the data points at each scanning time in the vehicle coordinate system.

[0024] Optionally, obtaining the compensated scene information includes:

[0025] The data points at each target time in the vehicle coordinate system are projected onto the original coordinate system to obtain multiple data points at target times in the original coordinate system, which are used as the compensated scene information.

[0026] Optionally, obtaining the location of the vehicle includes:

[0027] Based on the compensated scene information, determine the filtering parameters of the preset filter;

[0028] The inertial motion information of the vehicle is processed according to the filtering parameters to obtain the vehicle's positioning.

[0029] Optionally, determining the filtering parameters of the preset filter based on the compensated scene information includes:

[0030] Based on the compensated scene information, residual information is obtained to determine the filtering parameters.

[0031] Optionally, obtaining residual information based on the compensated scene information includes:

[0032] Multiple compensated data points in the compensated scene information are projected from the original coordinate system to the reference coordinate system to obtain reference scene information;

[0033] The residual information is obtained based on the reference scenario information.

[0034] Optionally, obtaining the residual information based on the reference scene information includes:

[0035] For each reference data point in the reference scene information, determine the nearest plane of the reference data point in the reference coordinate system;

[0036] The residual information is obtained based on the distance between the reference data point and the adjacent plane corresponding to the reference data point.

[0037] Optionally, determining the filtering parameters includes:

[0038] The filtering parameters are determined based on residual information that is less than or equal to a preset threshold.

[0039] Optionally, determining the filtering parameters includes:

[0040] Based on the residual information, the prior estimate of the filter parameters is corrected to obtain the posterior estimate of the filter parameters.

[0041] Optionally, the prior estimation of the filtering parameters includes a prior estimation of the nominal state vector, and the posterior estimation of the filtering parameters includes a posterior estimation of the error state vector.

[0042] Optionally, processing the inertial motion information of the vehicle according to the filtering parameters to obtain the vehicle's positioning includes:

[0043] Based on the inertial motion information of the vehicle, a priori estimate of the nominal state vector is obtained;

[0044] The prior estimate of the nominal state vector is corrected based on the posterior estimate of the error state vector to obtain the posterior estimate of the nominal state vector, thereby obtaining the positioning of the vehicle.

[0045] Optionally, obtaining the prior estimate of the nominal state vector based on the vehicle's inertial motion information includes:

[0046] The inertial motion information is pre-integrated, and based on the result of the pre-integration and the preset nominal state vector, a prior estimate of the nominal state vector at the target time is obtained.

[0047] Optionally, the nominal state vector includes at least one of the following: the vehicle's position, velocity, attitude, bias of the angular velocity measurement value of the inertial measurement unit, bias of the acceleration measurement value of the inertial measurement unit, and gravitational acceleration.

[0048] Optionally, the error state vector is determined based on the errors of each item in the nominal state vector.

[0049] Optionally, determining the vehicle's pose information based on the vehicle's driving information includes:

[0050] The vehicle's position and orientation information are determined based on the vehicle's driving information and a preset transmission ratio.

[0051] Optionally, determining the vehicle's pose information based on the vehicle's drive information and a preset transmission ratio includes:

[0052] The vehicle speed information is obtained based on the driving information and the preset transmission ratio;

[0053] Based on the vehicle's inertial motion information and speed information, the vehicle's pose information is determined.

[0054] Optionally, the method further includes:

[0055] If the vehicle speed information is less than a preset speed threshold, the step of determining the vehicle's pose information based on the vehicle's inertial motion information and the speed information is executed.

[0056] Optionally, the drive information includes at least one of the motor speed and the engine speed.

[0057] Optionally, the scene information includes point cloud data or visual data of the environment surrounding the vehicle.

[0058] According to a second aspect of this application, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, causes the computer to implement any of the vehicle positioning methods provided in the embodiments of this application.

[0059] According to a third aspect of this application, embodiments of this application also provide a computer program product storing instructions that, when executed by a computer, cause the computer to implement any of the vehicle positioning methods described in the embodiments of this application.

[0060] According to a fourth aspect of this application, embodiments of this application also provide an electronic device, comprising:

[0061] A memory on which computer programs are stored;

[0062] A processor is configured to execute the computer program in the memory to implement any of the vehicle positioning methods provided in the embodiments of this application.

[0063] According to a fifth aspect of this application, embodiments of this application also provide a vehicle, including the aforementioned electronic device, or for performing the aforementioned vehicle positioning method.

[0064] Some embodiments in this specification include at least the following beneficial effects: By utilizing the driving information (such as wheel speed, steering angle, etc.) output by the vehicle drive module, the vehicle's motion state and trend can be directly reflected, which helps to provide relatively stable pose information. Vehicle positioning based on the vehicle's own motion characteristics has stronger environmental adaptability and reliability compared to methods relying on external signals (such as GPS) or single sensors (such as vision, LiDAR). Simultaneously, by combining the vehicle's location scene information (such as road type, surrounding environmental features, etc.), the positioning results are further optimized, enabling it to maintain high accuracy in complex scenarios such as urban canyons, tunnels, and underground parking lots. This eliminates the need for additional auxiliary positioning equipment, reducing system costs and improving the practicality and economy of the positioning system.

[0065] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.

[0068] Figure 1These are application scenario diagrams of the vehicle positioning method shown in some embodiments of this specification;

[0069] Figure 2 This is an exemplary flowchart of a vehicle positioning method according to some embodiments of this specification;

[0070] Figure 3 This is an exemplary flowchart of motion compensation according to some embodiments of this specification;

[0071] Figure 4 This is an exemplary flowchart illustrating the determination of vehicle pose information according to some embodiments of this specification;

[0072] Figure 5 This is an exemplary schematic diagram of a vehicle positioning method according to some embodiments of this specification;

[0073] Figure 6 This is a schematic diagram of the structure of an electronic device according to some embodiments of this specification;

[0074] Figure 7 This is an exemplary schematic diagram of a vehicle provided in an exemplary embodiment of this application. Detailed Implementation

[0075] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0076] To facilitate understanding of the implementation schemes provided in this application, the relevant application background of the vehicle positioning method provided in this application will be explained first.

[0077] Currently, in fields such as intelligent transportation systems, autonomous driving, and robotics, there is a need for accurate estimation of vehicle attitude, speed, and position. Single sensors often suffer from problems such as high measurement noise and susceptibility to environmental interference, making it difficult to meet the demands of high-precision positioning. For example, inertial measurement unit (IMU) measurements are subject to random walk bias, and the attitude calculated through integration suffers from cumulative errors, resulting in low reliability of long-term integration results. LiDAR sensors can provide high-precision, dense 3D point clouds, but scene information within different frames can exhibit motion distortion, affecting positioning accuracy. Wheel speed sensors measure wheel rotation speed, indirectly estimating vehicle speed, but are easily affected by factors such as tire slippage.

[0078] In the fusion method of related technologies, a pose information is obtained by matching the point cloud of LiDAR, and then the obtained pose information is fused with the data of IMU and odometry. In the case of errors in dynamic or degenerate environments, the erroneous matched pose is used for fusion, which can easily lead to unreliable positioning and poor accuracy.

[0079] Therefore, some embodiments of this specification provide a vehicle positioning method that fuses LiDAR, IMU, and vehicle driving information using tight coupling. In practical applications, the LiDAR needs to accumulate a complete frame of scene information to constitute an effective measurement, which can lead to significant distortion introduced by vehicle motion during data acquisition. To reduce the impact of this distortion, motion compensation needs to be performed using the IMU and driving information after the LiDAR has scanned a complete frame of scene information. Specifically, during the scanning duration (t... k-1 , t k Within [the specified context], only IMU sensor data and driving information are fused. When the lidar is at t k After scanning a frame of scene information at each step, motion compensation is performed on the scene information using the fusion result, transforming all feature points to t. k This real-time positioning system not only effectively reduces motion distortion and improves the quality of scene information, but also further optimizes positioning by integrating IMU and driving information. It has advantages such as high computational efficiency and good real-time performance, and is suitable for positioning and navigation in dynamic environments and complex scenarios.

[0080] Figure 1 This is an application scenario diagram of the vehicle positioning method shown in some embodiments of this specification.

[0081] The vehicle positioning method provided in this application can be applied to vehicle positioning scenarios, as well as other application scenarios, such as vehicle navigation.

[0082] In some feasible embodiments, the vehicles involved in the embodiments of this application can be autonomous vehicles or non-autonomous vehicles. Autonomous vehicles, also known as driverless cars, computer-driven cars, or wheeled mobile robots, are intelligent vehicles that achieve driverless operation through computer systems. In practical applications, autonomous vehicles rely on the collaborative efforts of artificial intelligence, computer vision, radar, monitoring devices, and global positioning systems to enable computer equipment to automatically and safely operate motor vehicles without any active human intervention.

[0083] The vehicle positioning method provided in this application can be implemented by an electronic device in a vehicle, or by a vehicle positioning device in the aforementioned electronic device. For example, the vehicle positioning device can be implemented using software and / or hardware.

[0084] The electronic devices mentioned above in the embodiments of this application may include, but are not limited to, the main control computer (or industrial control computer) in the vehicle.

[0085] In this embodiment, the vehicle may include at least a sensing system. The sensing system is used to detect the vehicle's state. Specifically, the sensors may include internal sensors and external sensors; the internal sensors, used to monitor the vehicle's state, may include at least one of the following: a vehicle speed sensor, an acceleration sensor, an angular velocity sensor, a chassis motor sensor, etc. The external sensors are mainly used to monitor the external environment surrounding the vehicle. For example, they may include a radar device, which acquires and monitors electromagnetic wave data of the vehicle's surrounding environment. This is mainly achieved by emitting electromagnetic waves and then receiving electromagnetic waves reflected from surrounding objects to detect data such as the distance between surrounding objects and the vehicle, and the shape of surrounding objects.

[0086] For example, multiple radar units are distributed throughout the exterior of the vehicle. Multiple radar units are coupled to the front of the vehicle to locate objects in front of the vehicle. One or more other radar units may be located at the rear of the vehicle to locate objects behind the vehicle as the vehicle reverses. Other radar units may be located on the sides of the vehicle to locate objects approaching the vehicle from the sides, such as other vehicles. For example, the radar units are LiDAR (light detection and ranging) sensors, which can be mounted on the vehicle, for example, in a rotating structure mounted on the roof of the vehicle. Rotating the LiDAR sensor allows it to transmit light signals around the vehicle in a 360° pattern, thus continuously mapping all objects around the vehicle as it moves.

[0087] In some embodiments, the sensing system may further include: an inertial measurement unit (IMU), etc., wherein,

[0088] Inertial measurement unit (IMU) sensors can be used to sense changes in a vehicle's position and orientation based on inertial acceleration. In one embodiment, the IMU sensor can be a combination of accelerometers and gyroscopes. Specifically, the IMU sensor may include three single-axis accelerometers and three single-axis gyroscopes, where the accelerometers detect the vehicle's independent three-axis acceleration signals in the vehicle coordinate system, and the gyroscopes detect the angular velocity (or attitude angle) signals of the vehicle relative to the navigation coordinate system. In other words, the IMU sensor can measure the vehicle's rotational angular rate and linear acceleration in three-dimensional space. Information such as the vehicle's attitude, velocity, and displacement can be calculated from these signals. Specifically, relying on the IMU sensor, the vehicle's position and orientation at the current moment can be inferred from its previous position and orientation. The IMU sensor measures the route the vehicle has traveled relative to its starting point; therefore, the IMU sensor provides a relative positioning. Here, the vehicle coordinate system is a coordinate system centered on the vehicle, used to describe the relationship between the vehicle and surrounding objects, with its origin fixed to the vehicle. The navigation coordinate system can be a ground-fixed coordinate system, a geographic coordinate system (also known as a local horizontal coordinate system), etc. Commonly used geographic coordinate systems include the "Northeast-Sky" coordinate system and the "Northeast-Earth" coordinate system. The vehicle's attitude angles include roll angle, pitch angle, and yaw angle, all defined in the geographic coordinate system. It should be understood that different coordinate systems have definite relationships; one coordinate system can be transformed into another through a series of rotations and translations, which will not be detailed here. In the embodiments of this application, the positioning technology based on data measured by IMU sensors belongs to inertial positioning technology.

[0089] A chassis motor sensor is a device used to measure information related to the drive motor and project it onto a measurable electrical signal. In some embodiments, the chassis motor sensor can output a speed signal of the drive motor on the chassis.

[0090] The processing unit of the aforementioned sensor outputs data in real time. The data is transmitted to the vehicle positioning device (such as an embedded platform) via a wired method (such as a serial port, Ethernet port, or Controller Area Network (CAN) bus). The vehicle positioning device obtains the vehicle's location using the vehicle positioning method proposed in this application.

[0091] It is important to note that the application scenario 100 of the vehicle positioning method is provided for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can make various changes and modifications based on the description in this specification. For example, application scenario 100 may also include a database, information source, etc. Furthermore, application scenario 100 may be implemented on other devices to achieve similar or different functions. However, these changes and modifications will not depart from the scope of this specification.

[0092] Figure 2 This is an exemplary flowchart of a vehicle positioning method according to some embodiments of this specification. In some embodiments, process 200 can be executed according to an electronic device. Figure 2 As shown, process 200 includes the following steps.

[0093] Step 210: Determine the vehicle's pose information based on the vehicle's driving information, which is the information output by the vehicle's driving module.

[0094] Drive information refers to various relevant data of the vehicle drive module during operation.

[0095] In some embodiments, the drive information includes at least one of the motor speed and the engine speed.

[0096] The motor can be a vehicle's drive motor or a generator, etc.

[0097] Vehicle pose information refers to the estimated position and orientation of a vehicle in space based on sensor data.

[0098] For example, vehicle pose information can include the vehicle's position information and attitude information.

[0099] Location information refers to a vehicle's geographical location within a coordinate system. For example, location information can be represented by three coordinate values: the X, Y, and Z axes, corresponding to east-west and north-south horizontally, and height or altitude vertically. Another example is using coordinates (X, Y) in a two-dimensional plane to represent the vehicle's location. The X-axis coordinate represents the vehicle's position along the east-west direction. The Y-axis coordinate represents the vehicle's position along the north-south direction. The Z-axis coordinate represents the vehicle's height or depth.

[0100] Attitude information describes the vehicle's orientation and angle relative to a coordinate system. For example, attitude information may include yaw angle, pitch angle, and roll angle.

[0101] The yaw angle represents the vehicle's rotation about its vertical axis (Z-axis), and is the angle between the vehicle's direction of travel and true north. For vehicles on the ground, attitude information includes at least the yaw angle. The pitch angle represents the vehicle's rotation about its lateral axis (Y-axis), indicating the degree to which the front of the vehicle tilts up or down. The roll angle represents the vehicle's rotation about its longitudinal axis (X-axis), reflecting the vehicle's left and right roll.

[0102] In some embodiments, the vehicle's pose information can be determined based on the vehicle's driving information in various ways, such as through manual analysis, theoretical calculation, or modeling.

[0103] Step 220: Based on the pose information and the scene information where the vehicle is located, obtain the vehicle's location.

[0104] Scene information refers to information about the environment surrounding the vehicle. For example, scene information can include one or any combination of the characteristics, location, shape, and relative relationship to the vehicle of static and dynamic elements. Static elements can include road markings, signs, and static obstacles on the road, while dynamic elements can include other vehicles, pedestrians, and dynamic obstacles on the road.

[0105] In some embodiments, scene information may include information about multiple data points related to the vehicle's surrounding environment.

[0106] In some embodiments, scene information includes point cloud data or visual data of the environment surrounding the vehicle.

[0107] Point cloud data refers to a collection of a large number of data points in the three-dimensional space around a vehicle, with each data point representing the location of a reflected laser signal.

[0108] In some embodiments, each data point includes the vehicle's three-dimensional coordinates in the original coordinate system, reflection intensity information, and echo count information. The original coordinate system can be the radar coordinate system.

[0109] Visual data refers to images and video streams of the environment surrounding a vehicle.

[0110] Vehicle positioning refers to the final determined position and orientation of a vehicle in space.

[0111] In some embodiments, the vehicle's location can be obtained by correcting the scene information based on pose information. For example, the vehicle's location can be determined through multi-sensor fusion, deep learning models, or other methods.

[0112] In some embodiments of this specification, the accurate speed information provided by the driving information can significantly improve the accuracy of vehicle pose estimation; the scene information provides rich environmental geometric information, which, combined with the pose information, can further correct and optimize the positioning results, and help the vehicle to be positioned in complex environments or under high dynamic conditions.

[0113] Figure 3 This is an exemplary flowchart illustrating motion compensation according to some embodiments of this specification. In some embodiments, process 300 can be executed according to an electronic device. Figure 3 As shown, process 300 includes the following steps.

[0114] In some embodiments, the vehicle's location is obtained based on pose information and the scene information in which the vehicle is located, including:

[0115] Step 310: Perform motion compensation on the scene information based on the pose information to obtain the compensated scene information, thereby obtaining the vehicle's location.

[0116] Motion compensation refers to transforming each point in the scene information to the coordinate system at the same moment based on the pose changes of the vehicle during point cloud scanning, thereby removing motion distortion.

[0117] Compensated scene information refers to motion compensation processing performed on each data point in the original scene information, transforming the coordinates of the data points into a unified coordinate system (usually the coordinate system at the start or end of the scan). Compensated scene information no longer contains distortions introduced by vehicle motion, and can more accurately represent information about the vehicle's surrounding environment.

[0118] Since the vehicle's pose is constantly changing during its movement, and it takes a long time for the LiDAR to form each frame of point cloud, the scene information generated during this process is distorted (i.e., the scene information is very different from the objects in the real world). Therefore, in some embodiments, the vehicle's pose at a certain target moment is determined and used as a reference pose. All data points in each frame of scene information are represented as the state collected when the target vehicle is in the reference pose, thereby achieving the purpose of correcting the distortion of the initial scene information.

[0119] In some embodiments, if no scene information is detected, the vehicle's pose information is determined based on the vehicle's driving information; if scene information is detected, motion compensation is performed on the scene information based on the latest pose information to obtain compensated scene information, and the vehicle's positioning is obtained based on the compensated scene information.

[0120] In some embodiments of this specification, motion compensation is performed by effectively combining vehicle pose information and scene information, and the corrected scene information helps to achieve high-precision vehicle positioning.

[0121] In some embodiments, scene information includes data points from multiple scan times.

[0122] Motion compensation is performed on the scene information based on the pose information to obtain the compensated scene information, including:

[0123] Based on the pose information within a preset time period, data points at multiple scanning times are compensated to obtain compensated scene information.

[0124] The preset time period is a predefined period of time.

[0125] In some embodiments, the preset time period is the scanning duration for acquiring scene information. For example, the preset time period is the time interval from the start of LiDAR scanning to the completion of a full week of scanning.

[0126] In some embodiments, the vehicle's pose information during the scanning process is acquired from an inertial measurement unit or other sensors; each data point in the scene information is aligned with its corresponding pose information, for example, the pose information corresponding to each data point is calculated by interpolation based on the timestamp of the scene information; each data point is transformed from the coordinate system of its original scanning time to the coordinate system of the target time (e.g., the coordinate system of the start scanning time or the last scanning time).

[0127] In some embodiments of this specification, motion compensation is performed on the data points in a frame of scene information to ensure that all data points reflect the environmental conditions at the same moment, thus eliminating deviations caused by vehicle movement.

[0128] In some embodiments, based on pose information within a preset time period, data points at multiple scanning times are compensated to obtain compensated scene information, including:

[0129] Based on the pose information within a preset time period, the data points of each scanning moment are projected onto the target moment to obtain the compensated scene information.

[0130] The target time refers to a selected time reference point, and all data points from the scanned times will be transformed to the coordinate system corresponding to the target time.

[0131] In some embodiments, the start scan time and the last scan time are selected from a frame of scene information, and either the start scan time or the last scan time, or any one of the start scan time and the last scan time, is taken as the target time.

[0132] In some embodiments of this specification, point cloud distortion caused by vehicle motion is eliminated by converting each data point in the scene information from its original scanning time to a unified target time (typically the start or end time of the scan).

[0133] In some embodiments, the target time is the last scan time within a preset time period.

[0134] During motion compensation, the selected target time provides a unified time reference for all data points, ensuring the consistency and accuracy of scene information. By projecting the data points from each scan time onto the data points from the target time, errors caused by vehicle movement can be effectively eliminated, generating scene information that more realistically reflects the environmental conditions.

[0135] In some embodiments, based on pose information within a preset time period, data points at each scanning moment are projected onto a target moment to obtain compensated scene information, including:

[0136] Based on the pose information within a preset time period, determine the projection transformation matrix that projects the data points at each scanning moment to the target moment.

[0137] Based on the projection transformation matrix corresponding to each scanning time, the data points of each scanning time are projected onto the target time to obtain the compensated scene information.

[0138] A projection transformation matrix refers to information about the position and orientation changes of an object (such as a vehicle or robot) from one time point to another. The projection transformation matrix indicates the motion changes of a vehicle or sensor between different points in time. In some implementations, the projection transformation matrix includes rotation and translation parameters, which can be used to transform a data point from one coordinate system to another.

[0139] In some embodiments, initial pose information (pose information corresponding to the adjacent moments of the start of the scan duration) and end pose information (pose information corresponding to the adjacent moments of the last scan duration) are determined in the pose information. Linear interpolation is performed based on the initial pose information, the end pose, and all pose information in between to obtain the pose information corresponding to each scan moment of the scan duration. This yields the projection transformation matrix corresponding to the transformation of the vehicle from each scan moment to the target moment. The linear interpolation can use a straight line passing through two points to approximate the linear relationship between time and pose information. Based on this linear relationship, the pose information at each scan moment can be determined.

[0140] Specifically, the adjacent times of the start scan time / last scan time can be times corresponding to pose information adjacent to the start scan time / last scan time. For example, if the scan duration is 50-60ms, 50.1ms is the adjacent time of pose information at the start scan time, and 99.9ms is the adjacent time of pose information at the last scan time. Alternatively, the adjacent times of the start scan time / last scan time can be times outside the scan duration (i.e., before the scan starts and after the scan ends). For example, if the scan duration is 50-60ms, 49.9ms is the adjacent time of pose information at the start scan time, and 60.1ms is the adjacent time of pose information at the last scan time.

[0141] In some embodiments, the scene information is subjected to coordinate transformation processing according to the projection transformation matrix to obtain the scene information at the target time.

[0142] The purpose of coordinate transformation is to project all data points in the scene information to the same time, so that all data points in the scene information of a point cloud frame appear as if they were collected at the same time, thus achieving the effect of point cloud distortion correction.

[0143] In some embodiments, scene information motion compensation is typically applied to the start or end of the LiDAR motion scan duration. In other words, each compensation requires the motion of each data point within the scan duration to be compensated to the end of the scan duration (or the motion of each data point within the scan duration to be compensated to the start of the scan duration).

[0144] In some embodiments of this specification, by accurately calculating and applying the projection transformation matrix, motion compensation can be achieved for each data point in the scene information, thereby improving the accuracy of environmental perception, map building, and positioning.

[0145] In some embodiments, the projection transformation matrix is ​​a matrix in the vehicle coordinate system.

[0146] Based on the projection transformation matrix corresponding to each scanning time, the data points at each scanning time are projected onto the target time to obtain the compensated scene information, including:

[0147] Based on the data points at each scan time in the vehicle coordinate system and the corresponding projection transformation matrix, data points at multiple target times in the vehicle coordinate system are obtained to obtain the compensated scene information.

[0148] In some embodiments, the method further includes:

[0149] The data points at each scan time in the original coordinate system are projected to the vehicle coordinate system to obtain the data points at each scan time in the vehicle coordinate system.

[0150] For example, the data points corresponding to the scanning time can be projected from the original coordinate system to the vehicle coordinate system according to the first transformation matrix.

[0151] The first transformation matrix is ​​the transformation matrix projected from the original coordinate system to the vehicle coordinate system.

[0152] In some embodiments, the compensated scene information includes:

[0153] The data points at each target time in the vehicle coordinate system are projected onto the original coordinate system to obtain data points at multiple target times in the original coordinate system, which serve as the compensated scene information.

[0154] In some embodiments, the data points at each scanning time in the vehicle coordinate system can be projected to the data points at the target time in the vehicle coordinate system according to the projection transformation matrix.

[0155] In some embodiments, the data points at the target time in the vehicle coordinate system can be projected from the vehicle coordinate system to the original coordinate system according to the second transformation matrix.

[0156] The first transformation matrix is ​​the transformation matrix projected from the vehicle coordinate system to the original coordinate system.

[0157] The first transformation matrix and the second transformation matrix can be obtained from the calibration.

[0158] In some embodiments, the compensated scene information can be represented according to formula (1):

[0159]

[0160] in, Let J be the coordinates of the data point corresponding to scan time j in the original coordinate system. Let be the first transformation matrix. Let J be the projection transformation matrix corresponding to scan time j. This is the second transformation matrix.

[0161] In some embodiments of this specification, the scene information is effectively motion-compensated based on the pose information within a preset time period by using the transformation matrix between various coordinate systems, ensuring that all data points reflect the environmental conditions at the same moment, thereby improving the accuracy and consistency of the scene information.

[0162] In some embodiments, obtaining the location of the vehicle includes:

[0163] Based on the compensated scene information, determine the filtering parameters of the preset filter;

[0164] The vehicle's inertial motion information is processed based on the filtering parameters to obtain the vehicle's location.

[0165] In some embodiments, in (t k-1 , t k Within a preset time period, the LiDAR continuously scans and generates scene information. Simultaneously, the IMU sensor and vehicle drive module (e.g., drive motor) also generate corresponding sensor data. Since the LiDAR needs to accumulate a complete point cloud frame to form a measurement, within (t... k-1 , t k Within a preset time period, only the data from the IMU sensor and the speed sensor are fused to obtain an initial fusion result. When t k At time t, after the lidar completes scanning of one frame of scene information, it uses (t) k-1 , t k The fusion results of IMU sensor data and rotation sensor data within a preset time period are used to perform motion compensation on the cloud data of the lidar points, and (t) k-1 , t k Transform all feature points of the preset time period to t k At time, and then at t kThe system continuously fuses data from the IMU sensor and compensated scene information from the LiDAR to obtain localization information.

[0166] The preset filter can be a Kalman filter, an extended Kalman filter, or an error-state Kalman filter.

[0167] In some embodiments of this specification, the compensated scene information can more accurately reflect the actual state of the environment, thereby improving positioning accuracy; by dynamically updating the filter parameters, it can better adapt to environmental changes and vehicle movement, enhancing the robustness of the system.

[0168] In some embodiments, determining the filtering parameters of the preset filter based on the compensated scene information includes:

[0169] Based on the compensated scene information, residual information is obtained to determine the filtering parameters.

[0170] Residual information refers to the difference between the compensated scene information and the predicted value based on state information.

[0171] In some embodiments, residual information is obtained based on the compensated scene information, including:

[0172] Multiple compensated data points in the compensated scene information are projected from the original coordinate system to the reference coordinate system to obtain the reference scene information;

[0173] The residual information is obtained based on the reference scenario information.

[0174] The reference coordinate system can be the global map coordinate system, the world coordinate system, or other known fixed coordinate systems.

[0175] Reference scene information refers to the scene information after the compensated scene information is projected onto a global or known fixed coordinate system (i.e., the reference coordinate system).

[0176] Under the reference coordinate system, scene information collected at different time points can be registered and matched to construct a more accurate 3D environment model.

[0177] In some embodiments, multiple compensation data points in the compensated scene information can be transformed from the original coordinate system to the vehicle coordinate system according to the first transformation matrix; and multiple compensation data points in the compensated scene information in the vehicle coordinate system can be transformed from the vehicle coordinate system to the reference coordinate system according to the third transformation matrix.

[0178] The third transformation matrix is ​​the projection from the vehicle coordinate system to the reference coordinate system.

[0179] The third transformation matrix can be obtained from the calibration.

[0180] In some embodiments, the reference scene information can be determined according to formula (2):

[0181]

[0182] in, This is the third transformation matrix.

[0183] In some embodiments, significant point cloud features, such as edges and planes, can be extracted from reference scene information and used as reference markers for localization. The real-time collected point cloud features are then matched with scene information in a high-precision map using a preset algorithm. For example, the preset algorithm may include Iterative Closest Point (ICP) and Generalized Iterative Closest Point (GICP) algorithms, using the error between the point cloud features and the matched information in the high-precision map as residual information.

[0184] In some embodiments of this specification, by transforming scene information from the original coordinate system to a reference coordinate system (such as a global coordinate system or a map coordinate system), the reference scene information can more accurately reflect the actual state of the environment around the vehicle, and at the same time help to construct observation equations, thereby improving positioning accuracy.

[0185] In some embodiments, residual information is obtained based on reference scenario information, including:

[0186] For each reference data point in the reference scene information, determine the nearest plane of the reference data point in the reference coordinate system;

[0187] The residual information is obtained based on the distance between the reference data point and the adjacent plane corresponding to the reference data point.

[0188] The nearest neighbor plane is the plane in the reference coordinate system that is closest to the reference data point. The nearest neighbor plane can be obtained by fitting prior maps or local scene information, for example, through the least squares method or other plane fitting algorithms.

[0189] In some embodiments, for each reference data point, neighboring points within a certain range around it are selected (e.g., a fixed number of neighboring points are selected using the K-nearest neighbor algorithm); based on the neighboring points, a neighboring plane is fitted using a plane fitting method.

[0190] In some embodiments, after motion compensation, all data points in a scan frame can be considered as being sampled at the same time, and then projected onto a reference coordinate system (such as a global coordinate system) to obtain residual information, as shown in formula (3):

[0191]

[0192] in This represents the measured value of the j-th data point in the original coordinate system. This is the third transformation matrix for rotation and translation from the vehicle coordinate system to the reference coordinate system. Let z be the measured value of the j-th data point in the reference coordinate system. j For residual information, G is a point on the nearest plane. j It is the normal vector of the adjacent plane.

[0193] In some embodiments of this specification, the geometric information in the scene information can be effectively utilized based on the distance between each reference data point in the reference scene information and its corresponding neighboring plane, thereby improving positioning accuracy and system robustness.

[0194] In some embodiments, determining the filtering parameters includes:

[0195] The filtering parameters are determined based on the residual information that is less than or equal to a preset threshold.

[0196] In some embodiments, an observation equation can be established based on the residual information; the observation equation can be linearized to obtain the Jacobian matrix, which is then used to update the filtering parameters of the preset filter.

[0197] The observation equation describes the relationship between the estimated data of the system state (such as position, attitude, etc.) and the observed data. The observation equation can be expressed as formula (4):

[0198] Z k =h(X) k )+V k (4)

[0199] Among them, Z k It is the observation vector, representing the observed value at time k.

[0200] X k It is the system state vector, representing the estimated value of the state at time k.

[0201] V k It is the observation noise vector, usually assumed to be Gaussian noise with zero mean.

[0202] h is the observation function, used to express the relationship between the system state vector and the observation vector.

[0203] In practical applications, the observation equations may be nonlinear. To apply linear filtering techniques such as Kalman filtering, it is necessary to linearize the nonlinear observation equations. For example, this can be achieved through Taylor series expansion. A pose (e.g., an estimate of the current pose) is selected. A first-order Taylor expansion of the observation equations at that pose yields a linearized approximation.

[0204] In some embodiments of this specification, by using geometric information (e.g., the distance from a point to a plane) in the scene information as observation data, the positioning accuracy of the vehicle can be significantly improved, and positioning errors can be reduced by effectively utilizing the geometric features in the environment.

[0205] In some embodiments, determining the filtering parameters includes:

[0206] Based on the residual information, the prior estimates of the filter parameters are corrected to obtain the posterior estimates of the filter parameters.

[0207] The preset threshold can be a value determined based on experiments or experience.

[0208] Considering reference data points with residual information less than a preset threshold refers to selecting points that are close to neighboring planes (or other reference geometries) during scene information processing. These data points are reliable and can be used to establish observation equations, thereby improving the accuracy of localization and map building.

[0209] In some embodiments, the nearest point in the map can be searched by constructing a KD-tree of the latest map points. Only residuals with a norm below a certain threshold are considered; residuals exceeding this threshold can be outliers or newly observed points.

[0210] In some embodiments of this specification, points that are farther away may be more affected by noise or environmental changes. By selecting points that are closer together, the impact of noise can be reduced, and the robustness of the system can be improved.

[0211] In some embodiments, determining the filtering parameters includes:

[0212] Based on the residual information, the prior estimates of the filter parameters are corrected to obtain the posterior estimates of the filter parameters.

[0213] Prior estimation of filter parameters refers to the initial estimation of filter parameters before the arrival of observation data.

[0214] For example, prior estimates of filter parameters may include at least one of prior estimates of vehicle position, prior estimates of attitude, and prior estimates of error covariance.

[0215] For example, the posterior estimate of the filter parameters may include at least one of the following: the posterior estimate of the vehicle's position, the posterior estimate of its attitude, and the posterior estimate of its error covariance.

[0216] In some embodiments, the prior estimation of the filter parameters includes a prior estimation of the nominal state vector, and the posterior estimation of the filter parameters includes a posterior estimation of the error state vector.

[0217] In some embodiments, the preset filter may be an error-state Kalman filter.

[0218] In some embodiments, the nominal state vector includes at least one of the following: vehicle position, velocity, attitude, bias of angular velocity measurements from the inertial measurement unit, bias of acceleration measurements from the inertial measurement unit, and gravitational acceleration.

[0219] In some embodiments, the nominal state vector includes at least one of the following: the vehicle's position in the reference coordinate system, the vehicle's velocity in the reference coordinate system, the vehicle's attitude in the reference coordinate system, the bias of the angular velocity measurement value of the inertial measurement unit and the bias of the acceleration measurement value of the inertial measurement unit, and the gravitational acceleration in the reference coordinate system.

[0220] In some embodiments of this specification, by fusing data from multiple sensors, a more accurate estimation of vehicle pose can be provided.

[0221] In some embodiments, the error state vector is determined based on the errors of each term in the nominal state vector.

[0222] In some embodiments, the state of the Kalman filter truth value in the error state is xt = [Pt, Vt, Rt, bat, bgt, gt] T Where P represents the position coordinates in the reference coordinate system, V represents the velocity in the reference coordinate system, R is the system's attitude at this moment, b is the zero bias, g represents the gravitational acceleration, values ​​with the subscript t are the system's true values, subscript a represents the accelerometer, and subscript g represents the gyroscope. The IMU's gyroscope and accelerometer data are...

[0223] The discrete-time kinematic equations of the nominal state variables are established as shown in equation (5):

[0224]

[0225] b g (t+Δt)=b g (t);

[0226] b a (t+Δt)=b a (t);

[0227] g(t+Δt)=g(t); (5)

[0228] Among them, the discrete-time state equation of the error state variable is established as shown in formula (6):

[0229] δp(t+Δt)=δp+vΔt;

[0230]

[0231] δb g (t+Δt)=δb g +η g ;

[0232] δb a (t+Δt)=δb a +η g ;

[0233] δg(t+Δt)=δg;(6)

[0234] In some embodiments, δ represents the error of each nominal state variable. Simplifying equations (5) and (6), when the inertial motion information measured by the IMU arrives, it is pre-integrated to obtain the prior estimate of the nominal state vector and the prior estimate of the error state vector, thus obtaining the prediction model, as shown in equation (7):

[0235] δx pred =Fδx;

[0236] P pred =FPF T +Q;(7)

[0237] P is the posterior estimate of the error covariance, Ppred is the prior estimate of the error covariance, and δx is the prior estimate of the error state vector. pred It is recreated after each update and does not need to be included in the calculation.

[0238] In some embodiments of a multi-sensor fusion system, the sampling frequencies of different sensors often differ. For example, an IMU (Inertial Measurement Unit) typically has a high sampling frequency (e.g., 60 Hz or higher), while other sensors (e.g., LiDAR, speed sensors, etc.) may have lower sampling frequencies (e.g., 10 Hz or lower). In this case, the Kalman filtering prediction and update process for the error state needs to be adjusted according to the sensor's sampling frequency.

[0239] In some embodiments, the prediction process is performed based on the frequency of the IMU's inertial motion information. Whenever new IMU inertial motion information arrives, the predicted value of the nominal state vector (position, velocity, attitude, etc.) is calculated by integration based on the IMU's accelerometer data and drive information.

[0240] In some embodiments, a prior estimate of the nominal state vector is obtained based on the vehicle's inertial motion information, including:

[0241] The inertial motion information is pre-integrated, and based on the pre-integration result and the preset nominal state vector, a priori estimate of the nominal state vector at the target time is obtained.

[0242] The posterior estimate of the pre-defined nominal state vector can be the prior estimate of the nominal state vector corresponding to the scene information of the previous frame.

[0243] In some embodiments, the prediction of the nominal state vector can be achieved according to formula (8) to obtain a prior estimate of the nominal state vector at the current time.

[0244]

[0245] Let i be the sequence number of the inertial motion information of the IMU.

[0246] in It is the prior estimate of the nominal state vector at time i+1. It is the prior estimate of the nominal state vector at time i, u i It is the control input at time i, where Δt is the time step. It is the system's state transition function. (Symbol) These are the defined operators. Different variables have different operation methods; for example, vector addition is used for position and velocity, while quaternion multiplication is used for attitude.

[0247] In some embodiments, the prior estimate of the nominal state vector at the next time step can be calculated based on the prior estimate of the nominal state vector at the current time step, the control input, and the system's state transition function, by recursively starting from the initial state. Begin by making predictions step by step forward. This represents the posterior state estimate of the nominal state vector after the most recent LiDAR scan (e.g., the (k-1)th frame) or the posterior estimate of the nominal state vector after the most recent driving information.

[0248] In some embodiments of this specification, the state prediction process is a step in Kalman filtering and multi-sensor fusion, used to predict the future state of the system based on the system's dynamic model and control input.

[0249] In some embodiments, the model is updated only when low-frequency observation data (such as point cloud data or other driving information) is detected. When new observation data (such as scene information or driving information) arrives, the model is updated as shown in Equation (9):

[0250]

[0251] δx=K(zh(x t ));

[0252] P=(I-KH)P pred (9)

[0253] In some embodiments, K represents the gain coefficient, Ppred Let H represent the prior estimate of the error covariance, H represent the Jacobian matrix, and V represent the preset measurement noise matrix; δx represent the posterior estimate of the error state vector, z represent the observed data (e.g., scene information or driving information), and h(x) represent the prior estimate of the error state vector. t ) represents the predicted value of the observed data, and P represents the posterior estimate of the error covariance.

[0254] It should be noted that the sampling frequency of the inertial measurement unit and the sampling frequency of the drive information are higher than the acquisition frequency of the scene information. The sampling frequency of the inertial measurement unit can be greater than or equal to the sampling frequency of the drive information.

[0255] For example, if the frequency of a lidar is 10 Hz, then every 100 ms scan of the radar point cloud constitutes one frame of radar scene information. One frame lasts for 100 ms, and the scene information can be understood as the data obtained at the end of the frame.

[0256] In some embodiments, the inertial motion information of the vehicle is processed according to filtering parameters to obtain the vehicle's positioning, including:

[0257] Based on the vehicle's inertial motion information, a priori estimate of the nominal state vector is obtained;

[0258] The prior estimate of the nominal state vector is corrected based on the posterior estimate of the error state vector to obtain the posterior estimate of the nominal state vector, thereby obtaining the vehicle's positioning.

[0259] In some embodiments, the posterior estimate of the nominal state vector can be obtained according to formula (10).

[0260] p k|k =p k|k-1 +δp k|k ;

[0261] v k|k =v k|k-1 +δv k|k ;

[0262] R k|k =R k|k-1 Exp(δθ k|k );

[0263] b g，k|k =b g，k|k-1 +δb g，k|k ;

[0264] b a，k|k =b a，k|k-1 +δb a，k|k ;

[0265] g k|k =g k|k-1 +δgk|k (10)

[0266] Where P represents the position coordinates in the reference coordinate system, V represents the velocity in the reference coordinate system, R is the rotation matrix of the system at this time, b is the zero bias, g represents the gravitational acceleration, the subscript k|k represents the posterior estimate of the system, and the subscript k|k-1 represents the prior estimate of the system.

[0267] In some embodiments of this specification, by using the posterior estimate of the error state vector to correct the prior estimate of the nominal state vector, positioning accuracy can be significantly improved, observation data can be effectively utilized, error accumulation can be reduced, and the overall performance of the system can be improved.

[0268] In some embodiments, the method further includes:

[0269] Based on the posterior estimate of the error state vector obtained each time, determine the difference between the posterior estimate of the error state vector and the posterior estimate of the previous error state vector.

[0270] When the difference is greater than or equal to a preset difference threshold, the error state vector is iteratively updated.

[0271] In some embodiments, a preset difference threshold can be set to determine whether the difference is small enough. Typically, the preset difference threshold can be a very small positive number, representing the maximum permissible difference between the posterior estimates of the error state vectors.

[0272] In some embodiments, if the norm of the difference (e.g., Euclidean norm, etc.) is greater than or equal to a preset difference threshold, it is considered that the Kalman filter in the error state has not yet converged and needs to be iterated again to determine the filtering parameters of the preset filter.

[0273] In some embodiments of this specification, by determining the difference between the posterior estimate of the error state vector and the previous posterior estimate of the error state vector, and by iterating again when the difference is greater than or equal to a preset difference threshold, the accuracy and robustness of the Kalman filter can be significantly improved.

[0274] In some embodiments, the method further includes:

[0275] When the difference is less than a preset difference threshold, the posterior estimate of the obtained error state vector is used to correct the prior estimate of the corresponding nominal state vector to obtain the posterior estimate of the nominal state vector, which is used as a localization step.

[0276] In some embodiments, when the difference is less than a preset difference threshold, the vehicle is controlled to park based on the posterior estimate of the nominal state vector, or the global point cloud is downsampled and added to the KD tree, and then accumulated with the previous map to obtain a new map.

[0277] In some embodiments of this specification, the accuracy of positioning and navigation can be significantly improved by ensuring that the posterior estimate of the error state vector is sufficiently close to the true state after each update.

[0278] In some embodiments, determining the vehicle's pose information based on the vehicle's driving information includes:

[0279] The vehicle's position and posture information are determined based on the vehicle's driving information and preset transmission ratio.

[0280] In some embodiments, the drive information may include the speed and torque information of the motor (drive motor or generator).

[0281] In some embodiments, drive information can be obtained via the vehicle's CAN bus or directly from the motor controller. Alternatively, drive information can be obtained through a communication connection with the chassis motor's sensors.

[0282] In some embodiments of this specification, the high update frequency and high accuracy of the driving information can compensate for the lack of data from the IMU sensor and enhance the robustness of the system; by updating the vehicle speed parameters in real time, the system can quickly adapt to the dynamic changes of the vehicle and maintain high real-time performance.

[0283] The gear ratio represents the conversion relationship between drive information and wheel speed. For example, a preset gear ratio represents the conversion relationship between the speed of the drive motor output shaft and the speed of the wheels.

[0284] The preset transmission ratio can be determined based on experiments or experience.

[0285] In some embodiments of this specification, the vehicle speed information can be accurately calculated by using drive information and the ratio of the vehicle drive motor speed to the actual vehicle speed.

[0286] Figure 4 This is an exemplary flowchart illustrating the determination of vehicle pose information according to some embodiments of this specification. In some embodiments, process 400 can be executed according to an electronic device. Figure 4 As shown, process 400 includes the following steps.

[0287] In some embodiments, determining the vehicle's pose information based on the vehicle's driving information and a preset gear ratio includes:

[0288] Step 410: Obtain the vehicle speed information based on the drive information and the preset transmission ratio;

[0289] Step 420: Determine the vehicle's pose information based on the vehicle's inertial motion information and speed information.

[0290] Inertial motion information refers to data measured by an inertial measurement unit installed on a vehicle. For example, inertial motion information may include the vehicle's acceleration and angular velocity data.

[0291] In some embodiments, the electronic device can measure the linear acceleration of the vehicle in three directions using the accelerometer in the inertial measurement unit. By integrating the acceleration data, the vehicle's speed and position changes can be obtained.

[0292] In some embodiments, the electronic device can measure the angular velocity of the vehicle in three directions using a gyroscope integrated in the inertial measurement unit. By integrating the angular velocity data, the attitude changes of the vehicle (such as pitch, roll, and yaw) can be obtained.

[0293] Vehicle speed information refers to information that measures vehicle speed based on driving information.

[0294] In some embodiments, the vehicle speed information can be obtained based on the rotational speed of the drive motor and the preset transmission ratio in the drive information. Based on the vehicle speed information, the vehicle's inertial motion information, and the vehicle speed information, the vehicle's pose information (e.g., vehicle position, speed, etc.) can be obtained through filtering algorithms (e.g., Kalman filtering for error state, Kalman filtering, etc.).

[0295] In some embodiments, the method further includes:

[0296] If the vehicle speed is less than a preset speed threshold, the step of determining the vehicle's pose information based on the vehicle's driving information is executed.

[0297] The preset vehicle speed threshold can be determined based on experiments or experience.

[0298] In some embodiments of this specification, the accuracy of traditional wheel speed meters is insufficient when driving at low speeds, while the high-precision speed information provided by the drive information can significantly improve the positioning accuracy of the vehicle. By combining the IMU sensor and the drive information, the estimation accuracy of the vehicle pose information can be significantly improved, which is helpful for positioning when driving at low speeds or parking.

[0299] It should be noted that the above description of the process is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the process under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0300] Figure 5 This is an exemplary schematic diagram of a vehicle positioning method according to some embodiments of this specification.

[0301] In some embodiments, such as Figure 5As shown, a vehicle positioning method includes:

[0302] Motion compensation is performed on the point cloud of the LiDAR by fusing the inertial measurement unit and the vehicle speed information corresponding to the driving information to eliminate the point cloud distortion effect introduced by the movement of the carrier itself. Secondly, based on the scene information of the LiDAR, an observation model for the point-to-surface residual of the LiDAR is constructed to eliminate the adverse effects of dynamic objects on point cloud matching, thereby improving the point cloud matching accuracy in dynamic environments and effectively improving the positioning accuracy of the vehicle in dynamic environments.

[0303] In each discretization step, it is necessary to predict the error state and its covariance matrix (i.e., δx). pred =Fδx and P pred =FPF T To improve the sampling and calculation accuracy of the system, a high update frequency is required, especially important in high-dynamic systems. After the prediction update, if no measurement data (e.g., vehicle speed information or scene information) is detected, the prediction of the error state and its covariance matrix will be used as the optimal estimate of the vehicle state. After the prediction update, if observation data is available, the update process is executed (i.e., calculating the gain K = P). pred H T HP pred H T +V) -1 ; and state estimation δx=K(zh(x) t )) and P=(I-KH)P pred This method obtains the optimal estimate of the error state, and the update frequency depends on the measurement frequency of the sensor used to observe the data. A higher measurement frequency is better, but in practice it is often lower than the prediction frequency, further ensuring the stability and reliability of the fusion. It also eliminates the need for specialized sensor equipment, making it convenient and quick. When implementing the above vehicle parking method, using the vehicle positioning results obtained through the aforementioned vehicle positioning method for parking can improve parking accuracy and efficiency.

[0304] In some embodiments, such as Figure 4As shown, when no lidar scene information is detected, vehicle localization is performed using Kalman filtering of the error state based on the inertial motion information of the inertial measurement unit and the vehicle's driving information. This corrects and updates the prior estimate of the error state vector during the prediction process, yielding pose information. After lidar scene information is detected, the scene information undergoes preprocessing (e.g., noise and outlier removal, downsampling, ground detection and segmentation, clustering, feature extraction, etc.). Motion compensation is then applied to the preprocessed scene information based on the vehicle's pose information, resulting in compensated scene information. Based on the compensated scene information, an observation equation is constructed through residual calculation, updating the prior estimate of the error state vector to obtain a posterior estimate. The difference between two adjacent error state vectors is compared to a preset difference threshold to determine if iterative calculation is needed (whether the error state vector has converged). If the difference is greater than or equal to the preset difference threshold, the Kalman filtering update process is re-executed until the difference between two adjacent error state vectors is less than the preset difference threshold. The resulting posterior estimate of the error state vector is then used as the vehicle's localization.

[0305] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0306] like Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of an electronic device according to some embodiments of this specification.

[0307] Specifically, an electronic device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that... Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0308] The processor 601 is the vehicle's positioning center, connecting various parts of the electronic device via various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 602, and by calling data stored in the memory 602, thereby providing overall monitoring of the electronic device. It is understood that the processor 601 communicates with the controller via signal transmission. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 601.

[0309] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0310] In some embodiments of this application, the vehicle positioning device can be implemented as a computer program, which can be implemented as follows: Figure 6 The device operates on the electronic device shown. The memory of the electronic device can store various program modules that make up the vehicle positioning device. The computer program composed of the various program modules causes the processor to execute the steps in the vehicle positioning methods of the various embodiments of this application described in this specification.

[0311] The electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface is used to communicate with external electronic devices via a network connection. When the computer program is executed by the processor, it implements a vehicle positioning method.

[0312] The electronic device also includes a power supply 603 that supplies power to the various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0313] The electronic device may also include an input unit 604, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0314] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 602 according to the following instructions, and the processor 601 runs the applications stored in the memory 602 to realize various functions.

[0315] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0316] In another exemplary embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a computer, implements the steps of the touch event handling method described above.

[0317] In another exemplary embodiment, a computer program product is also provided, the computer program product storing instructions that, when executed by a computer, cause the computer to perform the touch event processing method described above.

[0318] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0319] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0320] Figure 7 This is an exemplary schematic diagram of a vehicle provided in an exemplary embodiment of this application.

[0321] like Figure 7 As shown, this application also provides a vehicle equipped with the electronic equipment provided in any of the above embodiments, or for executing the vehicle positioning method provided in any of the above embodiments. The vehicle may be a gasoline-powered vehicle, a plug-in hybrid electric vehicle, or a new energy vehicle, etc., and this specification does not specifically limit it.

[0322] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0323] The embodiments, implementation methods, and related technical features of this application can be combined and substituted for each other without conflict.

[0324] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Although the descriptions of each embodiment in this application have different focuses, and the parts not described in detail in a certain embodiment can be referred to the relevant embodiments of other embodiments, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of this application without departing from the content of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A vehicle positioning method characterized by comprising: The method comprises: determining pose information of the vehicle according to driving information of the vehicle, the driving information being information output by a vehicle driving module; obtaining positioning of the vehicle according to the pose information and scene information in which the vehicle is located.

2. The method of claim 1, wherein, The obtaining of the positioning of the vehicle according to the pose information and the scene information in which the vehicle is located comprises: motion compensating the scene information according to the pose information to obtain compensated scene information, so as to obtain the positioning of the vehicle.

3. The method of claim 2, wherein, The scene information comprises data points at multiple scanning time points, The motion compensating of the scene information according to the pose information to obtain the compensated scene information comprises: compensating the data points at the multiple scanning time points according to the pose information in a preset time period to obtain the compensated scene information.

4. The method of claim 3, wherein, The preset time period is a scanning duration for obtaining the scene information.

5. The method of claim 3, wherein, The compensating of the data points at the multiple scanning time points according to the pose information in the preset time period to obtain the compensated scene information comprises: projecting the data points at each of the scanning time points to a target time point according to the pose information in the preset time period to obtain the compensated scene information.

6. The method of claim 5, wherein, The target time point is a last scanning time point in the preset time period.

7. The method of claim 5, wherein, The projecting of the data points at each of the scanning time points to the target time point according to the pose information in the preset time period to obtain the compensated scene information comprises: determining a projection transformation matrix for projecting the data points at each of the scanning time points to the target time point according to the pose information in the preset time period; projecting the data points at each of the scanning time points to the target time point according to the projection transformation matrix corresponding to each of the scanning time points to obtain the compensated scene information.

8. The method of claim 7, wherein, The projection transformation matrix is a matrix in a vehicle coordinate system, The projecting of the data points at each of the scanning time points to the target time point according to the projection transformation matrix corresponding to each of the scanning time points to obtain the compensated scene information comprises: obtaining data points at multiple target time points in the vehicle coordinate system according to the data points at each of the scanning time points in the vehicle coordinate system and the projection transformation matrix corresponding to each of the scanning time points, so as to obtain the compensated scene information.

9. The method of claim 8, wherein, The method further comprises: projecting the data points at each of the scanning time points from an original coordinate system to the vehicle coordinate system to obtain the data points at each of the scanning time points in the vehicle coordinate system.

10. The method of claim 8, wherein, The obtaining of the compensated scene information comprises: projecting the data points at each of the target time points in the vehicle coordinate system to the original coordinate system to obtain data points at multiple target time points in the original coordinate system as the compensated scene information.

11. The method according to any one of claims 2-10, characterized in that, The obtaining of the positioning of the vehicle comprises: determining filter parameters of a preset filter according to the compensated scene information; processing inertial motion information of the vehicle according to the filter parameters to obtain the positioning of the vehicle.

12. The method of claim 11, wherein, The determining of the filter parameters of the preset filter according to the compensated scene information comprises: According to the compensated scene information, residual information is obtained to determine the filtering parameter.

13. The method of claim 12, wherein, The obtaining of the residual information according to the compensated scene information comprises: The compensated scene information is projected from an original coordinate system to a reference coordinate system to obtain reference scene information. The residual information is obtained according to the reference scene information.

14. The method of claim 13, wherein, The obtaining of the residual information according to the reference scene information comprises: For each reference data point in the reference scene information, a neighboring plane of the reference data point in the reference coordinate system is determined. The residual information is obtained according to a distance between the reference data point and the neighboring plane corresponding to the reference data point.

15. The method of claim 12, wherein, The determination of the filtering parameter comprises: The filtering parameter is determined according to residual information less than or equal to a preset threshold.

16. The method of claim 12, wherein, The determination of the filtering parameter comprises: The filtering parameter is determined according to residual information less than or equal to a preset threshold.

17. The method of claim 16, wherein, The filtering parameter is determined according to residual information less than or equal to a preset threshold.

18. The method of claim 16, wherein, The filtering parameter is determined according to residual information less than or equal to a preset threshold. The filtering parameter is determined according to residual information less than or equal to a preset threshold. The processing of the inertial motion information of the vehicle according to the filtering parameter to obtain the positioning of the vehicle comprises:

19. The method of claim 18, wherein, The inertial motion information of the vehicle is used to obtain a prior estimation of a nominal state vector; The prior estimation of the nominal state vector is corrected according to a posterior estimation of an error state vector to obtain a posterior estimation of the nominal state vector, so as to obtain the positioning of the vehicle.

20. The method of claim 18, wherein, The obtaining of the prior estimation of the nominal state vector based on the inertial motion information of the vehicle comprises:

21. The method of claim 18, wherein, The inertial motion information is pre-integrated, and the prior estimation of the nominal state vector at a target time is obtained according to a result of the pre-integration and a preset nominal state vector.

22. The method of claim 1, wherein, The nominal state vector comprises at least one of a position, a speed, an attitude, a bias of an angular velocity measurement of an inertial measurement unit, a bias of an acceleration measurement of the inertial measurement unit, and a gravity acceleration of the vehicle. The error state vector is determined according to errors of the items in the nominal state vector.

23. The method of claim 22, wherein, The determination of the pose information of the vehicle according to the driving information of the vehicle comprises: The driving information of the vehicle and a preset transmission ratio are used to determine the pose information of the vehicle. The determination of the pose information of the vehicle according to the driving information of the vehicle and the preset transmission ratio comprises:

24. The method of claim 23, wherein, The driving information of the vehicle and the preset transmission ratio are used to obtain vehicle speed information of the vehicle; The inertial motion information of the vehicle and the vehicle speed information are used to determine the pose information of the vehicle.

25. The method of claim 1, wherein, The method further comprises:

26. The method of any one of claims 1-25, wherein, In a case where the vehicle speed information of the vehicle is less than a preset vehicle speed threshold, the step of determining the pose information of the vehicle based on the inertial motion information of the vehicle and the vehicle speed information is performed. The driving information comprises at least one of a rotational speed of a motor and a rotational speed of an engine. The scene information comprises point cloud data or visual data of an environment around the vehicle.

27. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a computer, causes the computer to implement the vehicle positioning method of any one of claims 1 to 26.

28. A computer program product, characterised in that, The computer program product stores instructions which, when executed by a computer, cause the computer to implement the vehicle positioning method of any one of claims 1 to 26.

29. An electronic device, comprising: comprising: a memory having stored thereon a computer program; a processor configured to execute the computer program in the memory to implement the vehicle positioning method of any one of claims 1 to 26.

30. A vehicle characterized by comprising: The electronic device of claim 29, or configured to implement the vehicle positioning method of any one of claims 1 to 26.

Images