Vehicle pose determination method and device

Abstract

The invention discloses a vehicle pose determination method and device, and relates to the technical field of automatic driving. A specific embodiment of the method comprises the steps of performing semantic segmentation processing on an obtained current environment point cloud of a vehicle, and determining a target environment point cloud based on a semantic segmentation result; performing ground segmentation processing on the target environment point cloud to obtain a ground segmentation result; and determining a semantic point cloud map according to the obtained initial pose of the vehicle, and determining a target pose of the vehicle based on the ground segmentation result and the semantic point cloud map. According to the embodiment, the error of the vehicle pose determination result can be reduced, the success rate is improved, the time consumed in the determination process is shortened, and the efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more particularly to a method and apparatus for determining vehicle pose. Background Technology

[0002] In scenarios involving vehicle pose recognition, the current approach primarily relies on machine learning techniques to train corresponding models for relocalization and to determine the vehicle's pose.

[0003] In the process of realizing this invention, the inventors discovered at least the following problems in the related technology:

[0004] The determination of vehicle pose has a large error and a low success rate, and the determination process is time-consuming and inefficient. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for determining vehicle pose, which can reduce the error of the vehicle pose determination result, improve the success rate, shorten the time consumed in the determination process, and improve efficiency.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for determining a vehicle's pose is provided, comprising:

[0007] The acquired current environment point cloud of the vehicle is semantically segmented, and the target environment point cloud is determined based on the semantic segmentation results.

[0008] Ground segmentation is performed on the target environment point cloud to obtain the ground segmentation result;

[0009] The semantic point cloud map is determined based on the initial pose of the vehicle, and the target pose of the vehicle is determined based on the ground segmentation results and the semantic point cloud map.

[0010] Optionally, the target environment point cloud is determined based on the semantic segmentation results, including:

[0011] The target environment point cloud is obtained by removing the target obstacle from the current environment point cloud based on the semantic segmentation result.

[0012] Optionally, the target environment point cloud is subjected to ground segmentation processing to obtain ground segmentation results, including:

[0013] The target environment point cloud is divided using a ground segmentation algorithm to generate ground segmentation results; the ground segmentation results include target ground points and target non-ground points.

[0014] Optionally, the semantic point cloud map includes map ground points, and before determining the vehicle's target pose based on the ground segmentation results and the semantic point cloud map, it also includes:

[0015] The target ground point and the map ground point are iteratively matched for nearest point, and the initial pose is updated based on the matching result to obtain the first pose of the vehicle.

[0016] Update the semantic point cloud map based on the first pose.

[0017] Optionally, the semantic point cloud map includes non-ground points on the map. The target pose of the vehicle is determined based on the ground segmentation results and the semantic point cloud map, including:

[0018] Semantic voxel map determined from non-ground points on the map;

[0019] The target pose is determined using a branch and bound algorithm based on the target's non-ground points and semantic voxel map.

[0020] Optionally, the target pose is determined using a branch and bound algorithm based on the target's non-ground points and semantic voxel map, including:

[0021] Multiple candidate poses are determined from a semantic voxel map using a branch and bound algorithm;

[0022] Based on the candidate pose, the target non-ground points are projected onto the semantic voxel map, and the probability score corresponding to each target non-ground point is calculated; the probability score refers to the degree of matching between the target non-ground point and the semantic voxel map.

[0023] Based on a pre-configured weighting strategy, the probability scores of each target non-ground point are weighted and summed to obtain the comprehensive score of each candidate pose.

[0024] The candidate pose with the highest overall score is selected as the target pose.

[0025] According to a second aspect of the present invention, a vehicle pose determination device is provided, comprising:

[0026] The semantic segmentation module is used to perform semantic segmentation processing on the acquired current environment point cloud of the vehicle, and determine the target environment point cloud based on the semantic segmentation results;

[0027] The ground segmentation module is used to perform ground segmentation processing on the point cloud of the target environment to obtain the ground segmentation result;

[0028] The pose determination module is used to determine the semantic point cloud map based on the initial pose of the vehicle, and to determine the target pose of the vehicle based on the ground segmentation results and the semantic point cloud map.

[0029] According to a third aspect of the present invention, an electronic device is provided, comprising:

[0030] One or more processors;

[0031] Memory, used to store one or more programs.

[0032] When one or more programs are executed by one or more processors, the one or more processors implement the methods of any of the above embodiments.

[0033] According to a fourth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method of any of the above embodiments.

[0034] According to a fifth aspect of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the method of any of the above embodiments.

[0035] One embodiment of the above invention has the following advantages or beneficial effects: by performing semantic segmentation processing on the acquired current environment point cloud of the vehicle, the target environment point cloud is determined based on the semantic segmentation result; ground segmentation processing is performed on the target environment point cloud to obtain the ground segmentation result; a semantic point cloud map is determined based on the acquired initial pose of the vehicle, and the target pose of the vehicle is determined based on the ground segmentation result and the semantic point cloud map; by performing semantic segmentation on the point cloud data of the vehicle's current environment, the embodiment of the present invention can identify and filter out irrelevant or noisy data, retaining only the information useful for vehicle positioning, reducing unnecessary data processing and complex calculations and improving data accuracy; furthermore, performing ground segmentation processing on the semantically segmented target environment point cloud further simplifies data processing, extracting the actual ground where the vehicle is located, making the vertical positioning of the vehicle in three-dimensional space more accurate; in addition, by accurately matching the terrain features of the ground segmentation result with the semantic point cloud map, the error of the vehicle pose determination result can be reduced, the success rate can be improved, and the time consumption of the determination process can be shortened, thus improving efficiency.

[0036] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0037] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0038] Figure 1 This is a schematic diagram of the main flow of the vehicle pose determination method according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the main flow of a vehicle pose determination method according to a possible embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the main flow of a method for determining vehicle pose according to a preferred embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of the main modules of a vehicle pose determination device according to an embodiment of the present invention;

[0042] Figure 5 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0043] Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0044] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0045] It should be noted that the acquisition, storage, and application of personal information involved in the embodiments of the present invention comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0046] In current vehicle pose recognition scenarios, existing solutions typically rely on deep learning technology, building a model capable of recognizing vehicle position and orientation using a large amount of training data. Specifically, sensors collect environmental data around the vehicle, inputting this data into a trained model. The model then predicts the vehicle's pose based on the learned features. These models require extensive labeled data for training to ensure accurate vehicle pose recognition under various environmental conditions. However, due to the complexity and diversity of vehicle driving environments, such as lighting variations, dynamic obstacles, and weather effects, models struggle to fully cover all possible scenarios. Especially under extreme conditions (such as severe weather or complex urban environments), the model's generalization ability is limited. In practical applications, when the environment differs from the training data, the model's predictions can deviate significantly. Furthermore, in complex or unfamiliar scenarios, insufficient or imbalanced training data can significantly degrade model performance, reducing the overall success rate. When processing complex 3D point cloud data and a large number of features, especially when precise localization is required, the model needs multiple iterations or complex post-processing steps to improve accuracy, extending processing time. Therefore, current machine learning solutions have large errors and low success rates in determining vehicle pose, and the determination process is time-consuming and inefficient.

[0047] In view of this, according to one aspect of the present invention, a method for determining vehicle pose is provided.

[0048] Figure 1 This is a schematic diagram illustrating the main flow of a vehicle pose determination method according to an embodiment of the present invention. Figure 1 As shown, the method for determining the vehicle pose according to an embodiment of the present invention includes the following steps S101 to S103.

[0049] Step S101: Perform semantic segmentation on the acquired current environment point cloud of the vehicle, and determine the target environment point cloud based on the semantic segmentation results.

[0050] The vehicle can be any type of mobile device, such as a car, a driverless car, a drone, or an automated transport robot. The current environment point cloud refers to the 3D point cloud data of the vehicle's current surrounding environment, acquired through sensors (such as LiDAR, depth cameras, etc.), which includes the spatial location information of surrounding objects.

[0051] Semantic segmentation classifies each point in a point cloud into different semantic categories, such as roads, buildings, pedestrians, and vehicles. Semantic segmentation can be processed using feature-based machine learning methods or deep learning-based convolutional neural network methods. These methods construct complex semantic segmentation models to extract features from the current environment's point cloud and perform accurate semantic segmentation. Additionally, rule-based segmentation methods and graph-based methods can be used. Rule-based segmentation methods use manually designed rules and thresholds to distinguish different parts based on the geometric characteristics of the current environment's point cloud (such as height, density, and curvature). For example, setting a height threshold can separate ground points from non-ground points, or a density threshold can be used to detect and segment obstacles. Graph-based methods treat the current environment's point cloud as a graph structure, where edges between points represent their adjacency relationships. Graph segmentation algorithms, such as graph cutting or minimum spanning tree segmentation, can divide the current environment's point cloud into different parts.

[0052] The result of semantic segmentation can be a point cloud with semantic labels, where each point is assigned a category label. Based on the semantic segmentation results, the target environment point cloud can be determined. The target environment point cloud refers to the point cloud data after removing obstacles of a specific category. For example, in autonomous driving, dynamic obstacles (such as pedestrians and other vehicles) in the current environment point cloud can be removed based on the semantic segmentation results, retaining only the point cloud data of the static environment (such as roads, buildings, etc.) to reduce unnecessary interference and make the results more accurate. Specifically, after semantic segmentation, points of non-target categories can be filtered out according to predefined target categories. For example, if the target environment point cloud only needs to contain point cloud data of roads and buildings, points of other categories (such as pedestrians, vehicles, trees, etc.) can be filtered out. The final point cloud data obtained is the target environment point cloud.

[0053] Step S102: Perform ground segmentation processing on the target environment point cloud to obtain the ground segmentation result.

[0054] Ground segmentation refers to the process of identifying and separating ground points from non-ground points in a target environment point cloud, in order to distinguish between ground and non-ground parts and better understand and utilize environmental information. The ground segmentation result includes outputs such as ground point clouds and non-ground point clouds. The ground point cloud is the set of points identified as ground, and the non-ground point cloud is the set of points identified as non-ground.

[0055] Ground segmentation of target environment point clouds can be performed in various ways, such as simple filtering based on height thresholds, model-based segmentation, and machine learning-based classification. Simple filtering based on height thresholds simply identifies points below a certain height threshold as ground points by setting one or more thresholds. Model-based segmentation, such as RANSAC (Random Sample Consensus), identifies ground points by fitting a planar model. Machine learning-based classification utilizes a trained model to classify points based on the geometric features and contextual information of the target environment point cloud, offering high segmentation accuracy and adaptability. Specifically, the target environment point cloud can be preprocessed, including noise removal and downsampling; based on the selected ground segmentation method, feature values ​​such as height and curvature are calculated for each point; the selected ground segmentation algorithm is applied to classify the points in the point cloud into ground points and non-ground points according to the feature values ​​and algorithm rules; the segmentation results are saved as two independent point cloud files or data structures, representing the ground point cloud and the non-ground point cloud respectively.

[0056] Step S103: Determine the semantic point cloud map based on the obtained initial pose of the vehicle, and determine the target pose of the vehicle based on the ground segmentation results and the semantic point cloud map.

[0057] The initial pose of a vehicle refers to its known position and orientation when it begins a navigation or localization task, including its position coordinates and orientation angle in the global coordinate system. The initial pose can be obtained through GPS (Global Positioning System), IMU (Inertial Measurement Unit), or similar methods. A semantic point cloud map is a map containing three-dimensional spatial information of points in the environment, along with semantic labels. Semantic labels refer to the annotations of the category (such as road, building, pedestrian, vehicle, etc.) to which each point in the point cloud belongs.

[0058] Specifically, the initial pose can be obtained using visual inertial odometry or wheel speed measurement, and the environment can be scanned and mapped by combining the initial pose with sensor data (such as LiDAR point cloud and camera images). The obtained point cloud data can be processed by semantic segmentation model or other semantic segmentation methods, and each point can be assigned a corresponding semantic label. Then, a semantic point cloud map can be built and updated based on the initial pose.

[0059] The vehicle's target pose is determined based on ground segmentation results and a semantic point cloud map. Ground segmentation divides the current environment point cloud into ground points and non-ground points. Ground points can be used for coarse matching to correct the vehicle's vertical and horizontal tilt. The segmented current environment point cloud is then matched with the corresponding parts in the semantic point cloud map. Algorithms such as ICP (Iterative Closest Point) are used to align the ground points in the current environment point cloud with those in the semantic point cloud map, thus obtaining a preliminary pose estimate. Based on the semantic information in the semantic point cloud map, the non-ground points in the current environment point cloud are matched with the semantic point cloud map in a more refined manner. By calculating the matching probability between each non-ground point and the corresponding category point in the map, a branch and bound algorithm is used to search the pose space to find the pose that achieves the highest matching score. This score integrates the geometric and semantic consistency between the point cloud and the map. Finally, the iterative optimization process converges to a globally optimal target pose, which is the vehicle's accurate position and orientation in the current environment. Graph optimization and particle filtering techniques can also be used. Graph optimization methods construct an optimization problem that includes multiple frames of point clouds and corresponding poses. Using nonlinear optimization algorithms, the error of the entire graph is minimized, thereby accurately estimating the current pose. Particle filtering techniques maintain a set of particles, each representing a possible vehicle pose. Based on the ground segmentation results and semantic point cloud map, the weight of each particle is updated according to its matching degree, eventually converging to the target pose.

[0060] This invention embodiment performs semantic segmentation on the acquired vehicle's current environment point cloud, determines the target environment point cloud based on the semantic segmentation results, performs ground segmentation on the target environment point cloud to obtain ground segmentation results, determines a semantic point cloud map based on the acquired vehicle's initial pose, and determines the vehicle's target pose based on the ground segmentation results and the semantic point cloud map. This invention embodiment, by performing semantic segmentation on the vehicle's current environment point cloud data, integrates semantic information into the specific determination process, increases feature discriminativeness, identifies and filters irrelevant or noisy data, retains only information useful for vehicle positioning, reduces unnecessary data processing and complex calculations, and improves data accuracy. Furthermore, performing ground segmentation on the semantically segmented target environment point cloud further simplifies data processing, extracts the actual ground where the vehicle is located, and makes the vertical positioning of the vehicle in three-dimensional space more accurate. In addition, by accurately matching the ground segmentation results with the terrain features of the semantic point cloud map, the error in the vehicle pose determination result can be reduced, the success rate can be improved, and the determination process time can be shortened, thus improving efficiency.

[0061] Optionally, determining the target environment point cloud based on the semantic segmentation result includes: removing target obstacles from the current environment point cloud based on the semantic segmentation result to obtain the target environment point cloud.

[0062] Specifically, a deep learning-based semantic segmentation model can be used. This model can identify and label objects of different categories, such as pedestrians, vehicles, and buildings. By analyzing the segmentation results, the point cloud data of the target obstacle can be removed from the overall environment point cloud, resulting in a target environment point cloud without the target obstacle. Rule-based filtering techniques, such as height, shape, and density, can also be used to identify and remove obstacles from the current environment point cloud. For example, a height threshold can be set to remove points above or below a certain range, thereby filtering out obvious vehicle or pedestrian point clouds. This embodiment integrates semantic information into the determination process, increasing the distinguishability of different object categories and improving the success rate of the determination while reducing time consumption.

[0063] Optionally, the target environment point cloud is subjected to ground segmentation processing to obtain ground segmentation results, including: dividing the target environment point cloud using a ground segmentation algorithm to generate ground segmentation results; wherein, the ground segmentation results include target ground points and target non-ground points.

[0064] Ground segmentation algorithms include plane fitting-based algorithms, height statistics-based thresholding methods, and deep learning-based segmentation models. For example, when using plane fitting-based algorithms, ground points are identified by fitting a plane model and detecting points that conform to the model; points that do not conform to the plane model are marked as non-ground points. Height statistics methods can be based on the height distribution of the target environment point cloud, setting a height threshold to classify points below the threshold as ground points and points above the threshold as non-ground points. Deep learning methods can train neural networks to directly identify ground and non-ground regions from the target environment point cloud. The ground segmentation algorithm completes the processing, generating ground segmentation results including target ground points and target non-ground points. Target ground points are the point cloud data identified as ground areas, possessing flat and continuous characteristics. Target non-ground points include all point cloud data considered not to belong to the ground, potentially representing obstacles such as vehicles, pedestrians, and buildings. This ground segmentation process improves the utilization efficiency of the target environment point cloud, enabling subsequent processing to be accurate and efficient.

[0065] Optionally, Figure 2 This is a schematic diagram illustrating the main flow of a vehicle pose determination method according to a possible embodiment of the present invention. Figure 2 As shown, the semantic point cloud map includes map ground points. Before determining the target pose of the vehicle based on the ground segmentation results and the semantic point cloud map, steps S201 to S202 are also included.

[0066] Step S201: Iteratively match the nearest point between the target ground point and the map ground point, and update the initial pose based on the matching result to obtain the first pose of the vehicle.

[0067] Step S202: Update the semantic point cloud map based on the first pose.

[0068] Among them, Iterative Closest Point (ICP) is an algorithm for point cloud registration. It continuously adjusts the position and pose of one point cloud relative to another to minimize the distance between them. Through the matching process, the ground points in the current frame can be aligned with the ground points in the map, thereby correcting the vehicle's pose. In the matching process, the initial pose is the starting point of the algorithm. As the ICP algorithm iterates, the vehicle's pose can be gradually updated to better match the two point clouds. The updated pose is determined as the vehicle's first pose, which represents the vehicle's position and orientation after the initial correction.

[0069] Updating the semantic point cloud map refers to repositioning and fusing the current point cloud data based on the new vehicle pose, making the semantic point cloud map more accurate and consistent. Specifically, the point cloud data in the current environment can be transformed into a global coordinate system and merged with existing map data, retaining the latest environmental information.

[0070] The embodiments of the present invention enable the semantic point cloud map to accurately reflect the environment around the vehicle during continuous updating and optimization, providing a reliable data foundation for subsequent determination of the vehicle's target pose.

[0071] Optionally, the semantic point cloud map includes map non-ground points. The target pose of the vehicle is determined based on the ground segmentation results and the semantic point cloud map, including: determining a semantic voxel map based on the map non-ground points; and determining the target pose using a branch and bound algorithm based on the target non-ground points and the semantic voxel map.

[0072] Semantic point cloud maps include not only ground points but also non-ground points on the map. A semantic voxel map is determined based on these non-ground points. A semantic voxel map is a three-dimensional grid structure that divides non-ground point cloud data into voxel (i.e., three-dimensional pixel) grids according to a certain spatial resolution, storing corresponding semantic labels and statistical information in each grid. This allows for more efficient organization and processing of large amounts of point cloud data, making subsequent processing and analysis more efficient and accurate. Specifically, non-ground points can be directly assigned to predefined voxel grids based on their spatial location, with each voxel grid storing the semantic labels and location information of these points. Alternatively, a Gaussian process can be used to perform multi-Gaussian fitting on each voxel to capture the spatial distribution characteristics of the point cloud while preserving the semantic label information.

[0073] Branch and bound algorithms are optimization algorithms used to find the optimal solution among a large number of candidate solutions. Specifically, one approach is a hierarchical search from coarse-grained to fine-grained. First, the search space is divided into large coarse-grained grids. Each grid is initially evaluated to identify regions with high matching degrees. Then, based on this initial screening, these high-matching regions are further refined, gradually narrowing the search range to smaller, fine-grained grids. At each level of the search, the matching degree between the current point cloud and the semantic voxel map is evaluated, progressively approaching the optimal pose. This approach effectively reduces computational cost while ensuring the accuracy of the search results. Another approach is a branch and bound algorithm based on heuristic search. This uses a heuristic function to evaluate the potential matching degree of each candidate pose and guides the search process. First, the heuristic function is used to calculate the estimated score of each candidate pose, prioritizing the expansion of candidate poses with higher scores. As the search progresses, the estimated scores of candidate poses are continuously updated and adjusted, progressively approaching the global optimum. Through a reasonable heuristic strategy, both search efficiency and accuracy are effectively improved.

[0074] Optionally, the target pose is determined using a branch and bound algorithm based on the target non-ground points and the semantic voxel map, including: determining multiple candidate poses from the semantic voxel map using the branch and bound algorithm; projecting the target non-ground points onto the semantic voxel map based on the candidate poses, and calculating the probability score corresponding to each target non-ground point; the probability score refers to the degree of matching between the target non-ground point and the semantic voxel map; based on a pre-configured weighting strategy, the probability scores of each target non-ground point are weighted and summed to obtain the comprehensive score of each candidate pose; and the candidate pose with the highest comprehensive score is determined as the target pose.

[0075] Specifically, a branch-and-bound algorithm is used to determine multiple candidate poses from a semantic voxel map. A preliminary search on a lower-resolution semantic voxel map generates a set of possible candidate poses. The search space is divided into multiple regions, and the possible poses are identified by progressively narrowing the search range, thus reducing computational complexity. Based on these candidate poses, target non-ground points are projected onto the corresponding semantic voxel map. The projection process maps the target non-ground point cloud data onto a voxel grid, and a probability score is calculated for each target non-ground point. This probability score is determined by comparing the matching degree between the target non-ground point and its corresponding location in the semantic voxel map. Based on a pre-configured weighting strategy, the probability scores of each target non-ground point are weighted and summed. The weighting strategy sets the importance of different semantic labels, and the weighting reflects the contribution of different types of point clouds to the matching results. The weighted sum generates a comprehensive score for each candidate pose, representing the degree of matching between the candidate pose and the current environment point cloud. The candidate pose with the highest comprehensive score is determined as the target pose. This can be achieved by comparing the comprehensive scores of all candidate poses and selecting the pose with the highest score as the final vehicle target pose. The target pose selected in this embodiment of the invention has the best matching effect in the current environment, thereby achieving high-precision positioning results and vehicle pose determination results.

[0076] Figure 3 This is a schematic diagram illustrating the main flow of a method for determining vehicle pose according to a preferred embodiment of the present invention. Figure 3As shown, to achieve precise vehicle positioning, environmental data is first acquired using LiDAR and GPS. The point cloud data generated by LiDAR undergoes semantic segmentation to identify and label different categories of objects, such as ground, buildings, and obstacles. GPS provides a semantic point cloud map within a certain range, and these map points are transformed into the LiDAR coordinate system. Next, ground segmentation processing is performed on the LiDAR data to distinguish between ground points and non-ground points. Then, ICP matching is performed between the ground point cloud of the current frame and the ground point cloud in the map to obtain the three degrees of freedom of the vehicle pose: z, roll, and pitch (coordinates, roll angle, pitch angle), and the initial pose T0 is updated to T1. A semantic voxel Gaussian map (MVG) is generated, and non-ground points (MNG) are transformed into the current frame. A multi-resolution semantic voxel Gaussian map (MVG) is generated using max-pooling. Using a branch and bound approach, the search proceeds from low resolution to high resolution, calculating the optimal position yaw, x, y (yaw angle, x-coordinate, y-coordinate) at each resolution, updating T0, and finally determining the vehicle's precise pose T*.

[0077] Specifically, in step S301, the environmental point cloud and GPS positioning information T0 around the vehicle are obtained through the lidar and GPS devices configured on the vehicle.

[0078] Step S302: Use a semantic segmentation model to perform semantic segmentation on the point cloud of the current environment acquired by the lidar device, and use the semantic segmentation results to remove dynamic obstacles (vehicles, pedestrians, etc.) from the point cloud.

[0079] Step S303: Perform ground segmentation on the point cloud of the current environment acquired by the lidar device, dividing it into ground points PG and non-ground points PNG.

[0080] Step S304: Using the initial pose T0 as the center, acquire a high-precision semantic point cloud map within a certain range. The semantic point cloud map includes ground point cloud (MG) and non-ground point cloud (MNG), and transform it to the radar coordinate system. The "certain range" is determined based on the GPS device's positioning status and accuracy. GPS positioning accuracy varies in different environments. In open, unobstructed environments, GPS accuracy is high, so the range is set relatively small, for example, 5m. In obstructed environments, such as when the vehicle is parked next to a tall building or has trees on top, GPS accuracy is low, so the range is set larger, for example, 10m.

[0081] In step S305, the current frame ground point PG obtained in step S303 is matched with the map ground point MG obtained in step S304 to determine the current vehicle's position coordinates z and directions roll and pitch, and T0 is updated to obtain T1.

[0082] Step S306: Divide the non-terrestrial point cloud MNG obtained in step S304 into voxels of a certain resolution in 3D space, and perform multi-Gaussian fitting along the height direction within each voxel grid. Store the semantic labels in the corresponding voxel grid (if a voxel grid contains multiple semantic labels, all are stored). Based on the high-resolution voxel map, construct multi-layer low-resolution voxel maps, where the semantic label of each voxel grid in the low-resolution map is the union of its corresponding high-resolution voxel labels.

[0083] In step S307, x, y, and yaw are searched from low resolution to high resolution using a branch-and-bound method with a certain step size. The current point cloud PNG projection voxel mesh is used with six degrees of freedom pose, as shown in the following formula. The probability corresponding to each point is calculated as a score, and weighted by semantic information to obtain the total score. The low-resolution mesh with the highest score is then expanded until the highest resolution layer is found, corresponding to the highest score.

[0084]

[0085]

[0086] in, Indicates the current pose. Indicates the current point cloud One point, Indicates the weight. The semantic label for the i-th point in the current frame. The semantic labels are the corresponding voxel grids in the map.

[0087] Step S308: Obtain the position and direction of the highest score, and convert them to the global map coordinate system to obtain the final pose T* of the current vehicle.

[0088] This invention integrates semantic information into global relocalization and obtains the six-DOF global pose through ground point matching and branch-and-bound search. By using semantic information to increase the discriminative power of features, the success rate of relocalization is improved while reducing time consumption.

[0089] According to a second aspect of the embodiments of the present invention, such as Figure 4 As shown, a vehicle pose determination device 400 is provided, comprising:

[0090] The semantic segmentation module 401 is used to perform semantic segmentation processing on the acquired current environment point cloud of the vehicle, and determine the target environment point cloud based on the semantic segmentation result;

[0091] The ground segmentation module 402 is used to perform ground segmentation processing on the target environment point cloud to obtain the ground segmentation result;

[0092] The pose determination module 403 is used to determine the semantic point cloud map based on the acquired initial pose of the vehicle, and to determine the target pose of the vehicle based on the ground segmentation results and the semantic point cloud map.

[0093] Optionally, the semantic segmentation module 401 is also used for:

[0094] The target environment point cloud is obtained by removing the target obstacle from the current environment point cloud based on the semantic segmentation result.

[0095] Optionally, the ground segmentation module 402 is also used for:

[0096] The target environment point cloud is divided using a ground segmentation algorithm to generate ground segmentation results; the ground segmentation results include target ground points and target non-ground points.

[0097] Optionally, the semantic point cloud map includes map ground points, and the vehicle pose determination device 400 further includes a pose update module, which is used for:

[0098] The target ground point and the map ground point are iteratively matched for nearest point, and the initial pose is updated based on the matching result to obtain the first pose of the vehicle.

[0099] Update the semantic point cloud map based on the first pose.

[0100] Optionally, the semantic point cloud map includes non-ground points on the map, and the pose determination module 403 is also used for:

[0101] Semantic voxel map determined from non-ground points on the map;

[0102] The target pose is determined using a branch and bound algorithm based on the target's non-ground points and semantic voxel map.

[0103] Optionally, the pose determination module 403 is also used for:

[0104] Multiple candidate poses are determined from a semantic voxel map using a branch and bound algorithm;

[0105] Based on the candidate pose, the target non-ground points are projected onto the semantic voxel map, and the probability score corresponding to each target non-ground point is calculated; the probability score refers to the degree of matching between the target non-ground point and the semantic voxel map.

[0106] Based on a pre-configured weighting strategy, the probability scores of each target non-ground point are weighted and summed to obtain the comprehensive score of each candidate pose.

[0107] The candidate pose with the highest overall score is selected as the target pose.

[0108] It should be noted that the specific implementation of the vehicle position determination device of the present invention has been described in detail in the above vehicle position determination method, so the content will not be repeated here.

[0109] According to a third aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the present invention.

[0110] According to a fourth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method provided in the first aspect of the present invention.

[0111] According to a fifth aspect of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided in the first aspect of the present invention.

[0112] Figure 5 An exemplary system architecture 500 is shown, in which the vehicle pose determination method or vehicle pose determination apparatus of the embodiments of this application can be applied.

[0113] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, and 503, a network 504, and a server 505. Network 504 serves as the medium for providing communication links between terminal devices 501, 502, and 503 and server 505. Network 504 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0114] Users can use terminal devices 501, 502, and 503 to interact with server 505 via network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, and 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0115] Terminal devices 501, 502, and 503 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0116] Server 505 can be a server that provides various services, such as a backend management server that supports shopping websites browsed by users using terminal devices 501, 502, and 503 (for example only). The backend management server can analyze and process data such as received software startup requests, and feed back the processing results (such as target pose - for example only) to the terminal devices.

[0117] It should be noted that the vehicle pose determination method provided in this application embodiment is generally executed by server 505, and correspondingly, the vehicle pose determination device is generally set in server 505.

[0118] It should be understood that Figure 5 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0119] The following is for reference. Figure 6 It shows a schematic diagram of the structure of a computer system 600 suitable for implementing a terminal device according to the embodiments of this application. Figure 6 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0120] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the computer system 600. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0121] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0122] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs the functions defined above in the system of this application.

[0123] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0125] The units described in the embodiments of this application can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor can be described as including a pose determination module, a ground segmentation module, and a semantic segmentation module. The names of these modules do not necessarily limit the module itself; for example, the semantic segmentation module can also be described as "a module for performing semantic segmentation processing on the acquired current environment point cloud of the vehicle, and determining the target environment point cloud based on the semantic segmentation results."

[0126] In another aspect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: performing semantic segmentation processing on the acquired current environment point cloud of the vehicle, and determining a target environment point cloud based on the semantic segmentation result; performing ground segmentation processing on the target environment point cloud to obtain a ground segmentation result; determining a semantic point cloud map based on the acquired initial pose of the vehicle, and determining the target pose of the vehicle based on the ground segmentation result and the semantic point cloud map.

[0127] The computer program product provided in this embodiment of the invention includes a computer program that, when executed by a processor, implements the vehicle pose determination method in this embodiment of the invention.

[0128] According to the technical solution of the present invention, the following advantages or beneficial effects are achieved: by performing semantic segmentation processing on the current environment point cloud of the acquired vehicle, the target environment point cloud is determined based on the semantic segmentation result; ground segmentation processing is performed on the target environment point cloud to obtain the ground segmentation result; a semantic point cloud map is determined according to the initial pose of the acquired vehicle, and the target pose of the vehicle is determined based on the ground segmentation result and the semantic point cloud map; thereby, the error of the vehicle pose determination result can be reduced, the success rate can be improved, and the time consumption of the determination process can be shortened, thus improving efficiency.

[0129] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

[0130] It should be noted that the acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

Claims

1. A method for determining the position and orientation of a vehicle, characterized in that, The method comprises the following steps: performing semantic segmentation on the obtained current environment point cloud of the vehicle, and determining a target environment point cloud based on the semantic segmentation result; performing ground segmentation on the target environment point cloud to obtain a ground segmentation result; determining a semantic point cloud map according to the obtained initial pose of the vehicle, and determining a target pose of the vehicle based on the ground segmentation result and the semantic point cloud map.

2. The method of claim 1, wherein, The method for determining the target environment point cloud based on the semantic segmentation result comprises the following steps: removing target obstacles in the current environment point cloud according to the semantic segmentation result to obtain the target environment point cloud.

3. The method of claim 1, wherein, The method for performing ground segmentation on the target environment point cloud to obtain a ground segmentation result comprises the following steps: dividing the target environment point cloud by using a ground segmentation algorithm to generate a ground segmentation result; wherein the ground segmentation result comprises target ground points and target non-ground points.

4. The method of claim 3, wherein, The semantic point cloud map comprises map ground points, and before determining the target pose of the vehicle based on the ground segmentation result and the semantic point cloud map, the method further comprises the following steps: performing iterative closest point matching between the target ground points and the map ground points, updating the initial pose based on the matching result to obtain a first pose of the vehicle; updating the semantic point cloud map according to the first pose.

5. The method of claim 4, wherein, The semantic point cloud map comprises map non-ground points, and the method for determining the target pose of the vehicle based on the ground segmentation result and the semantic point cloud map comprises the following steps: determining a semantic voxel map based on the map non-ground points; determining the target pose by using a branch and bound algorithm according to the target non-ground points and the semantic voxel map.

6. The method of claim 5, wherein, The method for determining the target pose by using a branch and bound algorithm according to the target non-ground points and the semantic voxel map comprises the following steps: determining a plurality of candidate poses from the semantic voxel map by using a branch and bound algorithm; projecting the target non-ground points into the semantic voxel map based on the candidate poses, and calculating a probability score corresponding to each target non-ground point; the probability score refers to the matching degree between the target non-ground point and the semantic voxel map; weighting and summing the probability scores of each target non-ground point based on a preconfigured weight strategy to obtain a comprehensive score of each candidate pose; determining the candidate pose with the highest comprehensive score as the target pose.

7. A device for determining a pose of a vehicle, characterized in that The method comprises the following steps: a semantic segmentation module is configured to perform semantic segmentation on the obtained current environment point cloud of the vehicle, and determine a target environment point cloud based on the semantic segmentation result; a ground segmentation module is configured to perform ground segmentation on the target environment point cloud to obtain a ground segmentation result; a pose determination module is configured to determine a semantic point cloud map according to the obtained initial pose of the vehicle, and determine a target pose of the vehicle based on the ground segmentation result and the semantic point cloud map.

8. An electronic device, comprising: The method comprises the following steps: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-6.

9. A computer readable medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method according to any one of claims 1-6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method according to any one of claims 1-6.

Images