Point cloud-based object recognition method, apparatus, computer program product, and vehicle

Abstract

The application provides a point cloud-based object recognition method, device, computer program product and vehicle. The method comprises: obtaining point cloud data by acquiring point cloud collected by a laser radar of a vehicle; identifying the point cloud data by using a primary recognition mode to obtain a candidate object region corresponding to the point cloud data, wherein the primary recognition mode is a mode of using point cloud bounding box recognition and / or semantic segmentation recognition; performing secondary recognition on the candidate object region, and merging at least two candidate object regions with the highest similarity to obtain an object region corresponding to the point cloud data, wherein the secondary recognition at least comprises neural network model recognition. The scheme solves the problem of low accuracy in separating obstacles from point cloud in the prior art.

Description

Technical Field

[0001] This application relates to the field of autonomous driving point cloud recognition technology, and more specifically, to a point cloud-based object recognition method, a point cloud-based object recognition device, a computer program product, and a vehicle. Background Technology

[0002] In autonomous driving and intelligent robot perception systems, obstacle segmentation of LiDAR point clouds is a core component of environmental understanding, aiming to accurately identify and separate individual obstacle entities from the raw point cloud. Current mainstream technologies fall into two main categories: bounding box-based detection methods and semantic segmentation-based methods.

[0003] Bounding box detection methods directly regress the 3D bounding boxes of obstacles using deep neural networks, and then group the point cloud based on the bounding box range to extract obstacles. Semantic segmentation methods output category labels for each point or grid, and then merge points belonging to the same object using clustering algorithms.

[0004] However, current solutions still suffer from misclassification or oversegmentation when dealing with irregularly shaped vehicles (such as large cargo trucks, sweepers, and crash buffers), causing obstacles to be incorrectly segmented into multiple subsets. Therefore, the current method has a low accuracy rate in separating obstacles from point clouds. Summary of the Invention

[0005] The main objective of this application is to provide a point cloud-based object recognition method, a point cloud-based object recognition device, a computer program product, and a vehicle, so as to at least solve the problem of low accuracy in separating obstacles from point clouds in the prior art.

[0006] To achieve the above objectives, according to one aspect of this application, a point cloud-based object recognition method is provided, comprising: acquiring point clouds collected by a vehicle's lidar to obtain point cloud data; identifying the point cloud data using an initial recognition method to obtain candidate object regions corresponding to the point cloud data, wherein the initial recognition method is point cloud bounding box recognition and / or semantic segmentation recognition; and re-identifying the candidate object regions by merging at least two candidate object regions with the highest similarity to obtain the object region corresponding to the point cloud data, wherein the re-identification includes at least identification using a neural network model.

[0007] Optionally, the point cloud data is identified using an initial identification method to obtain candidate object regions corresponding to the point cloud data, including one of the following: identifying the point cloud data using a point cloud bounding box identification method to obtain the candidate object regions; identifying the point cloud data using a semantic segmentation identification method to obtain the candidate object regions; identifying the point cloud data using a point cloud bounding box identification method, and then re-identifying the point cloud data after point cloud bounding box identification using a semantic segmentation identification method to obtain the candidate object regions; or identifying the point cloud data using a semantic segmentation identification method, and then re-identifying the point cloud data after semantic segmentation identification using a point cloud bounding box identification method to obtain the candidate object regions.

[0008] Optionally, the candidate object regions are re-identified, and at least two candidate object regions with the highest similarity are merged to obtain the object region corresponding to the point cloud data. This includes: calculating the confidence that any two candidate object regions belong to the same object to obtain a first merging probability; and merging the two candidate object regions to obtain the object region when the first merging probability is greater than or equal to a first preset probability threshold.

[0009] Optionally, calculating the confidence that any two candidate object regions belong to the same object to obtain a first merging probability includes: acquiring multiple evaluation parameters, wherein the evaluation parameters include one or more of the following: distance between two candidate object regions, outline between two candidate object regions, size between two candidate object regions, orientation between two candidate object regions, point cloud density between two candidate object regions, continuity between two candidate object regions, consistency between two candidate object regions, and semantic label similarity between two candidate object regions; scoring the similarity between two candidate object regions according to the evaluation parameters to obtain multiple evaluation scores, wherein the evaluation scores correspond one-to-one with the evaluation parameters, and the distance between two candidate object regions is related to the corresponding evaluation score. The scores are positively correlated as follows: the similarity of the outlines between two candidate object regions is positively correlated with the corresponding evaluation score; the similarity of the dimensions between two candidate object regions is positively correlated with the corresponding evaluation score; the similarity of the orientation between two candidate object regions is positively correlated with the corresponding evaluation score; the similarity of the point cloud density between two candidate object regions is positively correlated with the corresponding evaluation score; the density of the continuity between two candidate object regions is positively correlated with the corresponding evaluation score; the strength of the consistency between two candidate object regions is positively correlated with the corresponding evaluation score; and the similarity of the semantic labels between two candidate object regions is positively correlated with the corresponding evaluation score. A weighted average of multiple evaluation scores is calculated to obtain the first merging probability.

[0010] Optionally, when the first merging probability is greater than or equal to a first preset probability threshold, merging the two candidate object regions to obtain the object region includes: when the first merging probability is greater than or equal to the first preset probability threshold, extracting the union of the two candidate object regions; and determining the extracted union as the object region.

[0011] Optionally, the candidate object regions are re-identified, and at least two candidate object regions with the highest similarity are merged to obtain the object region corresponding to the point cloud data. This includes: extracting any one of the merged candidate object regions; extracting any one of the unmerged candidate object regions; calculating the confidence that the merged candidate object region and the unmerged candidate object region belong to the same object to obtain a second merging probability; and merging the merged candidate object region and the unmerged candidate object region when the second merging probability is greater than or equal to a second preset probability threshold to obtain the object region.

[0012] Optionally, the candidate object regions are re-identified, and at least two candidate object regions with the highest similarity are merged to obtain the object region corresponding to the point cloud data. This includes: extracting any two merged candidate object regions; calculating the confidence that the two merged candidate object regions belong to the same object to obtain a third merging probability; and merging the two merged candidate object regions again if the third merging probability is greater than or equal to a third preset probability threshold to obtain the object region.

[0013] According to another aspect of this application, a point cloud-based object recognition device is provided, comprising: an acquisition unit for acquiring point clouds collected by a vehicle's lidar to obtain point cloud data; a first recognition unit for recognizing the point cloud data using an initial recognition method to obtain candidate object regions corresponding to the point cloud data, wherein the initial recognition method is point cloud bounding box recognition and / or semantic segmentation recognition; and a second recognition unit for re-recognizing the candidate object regions by merging at least two candidate object regions with the highest similarity to obtain object regions corresponding to the point cloud data, wherein the re-recognition includes at least recognition using a neural network model.

[0014] According to another aspect of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of any of the point cloud-based object recognition methods.

[0015] According to another aspect of this application, a vehicle is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any of the point cloud-based object recognition methods described above.

[0016] The technical solution of this application adds a refined recognition method on the basis of the existing solution (bounding box recognition or segmentation recognition). That is, coarse recognition is performed first, followed by fine recognition. The original point cloud is not reprocessed. Instead, the candidate object region obtained from the initial recognition is used as the basic unit. If two object regions are similar, they are merged. In this way, the same object will not be segmented and a complete object region can be formed, which improves the accuracy of separating obstacles from the point cloud. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This diagram illustrates the process of identifying point cloud data using point cloud bounding box recognition in the prior art.

[0019] Figure 2 This diagram illustrates the process of identifying point cloud data using semantic segmentation in the prior art.

[0020] Figure 3 A hardware structure block diagram of a mobile terminal performing a point cloud-based object recognition method is shown in an embodiment of this application.

[0021] Figure 4 A flowchart illustrating a point cloud-based object recognition method according to an embodiment of this application is shown.

[0022] Figure 5 A flowchart illustrating another point cloud-based object recognition method of this scheme is shown.

[0023] Figure 6 A structural block diagram of a point cloud-based object recognition device provided according to an embodiment of this application is shown.

[0024] The above figures include the following reference numerals:

[0025] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0030] Most current point cloud obstacle segmentation models based on LiDAR rely on bounding box detection or semantic segmentation overlay post-processing methods.

[0031] like Figure 1 As shown, given a LiDAR point cloud as input, a bounding box detection method outputs several bounding boxes for the LiDAR point cloud through a deep learning model. Based on the bounding box results, the LiDAR point cloud is grouped / segmented to obtain several subsets of the point cloud, each subset representing an obstacle. Specifically, the original 3D point cloud acquired by the LiDAR is first processed using a deep learning model. Using any common neural network model, the point cloud is divided into point clusters. Then, the 3D bounding box of each potential obstacle is predicted. The bounding box is defined by parameters such as center coordinates, length, width, height, and heading angle. Finally, all point clouds falling within the same bounding box are considered as one obstacle.

[0032] like Figure 2 As shown, semantic segmentation-based methods output the classification result for each LiDAR point or each grid cell at a given resolution using a deep learning model. Based on the point-by-point or grid-by-grid classification results, the LiDAR point cloud is clustered to obtain several subsets, each representing an obstacle. Methods based on bounding box detection or semantic segmentation are generally combined with rule-based post-processing to address inaccuracies in bounding box detection or semantic segmentation results. Specifically, semantic segmentation-based methods use any common neural network model to predict the semantic category (e.g., "vehicle," "pedestrian," "curb," "ground," etc.) for each point in the point cloud or a grid cell divided at a fixed resolution. Then, based on semantic consistency and spatial connectivity, a clustering algorithm is used to merge spatially adjacent points of the same category into continuous point cloud regions, each region representing an obstacle.

[0033] Existing technologies are prone to errors in bounding box or semantic segmentation outputs when dealing with irregularly shaped vehicle obstacles such as large cargo trucks, sweeping trucks, and crash buffer trucks. In such cases, obstacle detection becomes highly dependent on post-processing. Rule-based post-processing methods generally struggle to handle various irregularly shaped obstacles, leading to over-segmentation of obstacles (i.e., one obstacle is segmented into multiple obstacles).

[0034] As described in the background section, the accuracy of separating obstacles from point clouds in the prior art is low. To solve the above problems, embodiments of this application provide a point cloud-based object recognition method, a point cloud-based object recognition device, a computer program product, and a vehicle.

[0035] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0036] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 3 This is a hardware structure block diagram of a mobile terminal for a point cloud-based object recognition method according to an embodiment of the present invention. Figure 3 As shown, a mobile terminal may include one or more ( Figure 3 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 3 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 3 The more or fewer components shown, or having the same Figure 3 The different configurations shown.

[0037] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the point cloud-based object recognition method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0038] This embodiment provides a point cloud-based object recognition method that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0039] Figure 4 This is a flowchart illustrating a point cloud-based object recognition method according to an embodiment of this application. Figure 4 As shown, the method includes the following steps:

[0040] Step S401: Obtain the point cloud data collected by the vehicle's lidar.

[0041] Specifically, a raw set of three-dimensional spatial points can be obtained from the lidar sensor installed on the vehicle. Each point contains three-dimensional coordinates (x, y, z) and optional reflection intensity information, forming discrete point cloud data that reflects the geometry of the surrounding environment.

[0042] Step S402: The point cloud data is identified using an initial identification method to obtain the candidate object region corresponding to the point cloud data. The initial identification method is point cloud bounding box identification and / or semantic segmentation identification.

[0043] Specifically, the "initial identification method" refers to the first round of rapid identification of point cloud data through a pre-trained deep learning model, which includes two optional paths: one is point cloud bounding box identification, which outputs several axis-aligned or rotated 3D bounding boxes, each containing a set of point clouds within it, representing a potential object; the other is semantic segmentation identification, which assigns a category label (such as "vehicle", "pedestrian", "road") to each point or its voxel raster, and then extracts the region composed of point clouds of the same category through connected component analysis; both can be used independently or in combination, and the output results are several incomplete "candidate object regions" that may have breaks or oversegmentation.

[0044] Step S403: Re-identify the above candidate object regions, merge at least two of the above candidate object regions with the highest similarity to obtain the object region corresponding to the above point cloud data, wherein the re-identification includes at least identification using a neural network model.

[0045] Specifically, "re-identification" refers to extracting features and modeling relationships between candidate object regions without using manually designed rules (such as distance thresholds, angle constraints, etc.). Instead, it uses a neural network model to extract features and model relationships between these regions. First, it extracts the geometric feature vectors (such as size, shape entropy, point density distribution, surface curvature, etc.) for each candidate region. Then, it combines all candidate regions into pairs to form feature pairs, which are input into a neural network structure such as a Transformer model. The output is the probability value that each pair of candidate regions should be merged. The pairs with the highest probabilities are selected, and a graph structure is constructed based on their connectivity. Finally, through connected component analysis, multiple candidate regions belonging to the same real object are automatically aggregated into a complete "object region". This method, through data-driven similarity learning, enables the model to adaptively identify the real relationships between non-standard shaped obstacles such as large cargo trucks, sweepers, and crash buffer vehicles, effectively solving the over-segmentation problem caused by rigid rules in existing technologies.

[0046] This embodiment adds a refined recognition method to the existing scheme (bounding box recognition or segmentation recognition). That is, coarse recognition is performed first, followed by fine recognition. The original point cloud is not reprocessed. Instead, the candidate object region obtained from the initial recognition is used as the basic unit. If two object regions are similar, they are merged. In this way, the same object will not be segmented and a complete object region can be formed, which improves the accuracy of separating obstacles from the point cloud.

[0047] Furthermore, the solution of this application can plan a driving path for the vehicle based on the object region. The point cloud of the object region is projected onto a two-dimensional ground plane in the vehicle coordinate system to generate an obstacle occupancy grid map; based on the obstacle occupancy grid map, combined with the vehicle's kinematic constraints and a preset safety distance, the drivable area is calculated; within the drivable area, a path planning algorithm is used to generate a collision-free driving path from the vehicle's current position to the target position.

[0048] In the specific implementation process, the point cloud data is identified using an initial recognition method to obtain the candidate object regions corresponding to the point cloud data. This can be achieved through one of the following steps: identifying the point cloud data using a point cloud bounding box recognition method to obtain the candidate object regions; identifying the point cloud data using a semantic segmentation recognition method to obtain the candidate object regions; identifying the point cloud data using a point cloud bounding box recognition method, and then re-identifying the point cloud data after point cloud bounding box recognition using a semantic segmentation recognition method to obtain the candidate object regions; and finally, identifying the point cloud data using a semantic segmentation recognition method, and then re-identifying the point cloud data after semantic segmentation recognition using a point cloud bounding box recognition method to obtain the candidate object regions.

[0049] This scheme takes given LiDAR point cloud data as input, outputs several bounding boxes for the LiDAR point clouds based on bounding box detection, and groups / segments the LiDAR point clouds based on the bounding box results to obtain several point cloud subsets, each subset representing an obstacle, i.e., a candidate object region. Semantic segmentation outputs the classification result for each LiDAR point or each grid cell at a given resolution, and clusters the LiDAR point clouds based on the point-by-point or grid-by-grid classification results to obtain several point cloud subsets, each subset representing an obstacle, i.e., a candidate object region. Of course, bounding box detection and semantic segmentation can also be combined; this scheme can flexibly choose either method to complete the initial identification.

[0050] In some embodiments, the candidate object regions are re-identified, and at least two candidate object regions with the highest similarity are merged to obtain the object region corresponding to the point cloud data. Specifically, this can be achieved through the following steps: calculating the confidence that any two candidate object regions belong to the same object to obtain a first merging probability; and merging the two candidate object regions when the first merging probability is greater than or equal to a first preset probability threshold to obtain the object region.

[0051] In this scheme, the confidence level of two candidate object regions belonging to the same real object can be calculated. The value range is from 0 to 1. When the two candidate object regions are greater than or equal to the first preset probability threshold, it is more likely that the two candidate object regions are the same object. Therefore, merging the two candidate object regions results in an object region that better ensures the integrity of the object.

[0052] Specifically, the first preset probability threshold can be 0.6, or of course, any other feasible value.

[0053] In addition, this scheme can also obtain the radar echo intensity features and geometric features of the aforementioned candidate object regions; fuse the radar echo intensity features with the aforementioned geometric features to form a joint feature vector; input the aforementioned joint feature vector into a deep learning model, and output the aforementioned first merging probability. The aforementioned radar echo intensity features include the mean, variance, intensity distribution histogram, and intensity gradient change rate of the lidar reflection intensity at each point in the aforementioned candidate object regions. In the feature fusion stage, the aforementioned radar echo intensity features and the aforementioned geometric features are concatenated or weighted in the same feature space to form a joint feature vector with a dimension of 256, which is used as the input to the Transformer network.

[0054] For example, although the body and brush device of a sweeper are geometrically divided into two regions due to movement or obstruction, they have a high degree of consistency in lidar echo intensity (mean difference less than 5%, similar intensity variance), and their geometric structure is spatially continuous. Based on this, the model outputs a merging probability of 0.83. On the other hand, although a parked truck and an adjacent metal trash can partially overlap in geometric projection, their echo intensity differs significantly (the trash can's mean intensity is more than 30% lower than the truck's, and its intensity variance is high). The model outputs a merging probability of 0.21 and determines that they should not be merged.

[0055] By introducing radar echo intensity characteristics as an auxiliary basis for merging and judging, and jointly modeling them with geometric features, the accuracy of recognition can be further enhanced.

[0056] In the specific implementation process, calculating the confidence that any two candidate object regions belong to the same object to obtain the first merging probability can be achieved through the following steps: obtaining multiple evaluation parameters, including one or more of the following: distance between two candidate object regions, outline between two candidate object regions, size between two candidate object regions, orientation between two candidate object regions, point cloud density between two candidate object regions, continuity between two candidate object regions, consistency between two candidate object regions, and semantic label similarity between two candidate object regions; scoring the similarity between two candidate object regions based on the above evaluation parameters to obtain multiple evaluation scores, wherein the evaluation scores correspond one-to-one with the above evaluation parameters, and the distance between two candidate object regions is related to the similarity of their semantic labels. The evaluation scores are positively correlated with the following: the similarity of the outlines between two candidate object regions is positively correlated with the corresponding evaluation scores; the similarity of the dimensions between two candidate object regions is positively correlated with the corresponding evaluation scores; the similarity of the orientations between two candidate object regions is positively correlated with the corresponding evaluation scores; the similarity of the point cloud densities between two candidate object regions is positively correlated with the corresponding evaluation scores; the density of the continuity between two candidate object regions is positively correlated with the corresponding evaluation scores; the strength of the consistency between two candidate object regions is positively correlated with the corresponding evaluation scores; and the similarity of the semantic labels between two candidate object regions is positively correlated with the corresponding evaluation scores. A weighted average of the multiple evaluation scores is calculated to obtain the first merging probability.

[0057] This scheme achieves a refined evaluation of confidence by constructing a multi-dimensional and weighted evaluation parameter system, thereby enabling a comprehensive evaluation of confidence and ensuring a high accuracy of the obtained first pooling probability.

[0058] Specifically, the following eight evaluation parameters are calculated for any two candidate object regions:

[0059] (1) The Euclidean distance of the center point is normalized to the interval 0–1. The smaller the distance, the higher the score.

[0060] (2) Shape similarity of the outline; the closer the outlines are, the higher the score.

[0061] (3) Size similarity, with the mean square error of volume ratio and length-width-height ratio as the indicator. The closer the ratio is to 1, the higher the score.

[0062] (4) Orientation similarity is calculated using the cosine of the angle between the main axes. For example, full marks are awarded when the angle is less than 15°.

[0063] (5) Point cloud density: Calculate the ratio of the number of points per unit volume in each region. For example, full marks are given when the difference is less than 20%.

[0064] (6) Continuity is assessed by the connectivity of adjacent points in space. For example, if the distance between the nearest points in two regions is less than 0.5 meters and there is a point cloud interpolation trend, the score will be improved.

[0065] (7) Consistency, based on the consistency of dynamic behavior (such as velocity and acceleration) in the time series, if the motion trends of two regions are consistent in multi-frame data, the score increases;

[0066] (8) Semantic label similarity: If two regions are classified as the same category (such as “vehicles”) by the preliminary semantic segmentation model, then full marks are obtained.

[0067] Each parameter is independently normalized to the 0–1 range and assigned a fixed weight, such as distance (0.2), contour (0.15), size (0.1), orientation (0.1), density (0.1), continuity (0.15), consistency (0.1), and semantics (0.1). The first merging probability is output after weighted summation.

[0068] In some embodiments, when the first merging probability is greater than or equal to the first preset probability threshold, the two candidate object regions are merged to obtain the object region. Specifically, this can be achieved through the following steps: when the first merging probability is greater than or equal to the first preset probability threshold, the union of the two candidate object regions is extracted; and the extracted union is determined as the object region.

[0069] In this scheme, "union" refers to the set of all LiDAR point clouds contained in two candidate object regions. The point cloud coordinates of the two regions can be directly merged according to their original sampling accuracy to form a unified point cloud set, thereby obtaining a complete object region for the same object.

[0070] Specifically, the first preset probability threshold can be 0.8, or of course any other feasible value.

[0071] Specifically, this solution uses a transformer model to further refine the preliminary obstacle segmentation results obtained from bounding box or semantic segmentation using LiDAR, replacing the rule-based post-processing stage commonly used in existing technologies. Specifically, after using the coarse obstacle segmentation results obtained from bounding box or semantic segmentation, this method follows the process to replace the original rule-based post-processing, as follows: Figure 5 As shown, it consists of three consecutive steps:

[0072] First, a feature extraction network (any existing neural network model capable of processing point clouds) is used to extract the geometric features of each coarsely segmented obstacle (i.e., candidate object region, a subset of the point cloud; if the input is a bounding box, the point cloud subset inside is obtained, semantic segmentation is performed, and the point cloud covered by the semantic segmentation result is used; the actual input is a subset of the point cloud, and the bounding box and semantic box above are some selected point clouds). Specifically, a lightweight geometric feature extraction network is used to encode the features of each candidate object region output by the coarse segmentation stage. This network takes the point cloud coordinates, reflection intensity, normal vector distribution, point density, bounding box size, projected area, surface curvature, and other geometric attributes of each candidate region as input, and calculates a fixed-dimensional vector representation through a multilayer perceptron or point cloud feature aggregation module, which can be called the "obstacle geometric features";

[0073] Second, combine the n (n greater than or equal to 1) obstacle geometric features in pairs to form n (n-1) / 2 obstacle feature pairs are input into a transformer network to obtain the probability of whether each obstacle pair should be merged, i.e., the first merging probability. Specifically, each feature pair represents the potential correlation between two candidate regions. These feature pairs are fed as input sequences into the Transformer encoder structure. The self-attention mechanism in the Transformer allows each feature pair to interact with all other feature pairs globally, dynamically calculating the relevance weights between each pair of candidate regions to determine whether they should be merged. Specifically, the model calculates the attention score for each pair of regions through three sets of linear transformations: query, key, and value, thereby identifying obstacle pairs that are highly similar in shape, size, spatial relative position, and motion trend, even if they are not spatially adjacent (e.g., occluded by other obstacles but still belonging to the same entity). Finally, the model outputs a real probability value, representing the likelihood that the obstacle will be merged. The probability ranges from 0 to 1 and is normalized by the Softmax or Sigmoid function. The above feature pairs can be concatenated in any way (e.g., concatenation, subtraction, multiplication, connection).

[0074] Third, based on the probability of whether each obstacle pair (i.e., candidate object region) should be merged, some obstacle pairs (i.e., candidate object regions) are connected. Ultimately, all obstacles (i.e., candidate object regions) will form several connected components. The obstacles in each connected component are then merged. Specifically, based on the merging probabilities of all obstacle pairs, an undirected graph structure is constructed: each candidate obstacle is a node in the graph, and the merging probability between each pair of obstacles is used as the weight of the connecting edge. Subsequently, a threshold filtering mechanism (e.g., retaining edges with a probability greater than 0.6) is used to retain high-confidence merging relationships, forming a connected subgraph. For each connected component, a merging operation is performed, that is, the point cloud sets of all candidate regions within that component are uniformly merged into a complete object region. The merged region no longer retains the original coarse segmentation boundaries; instead, its bounding box or semantic label is recalculated based on all point clouds as the final output.

[0075] Specifically, the Transformer architecture can include an input layer, an encoder layer, and an output layer.

[0076] Input Embedding Layer: Each obstacle pair consists of geometric feature vectors from two candidate regions. Each feature vector has a dimension of d (e.g., d=128) and represents geometric attributes such as the size, shape distribution, point density, ground projection overlap, and normal vector consistency of the region. The two feature vectors are concatenated to form an input vector of length 2×d, which serves as the single input sequence element for the obstacle pair.

[0077] Transformer encoder: This scheme adopts a single-layer or multi-layer Transformer encoder structure, with each layer containing two sub-modules: a multi-head self-attention mechanism and a feedforward neural network.

[0078] Multi-Head Self-Attention (MHSA): This mechanism performs global interaction on the input features of all obstacle pairs. For example, when the model evaluates whether "obstacles A and B should be merged", it not only considers the direct features of A and B, but also refers to the contextual information of other pairs such as "obstacles A and C" and "obstacles B and D" through attention weights, thereby determining whether the possibility of merging A and B is affected by other regions (e.g., B and C should be merged, but A and C are far apart, indicating that A is an independent object).

[0079] Feed-Forward Network (FFN): Consists of two fully connected layers with a ReLU activation function in between. This function performs non-linear transformation and dimensionality compression on the features output by self-attention, enhancing feature representation. Each layer includes residual connections and layer normalization to stabilize the training process and mitigate gradient vanishing.

[0080] Output Layer: The Transformer encoder outputs the contextual augmentation features of each obstacle pair, which are passed through a fully connected layer and then through a Sigmoid activation function to output a scalar probability value in the range [0,1], representing the probability that the obstacle pair should be merged.

[0081] This scheme models the "merge or not" decision as a pairwise classification task and introduces the Transformer as the classifier. At the input level, n×(n-1) / 2 pairs of all candidate obstacles are combined as the input sequence, and each "pair" is a feature vector representing the geometric similarity between the two regions.

[0082] At the modeling level, Transformer's self-attention mechanism enables the model to dynamically focus on which obstacle pairs have discriminative value in their relationships. For example, when a sweeper truck and a trailer truck behind it are divided into two regions due to occlusion, the model uses the attention mechanism to find that the two are highly similar in multiple dimensions such as "ground projection overlap", "height distribution", and "reflection intensity trend". It also refers to other obstacles (such as nearby vehicles) as negative samples, thus determining that the two should be merged, rather than simply judging based on distance.

[0083] Each probability value output by the model at the output level does not depend on a preset threshold, but is adaptively learned from a large number of positive and negative samples (labeled whether they belong to the same object) in the training data. Therefore, it has good generalization ability for different vehicle models, different environments, and different sensor noise. At the integration level, this Transformer module is deployed as an independent sub-network for post-processing after coarse segmentation (bounding box or semantic segmentation) and before the final output. It only needs to receive candidate region features and does not need to modify the front-end network.

[0084] In the specific implementation process, the candidate object regions are re-identified, and at least two candidate object regions with the highest similarity are merged to obtain the object region corresponding to the point cloud data. This can be achieved through the following steps: extracting any one of the merged candidate object regions; extracting any one of the unmerged candidate object regions; calculating the confidence that the merged candidate object region and the unmerged candidate object region belong to the same object to obtain a second merging probability; if the second merging probability is greater than or equal to a second preset probability threshold, merging the merged candidate object region and the unmerged candidate object region to obtain the object region.

[0085] In this scheme, all combinations between merged and unmerged regions are traversed, and the second merging probability is calculated for each pair. In each iteration, the pair of regions with the highest second merging probability (i.e., the combination most likely to belong to the same object) is selected. If the probability is greater than the second merging probability, the point clouds of the two are merged as the new merged region, while the remaining regions remain unchanged. This can further ensure that the obtained object region can better guarantee the integrity of the object.

[0086] Specifically, the method for calculating the confidence that the merged candidate object region and the unmerged candidate object region belong to the same object is the same as the method for calculating the confidence that any two candidate object regions belong to the same object, and will not be elaborated here.

[0087] Specifically, the second preset probability threshold can be 0.8, or of course any other feasible value.

[0088] In some embodiments, the candidate object regions are re-identified, and at least two candidate object regions with the highest similarity are merged to obtain the object region corresponding to the point cloud data. Specifically, this can be achieved through the following steps: extracting any two merged candidate object regions; calculating the confidence that the two merged candidate object regions belong to the same object to obtain a third merging probability; and merging the two merged candidate object regions again when the third merging probability is greater than or equal to a third preset probability threshold to obtain the object region.

[0089] In this scheme, the distinction between the original region and the merged region is no longer made. All the formed object regions are treated as new processing units, and the third merging probability is calculated for each pair. In each iteration, all merged region pairs are traversed, and the pair with the highest third merging probability is selected. If the probability is greater than the third probability threshold, the merging is performed again to form a larger-scale object region. This can further ensure that the obtained object region has better integrity of the object.

[0090] Specifically, the method for calculating the confidence that the two merged candidate object regions belong to the same object is the same as the method for calculating the confidence that the merged candidate object region and the unmerged candidate object region belong to the same object, which will not be elaborated here.

[0091] Specifically, the third preset probability threshold can be 0.8, and of course it can also be any other feasible value.

[0092] In this solution, whether an obstacle should be merged is jointly determined by the merging probability output by the Transformer model and the preset threshold. Its decision-making mechanism is based on connected component clustering in graph theory. The entire process is completely data-driven and does not rely on artificial rules. The specific implementation is as follows:

[0093] In the model inference stage, for n candidate obstacles obtained by coarse segmentation, all possible unordered obstacle pairs are generated, totaling n×(n - 1) / 2. Each obstacle pair is input into the Transformer network, and a merging probability value between 0 and 1 is output, indicating the possibility that the pair of obstacles belongs to the same real object. This probability value is learned by the model in the training stage and reflects the true statistical law of the geometric and spatial relationships between obstacles, rather than a fixed threshold set artificially.

[0094] The merging decision adopts a "probability threshold + connected component merging" strategy. First, a global merging threshold T (for example, 0.6) is set. For each obstacle pair, if its output probability ≥ T, a connection edge is established between the two obstacles. For example, if the merging probability of obstacle A and B is 0.75, A and D is 0.62, B and C is 0.58, and C and D is 0.65, then only when the probability ≥ T are the edges established: A–B, A–D, C–D are connected, and B–C is not connected.

[0095] Subsequently, based on the established edges, an undirected graph is constructed. Each node in the undirected graph is a candidate obstacle, and each edge represents a "should be merged" relationship. Connected component analysis (such as union-find or depth-first search) is performed on the graph. All nodes that can be reached through any path (i.e., connected to each other through one or more edges) are regarded as the same real obstacle and merged into a final output region.

[0096] For example, if A–B, A–D, and C–D are all connected, then A, B, D, and C form a connected component, and the four obstacles are merged into a whole; if the merging probability of E and all other obstacles is < T, then E is an independent connected component and does not participate in the merging; if A–B, B–C, and C–D are all connected, even if there is no direct connection between A and D, because of the path A–B–C–D being connected, A and D are still merged; regardless of how long the merging chain is, as long as there is a path connection, it is regarded as the same object, and there is no need to judge the global consistency pair by pair.

[0097] During the training phase, the merging threshold T is not set manually, but is automatically optimized through offline data-driven methods. A LiDAR point cloud dataset containing real-world scenes is collected, and each obstacle is manually labeled by annotators to identify which coarsely segmented regions belong to the same real object (e.g., the front and body of a sweeper are still labeled as the same even if the point cloud is broken). This constructs "positive sample pairs" (obstacle pairs belonging to the same object) and "negative sample pairs" (obstacle pairs belonging to different objects).

[0098] For each possible threshold T (tested in increments of 0.01, from 0.01 to 0.99), perform the following operations: Compare the predicted probabilities of all obstacle pairs with the threshold to generate a merge map; for each connected component, match it with the label; if all obstacles within the component belong to the same real object label, it is a correct merge; if it contains multiple real object labels, it is an incorrect merge; if obstacles within the component should be independent but are not merged, it is an unmerged component; calculate the overall accuracy (number of correctly merged / total number of obstacle pairs) or F1 score, and select the threshold T that optimizes the evaluation metrics as the final application threshold.

[0099] This process involves hyperparameter tuning during the offline training phase and only needs to be completed once. After training, the model parameters and threshold T are fixed. After deployment to the vehicle system, only forward inference is performed. The input is the coarse segmentation result, the output is the probability, edges are established according to the fixed threshold T, connected components are calculated, and the merged result is output. At this point, no labels or manual intervention are required; the model automatically completes the merging based on the statistical patterns learned during training.

[0100] The Transformer network in this scheme can also be replaced by other deep learning / machine learning models capable of classifying obstacle feature pairs. Although the current scheme uses Transformer as the core model, any deep learning architecture capable of effectively classifying input feature pairs can be used as a substitute. For example, graph neural networks can model obstacles as graph nodes and merging relationships as edges, updating node states through message passing mechanisms; or multilayer perceptrons can be used in combination with paired feature concatenation to directly output merging probabilities; or even traditional machine learning methods such as random forests can be used. These alternative schemes do not deviate from the technical essence of this scheme, which is "transforming post-processing into obstacle pair classification," and therefore all fall within the scope of protection of this scheme and are reasonable variations.

[0101] In summary, compared with existing technologies, the proposed solution uses a transformer-based model for post-processing of coarsely segmented obstacles. Because model-based methods can learn from massive amounts of data how to post-process various coarsely segmented obstacles, they can adaptively merge coarsely segmented obstacle pairs. In contrast, existing technologies perform post-processing based on given rules, which is difficult to adapt to various long-tail scenarios.

[0102] Specifically, the core improvement of this solution compared to existing technologies lies in transforming the traditional post-processing process, which relies on manual rules, into an obstacle pair classification task based on a deep learning model. This fundamentally enhances the generalization ability and scene adaptability of the post-processing. Existing post-processing modules typically set fixed thresholds or logical conditions based on engineers' experience. While these rules perform reasonably well in standard scenarios (such as passenger cars and standard trucks), they easily fail when faced with non-standard, non-rigid, and structurally complex obstacles such as engineering vehicles, sweepers, trailers, and temporary storage objects. This is due to the diverse shapes, fragmented point clouds, severe occlusion, and inconsistent motion states, leading to significant oversegmentation.

[0103] This approach transforms the post-processing of coarsely segmented obstacles into a classification problem involving several pairs of obstacles, making the problem easier to handle with a model. Instead of directly determining the integrity of an object, this approach decomposes the problem into independent judgments of pairwise combinations of all candidate regions: each obstacle pair is treated as a sample, with inputs consisting of the geometric feature vectors of the two regions (such as size, distribution, spatial relative pose, surface continuity, point density gradient, etc.), and outputting a probability value representing the likelihood that the two regions belong to the same real object. The key significance of this transformation lies in the fact that classification tasks have mature, stable, and scalable model architectures in the deep learning field. Models can automatically learn which geometric and spatial relationship features are strongly correlated with "should be merged" through massive amounts of training data containing real annotations. For example, even if two point clouds are segmented, if their ground projection overlaps significantly, their normal vectors are continuous, their height distribution is consistent, and their motion trajectories fit well, the model can still learn that they belong to the same object, even if the center distance exceeds a traditional threshold. This learning process does not rely on pre-set conditions but automatically derives complex, nonlinear, and multi-dimensional merging criteria from the data.

[0104] Furthermore, the Transformer model, due to its powerful global modeling capabilities, can simultaneously consider long-range dependencies between multiple obstacles. For example, in a scenario where a tractor-trailer and a trailer are separated in their point clouds due to occlusion, the model can not only determine whether the tractor-trailer pair should be merged, but also simultaneously evaluate whether the trailer and a nearby parked truck "should not be merged," thus avoiding a chain reaction of erroneous merging caused by local misjudgments. This model architecture supports parallel processing of all obstacle pairs, has high computational efficiency, and the training process can be optimized end-to-end, making the entire post-processing workflow an adaptively evolving part of the perception system.

[0105] This application also provides a point cloud-based object recognition device. It should be noted that the point cloud-based object recognition device of this application can be used to execute the point cloud-based object recognition method provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0106] The following describes the point cloud-based object recognition device provided in the embodiments of this application.

[0107] Figure 6 This is a structural block diagram of a point cloud-based object recognition device according to an embodiment of this application. Figure 6 As shown, the device includes:

[0108] The acquisition unit 10 is used to acquire the point cloud collected by the vehicle's lidar and obtain point cloud data;

[0109] The first recognition unit 20 is used to recognize the point cloud data using an initial recognition method to obtain the candidate object region corresponding to the point cloud data. The initial recognition method is to use point cloud bounding box recognition and / or semantic segmentation recognition.

[0110] The second identification unit 30 is used to re-identify the above-mentioned candidate object regions, merge at least two of the above-mentioned candidate object regions with the highest similarity, and obtain the object region corresponding to the above-mentioned point cloud data. The re-identification includes at least identification using a neural network model.

[0111] This embodiment adds a refined recognition method to the existing scheme (bounding box recognition or segmentation recognition). That is, coarse recognition is performed first, followed by fine recognition. The original point cloud is not reprocessed. Instead, the candidate object region obtained from the initial recognition is used as the basic unit. If two object regions are similar, they are merged. In this way, the same object will not be segmented and a complete object region can be formed, which improves the accuracy of separating obstacles from the point cloud.

[0112] In the specific implementation process, the first recognition unit includes a first recognition module, a second recognition module, a third recognition module, and a fourth recognition module. The first recognition module is used to recognize the point cloud data using a point cloud bounding box recognition method to obtain the candidate object region. The second recognition module is used to recognize the point cloud data using a semantic segmentation recognition method to obtain the candidate object region. The third recognition module is used to recognize the point cloud data using a point cloud bounding box recognition method, and then uses a semantic segmentation recognition method to re-recognize the point cloud data after the point cloud bounding box recognition to obtain the candidate object region. The fourth recognition module is used to recognize the point cloud data using a semantic segmentation recognition method, and then uses a point cloud bounding box recognition method to re-recognize the point cloud data after the semantic segmentation recognition to obtain the candidate object region.

[0113] This scheme takes given LiDAR point cloud data as input, outputs several bounding boxes for the LiDAR point clouds based on bounding box detection, and groups / segments the LiDAR point clouds based on the bounding box results to obtain several point cloud subsets, each subset representing an obstacle, i.e., a candidate object region. Semantic segmentation outputs the classification result for each LiDAR point or each grid cell at a given resolution, and clusters the LiDAR point clouds based on the point-by-point or grid-by-grid classification results to obtain several point cloud subsets, each subset representing an obstacle, i.e., a candidate object region. Of course, bounding box detection and semantic segmentation can also be combined; this scheme can flexibly choose either method to complete the initial identification.

[0114] In some embodiments, the second identification unit includes a first calculation module and a first merging module. The first calculation module is used to calculate the confidence that any two candidate object regions belong to the same object to obtain a first merging probability. The first merging module is used to merge the two candidate object regions to obtain the object region when the first merging probability is greater than or equal to a first preset probability threshold.

[0115] In this scheme, the confidence level of two candidate object regions belonging to the same real object can be calculated. The value range is from 0 to 1. When the two candidate object regions are greater than or equal to the first preset probability threshold, it is more likely that the two candidate object regions are the same object. Therefore, merging the two candidate object regions results in an object region that better ensures the integrity of the object.

[0116] In the specific implementation, the first calculation module includes an acquisition submodule, an evaluation submodule, and a calculation submodule. The acquisition submodule is used to acquire multiple evaluation parameters, including one or more of the following: distance between two candidate object regions, outline between two candidate object regions, size between two candidate object regions, orientation between two candidate object regions, point cloud density between two candidate object regions, continuity between two candidate object regions, consistency between two candidate object regions, and semantic label similarity between two candidate object regions. The evaluation submodule is used to score the similarity between two candidate object regions based on the above evaluation parameters, obtaining multiple evaluation scores. Each evaluation score corresponds one-to-one with one of the evaluation parameters, and the distance between two candidate object regions corresponds to the corresponding evaluation score. The following parameters are positively correlated: the similarity of the outlines between two candidate object regions is positively correlated with their corresponding evaluation scores; the similarity of the dimensions between two candidate object regions is positively correlated with their corresponding evaluation scores; the similarity of the orientations between two candidate object regions is positively correlated with their corresponding evaluation scores; the similarity of the point cloud densities between two candidate object regions is positively correlated with their corresponding evaluation scores; the density of the continuity between two candidate object regions is positively correlated with their corresponding evaluation scores; the strength of the consistency between two candidate object regions is positively correlated with their corresponding evaluation scores; and the similarity of the semantic labels between two candidate object regions is positively correlated with their corresponding evaluation scores. The calculation submodule is used to calculate the weighted average of the above evaluation scores to obtain the first merging probability.

[0117] This scheme achieves a refined evaluation of confidence by constructing a multi-dimensional and weighted evaluation parameter system, thereby enabling a comprehensive evaluation of confidence and ensuring a high accuracy of the obtained first pooling probability.

[0118] In some embodiments, the first merging module includes an extraction submodule and a determination submodule. The extraction submodule is used to extract the union of two candidate object regions when the first merging probability is greater than or equal to the first preset probability threshold. The determination submodule is used to determine the extracted union as the object region.

[0119] In this scheme, "union" refers to the set of all LiDAR point clouds contained in two candidate object regions. The point cloud coordinates of the two regions can be directly merged according to their original sampling accuracy to form a unified point cloud set, thereby obtaining a complete object region for the same object.

[0120] In the specific implementation process, the second identification unit includes a first extraction module, a second extraction module, a second calculation module, and a second merging module. The first extraction module is used to extract any one of the merged candidate object regions. The second extraction module is used to extract any one of the unmerged candidate object regions. The second calculation module is used to calculate the confidence that the merged candidate object region and the unmerged candidate object region belong to the same object, and obtain a second merging probability. The second merging module is used to merge the merged candidate object region and the unmerged candidate object region when the second merging probability is greater than or equal to a second preset probability threshold, to obtain the object region.

[0121] In this scheme, all combinations between merged and unmerged regions are traversed, and the second merging probability is calculated for each pair. In each iteration, the pair of regions with the highest second merging probability (i.e., the combination most likely to belong to the same object) is selected. If the probability is greater than the second merging probability, the point clouds of the two are merged as the new merged region, while the remaining regions remain unchanged. This can further ensure that the obtained object region can better guarantee the integrity of the object.

[0122] In some embodiments, the second identification unit includes a third extraction module, a third calculation module, and a third merging module. The third extraction module is used to extract any two merged candidate object regions. The third calculation module is used to calculate the confidence that the two merged candidate object regions belong to the same object, and obtain a third merging probability. The third merging module is used to merge the two merged candidate object regions again when the third merging probability is greater than or equal to a third preset probability threshold, to obtain the object region.

[0123] In this scheme, the distinction between the original region and the merged region is no longer made. All the formed object regions are treated as new processing units, and the third merging probability is calculated for each pair. In each iteration, all merged region pairs are traversed, and the pair with the highest third merging probability is selected. If the probability is greater than the third probability threshold, the merging is performed again to form a larger-scale object region. This can further ensure that the obtained object region has better integrity of the object.

[0124] The aforementioned point cloud-based object recognition device includes a processor and a memory. The acquisition unit, the first recognition unit, and the second recognition unit, etc., are all stored as program units in the memory. The processor executes these program units stored in the memory to achieve the corresponding functions. All of the above modules are located in the same processor; alternatively, the modules may be located in different processors in any combination.

[0125] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the low accuracy of separating obstacles from point clouds in existing technologies.

[0126] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0127] This invention provides a computer-readable storage medium including a stored program, wherein the program, when running, controls the device containing the computer-readable storage medium to execute the point cloud-based object recognition method.

[0128] This invention provides a processor for running a program, wherein the program executes the point cloud-based object recognition method.

[0129] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements at least the steps of a point cloud-based object recognition method. The device described herein can be a server, PC, tablet, mobile phone, etc.

[0130] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform a program that initializes a method for object recognition with at least a point cloud-based approach.

[0131] This application also provides a vehicle including one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for performing any of the above-described point cloud-based object recognition methods.

[0132] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0133] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0135] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0136] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0137] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0138] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0139] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0140] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0142] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A point cloud-based object recognition method, characterized in that, include: The point cloud data is obtained by acquiring the point cloud data collected by the vehicle's LiDAR. The point cloud data is identified using an initial identification method to obtain the candidate object region corresponding to the point cloud data. The initial identification method is point cloud bounding box identification and / or semantic segmentation identification. The candidate object regions are re-identified, and at least two candidate object regions with the highest similarity are merged to obtain the object region corresponding to the point cloud data. The re-identification includes at least identification using a neural network model.

2. The method according to claim 1, characterized in that, The point cloud data is identified using an initial identification method to obtain candidate object regions corresponding to the point cloud data, including one of the following: The point cloud data is identified using a point cloud bounding box recognition method to obtain the candidate object region; The point cloud data is identified using a semantic segmentation and recognition method to obtain the candidate object region; The point cloud data is identified using a point cloud bounding box recognition method, and the point cloud data after point cloud bounding box recognition is further identified using a semantic segmentation recognition method to obtain the candidate object region. The point cloud data is identified using a semantic segmentation recognition method, and the point cloud data after semantic segmentation recognition is further identified using a point cloud bounding box recognition method to obtain the candidate object region.

3. The method according to claim 1, characterized in that, The candidate object regions are re-identified, and at least two candidate object regions with the highest similarity are merged to obtain the object region corresponding to the point cloud data, including: Calculate the confidence that any two candidate object regions belong to the same object to obtain the first merging probability; If the first merging probability is greater than or equal to the first preset probability threshold, the two candidate object regions are merged to obtain the object region.

4. The method according to claim 3, characterized in that, Calculate the confidence that any two candidate object regions belong to the same object to obtain the first merging probability, including: Multiple evaluation parameters are obtained, including one or more of the following: distance between two candidate object regions, shape contour between two candidate object regions, size between two candidate object regions, orientation between two candidate object regions, point cloud density between two candidate object regions, continuity between two candidate object regions, consistency between two candidate object regions, and semantic label similarity between two candidate object regions. The similarity between two candidate object regions is scored according to the evaluation parameters to obtain multiple evaluation scores. Each evaluation score corresponds one-to-one with a specific evaluation parameter. The distance between two candidate object regions is positively correlated with their respective evaluation scores; the similarity of their outlines is positively correlated with their respective evaluation scores; the similarity of their dimensions is positively correlated with their respective evaluation scores; the similarity of their orientations is positively correlated with their respective evaluation scores; the similarity of their point cloud densities is positively correlated with their respective evaluation scores; the density of their continuity is positively correlated with their respective evaluation scores; the strength of their consistency is positively correlated with their respective evaluation scores; and the similarity of their semantic labels is positively correlated with their respective evaluation scores. The first merging probability is obtained by calculating the weighted average of the multiple evaluation scores.

5. The method according to claim 3, characterized in that, When the first merging probability is greater than or equal to a first preset probability threshold, the two candidate object regions are merged to obtain the object region, including: If the first merging probability is greater than or equal to the first preset probability threshold, the union of the two candidate object regions is extracted. The extracted union is determined as the object region.

6. The method according to claim 3, characterized in that, The candidate object regions are re-identified, and at least two candidate object regions with the highest similarity are merged to obtain the object region corresponding to the point cloud data, including: Extract any one of the merged candidate object regions; Extract any one of the candidate object regions that has not yet been merged; Calculate the confidence that the merged candidate object region and the unmerged candidate object region belong to the same object to obtain the second merging probability; If the second merging probability is greater than or equal to the second preset probability threshold, the merged candidate object region and the unmerged candidate object region are merged to obtain the object region.

7. The method according to claim 3, characterized in that, The candidate object regions are re-identified, and at least two candidate object regions with the highest similarity are merged to obtain the object region corresponding to the point cloud data, including: Extract any two of the merged candidate object regions; Calculate the confidence that the two merged candidate object regions belong to the same object to obtain the third merging probability; If the third merging probability is greater than or equal to the third preset probability threshold, the two merged candidate object regions are merged again to obtain the object region.

8. A point cloud-based object recognition device, characterized in that, include: The acquisition unit is used to acquire the point cloud data collected by the vehicle's lidar. The first identification unit is used to identify the point cloud data using an initial identification method to obtain the candidate object region corresponding to the point cloud data, wherein the initial identification method is to use point cloud bounding box identification and / or semantic segmentation identification. The second identification unit is used to re-identify the candidate object regions, merge at least two candidate object regions with the highest similarity to obtain the object region corresponding to the point cloud data, wherein the re-identification includes at least identification using a neural network model.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the point cloud-based object recognition method according to any one of claims 1 to 7.

10. A vehicle, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing the point cloud-based object recognition method according to any one of claims 1 to 7.

Images