An adaptive semantic segmentation method for point clouds across vehicle-mounted lidar configurations

Abstract

The present invention proposes a point cloud adaptive semantic segmentation method for cross-vehicle laser radar configuration, specifically: establishing a deep implicit expression framework to reconstruct the three-dimensional scene continuous surface of the vehicle-mounted laser point cloud; constructing a self-supervised learning paradigm of the deep implicit expression framework based on the approximate unsigned distance field of local geometric statistics; constructing the unsigned distance field of the source scene and the target scene; obtaining the implicit expression space point cloud with the source scene and the target scene cloud point level distribution aligned; constructing a semantic segmentation dataset of the source scene implicit expression space point cloud; training a semantic segmentation model based on the implicit expression space dataset; inferring the semantic labels of the original target scene point cloud based on nearest neighbor radiation; the present invention reconstructs high-quality spatial continuous surfaces based on three-dimensional implicit expression, establishes a vehicle-mounted laser point cloud scene implicit expression method based on self-supervision, resamples the implicit expression space, realizes the point level distribution alignment of the point cloud across vehicle-mounted laser radar configurations, and avoids the explicit reconstruction of the standard domain.

Description

Technical Field

[0001] The present invention relates to the fields of point cloud deep learning and transfer learning technology, and in particular to a point cloud adaptive semantic segmentation method across vehicle-mounted laser radar configurations. Background Art

[0002] As a high-tech technology in surveying and mapping, mobile laser scanning systems can efficiently and accurately acquire point cloud data of road scenes and are widely used in mapping, 3D urban reconstruction, smart transportation, and other fields. However, due to the data characteristics of mobile laser point clouds (such as large data volume and uneven spatial distribution), the efficient and intelligent classification and identification of ground object information in mobile laser point cloud data still faces many challenges.

[0003] In recent years, with the development of deep learning for point clouds, deep learning-based semantic segmentation has also achieved significant breakthroughs. The goal of point cloud semantic segmentation is to label each point in the point cloud data with a semantic category. For vehicle-mounted laser point clouds, this primarily involves labeling each point with semantic labels such as road surface, traffic sign, traffic light, bus, truck, pedestrian, roadside tree, streetlight, fence, and building, thereby extracting road scene information from the vehicle-mounted laser point cloud data.

[0004] Compared to traditional point cloud classification methods, vehicle-mounted laser point cloud semantic segmentation methods based on deep learning have a higher degree of intelligence and better semantic segmentation performance. However, as a data-driven method, deep learning itself not only relies on a large amount of training data, but also requires that the training data and test data meet the assumption of independent and identical distribution. In addition, because different vehicle-mounted laser scanning systems may be equipped with different types or different numbers of laser scanners, or even the same vehicle-mounted laser scanning system at different operating speeds, there will be significant differences between the source scene vehicle-mounted laser point cloud used for training and the target scene vehicle-mounted laser point cloud used in actual applications.

[0005] However, there is currently no mature algorithm at home and abroad that can solve the adaptive semantic segmentation of point clouds across vehicle-mounted lidar configurations under a deep learning framework.

[0006] The present invention proposes solutions to the above problems. Summary of the Invention

[0007] The present invention proposes a point cloud adaptive semantic segmentation method across vehicle-mounted laser radar configurations to solve the application generalization problem of vehicle-mounted laser point cloud semantic segmentation methods based on deep learning across laser radar configurations when facing practical applications.

[0008] The present invention adopts the following technical solutions.

[0009] A point cloud adaptive semantic segmentation method for cross-vehicle laser radar configurations can adaptively segment point cloud data after acquiring point cloud data using a vehicle-mounted laser scanning system equipped with different laser scanners, including the following steps:

[0010] Step S1: Using a vehicle-mounted laser scanning system equipped with a laser radar, a point cloud for semantic segmentation model training is collected and used as the source scene vehicle-mounted laser point cloud; using a point cloud collected by a vehicle-mounted laser scanning system equipped with another laser radar and used for semantic segmentation model testing is used as the target scene vehicle-mounted laser point cloud; based on the source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud, a deep neural network model for implicitly representing the three-dimensional scene is established as a three-dimensional scene depth implicit expression framework, thereby reconstructing the three-dimensional scene continuous surface of the vehicle-mounted laser point cloud;

[0011] Step S2: using an approximate unsigned distance field based on local geometric statistics, constructing a self-supervised learning paradigm for implicit representation of 3D scene depth, ensuring high-fidelity reconstruction of the continuous scene surface in the implicit space;

[0012] Step S3: Using the three-dimensional scene depth implicit representation framework trained by the self-supervised learning paradigm based on the approximate unsigned distance field, perform query point unsigned distance inference on the given source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud, and establish the unsigned distance fields of the source scene and the target scene respectively;

[0013] Step S4: resample the source scene vehicle laser point cloud and the target scene vehicle laser point cloud using an unsigned distance threshold to obtain corresponding point clouds in the implicit expression space, thereby achieving point-level distribution alignment of the cross-vehicle laser radar configuration point clouds;

[0014] Step S5: Use the source scene vehicle-mounted laser point cloud label data to perform neighbor label voting on its implicit expression space point cloud to construct a semantic segmentation training dataset of the source scene in the implicit expression space;

[0015] Step S6: using the semantic segmentation training dataset to train and optimize the point cloud semantic segmentation model;

[0016] Step S7: Use the optimized and trained semantic segmentation model to perform point-by-point semantic label inference on the implicitly expressed spatial target scene point cloud, and perform nearest neighbor radiation on the inference results to complete the point cloud adaptive semantic segmentation of the cross-vehicle lidar configuration.

[0017] Step S1 specifically includes the following steps:

[0018] Step A1: Use an encoder based on a three-dimensional convolutional neural network (3DCNN) to map the given source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud into a feature representation that perceives the scene geometry. That is, establish a scene implicit expression framework that covers the source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud. Specifically:

[0019] For a given vehicle-mounted laser point cloud Using a 3DCNN-based encoder Mapping a given point cloud P into a scene geometry-aware feature representation And regard it as the latent space feature representation of the potential scene surface of a given point cloud P;

[0020] Step A2: randomly sampling query points from the minimum bounding box of the given point cloud; specifically:

[0021] Randomly sample K query points from the minimum bounding box of a given point cloud P, denoted as

[0022] Step A3: Using an implicit function decoder, with the scene geometry perception feature representation and the query point as input, regress an implicit value for each query point, specifically:

[0023] Combined implicit neural representation, define each query point q k The implicit value of ∈Q, that is, the distance from the query point to the potential scene surface, is as follows:

[0024] f P (q k )=ψ(q k ,v)

[0025] Among them, f P is the implicit function of the potential scene surface corresponding to the given point cloud P;

[0026] Step A4: unsigned distance from the query point to the potential scene surface, i.e., using the implicit function decoder ψ, with the latent spatial feature representation of the potential scene surface and the query point as input, an implicit value is regressed for each query point, where the implicit value is set to the unsigned distance from the query point to the potential scene surface.

[0027] Step S2 specifically includes the following steps:

[0028] Step B1: For a given vehicle-mounted laser point cloud P, calculate each point p i The average distance to its M nearest neighbors, denoted as d i , represents point p i The local geometric proximity of

[0029] Step B2: Count the local geometric neighbor averages of all points in the point cloud P as the approximate unsigned distance field D of the potential scene surface of the point cloud P. P The approximation threshold d t , and then realize the adaptive approximation threshold selection for the given point cloud sparsity;

[0030] Step B3: Compare the distance between the query point and its nearest neighbor. When the distance d between the query point q and its nearest neighbor is q Less than the approximation threshold d t When, along the nearest neighbor point p N Normal Calculate the query point q to the nearest neighbor point p N is the center of the sphere and approximates the threshold d t The distance of the tangent plane of the sphere with radius q is the unsigned distance of the query point q;

[0031] Step B4: When the distance between the query point and its nearest neighbor is greater than the approximation threshold, the unsigned distance of the query point is the difference between the distance of the nearest neighbor and the radius of the search sphere, that is, when the distance d between the query point q and its nearest neighbor is greater than the approximation threshold, the unsigned distance of the query point is the difference between the distance of the nearest neighbor and the radius of the search sphere. q Greater than or equal to the approach threshold d t When , the unsigned distance of the query point q is directly defined as (d q -d t ); specifically as follows:

[0032]

[0033] in, Indicates that the starting point is p N , the vector whose endpoint is the query point q;

[0034] Step B5: Use the constructed approximate unsigned distance field of the query point as the learning signal for the self-supervised learning of the three-dimensional scene depth implicit expression model, that is, use the established approximate unsigned distance field as the self-supervised learning signal of the vehicle-mounted laser point cloud scene implicit expression framework.

[0035] Step S3 specifically includes the following steps:

[0036] Step C1: Use the trained 3D scene depth implicit expression model to perform unsigned distance inference on the given source scene vehicle laser point cloud to establish the source scene unsigned distance field DF s ;

[0037] Step C2: Use the trained 3D scene depth implicit expression model to perform unsigned distance inference on the given target scene vehicle laser point cloud and establish the target scene unsigned distance field DF t .

[0038] Step S4 specifically includes the following steps:

[0039] Step D1: For the source scene vehicle laser point cloud P s and target scene vehicle-mounted laser point cloud P t , respectively set the unsigned distance threshold ε;

[0040] Step D2: convert the source scene unsigned distance field DF s The query points smaller than the threshold ε are sampled as the point cloud of the implicit expression space of the source scene

[0041] Step D3: The target scene unsigned distance field DF t The query points smaller than the threshold ε are sampled as the point cloud of the implicit expression space of the source scene

[0042] Step D4: Sampling based on the same implicit expression space, resulting in and Obey the same spatial distribution, thus achieving point-level distribution alignment across perception configuration domains.

[0043] Step S5 specifically includes the following steps:

[0044] Step E1: Implicitly express spatial point cloud for the source scene Each point in Search it to the original source scene vehicle laser point cloud {P s ,Y s}N nearest neighbor points in;

[0045] Step E2: Using the semantic labels Y of the N neighboring points s Implicitly express spatial points for the corresponding source scene Vote for the tag category;

[0046] Step E3: Count the voting results of the label categories and define the label category with the most votes as the implicit expression space point of the source scene Semantic category label of ;

[0047] Step E4: Traverse All points in , construct the source scene semantic segmentation training dataset of the implicit expression space

[0048] Step S6 specifically includes the following steps:

[0049] Step F1: Use the implicitly expressed spatial point cloud as input to establish a point cloud semantic segmentation model; the semantic segmentation model can also be other existing point cloud semantic segmentation models that directly target discrete points;

[0050] Step F2: Using implicit expression of spatial point cloud semantic labels As a supervised learning signal, the cross entropy loss is used to constrain the semantic segmentation model to optimize training, that is, to construct a cross entropy loss function and train the semantic segmentation model.

[0051] Step S7 specifically includes the following steps:

[0052] Step G1: Use the above optimized trained semantic segmentation model to implicitly express the spatial target scene point cloud Pointwise semantic labels Make inferences;

[0053] Step G2: For each point P in the original target scene point cloud t(k) , search for its nearest neighbor point in the corresponding implicit expression space target scene point cloud

[0054] Step G3: Use nearest neighbor points Semantic label inference results Infer the semantic label Y of each point in the original target scene point cloud t , and finally complete the point cloud adaptive semantic segmentation of the cross-vehicle lidar configuration.

[0055] The beneficial effects of the present invention are as follows: through self-supervised deep implicit expression, the three-dimensional scene continuous surface of the open space vehicle-mounted laser point cloud can be reconstructed; and through the trained three-dimensional scene deep implicit expression model, the query point unsigned distance inference is performed on the given source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud, so that the unsigned distance field of the source scene and the target scene can be established; the source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud are resampled using the unsigned distance threshold to obtain the corresponding point cloud of the implicit expression space, and realize the point-level distribution alignment of the point cloud configured across the vehicle-mounted laser radar; based on the training and testing of the semantic segmentation model of the implicit expression space point cloud, the adaptive semantic segmentation of the point cloud configured across the vehicle-mounted laser radar can be realized.

[0056] Theoretically, the method described in the present invention can solve the problem of domain migration of any form of on-board laser point cloud perception configuration, and can effectively enhance the universality of the point cloud adaptive semantic segmentation method across on-board laser radar configurations. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0058] Attachment Figure 1 A schematic diagram of a flow chart of a point cloud adaptive semantic segmentation method configured across vehicle-mounted lidars as described in a specific embodiment;

[0059] Attachment Figure 2Schematic diagram of the implicit expression framework of the vehicle-mounted laser point cloud scene described in the specific implementation method;

[0060] Attachment Figure 3 This is a schematic diagram of the approximate unsigned distance field of the vehicle-mounted laser point cloud described in the specific implementation method. DETAILED DESCRIPTION

[0061] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.

[0062] First, some English abbreviations in this embodiment are explained:

[0063] 3DCNN (3D Convolutional Neural Networks): A three-dimensional convolutional neural network, a convolutional neural network that can be directly used in three-dimensional space rule systems.

[0064] As shown in the figure, a point cloud adaptive semantic segmentation method for cross-vehicle laser radar configurations can adaptively segment point cloud data after acquiring point cloud data using a vehicle-mounted laser scanning system equipped with different laser scanners, including the following steps:

[0065] Step S1: Using a vehicle-mounted laser scanning system equipped with a laser radar, a point cloud for semantic segmentation model training is collected and used as the source scene vehicle-mounted laser point cloud; using a point cloud collected by a vehicle-mounted laser scanning system equipped with another laser radar and used for semantic segmentation model testing is used as the target scene vehicle-mounted laser point cloud; based on the source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud, a deep neural network model for implicitly representing the three-dimensional scene is established as a three-dimensional scene depth implicit expression framework, thereby reconstructing the three-dimensional scene continuous surface of the vehicle-mounted laser point cloud;

[0066] Step S2: using an approximate unsigned distance field based on local geometric statistics, constructing a self-supervised learning paradigm for implicit representation of 3D scene depth, ensuring high-fidelity reconstruction of the continuous scene surface in the implicit space;

[0067] Step S3: Using the three-dimensional scene depth implicit representation framework trained by the self-supervised learning paradigm based on the approximate unsigned distance field, perform query point unsigned distance inference on the given source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud, and establish the unsigned distance fields of the source scene and the target scene respectively;

[0068] Step S4: resample the source scene vehicle laser point cloud and the target scene vehicle laser point cloud using an unsigned distance threshold to obtain corresponding point clouds in the implicit expression space, thereby achieving point-level distribution alignment of the cross-vehicle laser radar configuration point clouds;

[0069] Step S5: Use the source scene vehicle-mounted laser point cloud label data to perform neighbor label voting on its implicit expression space point cloud to construct a semantic segmentation training dataset of the source scene in the implicit expression space;

[0070] Step S6: using the semantic segmentation training dataset to train and optimize the point cloud semantic segmentation model;

[0071] Step S7: Use the optimized and trained semantic segmentation model to perform point-by-point semantic label inference on the implicitly expressed spatial target scene point cloud, and perform nearest neighbor radiation on the inference results to complete the point cloud adaptive semantic segmentation of the cross-vehicle lidar configuration.

[0072] like Figure 2 In the embodiment of this example, the vehicle-mounted laser point cloud includes: a source scene vehicle-mounted laser point cloud and a target scene vehicle-mounted laser point cloud;

[0073] Step S1 specifically includes the following steps:

[0074] Step A1: Use an encoder based on a three-dimensional convolutional neural network (3DCNN) to map the given source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud into a feature representation that perceives the scene geometry. That is, establish a scene implicit expression framework that covers the source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud. Specifically:

[0075] For a given vehicle-mounted laser point cloud Using a 3DCNN-based encoder Mapping a given point cloud P into a scene geometry-aware feature representation And regard it as the latent space feature representation of the potential scene surface of a given point cloud P;

[0076] Step A2: randomly sampling query points from the minimum bounding box of the given point cloud; specifically:

[0077] Randomly sample K query points from the minimum bounding box of a given point cloud P, denoted as

[0078] Step A3: Using an implicit function decoder, with the scene geometry perception feature representation and the query point as input, regress an implicit value for each query point, specifically:

[0079] Combined implicit neural representation, define each query point q k The implicit value of ∈Q, that is, the distance from the query point to the potential scene surface, is as follows:

[0080] f P (q k )=ψ(q k ,v)

[0081] Among them, f P is the implicit function of the potential scene surface corresponding to the given point cloud P;

[0082] Step A4: unsigned distance from the query point to the potential scene surface, i.e., using the implicit function decoder ψ, with the latent spatial feature representation of the potential scene surface and the query point as input, an implicit value is regressed for each query point, where the implicit value is set to the unsigned distance from the query point to the potential scene surface.

[0083] like Figure 3 As shown, step S2 specifically includes the following steps:

[0084] Step B1: For a given vehicle-mounted laser point cloud P, calculate each point p i The average distance to its M nearest neighbors, denoted as d i , represents point p i The local geometric proximity of

[0085] Step B2: Count the local geometric neighbor averages of all points in the point cloud P as the approximate unsigned distance field D of the potential scene surface of the point cloud P. P The approximation threshold d t , and then realize the adaptive approximation threshold selection for the given point cloud sparsity;

[0086] Step B3: Compare the distance between the query point and its nearest neighbor. When the distance d between the query point q and its nearest neighbor is q Less than the approximation threshold d t When, along the nearest neighbor point p N Normal Calculate the query point q to the nearest neighbor point p N is the center of the sphere and approximates the threshold d t The distance of the tangent plane of the sphere with radius q is the unsigned distance of the query point q;

[0087] Step B4: When the distance between the query point and its nearest neighbor is greater than the approximation threshold, the unsigned distance of the query point is the difference between the distance of the nearest neighbor and the radius of the search sphere, that is, when the distance d between the query point q and its nearest neighbor is greater than the approximation threshold, the unsigned distance of the query point is the difference between the distance of the nearest neighbor and the radius of the search sphere. q Greater than or equal to the approach threshold d t When , the unsigned distance of the query point q is directly defined as (d q -d t ); specifically as follows:

[0088]

[0089] in, Indicates that the starting point is p N , the vector whose endpoint is the query point q;

[0090] Step B5: Use the constructed approximate unsigned distance field of the query point as the learning signal for the self-supervised learning of the three-dimensional scene depth implicit expression model, that is, use the established approximate unsigned distance field as the self-supervised learning signal of the vehicle-mounted laser point cloud scene implicit expression framework.

[0091] Step S3 specifically includes the following steps:

[0092] Step C1: Use the trained 3D scene depth implicit expression model to perform unsigned distance inference on the given source scene vehicle laser point cloud to establish the source scene unsigned distance field DF s ;

[0093] Step C2: Use the trained 3D scene depth implicit expression model to perform unsigned distance inference on the given target scene vehicle laser point cloud and establish the target scene unsigned distance field DF t .

[0094] Step S4 specifically includes the following steps:

[0095] Step D1: For the source scene vehicle laser point cloud P s and target scene vehicle-mounted laser point cloud P t , respectively set the unsigned distance threshold ε;

[0096] Step D2: convert the source scene unsigned distance field DF s The query points smaller than the threshold ε are sampled as the point cloud of the implicit expression space of the source scene

[0097] Step D3: The target scene unsigned distance field DF t The query points smaller than the threshold ε are sampled as the point cloud of the implicit expression space of the source scene

[0098] Step D4: Sampling based on the same implicit expression space, resulting in and Obey the same spatial distribution, thus achieving point-level distribution alignment across perception configuration domains.

[0099] Step S5 specifically includes the following steps:

[0100] Step E1: Implicitly express spatial point cloud for the source scene Each point in Search it to the original source scene vehicle laser point cloud {P s ,Y s}N nearest neighbor points in;

[0101] Step E2: Using the semantic labels Y of the N neighboring pointss Implicitly express spatial points for the corresponding source scene Vote for the tag category;

[0102] Step E3: Count the voting results of the label categories and define the label category with the most votes as the implicit expression space point of the source scene Semantic category label of ;

[0103] Step E4: Traverse All points in , construct the source scene semantic segmentation training dataset of the implicit expression space

[0104] Step S6 specifically includes the following steps:

[0105] Step F1: Use the implicitly expressed spatial point cloud as input to establish a point cloud semantic segmentation model; the semantic segmentation model can also be other existing point cloud semantic segmentation models that directly target discrete points;

[0106] Step F2: Using implicit expression of spatial point cloud semantic labels As a supervised learning signal, the cross entropy loss is used to constrain the semantic segmentation model to optimize training, that is, to construct a cross entropy loss function and train the semantic segmentation model.

[0107] Step S7 specifically includes the following steps:

[0108] Step G1: Use the above optimized trained semantic segmentation model to implicitly express the spatial target scene point cloud Pointwise semantic labels Make inferences;

[0109] Step G2: For each point P in the original target scene point cloud t(k) , search for its nearest neighbor point in the corresponding implicit expression space target scene point cloud

[0110] Step G3: Use nearest neighbor points Semantic label inference results Infer the semantic label Y of each point in the original target scene point cloud t , and finally complete the point cloud adaptive semantic segmentation of the cross-vehicle lidar configuration.

[0111] It should be noted that, in this document, relational terms such as first and second, etc., are used solely to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. Furthermore, the terms "include," "comprise," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. Without further limitation, elements defined by the phrase "include..." or "comprising..." do not exclude the presence of additional elements in the process, method, article, or terminal device comprising the elements. Furthermore, in this document, "greater than," "less than," "exceeding," etc., are understood to exclude the number itself; "above," "below," "within," etc., are understood to include the number itself.

[0112] Those skilled in the art will appreciate that the above embodiments may be provided as methods, devices, or computer program products. These embodiments may take the form of fully hardware embodiments, fully software embodiments, or embodiments combining software and hardware. All or part of the steps in the methods involved in the above embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a storage medium readable by a computer device for executing all or part of the steps described in the methods of the above embodiments. The computer device includes, but is not limited to, personal computers, servers, general-purpose computers, special-purpose computers, network devices, embedded devices, programmable devices, smart mobile terminals, smart home devices, wearable smart devices, in-vehicle smart devices, etc.; the storage medium includes, but is not limited to, RAM, ROM, magnetic disks, magnetic tapes, optical disks, flash memory, USB flash drives, mobile hard disks, memory cards, memory sticks, network server storage, network cloud storage, etc.

[0113] The above embodiments are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a computer device to generate a machine, so that the instructions executed by the processor of the computer device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0114] These computer program instructions can also be stored in a computer device readable memory that can guide a computer device to work in a specific manner, so that the instructions stored in the computer device readable memory produce a product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions can also be loaded onto a computer device so that a series of operating steps are executed on the computer device to produce a computer-implemented process, whereby the instructions executed on the computer device provide for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0116] Although the above embodiments have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concepts. Therefore, the above descriptions are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the scope of patent protection of the present invention.

Claims

1. A method for adaptive semantic segmentation of point clouds across vehicle-mounted laser radar configurations, capable of adaptively segmenting point cloud data acquired by a vehicle-mounted laser scanning system equipped with different laser scanners, characterized by: The following steps are included: Step S1: Using a vehicle-mounted laser scanning system equipped with a laser radar, a point cloud for semantic segmentation model training is collected and used as the source scene vehicle-mounted laser point cloud; using a point cloud collected by a vehicle-mounted laser scanning system equipped with another laser radar and used for semantic segmentation model testing is used as the target scene vehicle-mounted laser point cloud; based on the source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud, a deep neural network model for implicitly representing the three-dimensional scene is established as a three-dimensional scene depth implicit expression framework, thereby reconstructing the three-dimensional scene continuous surface of the vehicle-mounted laser point cloud; Step S2: using an approximate unsigned distance field based on local geometric statistics, constructing a self-supervised learning paradigm for implicit representation of 3D scene depth, ensuring high-fidelity reconstruction of continuous scene surfaces in the implicit representation space; Step S3: Using a 3D scene depth implicit representation framework trained with a self-supervised learning paradigm based on approximate unsigned distance fields, perform query point unsigned distance inference on a given source scene vehicle-mounted laser point cloud and a target scene vehicle-mounted laser point cloud, and establish unsigned distance fields for the source scene and the target scene, respectively; Step S4: resample the source scene vehicle laser point cloud and the target scene vehicle laser point cloud using an unsigned distance threshold to obtain corresponding point clouds in the implicit expression space, thereby achieving point-level distribution alignment of the cross-vehicle laser radar configuration point clouds; Step S5: Use the source scene vehicle-mounted laser point cloud label data to perform neighbor label voting on its implicit expression space point cloud to construct a semantic segmentation training dataset of the source scene in the implicit expression space; Step S6: using the semantic segmentation training dataset to train and optimize the point cloud semantic segmentation model; Step S7: using the optimized and trained semantic segmentation model to perform point-by-point semantic label inference on the implicitly expressed spatial target scene point cloud, and performing nearest neighbor radiation on the inference results to complete the adaptive semantic segmentation of the point cloud configured across the on-board lidar; Step S1 specifically The following steps are included: Step A1: Use an encoder based on a three-dimensional convolutional neural network (3DCNN) to map the given source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud into a feature representation that perceives the scene geometry. That is, establish a scene implicit expression framework that covers the source scene vehicle-mounted laser point cloud and the target scene vehicle-mounted laser point cloud. Specifically: For a given vehicle-mounted laser point cloud A 3DCNN-based encoder θ is used to map a given vehicle-mounted laser point cloud P into a scene geometry-aware feature representation v = θ(P), which is considered as a latent spatial feature representation of the potential scene surface of the given vehicle-mounted laser point cloud P. Step A2: randomly sample query points from the minimum bounding box of the given vehicle-mounted laser point cloud; specifically: randomly sample K query points from the minimum bounding box of the given vehicle-mounted laser point cloud P, denoted as Step A3: Using an implicit function decoder, with the potential scene geometry perception feature representation and the query point as input, regress an implicit value for each query point, specifically: Combined implicit neural representation, define each query point q K The implicit value of ∈Q, that is, the distance from the query point to the potential scene surface, is as follows: f P (what K )=ψ(q K ,v) Among them, f P is the implicit function of the potential scene surface corresponding to the given vehicle-mounted laser point cloud P; Step A4: unsigned distance from the query point to the potential scene surface, i.e., using the implicit function decoder ψ, with the latent spatial feature representation of the potential scene surface and the query point as input, an implicit value is regressed for each query point, where the implicit value is set to the unsigned distance from the query point to the potential scene surface.

2. The point cloud adaptive semantic segmentation method for cross-vehicle LiDAR configuration according to claim 1, characterized in that: Step S2 specifically The following steps are included: Step B1: For a given vehicle-mounted laser point cloud P, calculate each point p i The average distance to its M nearest neighbors, denoted as d i , represents point p i The local geometric proximity of Step B2: Count the local geometric neighbor averages of all points in the vehicle-borne laser point cloud P as the approximate unsigned distance field D of the potential scene surface of the vehicle-borne laser point cloud P P The approximation threshold d t , and then realize the adaptive approximation threshold selection for the given point cloud sparsity; Step B3: Compare the distance between the query point and its nearest neighbor. When the distance d between the query point q and its nearest neighbor is q Less than the approximation threshold d t When, along the nearest neighbor point p N Normal Calculate the query point q to the nearest neighbor point p N is the center of the sphere and approximates the threshold d t The distance of the tangent plane of the sphere with radius q is the unsigned distance of the query point q; Step B4: When the distance between the query point and its nearest neighbor is greater than or equal to the approximation threshold, the unsigned distance of the query point is the difference between the distance of the nearest neighbor and the radius of the search sphere, that is, when the distance d between the query point q and its nearest neighbor is greater than or equal to the approximation threshold, the unsigned distance of the query point is the difference between the distance of the nearest neighbor and the radius of the search sphere. q Greater than or equal to the approach threshold d t When , the unsigned distance of the query point q is directly defined as (d q -d t ); specifically as follows: in, Indicates that the starting point is p N , the vector whose endpoint is the query point q; Step B5: Use the constructed approximate unsigned distance field of the query point as the learning signal for the self-supervised learning of the three-dimensional scene depth implicit expression model, that is, use the established approximate unsigned distance field as the self-supervised learning signal of the vehicle-mounted laser point cloud scene implicit expression framework.

3. The method for adaptive semantic segmentation of point clouds across vehicle-mounted LiDAR configurations according to claim 2, characterized in that: Step S3 specifically The following steps are included: Step C1: Use the trained 3D scene depth implicit expression model to perform unsigned distance inference on the given source scene vehicle laser point cloud to establish the source scene unsigned distance field DF s ; Step C2: Use the trained 3D scene depth implicit expression model to perform unsigned distance inference on the given target scene vehicle laser point cloud and establish the target scene unsigned distance field DF t .

4. The method for adaptive semantic segmentation of point clouds across vehicle-mounted LiDAR configurations according to claim 3, characterized in that: Step S4 specifically includes the following steps: Step D1: For the source scene vehicle laser point cloud P s and target scene vehicle-mounted laser point cloud P t , respectively set the unsigned distance threshold ε; Step D2: convert the source scene unsigned distance field DF s The query points smaller than the threshold ε in the sample are the point cloud of the implicit expression space of the source scene Step D3: The target scene unsigned distance field DF t The query points smaller than the threshold ε are sampled as the point cloud P of the implicit expression space of the target scene t imp ; Step D4: Sampling based on the same implicit expression space, resulting in and P t imp Obey the same spatial distribution, thus achieving point-level distribution alignment across perception configuration domains.

5. The point cloud adaptive semantic segmentation method for cross-vehicle LiDAR configuration according to claim 4, characterized in that: Step S5 specifically includes the following steps: Step E1: Implicitly express spatial point cloud for the source scene Each point in Search it to the original source scene vehicle laser point cloud {P s ,Y s }N nearest neighbor points in; Step E2: Using the semantic labels Y of the N neighboring points s Implicitly express spatial points for the corresponding source scene Vote for the tag category; Step E3: Count the voting results of the label categories and define the label category with the most votes as the implicit expression space point of the source scene Semantic category label of ; Step E4: Traverse All points in , construct the source scene semantic segmentation training dataset of the implicit expression space To implicitly express the semantic labels of spatial point clouds.

6. The point cloud adaptive semantic segmentation method for cross-vehicle LiDAR configuration according to claim 5, characterized in that: Step S6 specifically includes the following steps: Step F1: Use the implicit expression space point cloud as input to establish a point cloud semantic segmentation model; this semantic segmentation model is different from other existing point cloud semantic segmentation models that directly target discrete points; Step F2: Using implicit expression of spatial point cloud semantic labels As a supervised learning signal, the cross entropy loss is used to constrain the semantic segmentation model to optimize training, that is, to construct a cross entropy loss function and train the semantic segmentation model.

7. The point cloud adaptive semantic segmentation method for cross-vehicle LiDAR configuration according to claim 6, characterized in that: Step S7 specifically The following steps are included: Step G1: Use the above optimized trained semantic segmentation model to implicitly express the spatial target scene point cloud P t imp Pointwise semantic label Y t imp Make inferences; Step G2: For each point P in the original target scene point cloud t(k) , search for its nearest neighbor point in the corresponding implicit expression space target scene point cloud Step G3: Use nearest neighbor points Semantic label inference results Infer the semantic label Y of each point in the original target scene point cloud t , and finally complete the point cloud adaptive semantic segmentation of the cross-vehicle lidar configuration.

Images