Method, device and equipment for three-dimensional point cloud segmentation and storage medium

Abstract

Embodiments of the present application provide a three-dimensional point cloud segmentation method, device, equipment and storage medium, the method comprises: inputting a three-dimensional point cloud into a feature extraction model to obtain all first point feature vectors corresponding to the three-dimensional point cloud; determining the result of mapping the first point feature sequence to a new feature space based on a point feature mapping model; determining the category corresponding to the second point feature vector according to the first classification criterion and the prototype set; determining the category corresponding to the first point feature vector based on the category corresponding to the second point feature vector, and performing open set segmentation on the three-dimensional point cloud according to the category corresponding to the first point feature vector; the present application maps all point features corresponding to the three-dimensional point cloud to a new feature space by using a point feature mapping model, improves the distinguishing degree of the point features of the three-dimensional point cloud, correctly classifies all point features, marks the unknown category of the point features as an unknown category, and thus improves the resolution of the point features belonging to the known category or the unknown category in the three-dimensional point cloud.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and storage medium for three-dimensional point cloud segmentation. Background Technology

[0002] 3D point cloud semantic segmentation is an important research topic in the field of computer vision, aiming to assign a semantic category label to each point in a 3D point cloud.

[0003] Existing classic point cloud segmentation algorithms do not account for the existence of unknown data categories in open worlds. These methods tend to misclassify all points of unknown categories as known categories. This error can have extremely serious consequences in real-world applications, such as autonomous driving.

[0004] Therefore, improving the ability to identify unknown categories and correctly classifying them as unknown categories during the process of identifying known categories has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] In view of the technical problems existing in the prior art, the present invention provides a method, apparatus, device and storage medium for three-dimensional point cloud segmentation.

[0006] In a first aspect, the present invention provides a method for three-dimensional point cloud segmentation, comprising:

[0007] The three-dimensional point cloud is input into the feature extraction model to obtain all the first point feature vectors corresponding to the three-dimensional point cloud, which are used as the first point feature sequence; the first point feature vector is used to characterize any point feature of the three-dimensional point cloud that belongs to a known category;

[0008] Based on the point feature mapping model, the result of mapping the first point feature sequence to the new feature space is determined and used as the second point feature sequence.

[0009] According to the first classification criterion and the prototype set, the category corresponding to the second feature vector is determined; the prototype set includes all prototype vectors belonging to the known categories; the second feature vector is any vector in the second feature sequence.

[0010] Based on the category corresponding to the second point feature vector, the category corresponding to the first point feature vector is determined, and the three-dimensional point cloud is segmented into an open set according to the category corresponding to the first point feature vector;

[0011] The point feature mapping model is obtained by training a generative adversarial network based on the point features corresponding to the sample 3D point cloud. The generative adversarial network is used to optimize the point feature distribution of the 3D point cloud by introducing point features belonging to unknown categories during the training process.

[0012] Optionally, the generative adversarial network consists of a feature generation model and a feature discrimination model; the feature generation model is used to generate synthetic point features belonging to an unknown category, and the feature discrimination model is used to distinguish whether the input point features are true features or false features.

[0013] Optionally, the point feature mapping model is obtained based on the point features corresponding to the sample 3D point cloud and trained using a generative adversarial network, the training method including:

[0014] Based on the first noise set, a synthetic point feature vector corresponding to the first point feature vector is generated, and each of the synthetic point feature vectors belongs to an unknown category;

[0015] Based on the synthetic point feature vector and the prior distribution of point features belonging to unknown categories, a first objective function of the feature generation model is determined; the first objective function is used to characterize the similarity between the synthetic point features generated by the feature generation model and the first point features.

[0016] Based on the first point feature vector and the corresponding synthetic point feature vector, a second objective function of the feature discrimination model is determined; the second objective function is used to characterize the ability of the feature discrimination model to distinguish between the synthetic point feature and the first point feature.

[0017] Based on the first loss function of the prototype constraint module and the prior distribution of the unknown category point features, the third objective function of the point feature mapping model is determined; the third objective function is used to characterize the difference between the point features and the prototype vector.

[0018] The point feature mapping model is trained when the first objective function, the second objective function, and the third objective function meet preset conditions.

[0019] Optionally, before determining the first objective function of the feature generation model based on the synthesized point feature vector and the prior distribution of point features belonging to unknown categories, the process includes:

[0020] The synthesized point feature vectors with a first probability less than a preset screening threshold are selected as target synthesized point feature vectors; the first probability is the probability that the synthesized point feature vector is determined to be a true feature.

[0021] Based on the target synthetic point feature vector and the normalized result of the initialized prototype set, the distribution prior of the point features belonging to the unknown category is determined.

[0022] Optionally, determining the result of mapping the first point feature sequence to the new feature space based on the point feature mapping model includes:

[0023] Based on the nonlinear layer of the point feature mapping model, the equidimensional mapping result of the first point feature vector to the new feature space is determined and used as the second point feature vector.

[0024] Based on all the second point feature vectors, determine the second point feature sequence.

[0025] Optionally, before determining the category corresponding to the second feature vector according to the first classification criterion and the prototype set, the following steps are included:

[0026] Based on the feature fusion result of the second point feature vector belonging to the known category of the target in the second point feature sequence, the prototype vector belonging to the known category of the target is determined; the known category of the target is any one of all known categories to which the second point feature sequence belongs;

[0027] The prototype set is updated based on the prototype vectors belonging to the known category of the target.

[0028] Optionally, determining the category corresponding to the second feature vector according to the first classification criterion and the prototype set includes:

[0029] Based on the distance between the second feature vector and the prototype vector of the known category in the new feature space, the probability that the second feature vector belongs to the unknown category is determined as the second probability.

[0030] If the second probability is greater than or equal to the preset category threshold, then the category corresponding to the second feature vector is determined to be an unknown category;

[0031] If the second probability is less than a preset category threshold, then based on the target prototype vector corresponding to the first feature vector and the category to which the target prototype vector belongs, the category corresponding to the second feature vector is determined to be the category to which the target prototype vector belongs.

[0032] The second feature vector corresponds one-to-one with the first feature vector; the target prototype vector is the prototype vector corresponding to the target category when the probability value of the first feature vector belonging to the target category is the highest.

[0033] Optionally, the training method for the feature extraction model includes:

[0034] Obtain all first point feature vectors corresponding to the three-dimensional point cloud of the sample, and use them as the first point feature sequence;

[0035] Based on the distance between the first feature vector and the first category, the probability that the first feature vector belongs to the first category is determined as the first probability; the first feature vector is any feature vector in the first feature sequence; the first category is any one of all categories corresponding to the initial prototype set.

[0036] Based on the first probability, the first point feature vector, and the first category, the distance loss function, distribution loss function, and dimensionality loss function of the prototype constraint module are determined.

[0037] Based on the weight summation method, the distance loss function, the distribution loss function, and the dimension loss function, the first loss function of the prototype constraint module is determined.

[0038] If the first loss function converges, the training of the feature extraction module is completed.

[0039] Secondly, the present invention also provides a three-dimensional point cloud segmentation apparatus, comprising:

[0040] The acquisition module is used to input the 3D point cloud into the feature extraction model and obtain all the first point feature vectors corresponding to the 3D point cloud as the first point feature sequence; the first point feature vector is used to characterize any point feature of the 3D point cloud belonging to a known category;

[0041] The mapping module is used to determine the result of mapping the first point feature sequence to a new feature space based on the point feature mapping model, and use it as the second point feature sequence.

[0042] The determination module is used to determine the category corresponding to the second feature vector according to the first classification criterion and the prototype set; the prototype set includes all prototype vectors belonging to known categories; the second feature vector is any vector in the second feature sequence.

[0043] The segmentation module is used to determine the category corresponding to the first point feature vector based on the category corresponding to the second point feature vector, and to perform open set segmentation on the three-dimensional point cloud according to the category corresponding to the first point feature vector;

[0044] The point feature mapping model is obtained by training a generative adversarial network based on the point features corresponding to the sample 3D point cloud. The generative adversarial network is used to optimize the point feature distribution of the 3D point cloud by introducing point features belonging to unknown categories during the training process.

[0045] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the three-dimensional point cloud segmentation method described in the first aspect above.

[0046] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the three-dimensional point cloud segmentation method as described in the first aspect above.

[0047] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the three-dimensional point cloud segmentation method as described in the first aspect above.

[0048] The method, apparatus, device, and storage medium for three-dimensional point cloud segmentation provided by this invention map all point features corresponding to the three-dimensional point cloud to a new feature space using a point feature mapping model. Then, according to a first classification criterion and a prototype set composed of prototype vectors of all known categories, the category to which the mapped point features of the three-dimensional point cloud belong is determined. This improves the discriminative power of the point features in the three-dimensional point cloud, correctly classifies all point features, and marks point features of unknown categories as unknown categories, thereby enhancing the ability to distinguish whether point features in the three-dimensional point cloud belong to known or unknown categories. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the three-dimensional point cloud segmentation method provided in an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of the prototype constraint module provided in an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of the feature countermeasure module provided in an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram illustrating the implementation process of the three-dimensional point cloud segmentation method provided in this embodiment of the invention;

[0054] Figure 5 This is a schematic diagram of the structure of the three-dimensional point cloud segmentation device provided in an embodiment of the present invention;

[0055] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0056] In this embodiment of the invention, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0057] In this embodiment of the invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Figure 1 This is a flowchart illustrating the three-dimensional point cloud segmentation method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0060] Step 101: Input the 3D point cloud into the feature extraction model to obtain all the first point feature vectors corresponding to the 3D point cloud, as the first point feature sequence; the first point feature vector is used to characterize any point feature of the 3D point cloud belonging to a known category;

[0061] Step 102: Based on the point feature mapping model, determine the result of mapping the first point feature sequence to the new feature space, and use it as the second point feature sequence;

[0062] Step 103: Determine the category corresponding to the second feature vector according to the first classification criterion and the prototype set; the prototype set includes all prototype vectors belonging to the known categories; the second feature vector is any vector in the second feature sequence.

[0063] Step 104: Based on the category corresponding to the second point feature vector, determine the category corresponding to the first point feature vector, and perform open set segmentation on the three-dimensional point cloud according to the category corresponding to the first point feature vector;

[0064] The point feature mapping model is obtained by training a generative adversarial network based on the point features corresponding to the sample 3D point cloud. The generative adversarial network is used to optimize the point feature distribution of the 3D point cloud by introducing point features belonging to unknown categories during the training process.

[0065] Specifically, 3D point clouds are the most common and fundamental 3D model in spatial information representation. Point cloud models are often obtained directly from measurements, with each point corresponding to a measurement point, without any other processing methods, thus containing the largest amount of target information. 3D point clouds are mainly used for target monitoring, shape classification, and target segmentation. Among these, classification and segmentation tasks have been widely applied in surveying and mapping, navigation and positioning, and autonomous driving. The inability to effectively distinguish between seemingly similar targets refers to the phenomenon where two or more adjacent targets cannot be correctly classified and segmented due to their very similar appearance. For example, objects such as beams, columns, and walls have very similar appearances, differing only in local geometric structure and contextual details. If these details are not fully explored, it can easily lead to errors in target classification and segmentation.

[0066] This invention utilizes the Stanford Large-Scale 3D Indoor Spaces Dataset (S3DIS) and the SemanticKITTI dataset, which are representative of 3D point clouds, to initialize the prototype set, i.e., the set of prototype vectors, in this invention. The initialized prototype set can be represented as follows:

[0067] Where P represents the prototype set, P j Let represent the prototype belonging to the j-th known category. It can be represented as a vector, where C is the total number of known categories, and j takes the value [1, C].

[0068] Inputting the 3D point cloud into the trained feature extraction model yields all the point features corresponding to the 3D point cloud, which can be represented as the first point feature sequence, specifically as follows:

[0069]

[0070] Where F represents the set of all point features in the 3D point cloud, which can be represented as the first point feature sequence, f i The feature vector representing the i-th point in a 3D point cloud can be represented as a vector, i.e., the feature vector of the first point, N. c Let i represent the number of points of all known categories, where i ranges from [1, N]. c ].

[0071] The corresponding point features are obtained from the 3D point cloud, specifically represented as the first point feature vector. All first point feature vectors constitute the first point feature sequence. These point features are used to describe the local features of the corresponding object and can be the result of aggregation of points from any object, such as the multiple points included in the components that make up a car window. The point features will aggregate information from other points in its neighborhood. Point features are used to classify points, indicating which category the point belongs to.

[0072] By using a generative adversarial network (GAN) to train the point feature mapping model, point features belonging to unknown categories are introduced to optimize the point feature distribution of the 3D point cloud. Based on the point features corresponding to the sample 3D point cloud, and after training the point feature mapping model using the aforementioned GAN, this model can map the first point feature sequence obtained from the 3D point cloud to a new feature space, obtaining a second point feature sequence, thereby improving the discriminative power between the point features.

[0073] Next, according to the first classification criterion and the prototype set, the category corresponding to any feature vector in the second feature sequence is determined, i.e., the category corresponding to the second feature vector is determined. This allows for accurate classification of the categories corresponding to each feature. The first and second feature sequences are in one-to-one correspondence; the mapped first feature sequence is the second feature sequence. Determining the category corresponding to the second feature vector is equivalent to determining the category corresponding to the first feature vector. Therefore, based on the category corresponding to the first feature vector in the 3D point cloud, open-set segmentation is performed on the 3D point cloud. Because a point feature mapping model is used, the obtained point features have higher discriminative power, resulting in more accurate categories corresponding to each feature in the 3D point cloud. Furthermore, open-set segmentation correctly classifies all known category samples while identifying all unknown category samples as unknown categories, thus improving the distinction between known and unknown category point features.

[0074] The three-dimensional point cloud segmentation method provided by this invention maps all point features corresponding to the three-dimensional point cloud to a new feature space using a point feature mapping model. Then, according to a first classification criterion and a prototype set composed of prototype vectors of all known categories, the category to which the mapped point features of the three-dimensional point cloud belong is determined, thereby improving the discriminative power of the point features of the three-dimensional point cloud, correctly classifying all point features, and marking point features of unknown categories as unknown categories, thereby improving the resolution of whether point features in the three-dimensional point cloud belong to known or unknown categories.

[0075] Optionally, the training method for the feature extraction model includes:

[0076] Obtain all first point feature vectors corresponding to the three-dimensional point cloud of the sample, and use them as the first point feature sequence;

[0077] Based on the distance between the first feature vector and the first category, the probability that the first feature vector belongs to the first category is determined as the first probability; the first feature vector is any feature vector in the first feature sequence; the first category is any one of all categories corresponding to the initial prototype set.

[0078] Based on the first probability, the first point feature vector, and the first category, the distance loss function, distribution loss function, and dimensionality loss function of the prototype constraint module are determined.

[0079] Based on the weight summation method, the distance loss function, the distribution loss function, and the dimension loss function, the first loss function of the prototype constraint module is determined.

[0080] If the first loss function converges, the training of the feature extraction module is completed.

[0081] Specifically, before using a feature extraction model to obtain all point features corresponding to a 3D point cloud, the feature extraction model needs to be trained. Specific training methods include:

[0082] The sample 3D point cloud is used as input to jointly train a feature extraction model and a prototype constraint module (PCM). The sample 3D point cloud can be obtained from the S3DIS dataset or the SemanticKITTI dataset. The feature extraction model mainly obtains the point features of the 3D point cloud through a feature extraction network, which can use existing skeleton networks for extracting point cloud features. This will contain N... c The 3D point cloud of each point is input into the feature extraction model to obtain point-by-point features, represented as follows:

[0083]

[0084] Where F represents the set of all point features in the 3D point cloud, which can be represented as the first point feature sequence, f i The feature vector representing the i-th point in a 3D point cloud can be represented as a vector, i.e., the feature vector of the first point, N. c Let i represent the number of points of all known categories, where i takes values ​​in the range [1, N]. c ].

[0085] The initialized prototype set can be represented as Where C represents the number of known categories.

[0086] Based on the first feature vector and the prototype vector of the known class, the distance between them in the feature space is determined. The method for determining the distance between the first feature vector and the prototype vector of the known class combines Euclidean distance and the angle between the two feature vectors, and can be specifically expressed as:

[0087] Where d(f) i P c ) represents the eigenvector f of the first point. i and the prototype vector P of the known category c The distance between them, f i Let P represent the feature of the i-th point. c Let f represent the prototype vector corresponding to category c, ||·||2 represent the L2 norm, and f i ·P c f represents the eigenvector of the first point. i and the prototype vector P of the known category c The dot product between them, which is the eigenvector f of the first point. i and the prototype vector P of the known category c The angle between features in the feature space.

[0088] Based on the distance between the first point feature vector and the first category, the probability that the first point feature vector belongs to the first category is determined, which is used as the first probability. This is equivalent to predicting the category of the known point features, specifically expressed as follows:

[0089]

[0090] Where p(y) i =c|f i P) represents the eigenvector f of the first point. i Category y i The probability of f is c. i Let P represent the feature of the i-th point. c d(f) represents the prototype vector corresponding to category c. i P c ) represents the feature f of the i-th point. i The prototype vector P corresponding to category c c The distance in the feature space; the greater the distance, the stronger the feature f of the point. i Category y i The smaller the probability of being c, the more exp() represents an exponential function with the natural constant e as the base, exp(-d(f i P c )) represents the natural constant e with respect to -d(f i P c ) to the power of P j This represents the prototype vector belonging to the j-th known category.

[0091] To ensure that point features of a known class are closer to prototype vectors belonging to that known class and farther from prototype vectors of other classes, distance-based cross-entropy loss is used, i.e., the distance loss function of the prototype constraint module, which can be expressed as:

[0092]

[0093] in, N represents the distance loss function of the prototype constraint module. c p(y) represents the number of point features for all known categories; i =c|f i , P) First point eigenvector f i Category y i The probability of f is c. i Let P represent the feature of the i-th point. c This represents the prototype vector corresponding to category c.

[0094] Furthermore, to make the distribution of point features of the same category and prototype vectors belonging to the same category more compact in the feature space, a distribution loss function is also used, which can be expressed as:

[0095]

[0096] in, N represents the distributed loss function of the prototype constraint module. c This represents the number of point features across all known categories; f represents the eigenvector of the first point. i With prototype vector P c The square of the L2 norm. The smaller the distribution loss function, the more compact the distribution of point features of the same class and prototype vectors belonging to the same class in the feature space.

[0097] Furthermore, under the constraints of the distance loss function and the distribution loss function mentioned above, the network corresponding to the prototype constraint module may experience all-zero degeneration. To avoid this degeneration, this invention also introduces a dimensionality loss function to the prototype constraint module, which balances the distribution of features in each dimension using the L1 norm. Specifically, it can be expressed as:

[0098]

[0099] in, The dimension loss function of the prototype constraint module is represented by C, where C represents the total number of known categories; d represents the feature dimension of the prototype vector; P jk Represents the prototype vector P j The k-th eigenvalue.

[0100] Based on the weight summation method, the distance loss function, the distribution loss function, and the dimension loss function, the first loss function of the prototype constraint module is determined.

[0101] The first loss function of the prototype constraint module can be expressed as:

[0102]

[0103] in, This represents the first loss function of the prototype constraint module. This represents the distance loss function of the prototype constraint module. This represents the distributed loss function of the prototype constraint module. The dimension loss function representing the prototype constraint module; λ attr λ represents the weight hyperparameters corresponding to the distributed loss function. info This represents the weight hyperparameters corresponding to the dimensionality loss function. Generally, λ... attr and λ info The optimal value range is [0, 1], while the best value obtained from actual debugging is 0.1.

[0104] The feature extraction module and the prototype constraint module are jointly trained. The training of the feature extraction model is complete when the first loss function of the prototype constraint module converges.

[0105] During training, as the value of the first loss function changes, the relevant parameters of the feature extraction model are updated backpropagated, along with the prototype set composed of prototype vectors obtained from the prototype constraint module. A schematic diagram of the prototype constraint module is shown below. Figure 2 As shown.

[0106] Optionally, the generative adversarial network consists of a feature generation model and a feature discrimination model; the feature generation model is trained based on the prior distribution of point features belonging to unknown categories; the feature discrimination model is used to distinguish whether the input point features are true features or false features.

[0107] Specifically, the point feature mapping model is obtained based on the point features corresponding to the sample 3D point cloud and the generative adversarial network trained by it. The generative adversarial network consists of a feature generation model and a feature discrimination model.

[0108] The feature generation model is trained based on the prior distribution of point features belonging to unknown categories. Its goal is to simulate the distribution of point features of unknown categories. It can not only synthesize features similar to the supervision information (the real known category point features), but also make the generated synthetic point features aggregate in the middle of the feature space.

[0109] The feature discrimination model is used to distinguish whether the input point features are real or fake features. Real features indicate that the input point features are real features, just like the point features extracted from the 3D point cloud, while fake features indicate that the input point features are point features synthesized by the feature generation module.

[0110] Generative Adversarial Networks (GANs) use feature generation and feature discrimination models to enable the feature generation model to synthesize realistic fake features, thereby increasing the probability that the synthesized features generated by the feature generation model will be judged as real features by the feature discrimination model.

[0111] Optionally, the point feature mapping model is obtained based on the point features corresponding to the sample 3D point cloud and trained using a generative adversarial network, the training method including:

[0112] Based on the first noise set, a synthetic point feature vector corresponding to the first point feature vector is generated, and each of the synthetic point feature vectors belongs to an unknown category;

[0113] Based on the synthetic point feature vector and the prior distribution of point features belonging to unknown categories, a first objective function of the feature generation model is determined; the first objective function is used to characterize the similarity between the synthetic point features generated by the feature generation model and the first point features.

[0114] Based on the first point feature vector and the corresponding synthetic point feature vector, a second objective function of the feature discrimination model is determined; the second objective function is used to characterize the ability of the feature discrimination model to distinguish between the synthetic point feature and the first point feature.

[0115] Based on the first loss function of the prototype constraint module and the prior distribution of the unknown category point features, the third objective function of the point feature mapping model is determined; the third objective function is used to characterize the difference between the point features and the prototype vector.

[0116] The point feature mapping model is trained when the first objective function, the second objective function, and the third objective function meet preset conditions.

[0117] Specifically, after the feature extraction model is trained, its parameters can be considered fixed. Next, the feature generation model, feature discrimination model, point feature mapping model, and prototype constraint module need to be trained jointly. The structural diagram of the Feature Adversarial Module (FAM), which consists of the feature generation model, feature discrimination model, and point feature mapping model, is shown below. Figure 3As shown, the feature generation model generates synthetic point features belonging to unknown categories based on random noise. These features are then input into a feature discrimination model to determine if they are genuine features. Based on the discrimination model's output and the generated synthetic point features, the data is input into a feature filtering unit. This unit further filters synthetic point features deemed genuine, removing those with excessive similarity to known category features to avoid negatively impacting segmentation of known categories. Furthermore, a priori distribution of unknown category point features is introduced, causing the synthetic point features to aggregate in the middle region of the feature space. The filtered genuine synthetic point features and point features obtained from the 3D point cloud are input into a point feature mapping model to determine a new feature space. The model outputs the result of mapping the point features from the 3D point cloud to the new feature space. The output of the feature mapping model is used to determine the priori distribution of unknown category point features and the loss function of the prototype constraint module. Finally, the result of mapping the point features obtained from the 3D point cloud to the new feature space is input into the prototype constraint module to update the parameters of the prototype set composed of prototype vectors in the prototype constraint module, the feature generation model (generator G), the feature discrimination model (discriminator D), and the point feature mapping model (adversarial mapper M) in the feature adversarial module.

[0118] The feature generation model (generator G) will generate noise. As input, synthetic point features are generated. To ensure that the synthetic features generated by the feature generation model are infinitely close to the real features, and that the discriminator classifies the synthetic features as true, the feature generation model needs to be optimized. The formula corresponding to this optimization objective is:

[0119]

[0120] Wherein G(n i ) represents the synthetic features generated by the feature generation model, D(G(n) i )) represents the probability that the feature discrimination model identifies it as a true feature, G represents the generator, and N represents the value of the generator. c This represents the total number of point features belonging to the known class.

[0121] Furthermore, a prior distribution of point features belonging to unknown categories is introduced into the feature generation model, ensuring that the generated synthetic point features are located in the central region of the feature space. Specifically, the methods for determining the prior distribution of point features belonging to unknown categories include:

[0122] The synthesized point feature vectors with a first probability less than a preset screening threshold are selected as target synthesized point feature vectors; the first probability is the probability that the synthesized point feature vector is determined to be a true feature.

[0123] Based on the target synthetic point feature vector and the normalized result of the initialized prototype set, the distribution prior of the point features belonging to the unknown category is determined.

[0124] The synthesized point features output by the feature generation model are usually treated as point features belonging to unknown categories. To prevent the synthesized point features from having too high a similarity to the real point features of known categories, thus negatively impacting the segmentation effect of point features of known categories, this invention also employs a feature selection strategy to avoid this phenomenon, specifically expressed by the formula:

[0125] f s ={G(n i )|D(G(n i ))<λ s},

[0126] Among them, f s G(n) is the point feature selected from the synthesized point features. i The feature generation model is based on the noise signal n. i The generated synthetic point features, D(G(n) i )) represents the point features G(n) synthesized by the feature discrimination model. i The probability that ) is a true feature, λ s The selection threshold is a hyperparameter determined through experimental tuning. A higher probability that a synthesized point feature is classified as a true feature indicates a higher similarity between the synthesized feature and the true known category point features. This may negatively impact the segmentation of known category point features. Therefore, synthesized point features with excessively high similarity need to be filtered out, while those below the selection threshold are retained. For example, if the feature discrimination model outputs [0, 1], where 0 represents a false feature and 1 represents a true feature, then D(G(n_i)) represents the probability that the feature discrimination model classifies a synthesized point feature as a true feature, i.e., the probability that the feature discrimination model outputs a value of 1.

[0127] The prior distribution of the point features belonging to the unknown category can be expressed as:

[0128]

[0129]

[0130] Among them, Adv(f s P) represents the selected point features f s The prior distribution of point features belonging to the unknown category corresponding to the prototype set P, N s h(f) represents the total number of selected point features, C represents the total number of known categories, and h(f) represents the total number of selected point features. i s Pj ) represents the selected point features f s The result after reverting to the new feature space and the prototype vector P j The L2 norm normalization result between them, where ∈ represents the minimum value, is used to prevent missing values. Softmax() represents the L2 norm normalization result between them. Normalization is performed. Representing point features f s The result M(f) after reverting to the new feature space i s ) and prototype vector P j The square of the L2 norm between, M(f i s ) indicates that the point feature f s The result after reverting to the new feature space.

[0131] Furthermore, based on the synthesized point feature vectors and the prior distribution of point features belonging to unknown categories, the first objective function of the feature generation model is determined, which can be expressed as:

[0132]

[0133] Where, λ adv The weight parameters represent the prior distribution of point features belonging to an unknown category.

[0134] The feature discrimination model (discriminator D) is used to determine whether the input features are true or false features. A true feature indicates a high similarity between the synthesized point features and the true point features. A false feature indicates a low similarity between the synthesized and true point features. True point features are obtained from a 3D point cloud. The similarity between the synthesized and true point features is determined by whether the similarity is greater than or equal to a set threshold. Synthetic point features with a similarity greater than or equal to the threshold are considered true features, while those with a similarity less than the threshold are considered false features.

[0135] Based on the point features obtained from the 3D point cloud, namely the first point feature vector, and the synthetic point feature vector corresponding to the first point feature vector, a second objective function of the feature discrimination model is determined, which can be expressed as:

[0136]

[0137] Where, N c f represents the number of point features of all known classes. i D(f) represents the point features of a known class. i G(n) represents the probability that the feature discrimination model identifies a feature as a true feature. i) represents the synthetic point features generated by the feature generation model, D(G(n) i )) represents the probability that the feature discrimination model classifies it as a synthetic feature, n i Let represent the i-th noise, and D represent the feature discrimination model.

[0138] The optimization objective of the feature discrimination model is to correctly distinguish between real features (output 1) and synthetic features (output 0). The larger the maximum value determined by the second objective function, the stronger the discrimination ability of the feature discrimination model.

[0139] Furthermore, to ensure the segmentation ability of point features of known categories, the point feature mapping model also proposes a corresponding optimization objective, namely a third objective function. This third objective function is determined based on the first loss function of the prototype constraint module and the prior distribution of the point features of unknown categories, and can be specifically expressed as:

[0140]

[0141] in, This represents the loss function in the prototype constraint module. For other parameters, please refer to the meaning of the corresponding parameters in the aforementioned formula.

[0142] Thus, the training of the point feature mapping model is completed when the first objective function, the second objective function, and the third objective function meet the preset conditions.

[0143] In other words, the point feature mapping model is trained when the first objective function satisfies its maximum value, the second objective function satisfies its maximum value, and the third objective function satisfies its minimum value.

[0144] Optionally, determining the result of mapping the first point feature sequence to the new feature space based on the point feature mapping model includes:

[0145] Based on the nonlinear layer of the point feature mapping model, the equidimensional mapping result of the first point feature vector to the new feature space is determined and used as the second point feature vector.

[0146] Based on all the second point feature vectors, determine the second point feature sequence.

[0147] Specifically, after training the point feature mapping model, all point features extracted from the 3D point cloud, i.e., the first point feature sequence, are input into the point feature mapping model. Through the nonlinear layer of the point feature mapping model, the first point feature vectors are mapped to a new feature space to obtain the second point feature vectors. In this way, all the second point feature vectors constitute the final second point feature sequence. This second point feature sequence has higher discriminative power than the first point feature sequence.

[0148] Optionally, before determining the category corresponding to the second feature vector according to the first classification criterion and the prototype set, the following steps are included:

[0149] Based on the feature fusion result of the second point feature vector belonging to the known category of the target in the second point feature sequence, the prototype vector belonging to the known category of the target is determined; the known category of the target is any one of all known categories to which the second point feature sequence belongs;

[0150] The prototype set is updated based on the prototype vectors belonging to the known category of the target.

[0151] Specifically, after obtaining the second-point feature sequence corresponding to the 3D point cloud, the prototype constraint module performs feature fusion on the second-point feature vectors belonging to any known category in the second-point feature sequence to obtain prototype vectors belonging to that known category, and updates the prototype set. For example, if the second-point feature sequence includes multiple second-point feature vectors belonging to known categories A, B, and C, the second-point feature vector belonging to known category A is processed through the learnable prototype set in the prototype constraint module to obtain prototype vector A belonging to known category A. Similarly, the second-point feature vector belonging to known category B is input into the prototype constraint module, and prototype vector B belonging to known category B is output. The second-point feature vector belonging to known category C is input into the prototype constraint module, and prototype vector C belonging to known category C is output. Then, these prototype vectors A, B, and C are used to update the prototype set to obtain the updated prototype set. The updated prototype set is synchronized with the second-point feature sequence and, compared to the previously initialized prototype set, better represents the differences between categories, with higher distinguishability.

[0152] Optionally, determining the category corresponding to the second feature vector according to the first classification criterion and the prototype set includes:

[0153] Based on the distance between the second feature vector and the prototype vector of the known category in the new feature space, the probability that the second feature vector belongs to the unknown category is determined as the second probability.

[0154] If the second probability is greater than or equal to the preset category threshold, then the category corresponding to the second feature vector is determined to be an unknown category;

[0155] If the second probability is less than a preset category threshold, then based on the target prototype vector corresponding to the first feature vector and the category to which the target prototype vector belongs, the category corresponding to the second feature vector is determined to be the category to which the target prototype vector belongs.

[0156] The second feature vector corresponds one-to-one with the first feature vector; the target prototype vector is the prototype vector corresponding to the target category when the probability value of the first feature vector belonging to the target category is the highest.

[0157] Specifically, after obtaining the mapping result of the first point feature sequence composed of all point features corresponding to the 3D point cloud in the new feature space, and the updated prototype set, the category corresponding to each point feature in the 3D point cloud is determined according to the first classification criterion, which specifically includes:

[0158] The mapping result of the first feature sequence in the new feature space, i.e., the second feature sequence, can be represented as:

[0159]

[0160] in, Let M(F) represent the second feature sequence, and let M(F) represent the result of mapping the point features output by the feature extraction module to the new feature space. Let n represent the eigenvector of the i-th second point. c This represents the number of second-point feature vectors, and also the total number of point features for all known categories. There is a one-to-one correspondence between the first-point and second-point feature vectors.

[0161] The updated prototype set can also be represented as:

[0162] Based on the second feature vector The distance between the second feature vector and the prototype vector P of the known category in the new feature space, and the probability that the second feature vector belongs to the unknown category, can be expressed as:

[0163]

[0164] in, Represents the eigenvector of the second point The probability of belonging to an unknown category. This represents the i-th second-point feature vector, i.e., the i-th point feature in the new feature space, and the first-point feature vector f. i One-to-one correspondence, P j Let represent the j-th prototype vector, and also the prototype vector belonging to the j-th category. Represents the eigenvector of the second point and prototype vector P j The distance, which is obtained through the feature vector of the second point. and prototype vector P j The L2 norm is represented by the square of the L2 norm between the two values, where ||·||² represents the L2 norm. This represents the maximum distance between the second feature vector and each prototype vector in the prototype set, where k represents the identifier corresponding to the second feature vector when this distance is at its maximum value. That is, the maximum distance between the k-th second feature vector and each prototype vector in the prototype set.

[0165] After determining the probability that each second-point feature vector belongs to the unknown category, i.e., after determining the second probability, the preset category threshold λ is applied. u The category corresponding to each second feature vector is determined, which can be specifically represented as:

[0166]

[0167] When the second probability is determined to be greater than or equal to the preset category threshold, the category corresponding to the second feature vector is determined to be an unknown category;

[0168] When the second probability is determined to be less than a preset category threshold, since the first and second feature vectors correspond one-to-one, determining the category corresponding to the second feature vector is equivalent to determining the category corresponding to the first feature vector. In other words, based on the probability of any category corresponding to the first feature vector, when the probability of the first feature vector corresponding to a certain category j is maximized, that category j is the category corresponding to the first feature vector. That is, determining the first feature vector and a certain prototype vector P... j When the distance is minimized, the prototype vector P j The category to which it belongs, and each category corresponds to a prototype vector, then the prototype vector P j The category j to which it belongs is the category corresponding to the first feature vector. In other words, the category corresponding to the second feature vector, which is the same as the first feature vector, is category j.

[0169] The three-dimensional point cloud segmentation method provided by this invention maps all point features corresponding to the three-dimensional point cloud to a new feature space using a point feature mapping model. Then, according to a first classification criterion and a prototype set composed of prototype vectors of all known categories, the category to which the mapped point features of the three-dimensional point cloud belong is determined, thereby improving the discriminative power of the point features of the three-dimensional point cloud, correctly classifying all point features, and marking point features of unknown categories as unknown categories, thereby improving the resolution of whether point features in the three-dimensional point cloud belong to known or unknown categories.

[0170] Figure 4 This is a schematic diagram illustrating the implementation process of the three-dimensional point cloud segmentation method provided in this embodiment of the invention, as follows: Figure 4 As shown, the method includes:

[0171] The first step involves using the original 3D point cloud as input to jointly train the feature extraction model and the prototype constraint module. The original 3D point cloud is input into the feature extraction model, and the extracted point-by-point features are input into the prototype constraint module. The loss function of the prototype constraint is calculated and backpropagated to update the parameters of the skeleton network in the feature extraction model and the prototype vector in the prototype constraint module.

[0172] The second step involves fixing the parameters of the feature extraction model and jointly training the prototype constraint module and the feature adversarial module. The point features output by the feature extraction model are input into the feature adversarial module, where a generative adversarial network (GAN) is used to synthesize point features belonging to the unknown category. The point features mapped to the new feature space by the point feature mapping model are then input into the prototype constraint module, updating the prototype vector in the module and the parameters of the feature generation model (generator G), feature discrimination model (discriminator D), and point feature mapping model (adversarial mapper M) in the feature adversarial module. The feature generation model (generator G) and the feature discrimination model (discriminator D) constitute the generative adversarial network.

[0173] To ensure the stability of the Generative Adversarial Network (GAN), the parameters of the feature extraction model are fixed in the second step, resulting in fixed point features output by the model, while the prototype vector continues to update during training. To guarantee the synchronous updating of point features and prototype vectors, a point feature mapping model (adversarial mapper M) maps the point features output by the feature extraction model to a new feature space.

[0174] Figure 5 This is a schematic diagram of the structure of the three-dimensional point cloud segmentation device provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the device includes:

[0175] The acquisition module 501 is used to input the three-dimensional point cloud into the feature extraction model and obtain all the first point feature vectors corresponding to the three-dimensional point cloud as the first point feature sequence; the first point feature vector is used to characterize any point feature corresponding to the three-dimensional point cloud that belongs to a known category;

[0176] Mapping module 502 is used to determine the result of mapping the first point feature sequence to a new feature space based on the point feature mapping model, and use it as the second point feature sequence;

[0177] The determination module 503 is used to determine the category corresponding to the second feature vector according to the first classification criterion and the prototype set; the prototype set includes all prototype vectors belonging to known categories; the second feature vector is any vector in the second feature sequence.

[0178] The segmentation module 504 is used to determine the category corresponding to the first point feature vector based on the category corresponding to the second point feature vector, and to perform open set segmentation on the three-dimensional point cloud according to the category corresponding to the first point feature vector.

[0179] The point feature mapping model is obtained by training a generative adversarial network based on the point features corresponding to the sample 3D point cloud. The generative adversarial network is used to optimize the point feature distribution of the 3D point cloud by introducing point features belonging to unknown categories during the training process.

[0180] The three-dimensional point cloud segmentation apparatus provided in this embodiment of the invention can execute the technical solution of the three-dimensional point cloud segmentation method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the three-dimensional point cloud segmentation method. Please refer to the implementation principle and beneficial effects of the three-dimensional point cloud segmentation method. It will not be repeated here.

[0181] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided in the embodiments of the present invention, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a three-dimensional point cloud segmentation method, which includes:

[0182] The three-dimensional point cloud is input into the feature extraction model to obtain all the first point feature vectors corresponding to the three-dimensional point cloud, which are used as the first point feature sequence; the first point feature vector is used to characterize any point feature of the three-dimensional point cloud that belongs to a known category;

[0183] Based on the point feature mapping model, the result of mapping the first point feature sequence to the new feature space is determined and used as the second point feature sequence.

[0184] According to the first classification criterion and the prototype set, the category corresponding to the second feature vector is determined; the prototype set includes all prototype vectors belonging to the known categories; the second feature vector is any vector in the second feature sequence.

[0185] Based on the category corresponding to the second point feature vector, the category corresponding to the first point feature vector is determined, and the three-dimensional point cloud is segmented into an open set according to the category corresponding to the first point feature vector;

[0186] The point feature mapping model is obtained by training a generative adversarial network based on the point features corresponding to the sample 3D point cloud. The generative adversarial network is used to optimize the point feature distribution of the 3D point cloud by introducing point features belonging to unknown categories during the training process.

[0187] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0188] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the three-dimensional point cloud segmentation method provided by the above methods. The method includes: inputting the three-dimensional point cloud into a feature extraction model to obtain all first point feature vectors corresponding to the three-dimensional point cloud as a first point feature sequence; the first point feature vectors are used to characterize any point feature corresponding to the three-dimensional point cloud that belongs to a known category.

[0189] Based on the point feature mapping model, the result of mapping the first point feature sequence to the new feature space is determined and used as the second point feature sequence.

[0190] According to the first classification criterion and the prototype set, the category corresponding to the second feature vector is determined; the prototype set includes all prototype vectors belonging to the known categories; the second feature vector is any vector in the second feature sequence.

[0191] Based on the category corresponding to the second point feature vector, the category corresponding to the first point feature vector is determined, and the three-dimensional point cloud is segmented into an open set according to the category corresponding to the first point feature vector;

[0192] The point feature mapping model is obtained by training a generative adversarial network based on the point features corresponding to the sample 3D point cloud. The generative adversarial network is used to optimize the point feature distribution of the 3D point cloud by introducing point features belonging to unknown categories during the training process.

[0193] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for three-dimensional point cloud segmentation provided by the methods described above. The method includes: inputting a three-dimensional point cloud into a feature extraction model to obtain all first point feature vectors corresponding to the three-dimensional point cloud as a first point feature sequence; the first point feature vectors are used to characterize any point feature corresponding to the three-dimensional point cloud that belongs to a known category.

[0194] Based on the point feature mapping model, the result of mapping the first point feature sequence to the new feature space is determined and used as the second point feature sequence.

[0195] According to the first classification criterion and the prototype set, the category corresponding to the second feature vector is determined; the prototype set includes all prototype vectors belonging to the known categories; the second feature vector is any vector in the second feature sequence.

[0196] Based on the category corresponding to the second point feature vector, the category corresponding to the first point feature vector is determined, and the three-dimensional point cloud is segmented into an open set according to the category corresponding to the first point feature vector;

[0197] The point feature mapping model is obtained by training a generative adversarial network based on the point features corresponding to the sample 3D point cloud. The generative adversarial network is used to optimize the point feature distribution of the 3D point cloud by introducing point features belonging to unknown categories during the training process.

[0198] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0199] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for three-dimensional point cloud segmentation, characterized in that, include: The three-dimensional point cloud is input into the feature extraction model to obtain all the first point feature vectors corresponding to the three-dimensional point cloud, which are used as the first point feature sequence. The first feature vector is used to characterize any point feature belonging to a known category in the three-dimensional point cloud; Based on the point feature mapping model, the result of mapping the first point feature sequence to the new feature space is determined and used as the second point feature sequence. According to the first classification criterion and the prototype set, the category corresponding to the second feature vector is determined; the prototype set includes all prototype vectors belonging to the known categories; the second feature vector is any vector in the second feature sequence. Based on the category corresponding to the second point feature vector, the category corresponding to the first point feature vector is determined, and the three-dimensional point cloud is segmented into an open set according to the category corresponding to the first point feature vector; The point feature mapping model is obtained by training a generative adversarial network based on the point features corresponding to the sample 3D point cloud. The generative adversarial network is used to introduce point features belonging to unknown categories during the training process to optimize the point feature distribution of the 3D point cloud.

2. The method for three-dimensional point cloud segmentation according to claim 1, characterized in that, The generative adversarial network consists of a feature generation model and a feature discrimination model; the feature generation model is trained based on the prior distribution of point features belonging to unknown categories; the feature discrimination model is used to distinguish whether the input point features are true features or false features.

3. The method for three-dimensional point cloud segmentation according to claim 2, characterized in that, The point feature mapping model is obtained based on the point features corresponding to the sample 3D point cloud and trained using a generative adversarial network. The training method of the point feature mapping model includes: Based on the first noise set, a synthetic point feature vector corresponding to the first point feature vector is generated, and each of the synthetic point feature vectors belongs to an unknown category; Based on the synthetic point feature vector and the prior distribution of point features belonging to unknown categories, a first objective function of the feature generation model is determined; the first objective function is used to characterize the similarity between the synthetic point features generated by the feature generation model and the first point features. Based on the first point feature vector and the corresponding synthetic point feature vector, a second objective function of the feature discrimination model is determined; the second objective function is used to characterize the ability of the feature discrimination model to distinguish between the synthetic point feature and the first point feature. Based on the first loss function of the prototype constraint module and the prior distribution of the unknown category point features, the third objective function of the point feature mapping model is determined; the third objective function is used to characterize the difference between the point features and the prototype vector. The point feature mapping model is trained when the first objective function, the second objective function, and the third objective function meet preset conditions.

4. The method for three-dimensional point cloud segmentation according to claim 3, characterized in that, Before determining the first objective function of the feature generation model based on the synthesized point feature vector and the prior distribution of point features belonging to unknown categories, the process includes: The synthesized point feature vectors with a first probability less than a preset screening threshold are selected as target synthesized point feature vectors; the first probability is the probability that the synthesized point feature vector is determined to be a true feature. Based on the target synthetic point feature vector and the normalized result of the initialized prototype set, the distribution prior of the point features belonging to the unknown category is determined.

5. The method for three-dimensional point cloud segmentation according to claim 1, characterized in that, The result of determining the mapping of the first point feature sequence to the new feature space based on the point feature mapping model includes: Based on the nonlinear layer of the point feature mapping model, the equidimensional mapping result of the first point feature vector to the new feature space is determined and used as the second point feature vector. Based on all the second point feature vectors, determine the second point feature sequence.

6. The method for three-dimensional point cloud segmentation according to claim 5, characterized in that, Before determining the category corresponding to the second feature vector according to the first classification criterion and the prototype set, the following steps are included: Based on the feature fusion result of the second point feature vector belonging to the known category of the target in the second point feature sequence, the prototype vector belonging to the known category of the target is determined; the known category of the target is any one of all known categories to which the second point feature sequence belongs; The prototype set is updated based on the prototype vectors belonging to the known categories of the target.

7. The method for three-dimensional point cloud segmentation according to claim 6, characterized in that, The step of determining the category corresponding to the second feature vector according to the first classification criterion and the prototype set includes: Based on the distance between the second feature vector and the prototype vector of the known category in the new feature space, the probability that the second feature vector belongs to the unknown category is determined as the second probability; If the second probability is greater than or equal to the preset category threshold, then the category corresponding to the second feature vector is determined to be an unknown category; If the second probability is less than a preset category threshold, then based on the target prototype vector corresponding to the first feature vector and the category to which the target prototype vector belongs, the category corresponding to the second feature vector is determined to be the category to which the target prototype vector belongs. The second feature vector corresponds one-to-one with the first feature vector; the target prototype vector is the prototype vector corresponding to the target category when the probability value of the first feature vector belonging to the target category is the highest.

8. The method for three-dimensional point cloud segmentation according to claim 3, characterized in that, The training method for the feature extraction model includes: Obtain all first point feature vectors corresponding to the three-dimensional point cloud of the sample, and use them as the first point feature sequence; Based on the distance between the first feature vector and the first category, the probability that the first feature vector belongs to the first category is determined as the first probability; the first feature vector is any feature vector in the first feature sequence; the first category is any one of all categories corresponding to the initial prototype set. Based on the first probability, the first point feature vector, and the first category, the distance loss function, distribution loss function, and dimensionality loss function of the prototype constraint module are determined. Based on the weight summation method, the distance loss function, the distribution loss function, and the dimension loss function, the first loss function of the prototype constraint module is determined. If the first loss function converges, the training of the feature extraction model is completed.

9. A device for three-dimensional point cloud segmentation, characterized in that, include: The acquisition module is used to input the 3D point cloud into the feature extraction model and obtain all the first point feature vectors corresponding to the 3D point cloud as the first point feature sequence. The first feature vector is used to characterize any point feature belonging to a known category in the three-dimensional point cloud; The mapping module is used to determine the result of mapping the first point feature sequence to a new feature space based on the point feature mapping model, and use it as the second point feature sequence. The determination module is used to determine the category corresponding to the second feature vector according to the first classification criterion and the prototype set; the prototype set includes all prototype vectors belonging to known categories; the second feature vector is any vector in the second feature sequence. The segmentation module is used to determine the category corresponding to the first point feature vector based on the category corresponding to the second point feature vector, and to perform open set segmentation on the three-dimensional point cloud according to the category corresponding to the first point feature vector; The point feature mapping model is obtained by training a generative adversarial network based on the point features corresponding to the sample 3D point cloud. The generative adversarial network is used to introduce point features belonging to unknown categories during the training process to optimize the point feature distribution of the 3D point cloud.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for three-dimensional point cloud segmentation as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for three-dimensional point cloud segmentation as described in any one of claims 1 to 8.

Images