Small sample point cloud semantic segmentation method and device, computer equipment and storage medium

By aligning and distribution correction of the supporting set and query set features in the semantic segmentation method of small sample point cloud, the problem of distribution deviation in point cloud segmentation is solved, which improves segmentation accuracy and reduces inference time.

CN119579881AActive Publication Date: 2025-03-07SHENZHEN INST OF ADVANCED TECH CHINESE ACAD OF SCI
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202311160173.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-09-07
Publication Date
2025-03-07
Estimated Expiration
2043-09-07

AI Technical Summary

Technical Problem

When dealing with complex point cloud structures, the existing small sample point cloud semantic segmentation method has inter-class distribution deviations and intra-class distribution deviations, resulting in low segmentation accuracy and long inference time.

Method used

By obtaining features that support set point cloud and query set point cloud, compute the distance between features to align them, adjust the feature distribution information of prototype and query set features, thereby reducing inter-class and intra-class distribution bias.

Benefits of technology

It effectively reduces point cloud segmentation error, improves the accuracy of small sample point cloud semantic segmentation, and reduces the inference time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119579881A_ABST
    Figure CN119579881A_ABST
Patent Text Reader

Abstract

The invention relates to a small sample point cloud semantic segmentation method and device, computer equipment and a storage medium. The method comprises the steps that support set point clouds and query set point clouds of corresponding masks are acquired, feature extraction is carried out on the support set point clouds and the query set point clouds, and support set features and query set features are obtained; according to the distance between the support set features and the query set features, aligning the support set features according to the distance to obtain the aligned support set features; obtaining a corresponding prototype according to the aligned mask supporting the set features, splicing the prototype with the query set features to obtain expanded query set features, extracting point cloud feature distribution information from the expanded query set features, and correcting the point cloud feature distribution information to obtain a point cloud feature distribution information set; and segmenting a corrected prototype and a corrected query set feature from the corrected point cloud feature distribution information. The method can reduce the reasoning time, and greatly reduces the point cloud segmentation error.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of computer vision technology, and in particular, relates to a method, apparatus, computer device and storage medium for semantic segmentation of a small sample point cloud. Background Art

[0002] The semantic segmentation method of small sample point cloud means that given a small number of labeled new category samples, the model can semantically segment the point cloud and obtain the point cloud belonging to the new category. In order to extract effective point cloud features, researchers are committed to improving the accuracy of prototypes. Multiple prototypes are redundant and take a lot of time in the reasoning process. Due to the complex structure of point clouds, there are usually inter-class distribution deviations and intra-class distribution deviations between the ideal feature distribution and the actual feature distribution. Specifically, due to the distribution shift caused by the difference between the support data and the query data, there is an inter-class distribution deviation, and due to the different local structures of the classes, such as the seat and legs on a chair, the features of the query data may be scattered, which means that the query features are not closely around the relevant prototypes, resulting in intra-class distribution deviations. Due to the complex feature distribution of point clouds, how to reduce these deviations is a challenge.

[0003] In the prior art, Snell et al. [J.Snell, K.Swersky, R.Zemel, Prototypical networks for few-shot learning, in: Advances in Neural Information Processing Systems (NeurIPS), 2017: 4080-4090.] proposed to use the average value of class features as the prototype and segment the point cloud by the distance between the point feature and the prototype. However, this method does not take into account the complexity of the structure of the point cloud itself, and the segmentation accuracy is not satisfactory. Zhao et al. [N.Zhao, T.-S.Chua, GHLee, Few-shot 3D point cloud semantic segmentation, in: IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 8869-8878.] designed a point cloud semantic segmentation method based on multiple prototypes, which uses multiple prototypes to describe complex point cloud distribution, but its reasoning time is relatively long. Summary of the invention

[0004] The present application provides a small sample point cloud semantic segmentation method, apparatus, computer equipment and storage medium, aiming to solve at least one of the above-mentioned technical problems in the prior art to a certain extent.

[0005] In order to solve the above problems, this application provides the following technical solutions:

[0006] A small sample point cloud semantic segmentation method, comprising:

[0007] Obtaining a support set point cloud and a query set point cloud corresponding to the mask, performing feature extraction on the support set point cloud and the query set point cloud to obtain support set features and query set features;

[0008] Calculating the distance between the support set features and the query set features, aligning the support set features according to the distance, and obtaining aligned support set features;

[0009] A corresponding prototype is obtained according to the mask of the aligned support set features, the prototype is spliced ​​with the query set features to obtain an expanded query set feature, point cloud feature distribution information is extracted from the expanded query set features, and the point cloud feature distribution information is corrected to obtain corrected point cloud feature distribution information, and the corrected prototype and the corrected query set features are segmented from the corrected point cloud feature distribution information.

[0010] The technical solution adopted by the embodiment of the present application also includes: the distance between the support set feature and the query set feature is calculated specifically as follows:

[0011]

[0012] In the above formula, F s represents the support set feature, F q represents the query set feature, i represents the support set feature F s The i-th feature in , j represents the query set feature F q The jth feature in d i,j Denotes the support set feature F s The i-th feature in the query set F q The distance of the jth feature in .

[0013] The technical solution adopted by the embodiment of the present application also includes: aligning the support set features according to the distance, and obtaining the aligned support set features specifically as follows:

[0014] For the support set feature F s With query set feature F q The distance is inverted and normalized to obtain the weight weight:

[0015] weight i,j =softmax(-d i,j )

[0016] According to the weight, the feature that tends to be in the query set point cloud is obtained, and the feature is fused with the support set feature to obtain the aligned support set feature:

[0017]

[0018] The technical solution adopted by the embodiment of the present application also includes: extracting point cloud feature distribution information from the expanded query set features, and correcting the point cloud feature distribution information, and obtaining the corrected point cloud feature distribution information is specifically:

[0019] The point cloud feature distribution information F is extracted from the expanded query set features using a shared multi-layer perceptron e :

[0020] F e =MLP(Con(F q ,P))

[0021] In the above formula, Con represents the concatenation operation;

[0022] The point cloud feature distribution information F e Multiply it with the transformed matrix to get the relationship matrix, and use the relationship matrix to calculate the point cloud feature distribution information F e Perform correction to obtain the corrected point cloud feature distribution information

[0023]

[0024] The technical solution adopted by the embodiment of the present application also includes: the segmentation of the corrected prototype and the corrected query set features from the corrected point cloud feature distribution information is specifically:

[0025]

[0026]

[0027] In the above formula, and They represent the corrected prototype after segmentation and the corrected query set features respectively.

[0028] Another technical solution adopted by the embodiment of the present application is: a small sample point cloud semantic segmentation device, comprising:

[0029] Feature extraction module: used to obtain the support set point cloud and the query set point cloud corresponding to the mask, perform feature extraction on the support set point cloud and the query set point cloud, and obtain support set features and query set features;

[0030] Feature alignment module: used to calculate the distance between the support set features and the query set features, align the support set features according to the distance, and obtain aligned support set features;

[0031] Distribution correction module: used to obtain the corresponding prototype according to the mask of the aligned support set features, splice the prototype with the query set features to obtain the expanded query set features, extract the point cloud feature distribution information from the expanded query set features, and correct the point cloud feature distribution information to obtain the corrected point cloud feature distribution information, and segment the corrected prototype and the corrected query set features from the corrected point cloud feature distribution information.

[0032] The technical solution adopted by the embodiment of the present application also includes: the feature alignment module calculates the distance between the support set feature and the query set feature, and aligns the support set feature according to the distance specifically as follows:

[0033]

[0034] In the above formula, F s represents the support set feature, F q represents the query set feature, i represents the support set feature F s The i-th feature in , j represents the query set feature F q The jth feature in d i,j Denotes the support set feature F s The i-th feature in the query set F q The distance of the jth feature in;

[0035] For the support set feature F s With query set feature F q The distance is inverted and normalized to obtain the weight weight:

[0036] weight i,j =softmax(-d i,j )

[0037] According to the weight, the feature that tends to be in the query set point cloud is obtained, and the feature is fused with the support set feature to obtain the aligned support set feature:

[0038]

[0039] Among them, α is a hyperparameter used to maintain the original information of the feature.

[0040] The technical solution adopted by the embodiment of the present application also includes: the distribution correction module extracts point cloud feature distribution information from the expanded query set features, and corrects the point cloud feature distribution information to obtain corrected point cloud feature distribution information, and the corrected prototype and the corrected query set features are segmented from the corrected point cloud feature distribution information. Specifically:

[0041] The point cloud feature distribution information F is extracted from the expanded query set features using a shared multi-layer perceptron e :

[0042] F e =MLP(Con(F q ,P))

[0043] In the above formula, Con represents the concatenation operation;

[0044] The point cloud feature distribution information F e Multiply it with the transformed matrix to get the relationship matrix, and use the relationship matrix to calculate the point cloud feature distribution information F e Perform correction to obtain the corrected point cloud feature distribution information

[0045]

[0046] Segment the corrected prototype and the corrected query set features from the corrected point cloud feature distribution information:

[0047]

[0048]

[0049] In the above formula, and Respectively represent the corrected prototype of the segmentation and the corrected query set features, and It represents the final prototype and query set features, and β is a hyperparameter used to maintain the original information of the features.

[0050] Another technical solution adopted by the embodiment of the present application is: a computer device, the computer device includes a processor and a memory coupled to the processor, wherein:

[0051] The memory stores program instructions for implementing the small sample point cloud semantic segmentation method;

[0052] The processor is used to execute the program instructions stored in the memory to control a small sample point cloud semantic segmentation method.

[0053] Another technical solution adopted by the embodiment of the present application is: a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute the small sample point cloud semantic segmentation method.

[0054] Compared with the prior art, the beneficial effects produced by the embodiments of the present application are as follows: the small sample point cloud semantic segmentation method, device, computer equipment and storage medium of the embodiments of the present application perform feature extraction on the support set point cloud and the query set point cloud through a feature extractor to obtain support set features and query set features, and use a feature alignment module to align the support set features and the query set features, thereby reducing the inter-class distribution deviation caused by the difference between the support set point cloud and the query set point cloud, and use a distribution correction module to adjust the feature distribution information of the prototype and query set features, thereby reducing the intra-class distribution deviation caused by the difference in feature distribution between the prototype and the query set features. While reducing the inference time, the point cloud segmentation error is greatly reduced, and the accuracy of the semantic segmentation of small sample point clouds is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a flowchart of a small sample point cloud semantic segmentation method according to an embodiment of the present application;

[0056] Figure 2 It is a schematic diagram of the framework of a small sample point cloud semantic segmentation system according to an embodiment of the present application;

[0057] Figure 3 This is a schematic diagram of a feature alignment operation of a feature alignment module in an embodiment of the present application;

[0058] Figure 4 is a characteristic correction schematic diagram of the distribution correction module in an embodiment of the present application;

[0059] Figure 5 This is a schematic diagram of the structure of a small sample point cloud semantic segmentation device according to an embodiment of the present application;

[0060] Figure 6 A schematic diagram of the structure of a computer device according to an embodiment of the present application;

[0061] Figure 7 A schematic diagram of the structure of a storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0063] The terms "first", "second", "third" in this application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In the embodiments of this application, all directional indications (such as up, down, left, right, front, back...) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or computer device that includes a series of steps or units is not limited to the steps or units listed, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or computer devices.

[0064] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0065] See also Figure 1 and Figure 2 , Figure 1 is a flowchart of a small sample point cloud semantic segmentation method according to an embodiment of the present application, Figure 2 Schematic diagram of the framework of the small sample point cloud semantic segmentation system of the embodiment of the present application. Specifically, the small sample point cloud semantic segmentation method of the embodiment of the present application includes the following steps:

[0066] S100: Obtain a support set point cloud and a query set point cloud corresponding to the mask, and perform feature extraction on the support set point cloud and the query set point cloud through a feature extractor to obtain support set features and query set features;

[0067] S110: calculating the distance between the support set features and the query set features through a feature alignment module, and aligning the support set features according to the distance to obtain aligned support set features;

[0068] In this step, if Figure 3 FIG. 1 is a schematic diagram of a feature alignment operation of a feature alignment module in an embodiment of the present application. Specifically, the feature alignment algorithm of the feature alignment module includes:

[0069] Step 1: Calculate the support set feature F s With query set feature F q Distance:

[0070]

[0071] In the above formula, F s represents the support set feature, F q represents the query set feature, i represents the support set feature F s The i-th feature in , j represents the query set feature F q The jth feature in d i,j Denotes the support set feature F s The i-th feature in the query set F q The distance of the jth feature in .

[0072] Step 2: For the support set feature F s With query set feature F q The distance is inverted and normalized to obtain the weight weight:

[0073] weight i,j =soft max(-d i,j ) (2)

[0074] The third step is to obtain the features that tend to be in the query set point cloud according to the weight, and fuse the features with the support set features to obtain the aligned support set features:

[0075]

[0076] Among them, α is a hyperparameter used to maintain the original information of the feature.

[0077] It can be understood that the embodiment of the present application uses a feature alignment module to align support set features and query set features in the first stage, thereby reducing the inter-class distribution deviation caused by the difference between the support set point cloud and the query set point cloud.

[0078] S120: acquiring a corresponding prototype according to the mask of the aligned support set feature, and splicing the prototype with the query set feature using a distribution correction module to obtain an expanded query set feature, extracting point cloud feature distribution information from the expanded query set feature, and correcting the point cloud feature distribution information to obtain corrected point cloud feature distribution information;

[0079] In this step, please also refer to Figure 4 , is a schematic diagram of feature correction of the distribution correction module in the embodiment of the present application. Specifically, the feature correction algorithm of the distribution correction module includes:

[0080] S121: Combine the prototype P with the query set feature F q Perform splicing to obtain the expanded query set features;

[0081] S122: Use a shared multi-layer perceptron (MLP) to extract point cloud feature distribution information F from the expanded query set features e :

[0082] F e =MLP(Con(F q ,P)) (4)

[0083] In formula (4), Con represents the concatenation operation.

[0084] S123: Point cloud feature distribution information F e Multiply it with the transformed matrix to get the relationship matrix, and use the relationship matrix to distribute the point cloud feature information F e Perform correction to obtain the corrected point cloud feature distribution information

[0085]

[0086] S130: Segmenting a corrected prototype and a corrected query set feature from the corrected point cloud feature distribution information through a distribution correction module;

[0087] In this step, in order to ensure the original feature information, the distribution correction module adds the original feature information to the corrected prototype and query set features during segmentation. The segmentation formula is as follows:

[0088]

[0089]

[0090] In the above formula, and Respectively represent the corrected prototype of the segmentation and the corrected query set features, and It represents the final prototype and query set features, and β is a hyperparameter used to maintain the original information of the features.

[0091] It can be understood that the embodiment of the present application uses the distribution correction module to adjust the feature distribution information in the second stage, which can reduce the intra-class deviation and obtain a better segmentation effect.

[0092] Based on the above, the small sample point cloud semantic segmentation method of the embodiment of the present application uses a feature extractor to extract features from the support set point cloud and the query set point cloud to obtain support set features and query set features, and uses a feature alignment module to align the support set features and the query set features, thereby reducing the inter-class distribution deviation caused by the difference between the support set point cloud and the query set point cloud, and uses a distribution correction module to adjust the feature distribution information of the prototype and query set features, thereby reducing the intra-class distribution deviation caused by the difference in feature distribution between the prototype and the query set features. While reducing the reasoning time, the point cloud segmentation error is greatly reduced, and the accuracy of the semantic segmentation of small sample point clouds is improved.

[0093] See also Figure 5 , is a schematic diagram of the structure of a small sample point cloud semantic segmentation device according to an embodiment of the present application. The small sample point cloud semantic segmentation device 40 according to an embodiment of the present application includes:

[0094] Feature extraction module 41: used to obtain a support set point cloud and a query set point cloud corresponding to a mask, perform feature extraction on the support set point cloud and the query set point cloud, and obtain support set features and query set features;

[0095] Feature alignment module 42: used to calculate the distance between the support set feature and the query set feature, align the support set features according to the distance, and obtain aligned support set features;

[0096] Distribution correction module 43: used to obtain the corresponding prototype according to the mask of the aligned support set features, splice the prototype with the query set features to obtain the expanded query set features, extract the point cloud feature distribution information from the expanded query set features, and correct the point cloud feature distribution information to obtain the corrected point cloud feature distribution information, and segment the corrected prototype and the corrected query set features from the corrected point cloud feature distribution information.

[0097] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0098] The device provided in the embodiment of the present application can be applied in the aforementioned method embodiment. For details, please refer to the description of the aforementioned method embodiment, which will not be repeated here.

[0099] See also Figure 6 , is a schematic diagram of the computer device structure of an embodiment of the present application. The computer device 50 includes:

[0100] A memory 51 storing executable program instructions;

[0101] A processor 52 connected to the memory 51;

[0102] The processor 52 is used to call the executable program instructions stored in the memory 51 and execute the following steps: obtain the support set point cloud and the query set point cloud of the corresponding mask, perform feature extraction on the support set point cloud and the query set point cloud to obtain the support set feature and the query set feature; calculate the distance between the support set feature and the query set feature, align the support set feature according to the distance, and obtain the aligned support set feature; obtain the corresponding prototype according to the mask of the aligned support set feature, splice the prototype with the query set feature to obtain the expanded query set feature, extract point cloud feature distribution information from the expanded query set feature, and correct the point cloud feature distribution information to obtain the corrected point cloud feature distribution information, and segment the corrected prototype and the corrected query set feature from the corrected point cloud feature distribution information.

[0103] The processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip having signal processing capabilities. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0104] See also Figure 7, is a schematic diagram of the structure of the storage medium of the embodiment of the present application. The storage medium of the embodiment of the present application stores program instructions 61 capable of implementing the following steps: obtaining a support set point cloud and a query set point cloud of a corresponding mask, extracting features from the support set point cloud and the query set point cloud, and obtaining support set features and query set features; calculating the distance between the support set features and the query set features, aligning the support set features according to the distance, and obtaining aligned support set features; obtaining the corresponding prototype according to the mask of the aligned support set features, splicing the prototype with the query set features, and obtaining an extended query set feature, extracting point cloud feature distribution information from the extended query set features, and correcting the point cloud feature distribution information to obtain corrected point cloud feature distribution information, and segmenting the corrected prototype and the corrected query set feature from the corrected point cloud feature distribution information. The program instructions 61 can be stored in the above storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network computer device, etc.) or a processor to perform all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program instructions, or terminal computer devices such as computers, servers, mobile phones, tablets, etc. Among them, the server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0105] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0106] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the description and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A small sample point cloud semantic segmentation method, characterized in that: include: Obtaining a support set point cloud and a query set point cloud corresponding to the mask, performing feature extraction on the support set point cloud and the query set point cloud to obtain support set features and query set features; Calculating the distance between the support set features and the query set features, aligning the support set features according to the distance, and obtaining aligned support set features; A corresponding prototype is obtained according to the mask of the aligned support set features, the prototype is spliced ​​with the query set features to obtain an expanded query set feature, point cloud feature distribution information is extracted from the expanded query set features, and the point cloud feature distribution information is corrected to obtain corrected point cloud feature distribution information, and the corrected prototype and the corrected query set features are segmented from the corrected point cloud feature distribution information.

2. The small sample point cloud semantic segmentation method according to claim 1, characterized in that: The calculation of the distance between the support set feature and the query set feature is specifically as follows: In the above formula, F s represents the support set feature, F q represents the query set feature, i represents the support set feature F s The i-th feature in , j represents the query set feature F q The jth feature in d i,j Denotes the support set feature F s The i-th feature in the query set F q The distance of the jth feature in .

3. The small sample point cloud semantic segmentation method according to claim 2, characterized in that: The support set features are aligned according to the distance to obtain the aligned support set features: For the support set feature F s With query set feature F q The distance is inverted and normalized to obtain the weight weight: weight i,j =softmax(-d i,j ) According to the weight, the feature that tends to be in the query set point cloud is obtained, and the feature is fused with the support set feature to obtain the aligned support set feature: Among them, α is a hyperparameter used to maintain the original information of the feature.

4. The small sample point cloud semantic segmentation method according to claim 3, characterized in that: The step of extracting point cloud feature distribution information from the expanded query set features and correcting the point cloud feature distribution information to obtain the corrected point cloud feature distribution information is specifically: The point cloud feature distribution information F is extracted from the expanded query set features using a shared multi-layer perceptron e : F e =MLP(Con(F q ,P)) In the above formula, Con represents the concatenation operation; The point cloud feature distribution information F e Multiply it with the transformed matrix to get the relationship matrix, and use the relationship matrix to calculate the point cloud feature distribution information F e Perform correction to obtain the corrected point cloud feature distribution information 5. The small sample point cloud semantic segmentation method according to claim 4, characterized in that: The step of segmenting the corrected prototype and the corrected query set features from the corrected point cloud feature distribution information is as follows: In the above formula, and Respectively represent the corrected prototype of the segmentation and the corrected query set features, and represents the final prototype and query set features, and β is a hyperparameter used to maintain the original information of the features.

6. A small sample point cloud semantic segmentation device, characterized in that: include: Feature extraction module: used to obtain the support set point cloud and the query set point cloud corresponding to the mask, perform feature extraction on the support set point cloud and the query set point cloud, and obtain support set features and query set features; Feature alignment module: used to calculate the distance between the support set features and the query set features, align the support set features according to the distance, and obtain aligned support set features; Distribution correction module: used to obtain the corresponding prototype according to the mask of the aligned support set features, splice the prototype with the query set features to obtain the expanded query set features, extract the point cloud feature distribution information from the expanded query set features, and correct the point cloud feature distribution information to obtain the corrected point cloud feature distribution information, and segment the corrected prototype and the corrected query set features from the corrected point cloud feature distribution information.

7. The small sample point cloud semantic segmentation device according to claim 6, characterized in that: The feature alignment module calculates the distance between the support set feature and the query set feature, and aligns the support set features according to the distance as follows: In the above formula, F s represents the support set feature, F q represents the query set feature, i represents the support set feature F s The i-th feature in , j represents the query set feature F q The jth feature in d i,j Denotes the support set feature F s The i-th feature in the query set F q The distance of the jth feature in; For the support set feature F s With query set feature F q The distance is inverted and normalized to obtain the weight weight: weight i,j =softmax(-d i,j ) According to the weight, the feature that tends to be in the query set point cloud is obtained, and the feature is fused with the support set feature to obtain the aligned support set feature:

8. The small sample point cloud semantic segmentation device according to claim 7, characterized in that: The distribution correction module extracts point cloud feature distribution information from the expanded query set features, and corrects the point cloud feature distribution information to obtain corrected point cloud feature distribution information, and segments the corrected prototype and the corrected query set features from the corrected point cloud feature distribution information as follows: The point cloud feature distribution information F is extracted from the expanded query set features using a shared multi-layer perceptron e : F e =MLP(Con(F q ,P)) In the above formula, Con represents the concatenation operation; The point cloud feature distribution information F e Multiply it with the transformed matrix to get the relationship matrix, and use the relationship matrix to calculate the point cloud feature distribution information F e Perform correction to obtain the corrected point cloud feature distribution information Segment the corrected prototype and the corrected query set features from the corrected point cloud feature distribution information: In the above formula, and They represent the corrected prototype after segmentation and the corrected query set features respectively.

9. A computer device, characterized in that: The computer device includes a processor and a memory coupled to the processor, wherein: The memory stores program instructions for implementing the small sample point cloud semantic segmentation method according to any one of claims 1 to 5; The processor is used to execute the program instructions stored in the memory to control a small sample point cloud semantic segmentation method.

10. A storage medium, characterized in that: The device stores program instructions executable by a processor, wherein the program instructions are used to execute the small sample point cloud semantic segmentation method according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Point cloud semantic segmentation method and device, electronic equipment and storage medium

    CN114782684A

  • Point cloud information processing device, point cloud information processing method, and point cloud information processing program

    US20220319023A1

  • Point cloud segmentation and virtual environment generation method and apparatus based on pointnet network

    WO2022252274A1