Point cloud domain self-adaption method and device based on geometric correlation prediction and medium

Abstract

The invention discloses a point cloud domain self-adaption method and device based on geometric correlation prediction and a medium. The method comprises the steps that a feature encoder is adopted to extract features of a source domain point cloud sample to obtain source domain features, and features of a target domain point cloud sample are extracted to obtain target domain features; training a supervised task head based on the source domain features, and training a self-supervised task head based on the target domain features; extracting features of a target domain point cloud sample by using the trained feature encoder, obtaining target domain features, processing the target domain features by using the trained supervised task head to obtain category probability distribution of the target domain point cloud sample, and obtaining entropy of the target domain point cloud sample; and taking the low-entropy target domain point cloud sample as a new source domain point cloud sample, taking the high-entropy target domain point cloud sample as a new target domain point cloud sample, carrying out secondary training, and processing the samples by adopting a supervised task head after the secondary training to obtain a classification result of the to-be-classified sample. According to the method disclosed by the invention, the prediction accuracy is improved.

Description

Technical Field

[0001] The invention relates to a point cloud domain adaptation method based on geometric association prediction, and belongs to the technical field of computer vision. Background Art

[0002] Point clouds have been widely used in autonomous driving, augmented reality, virtual reality, robotics, etc. However, due to factors such as data acquisition process, sensor perspective, and real-world noise, directly applying models trained on labeled point cloud datasets to unlabeled point cloud datasets often results in performance degradation due to domain differences.

[0003] The self-supervised domain adaptation method is the most widely used point cloud domain adaptation method. The source domain and the target domain share the same feature encoder. The source domain data is sent to the supervised task head for supervised task learning, and the target domain data is sent to the self-supervised task head for self-supervised task learning. Through this process, the feature alignment of the source domain and the target domain is completed.

[0004] However, traditional point cloud domain adaptation methods based on self-supervision generally have problems such as limited model expression ability, poor robustness, and neglect of distribution differences within the target domain, resulting in limited improvement in model performance after adaptation.

[0005] Therefore, it is necessary to conduct more in-depth research on point cloud domain adaptation methods to solve the above problems. Summary of the invention

[0006] In order to overcome the above problems, the inventors conducted in-depth research and designed a point cloud domain adaptation method based on geometric association prediction, which includes the following steps: S1. Use feature encoder to extract features of source domain point cloud samples to obtain source domain features, and extract features of target domain point cloud samples to obtain target domain features; S2, train the supervised task head based on the source domain features, and train the self-supervised task head based on the target domain features; S3, repeat S1-S2 until convergence, and obtain the trained feature encoder, supervised task head, and self-supervised task head; S4, using a feature encoder trained once to extract features of the target domain point cloud samples, obtaining target domain features, and using a supervised task head trained once to process the target domain features to obtain the category probability distribution of the target domain point cloud samples; S5. Obtaining the entropy of the target domain point cloud samples based on the category probability distribution of the target domain point cloud samples; S6. Divide the target domain point cloud samples into two groups according to the entropy level. The target domain point cloud samples in the low entropy group are used as new source domain point cloud samples, and the target domain point cloud samples in the high entropy group are used as new target domain point cloud samples. Repeat steps S1 and S2 until convergence to obtain the feature encoder and supervised task head after secondary training. S7. Use the feature encoder after secondary training to extract features from the samples to be classified to obtain sample features, and use the supervised task head after secondary training to process the samples to obtain classification results of the samples to be classified.

[0007] In a preferred embodiment, in S1, before extraction, it also includes: using the farthest point sampling method to downsample the source domain point cloud and the target domain point cloud samples to points.

[0008] In a preferred embodiment, in S2, the self-supervised task head is a geometric association prediction network, including multiple layers of convolutional layers, activation function layers, and dropout layers connected in sequence.

[0009] In a preferred embodiment, in S2, the training of the self-supervised task head comprises the following sub-steps: S21, use the self-supervised task head to process the target domain features and obtain the geometric correlation feature vector; S22, obtaining the covariance matrix of each point in the target domain point cloud sample, and selecting the geometric association eigenvector corresponding to the smallest eigenvalue in the covariance matrix of each point as the predicted value of the geometric association feature label; S23. Use the predicted values ​​of geometrically associated feature labels as supervision to train the self-supervised task head.

[0010] In a preferred embodiment, in S23, the loss function during training is set to cosine distance loss, which is expressed as:

[0011] in, represents the geometric association feature prediction loss, represents the number of samples in the target domain, represents the target domain, represents the point cloud samples selected from the target domain, represents a point in a point cloud sample, Represents the geometrically associated feature labels of points in the point cloud, Represents the predicted value of the geometrically associated feature label of a point in the point cloud, Table vector modulo operation.

[0012] In a preferred embodiment, in S3, the loss function during training is set to:

[0013] in, represents the overall loss, represents the supervised task loss, represents the geometric association feature prediction loss, represents the weight of the supervised task loss.

[0014] In a preferred embodiment, in S5, the entropy of the target domain point cloud sample obtained is expressed as:

[0015] in, represents the entropy of the point cloud sample, Represents the total number of categories of samples, c Indicates different categories, Represents the class probability distribution of samples.

[0016] In a preferred embodiment, in S6, the target domain samples are sorted according to the size of the entropy, and the samples are sorted from small to large. The samples of are regarded as new source domains, and the pseudo labels are confirmed according to the prediction results, and the remaining samples are regarded as new target domains.

[0017] The present invention also provides an electronic device, comprising: at least one processor; and A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the methods described above.

[0018] The present invention also provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any of the above-mentioned methods.

[0019] The beneficial effects of the present invention include: (1) To address the limited expressive power of self-supervised tasks in point cloud domain adaptation methods, a geometric association prediction task is proposed to enable the model to learn more comprehensive and robust features. (2) In order to address the problem that existing point cloud domain adaptation methods ignore the distribution differences within the target domain, an additional training phase specifically for the target domain is proposed to make the model further applicable to the target domain and improve the model prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1A schematic diagram of a point cloud domain adaptive method based on geometric association prediction according to a preferred embodiment of the present invention is shown; Figure 2 A schematic structural diagram of a point cloud domain adaptation method based on geometric association prediction according to a preferred embodiment of the present invention is shown. DETAILED DESCRIPTION

[0021] The present invention will be further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present invention will become more clear and distinct.

[0022] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.

[0023] According to a point cloud domain adaptation method based on geometric association prediction provided by the present invention, Figure 1 As shown, the following steps are included: S1. Use feature encoder to extract features of source domain point cloud samples to obtain source domain features, and extract features of target domain point cloud samples to obtain target domain features; S2, train the supervised task head based on the source domain features, and train the self-supervised task head based on the target domain features; S3, repeat S1-S2 until convergence, and obtain the trained feature encoder, supervised task head, and self-supervised task head; S4, using a feature encoder trained once to extract features of the target domain point cloud samples, obtaining target domain features, and using a supervised task head trained once to process the target domain features to obtain the category probability distribution of the target domain point cloud samples; S5. Obtaining the entropy of the target domain point cloud samples based on the category probability distribution of the target domain point cloud samples; S6. Divide the target domain point cloud samples into two groups according to the entropy level. The target domain point cloud samples in the low entropy group are used as new source domain point cloud samples, and the target domain point cloud samples in the high entropy group are used as new target domain point cloud samples. Repeat steps S1 and S2 until convergence to obtain the feature encoder and supervised task head after secondary training. S7. Use the feature encoder after secondary training to extract features from the samples to be classified to obtain sample features, and use the supervised task head after secondary training to process the samples to obtain classification results of the samples to be classified.

[0024] According to the present invention, similar to the traditional point cloud domain adaptation method, the source domain refers to the annotated point cloud data in the original environment or scene, and the target domain refers to the annotated point cloud data in the new environment or scene.

[0025] Preferably, in S1, before the extraction, it also includes: using the farthest point sampling method to downsample the source domain point cloud and the target domain point cloud samples to points in order to normalize the point cloud data to a fixed size so that the point cloud data can better fit the model.

[0026] Furthermore, in S1, features of the source domain point cloud and the target domain point cloud are extracted through a shared feature encoder.

[0027] In the present invention, any known point cloud feature extraction network can be used as a feature encoder to extract point cloud features. Preferably, a DGCNN network is used. The DGCNN network regards point cloud data as a dynamic graph, points as nodes in the graph, and edges are generated by the neighborhood relationship of each point. The point cloud features are extracted by graph convolution. DGCNN has shown excellent performance on multiple data sets and is a general and powerful point cloud encoder.

[0028] In S2, the supervised task head is trained for a supervised task, where the supervised task may be a classification task or a segmentation task, etc., preferably a classification task.

[0029] Preferably, in the classification task, the classification loss is set to the cross entropy loss, expressed as:

[0030] in, represents the classification loss, represents the number of samples in the source domain, represents the source domain, represents the source domain sample, Represents the total number of categories of samples, c Indicates different categories, and Represent the classification labels and predicted probability distributions of source domain samples respectively.

[0031] In the present invention, the specific structure of the supervised task head is not limited, and those skilled in the art can freely select it according to actual needs.

[0032] In S2, the self-supervised task head is a geometric association prediction network, including multiple layers of convolutional layers, activation function layers, and dropout layers connected in sequence.

[0033] Preferably, the multi-layer convolution layer is 4 layers of 1-dimensional convolution layers with a convolution kernel of 1.

[0034] More preferably, a normalization layer is set for each convolutional layer.

[0035] In S2, the training of the self-supervised task head includes the following sub-steps: S21, use the self-supervised task head to process the target domain features and obtain the geometric correlation feature vector; S22, obtaining the covariance matrix of each point in the target domain point cloud sample, and selecting the geometric association eigenvector corresponding to the smallest eigenvalue in the covariance matrix of each point as the predicted value of the geometric association feature label; S23. Use the predicted values ​​of geometrically associated feature labels as supervision to train the self-supervised task head.

[0036] Preferably, in S22, the kNN method is used to obtain the nearest neighbors of each point in the point cloud, and the nearest neighbors of each point in the point cloud are obtained according to the kNN method. Nearest neighbors compute the covariance matrix.

[0037] The kNN method (K-Nearest Neighbor) is a basic classification and regression method, which is widely used in machine learning. The specific process is not described in detail in the present invention.

[0038] Preferably, in the kNN method, Neighbors The value is 30.

[0039] Preferably, the calculating the covariance matrix comprises: S221. For each point in the point cloud and its Neighboring points , calculate this The mean of neighboring points , expressed as: ; in, Indicates the sequence number of the neighboring points, Indicate point No. neighboring points.

[0040] S222. Construct a covariance matrix, expressed as:

[0041] in, Indicate point The covariance matrix of Indicates transpose.

[0042] Calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and select the eigenvector corresponding to the minimum eigenvalue as the predicted value of the geometric association feature label.

[0043] According to the present invention, the predicted value of the geometrically associated feature label is obtained from the covariance matrix of the local point cluster, which is related to factors such as the position and angle of the points, and whether the two points are in the same plane, so that the model can learn more comprehensive and robust features.

[0044] In S23, the loss function during training is set to cosine distance loss, expressed as:

[0045] in, represents the geometric association feature prediction loss, represents the number of samples in the target domain, represents the target domain, represents the point cloud samples selected from the target domain, represents a point in a point cloud sample, Represents the geometrically associated feature labels of points in the point cloud, Represents the predicted value of the geometrically associated feature label of a point in the point cloud, Represents a vector modulo operation.

[0046] Preferably, in S3, the loss function during training is set to:

[0047] in, represents the overall loss, represents the supervised task loss, represents the geometric association feature prediction loss, represents the weight of the supervised task loss.

[0048] Preferably, The value is 0.67.

[0049] According to the present invention, steps S1-S3 complete the feature alignment of the source domain and the target domain. However, simply aligning the features of the source domain and the target domain is not enough. There are also distribution differences within the target domain, which causes the model to fail to achieve the best performance on the target domain.

[0050] In the present invention, feature alignment is completed in a more fine-grained manner through steps S4-S6, and an additional training phase specifically for the target domain is added after the feature alignment at the domain level, so that the model can learn the distribution differences within the target domain. After the training of S1-S3 is completed, the model has a certain generalization ability in the target domain. At this time, the target domain samples are sent to the model for prediction, and the samples are divided into two parts that are easier to divide and more difficult to divide according to the entropy of the predicted probability distribution. If the predicted category probability distribution is concentrated on a certain category, it means that the model has a high degree of confidence in the prediction result of the sample, and the sample is relatively regular and easy to be identified. At this time, the entropy of the sample is low and is used as a new source domain for subsequent training; if the predicted category probability distribution is scattered across various categories, it means that the model prediction confidence is low, the sample is irregular and not easy to be identified, and the entropy is relatively high at this time, and it is used as a new target domain for subsequent training.

[0051] In S5, the entropy of the target domain point cloud sample obtained is expressed as:

[0052] in, represents the entropy of the point cloud sample, Represents the total number of categories of samples, Represents the class probability distribution of samples.

[0053] In S6, the target domain samples are sorted according to the size of entropy, and the samples are sorted from small to large. The samples of are regarded as new source domains, and the pseudo labels are confirmed according to the prediction results, and the remaining samples are regarded as new target domains.

[0054] Preferably, Set to 0.8. Example

[0055] Example 1 The PointDA-10 dataset is used to conduct experiments on the domain adaptation task of point cloud classification model. The PointDA-10 dataset contains , , , , , There are 6 adaptive scenarios in total. The experiment includes the following steps: S1. Use feature encoder to extract features of source domain point cloud samples to obtain source domain features, and extract features of target domain point cloud samples to obtain target domain features; S2, train the supervised task head based on the source domain features, and train the self-supervised task head based on the target domain features; S3, repeat S1-S2 until convergence, and obtain the trained feature encoder, supervised task head, and self-supervised task head; S4, using a feature encoder trained once to extract features of the target domain point cloud samples, obtaining target domain features, and using a supervised task head trained once to process the target domain features to obtain the category probability distribution of the target domain point cloud samples; S5. Obtaining the entropy of the target domain point cloud samples based on the category probability distribution of the target domain point cloud samples; S6. Divide the target domain point cloud samples into two groups according to the entropy level. The target domain point cloud samples in the low entropy group are used as new source domain point cloud samples, and the target domain point cloud samples in the high entropy group are used as new target domain point cloud samples. Repeat steps S1 and S2 until convergence to obtain the feature encoder and supervised task head after secondary training. S7. Use the feature encoder after secondary training to extract features from the samples to be classified to obtain sample features, and use the supervised task head after secondary training to process the samples to obtain classification results of the samples to be classified.

[0056] In S1, before extraction, it also includes: using the farthest point sampling method to downsample the source domain point cloud and the target domain point cloud samples to Points are extracted from the source domain point cloud and the target domain point cloud features through a shared feature encoder, and the feature encoder adopts a DGCNN network.

[0057] In S2, the supervised task head is trained for the classification task. In the classification task, the classification loss is set to the cross entropy loss, which is expressed as:

[0058] The self-supervised task head is composed of 4 1-dimensional convolution layers with a convolution kernel of 1, each convolution layer is provided with a normalization layer, and also includes an activation function layer and a dropout layer.

[0059] In S2, the training of the self-supervised task head includes the following sub-steps: S21, use the self-supervised task head to process the target domain features and obtain the geometric correlation feature vector; S22, obtaining the covariance matrix of each point in the target domain point cloud sample, and selecting the geometric association eigenvector corresponding to the smallest eigenvalue in the covariance matrix of each point as the predicted value of the geometric association feature label; S23. Use the predicted values ​​of geometrically associated feature labels as supervision to train the self-supervised task head.

[0060] Among them, in S22, the kNN method is used to obtain the Neighbors, based on the point The nearest neighbor calculates the covariance matrix, Neighbors The value is 30.

[0061] The calculation of the covariance matrix includes: S221. For each point in the point cloud and its Neighboring points , calculate this The mean of neighboring points , expressed as: ; S222. Construct a covariance matrix, expressed as: In S23, the loss function during training is set to cosine distance loss, expressed as:

[0062] In S3, a total of 150 epochs are trained, and the loss function during training is set to:

[0063] in The value is 0.67.

[0064] In S5, the entropy of the target domain point cloud sample obtained is expressed as:

[0065] In S6, Set to 0.8.

[0066] Example 2 The same experiment as in Example 1 was performed, except that the GraspNetPC-10 dataset was used. The GraspNetPC-10 dataset contains , , , There are 4 adaptive scenarios in total; each set of training is trained for 120 epochs.

[0067] Comparative Example 1 The same experiment as Example 1 was carried out, except that the PointDAN method was used. For the specific process of PointDAN, please refer to the paper [Qin C, You H, Wang L, et al. Pointdan: A multi-scale 3d domain adaptation network for point cloud representation[J]. Advances in Neural Information Processing Systems, 2019, 32.].

[0068] Comparative Example 2 The same experiment as Example 1 was carried out, except that the DefRec method was used. For the specific process of DefRec, please refer to the paper [Achituve I, Maron H, Chechik G. Self-supervised learning for domain adaptation on point clouds[C] / / Proceedings of the IEEE / CVF winter conference on applications of computer vision. 2021: 123-133.].

[0069] Comparative Example 3 The same experiment as in Example 1 was performed, except that the PC-Adapter method was used. For the specific process of PC-Adapter, see the paper [Park J, Seo H, Yang E. PC-Adapter: Topology-Aware Adapter for Efficient Domain Adaption on Point Clouds with Rectified Pseudo-label [C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2023: 11530-11540.].

[0070] Comparative Example 4 The same experiment as Example 1 was performed, except that the GLRV method was used. For the specific process of GLRV, please refer to the paper [Fan H, Chang X, Zhang W, et al. Self-supervised global-local structuremodeling for point clouddomain adaptation with reliable voted pseudo labels[C] / / Proceedings of theIEEE / CVF Conference on Computer Vision and PatternRecognition. 2022: 6377-6386.].

[0071] Comparative Example 5 The same experiment as Example 1 was carried out, except that the GAST method was used. For the specific process of GAST, please refer to the paper [Zou L, Tang H, Chen K, et al. Geometry-aware self-training for unsupervised domain adaptation on object point clouds[C] / / Proceedings of theIEEE / CVF International Conference on Computer Vision. 2021: 6403-6412.].

[0072] Comparative Example 6 The same experiment as Example 1 was performed, except that the Implicit PCDA method was used. For the specific process of Implicit PCDA, please refer to the paper [Shen Y, Yang Y, Yan M, et al. Domain adaptation on point clouds via geometry-aware implicits [C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition. 2022: 7223-7232.].

[0073] Comparative Example 7 The same experiment as Example 1 was performed, except that the MLSP method was used. For the specific process of MLSP, please refer to the paper [Liang H, Fan H, Fan Z, et al. Point cloud domain adaptation via maskedlocal 3d structureprediction[C] / / European Conference on Computer Vision.Cham: Springer NatureSwitzerland, 2022: 156-172.].

[0074] Comparative Example 8 The same experiment as Example 2 was carried out, except that the PointDAN method was used. For the specific process of PointDAN, please refer to the paper [Qin C, You H, Wang L, et al. Pointdan: A multi-scale 3d domain adaptation network for point cloud representation[J]. Advances in Neural Information Processing Systems, 2019, 32.].

[0075] Comparative Example 9 The same experiment as Example 2 was carried out, except that the RS method was used. For the specific process of RS, please refer to the paper [Sauder J, Sievers B. Self-supervised deep learning on point clouds by reconstructing space [J]. Advances in Neural Information Processing Systems, 2019, 32.].

[0076] Comparative Example 10 The same experiment as Example 2 was carried out, except that the DefRec method was used. For the specific process of DefRec, please refer to the paper [Achituve I, Maron H, Chechik G. Self-supervised learning for domain adaptation on point clouds[C] / / Proceedings of the IEEE / CVF winter conference on applications of computer vision. 2021: 123-133.].

[0077] Comparative Example 11 The same experiment as Example 2 was carried out, except that the GAST method was used. For the specific process of GAST, please refer to the paper [Zou L, Tang H, Chen K, et al. Geometry-aware self-training for unsupervised domain adaptation on object point clouds[C] / / Proceedings of theIEEE / CVF International Conference on Computer Vision. 2021: 6403-6412.].

[0078] Comparative Example 12 The same experiment as Example 2 was performed, except that the Implicit PCDA method was used. For the specific process of Implicit PCDA, please refer to the paper [Shen Y, Yang Y, Yan M, et al. Domain adaptation on point clouds via geometry-aware implicits [C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition. 2022: 7223-7232.].

[0079] Comparative Example 13 The same experiment as in Example 2 was performed, except that the PC-Adapter method was used. For the specific process of PC-Adapter, see the paper [Park J, Seo H, Yang E. PC-Adapter: Topology-Aware Adapter for Efficient Domain Adaption on Point Clouds with Rectified Pseudo-label [C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2023: 11530-11540.].

[0080] Experimental Example 1 By comparing the results of Example 1 with those of Comparative Examples 1-7, the recognized metric standard is adopted to measure the model effect using the classification accuracy on each adaptive scenario and the average classification accuracy of all adaptive scenarios. The comparison results are shown in Table 1.

[0081] Table 1

[0082] As can be seen from Table 1, in the PointDA-10 dataset, the method in Example 1 has the highest average accuracy in the six adaptive scenarios compared to other methods, especially in , and The highest accuracy was achieved in all adaptive scenarios, proving the effectiveness of the method in Example 1.

[0083] Experimental Example 2 Comparing the results of Example 2 with Comparative Examples 8-13, the recognized metric standard is adopted, and the classification accuracy on each adaptive scene and the average classification accuracy of all adaptive scenes are used to measure the model effect. The comparison results are shown in Table 2.

[0084] Table 2

[0085] As can be seen from Table 2, in the GraspNetPC-10 dataset, the method in Example 2 has the highest average accuracy in all adaptive scenarios compared to other methods. and The highest accuracy is achieved in both adaptive scenarios. Combining Experimental Example 1 and Experimental Example 2, it can be seen that the method in the embodiment can achieve higher performance under different data sets.

[0086] Various embodiments of the method described above in the present invention can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0087] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0088] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0089] To provide interaction with a user, the methods and apparatus described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0090] The methods and apparatus described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0091] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0092] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document is not limited here.

Claims

1. A point cloud domain adaptation method based on geometric association prediction, characterized in that: The following steps are involved: S1. Use feature encoder to extract features of source domain point cloud samples to obtain source domain features, and extract features of target domain point cloud samples to obtain target domain features; S2, train the supervised task head based on the source domain features, and train the self-supervised task head based on the target domain features; S3, repeat S1-S2 until convergence, and obtain the trained feature encoder, supervised task head, and self-supervised task head; S4, using a feature encoder trained once to extract features of the target domain point cloud samples, obtaining target domain features, and using a supervised task head trained once to process the target domain features to obtain the category probability distribution of the target domain point cloud samples; S5. Obtaining the entropy of the target domain point cloud samples based on the category probability distribution of the target domain point cloud samples; S6. Divide the target domain point cloud samples into two groups according to the entropy level. The target domain point cloud samples in the low entropy group are used as new source domain point cloud samples, and the target domain point cloud samples in the high entropy group are used as new target domain point cloud samples. Repeat steps S1 and S2 until convergence to obtain the feature encoder and supervised task head after secondary training. S7. Use the feature encoder after secondary training to extract features from the samples to be classified to obtain sample features, and use the supervised task head after secondary training to process the samples to obtain classification results of the samples to be classified.

2. The point cloud domain adaptation method based on geometric association prediction according to claim 1, characterized in that: In S1, before extraction, it also includes: using the farthest point sampling method to downsample the source domain point cloud and the target domain point cloud samples to points.

3. The point cloud domain adaptation method based on geometric association prediction according to claim 1, characterized in that: In S2, the self-supervised task head is a geometric association prediction network, including multiple layers of convolutional layers, activation function layers, and dropout layers connected in sequence.

4. The point cloud domain adaptation method based on geometric association prediction according to claim 1, characterized in that: In S2, the training of the self-supervised task head includes the following sub-steps: S21, use the self-supervised task head to process the target domain features and obtain the geometric correlation feature vector; S22, obtaining the covariance matrix of each point in the target domain point cloud sample, and selecting the geometric association eigenvector corresponding to the smallest eigenvalue in the covariance matrix of each point as the predicted value of the geometric association feature label; S23. Use the predicted values ​​of geometrically associated feature labels as supervision to train the self-supervised task head.

5. The point cloud domain adaptation method based on geometric association prediction according to claim 4, characterized in that: In S23, the loss function during training is set to cosine distance loss, expressed as: ， in, represents the geometric association feature prediction loss, represents the number of samples in the target domain, represents the target domain, represents the point cloud samples selected from the target domain, represents a point in a point cloud sample, Represents the geometrically associated feature labels of points in the point cloud, Represents the predicted value of the geometrically associated feature label of a point in the point cloud, Represents a vector modulo operation.

6. The point cloud domain adaptation method based on geometric association prediction according to claim 1, characterized in that: In S3, the loss function during training is set to: ， in, represents the overall loss, represents the supervised task loss, represents the geometric association feature prediction loss, represents the weight of the supervised task loss.

7. The point cloud domain adaptation method based on geometric association prediction according to claim 1, characterized in that: In S5, the entropy of the target domain point cloud sample obtained is expressed as: ， in, represents the entropy of the point cloud sample, Represents the total number of categories of samples, c Indicates different categories, Represents the class probability distribution of samples.

8. The point cloud domain adaptation method based on geometric association prediction according to claim 1, characterized in that: In S6, the target domain samples are sorted according to the size of entropy, and the samples are sorted from small to large. The samples of are regarded as new source domains, and the pseudo labels are confirmed according to the prediction results, and the remaining samples are regarded as new target domains.

9. An electronic device, comprising: at least one processor; as well as A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the method of any one of claims 1-8.

10. A computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

Images