A point cloud domain adaptive method, device and medium based on geometric correlation prediction

Abstract

The application discloses a point cloud domain adaptive method and device based on geometric correlation prediction and a medium, and the method comprises the following steps: a feature encoder is used to extract the features of source domain point cloud samples to obtain source domain features, and the features of target domain point cloud samples are extracted to obtain target domain features; a supervised task head is trained based on the source domain features, and a self-supervised task head is trained based on the target domain features; the trained feature encoder is used to extract the features of the target domain point cloud samples to obtain the target domain features, the trained supervised task head is used to process the target domain features to obtain the category probability distribution of the target domain point cloud samples, and the entropy of the target domain point cloud samples is obtained; the target domain point cloud samples with low entropy are used as new source domain point cloud samples, the target domain point cloud samples with high entropy are used as new target domain point cloud samples, secondary training is performed, and the secondary trained supervised task head is used to process the samples to obtain the classification result of the samples to be classified. The method disclosed by the application improves the prediction accuracy.

Description

Technical Field

[0001] This invention relates to a point cloud domain adaptive method based on geometric correlation prediction, belonging to the field of computer vision technology. Background Technology

[0002] Point clouds have important applications in fields such as autonomous driving, augmented reality, virtual reality, and robotics. However, due to factors such as data acquisition process, sensor perspective, and real-world noise, directly applying a model trained on a labeled point cloud dataset to an unlabeled point cloud dataset often leads to performance degradation due to domain differences.

[0003] Self-supervised domain adaptation is the most widely used point cloud domain adaptation method. It shares the same feature encoder between the source and target domains. The source domain data is fed into the supervised task head for supervised learning, and the target domain data is fed into the self-supervised task head for self-supervised learning. Through this process, feature alignment between the source and target domains is achieved.

[0004] However, traditional self-supervised point cloud domain adaptation methods generally suffer from problems such as limited model expressive power, poor robustness, and neglect of distribution differences within the target domain, resulting in limited performance improvement after adaptation.

[0005] Therefore, it is necessary to conduct more in-depth research on point cloud domain adaptive methods to solve the above problems. Summary of the Invention

[0006] To overcome the above problems, the inventors conducted in-depth research and designed a point cloud domain adaptive method based on geometric correlation prediction, including the following steps:

[0007] S1. Use a feature encoder to extract features from source domain point cloud samples to obtain source domain features, and extract features from target domain point cloud samples to obtain target domain features.

[0008] S2. Train the supervised task head based on source domain features, and train the self-supervised task head based on target domain features;

[0009] S3. Repeat S1~S2 until convergence, to obtain the feature encoder, supervised task head and self-supervised task head after one training.

[0010] S4. Use a feature encoder trained once to extract features from point cloud samples in the target domain, obtain target domain features, and use a supervised task head trained once to process the target domain features to obtain the class probability distribution of point cloud samples in the target domain.

[0011] S5. Obtain the entropy of the target domain point cloud samples based on the category probability distribution of the target domain point cloud samples;

[0012] S6. Divide the target domain point cloud samples into two groups according to the entropy level. Use the target domain point cloud samples of the low-entropy group as the new source domain point cloud samples, and use the target domain point cloud samples of the high-entropy group as the new target domain point cloud samples. Repeat steps S1 and S2 until convergence, and obtain the feature encoder and supervised task head after secondary training.

[0013] S7. The feature encoder trained twice is used to extract features from the samples to be classified to obtain sample features. The supervised task head trained twice is used to process the samples to obtain the classification results of the samples to be classified.

[0014] In a preferred embodiment, S1 further includes, before extraction: downsampling the source domain point cloud and target domain point cloud samples to a minimum using the farthest point sampling method. One point.

[0015] In a preferred embodiment, in S2, the self-supervised task head is a geometric association prediction network, which includes multiple convolutional layers, activation function layers, and dropout layers connected in sequence.

[0016] In a preferred embodiment, step S2, training the self-supervised task head, includes the following sub-steps:

[0017] S21. Use a self-supervised task head to process the features of the target domain and obtain the geometric correlation feature vector;

[0018] S22. Obtain the covariance matrix of each point in the target domain point cloud sample, and select the geometric correlation feature vector corresponding to the smallest eigenvalue in the covariance matrix of each point as the predicted value of the geometric correlation feature label.

[0019] S23. Use the predicted values ​​of geometric correlation feature labels as supervision to train the self-supervised task head.

[0020] In a preferred embodiment, in S23, the loss function during training is set to cosine distance loss, expressed as:

[0021]

[0022] in, This represents the loss for predicting geometric correlation features. Indicates the number of samples in the target domain. Indicates the target domain. This represents the point cloud samples selected from the target domain. This represents the points in the point cloud sample. The geometric association feature labels of points in a point cloud. The predicted value representing the geometric association feature label of points in a point cloud. The table shows the modulo operation of vectors.

[0023] In a preferred embodiment, in S3, the loss function during training is set as follows:

[0024]

[0025] in, Indicates the overall loss. This indicates a loss of supervisory duties. This represents the loss for predicting geometric correlation features. This indicates the weight of the loss in the supervised task.

[0026] In a preferred embodiment, in S5, the entropy of the acquired target domain point cloud sample is represented as:

[0027]

[0028] in, Represents the entropy of a point cloud sample. This represents the total number of categories in the sample. c To represent different categories, This represents the probability distribution of the sample's class.

[0029] In a preferred embodiment, in S6, the target domain samples are sorted according to their entropy, from smallest to largest. The samples are regarded as the new source domain, and the pseudo-labels are confirmed based on the prediction results. The remaining samples are regarded as the new target domain.

[0030] The present invention also provides an electronic device, comprising:

[0031] At least one processor; and

[0032] A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in any of the preceding descriptions.

[0033] The present invention also provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in any of the preceding claims.

[0034] The beneficial effects of this invention include:

[0035] (1) To address the limited expressive power of self-supervised tasks in point cloud domain adaptive methods, a geometric association prediction task is proposed, enabling the model to learn more comprehensive and robust features;

[0036] (2) To address the problem that existing point cloud domain adaptive methods ignore the distribution differences within the target domain, we propose an additional training phase specifically for the target domain, which makes the model more applicable to the target domain and improves the model's prediction accuracy. Attached Figure Description

[0037] Figure 1 A schematic diagram of the process of a point cloud domain adaptive method based on geometric correlation prediction according to a preferred embodiment of the present invention is shown.

[0038] Figure 2 The diagram shows a schematic of a point cloud domain adaptive method based on geometric correlation prediction according to a preferred embodiment of the present invention. Detailed Implementation

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Through these descriptions, the features and advantages of the present invention will become clearer and more apparent.

[0040] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments. Although various aspects of embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated otherwise.

[0041] According to the present invention, a point cloud domain adaptive method based on geometric correlation prediction is provided, such as... Figure 1 As shown, it includes the following steps:

[0042] S1. Use a feature encoder to extract features from source domain point cloud samples to obtain source domain features, and extract features from target domain point cloud samples to obtain target domain features.

[0043] S2. Train the supervised task head based on source domain features, and train the self-supervised task head based on target domain features;

[0044] S3. Repeat S1~S2 until convergence, to obtain the feature encoder, supervised task head and self-supervised task head after one training.

[0045] S4. Use a feature encoder trained once to extract features from point cloud samples in the target domain, obtain target domain features, and use a supervised task head trained once to process the target domain features to obtain the class probability distribution of point cloud samples in the target domain.

[0046] S5. Obtain the entropy of the target domain point cloud samples based on the category probability distribution of the target domain point cloud samples;

[0047] S6. Divide the target domain point cloud samples into two groups according to the entropy level. Use the target domain point cloud samples of the low-entropy group as the new source domain point cloud samples, and use the target domain point cloud samples of the high-entropy group as the new target domain point cloud samples. Repeat steps S1 and S2 until convergence, and obtain the feature encoder and supervised task head after secondary training.

[0048] S7. The feature encoder trained twice is used to extract features from the samples to be classified to obtain sample features. The supervised task head trained twice is used to process the samples to obtain the classification results of the samples to be classified.

[0049] According to the present invention, similar to the traditional point cloud domain adaptation method, the source domain refers to the labeled point cloud data in the original environment or scene, and the target domain refers to the labeled point cloud data in the new environment or scene.

[0050] Preferably, in step S1, before extraction, the method further includes: downsampling the source domain point cloud and target domain point cloud samples to [value missing] using the farthest point sampling method. We use a fixed number of points to standardize the point cloud data to a fixed size, allowing the point cloud data to better fit the model.

[0051] Furthermore, in S1, features of the source domain point cloud and the target domain point cloud are extracted through a shared feature encoder.

[0052] In this invention, any known point cloud feature extraction network can be used as the feature encoder to extract point cloud features. Preferably, the DGCNN network is used. The DGCNN network treats point cloud data as a dynamic graph, with points as nodes in the graph and edges generated by the neighborhood relationships of each point. It extracts point cloud features using graph convolution. DGCNN has demonstrated excellent performance on multiple datasets and is a general and powerful point cloud encoder.

[0053] In S2, the supervised task head is trained on a supervised task, which can be a classification task or a segmentation task, preferably a classification task.

[0054] Preferably, in classification tasks, the classification loss is set as the cross-entropy loss, expressed as:

[0055]

[0056] in, Represents classification loss, This represents the number of samples in the source domain. Represents the source domain. Represents source domain samples, This represents the total number of categories in the sample. c To represent different categories, and These represent the classification label and predicted probability distribution of the source domain samples, respectively.

[0057] In this invention, the specific structure of the supervised task head is not limited, and those skilled in the art can freely choose according to actual needs.

[0058] In S2, the self-supervised task head is a geometric correlation prediction network, which includes multiple convolutional layers, activation function layers, and dropout layers connected in sequence.

[0059] Preferably, the multi-layer convolutional layer is a 4-layer 1D convolutional layer with a kernel of 1.

[0060] More preferably, a normalization layer is set for each convolutional layer.

[0061] In S2, training the self-supervised task head includes the following sub-steps:

[0062] S21. Use a self-supervised task head to process the features of the target domain and obtain the geometric correlation feature vector;

[0063] S22. Obtain the covariance matrix of each point in the target domain point cloud sample, and select the geometric correlation feature vector corresponding to the smallest eigenvalue in the covariance matrix of each point as the predicted value of the geometric correlation feature label.

[0064] S23. Use the predicted values ​​of geometric correlation feature labels as supervision to train the self-supervised task head.

[0065] Preferably, in S22, the kNN method is used to obtain the nearest neighbors of each point in the point cloud, based on the points... Calculate the covariance matrix using nearest neighbors.

[0066] The kNN (K-Nearest Neighbor) method is a basic classification and regression method widely used in machine learning. Its specific process will not be elaborated in this invention.

[0067] Preferably, in the kNN method, Neighbor The value is 30.

[0068] Preferably, the calculation of the covariance matrix includes:

[0069] S221. For each point in the point cloud and its Neighboring points Calculate this The mean of the nearest neighbors , represented as:

[0070] ;

[0071] in, Indicates the index of the nearest point. Point The 1 neighboring point.

[0072] S222. Construct the covariance matrix, expressed as:

[0073]

[0074] in, Point The covariance matrix, superscript This indicates transpose.

[0075] Calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and select the eigenvector corresponding to the smallest eigenvalue as the predicted value of the geometric association feature label.

[0076] According to the present invention, the predicted value of the obtained geometric association feature label is obtained from the covariance matrix of the local point cluster, which is related to the position and angle of the point, whether the two points are on the same plane, etc., which can enable the model to learn more comprehensive and robust features.

[0077] In S23, the loss function during training is set to cosine distance loss, expressed as:

[0078]

[0079] in, This represents the loss for predicting geometric correlation features. Indicates the number of samples in the target domain. Indicates the target domain. This represents the point cloud samples selected from the target domain. This represents the points in the point cloud sample. The geometric association feature labels of points in a point cloud. The predicted value representing the geometric association feature label of points in a point cloud. This represents the vector modulo operation.

[0080] Preferably, in S3, the loss function during training is set as follows:

[0081]

[0082] in, Indicates the overall loss. This indicates a loss of supervisory duties. This represents the loss for predicting geometric correlation features. This indicates the weight of the loss in the supervised task.

[0083] Preferably, The value is 0.67.

[0084] According to the present invention, steps S1-S3 complete the feature alignment between the source domain and the target domain. However, simply aligning the features of the source domain and the target domain is insufficient, as there are also distribution differences within the target domain, causing the model to fail to achieve optimal performance in the target domain.

[0085] In this invention, feature alignment is performed in a more granular manner through steps S4-S6. After feature alignment at the domain level, an additional training phase specifically targeting the target domain is added, allowing the model to learn the distributional differences within the target domain. After training in S1-S3 is completed, the model has a certain generalization ability in the target domain. At this point, the target domain samples are fed into the model for prediction. Based on the entropy of the predicted probability distribution, the samples are divided into two parts: those that are easier to classify and those that are more difficult to classify. If the predicted class probability distribution is concentrated on a certain class, it indicates that the model has high confidence in the prediction result for that sample. The sample is relatively regular and easy to identify. In this case, the entropy of the sample is low, and it is used as a new source domain for subsequent training. If the predicted class probability distribution is scattered across various classes, it indicates that the model has low confidence in the prediction. The sample is irregular and not easy to identify. In this case, the entropy is high, and it is used as a new target domain for subsequent training.

[0086] In S5, the entropy of the acquired target domain point cloud samples is represented as:

[0087]

[0088] in, Represents the entropy of a point cloud sample. This represents the total number of categories in the sample. This represents the probability distribution of the sample's class.

[0089] In S6, the target domain samples are sorted according to their entropy, from smallest to largest. The samples are regarded as the new source domain, and the pseudo-labels are confirmed based on the prediction results. The remaining samples are regarded as the new target domain.

[0090] Preferably, Set it to 0.8. Example

[0091] Example 1

[0092] Experiments were conducted using the PointDA-10 dataset to perform domain adaptation tasks for point cloud classification models. The PointDA-10 dataset contains... , , , , , A total of 6 adaptive scenarios were tested, and the experiment included the following steps:

[0093] S1. Use a feature encoder to extract features from source domain point cloud samples to obtain source domain features, and extract features from target domain point cloud samples to obtain target domain features.

[0094] S2. Train the supervised task head based on source domain features, and train the self-supervised task head based on target domain features;

[0095] S3. Repeat S1~S2 until convergence, to obtain the feature encoder, supervised task head and self-supervised task head after one training.

[0096] S4. Use a feature encoder trained once to extract features from point cloud samples in the target domain, obtain target domain features, and use a supervised task head trained once to process the target domain features to obtain the class probability distribution of point cloud samples in the target domain.

[0097] S5. Obtain the entropy of the target domain point cloud samples based on the category probability distribution of the target domain point cloud samples;

[0098] S6. Divide the target domain point cloud samples into two groups according to the entropy level. Use the target domain point cloud samples of the low-entropy group as the new source domain point cloud samples, and use the target domain point cloud samples of the high-entropy group as the new target domain point cloud samples. Repeat steps S1 and S2 until convergence, and obtain the feature encoder and supervised task head after secondary training.

[0099] S7. The feature encoder trained twice is used to extract features from the samples to be classified to obtain sample features. The supervised task head trained twice is used to process the samples to obtain the classification results of the samples to be classified.

[0100] In S1, before extraction, the process also includes: downsampling the source and target domain point cloud samples to their original values ​​using the farthest point sampling method. For each point, features of the source domain point cloud and the target domain point cloud are extracted through a shared feature encoder, which employs a DGCNN network.

[0101] In S2, the supervised task head is trained for a classification task. In the classification task, the classification loss is set as the cross-entropy loss, expressed as:

[0102]

[0103] The self-supervised task head consists of four 1D convolutional layers with a kernel of 1. Each convolutional layer has a normalization layer and also includes an activation function layer and a dropout layer.

[0104] In S2, training the self-supervised task head includes the following sub-steps:

[0105] S21. Use a self-supervised task head to process the features of the target domain and obtain the geometric correlation feature vector;

[0106] S22. Obtain the covariance matrix of each point in the target domain point cloud sample, and select the geometric correlation feature vector corresponding to the smallest eigenvalue in the covariance matrix of each point as the predicted value of the geometric correlation feature label.

[0107] S23. Use the predicted values ​​of geometric correlation feature labels as supervision to train the self-supervised task head.

[0108] In S22, the kNN method is used to obtain each point in the point cloud. Neighbors, based on points Nearest neighbor calculation of covariance matrix Neighbor The value is 30.

[0109] The calculation of the covariance matrix includes:

[0110] S221. For each point in the point cloud and its Neighboring points Calculate this The mean of the nearest neighbors , represented as:

[0111] ;

[0112] S222. Construct the covariance matrix, expressed as:

[0113]

[0114] In S23, the loss function during training is set to cosine distance loss, expressed as:

[0115]

[0116] In S3, a total of 150 epochs are trained, and the loss function during training is set as follows:

[0117]

[0118] in The value is 0.67.

[0119] In S5, the entropy of the acquired target domain point cloud samples is represented as:

[0120]

[0121] In S6, Set it to 0.8.

[0122] Example 2

[0123] The same experiments as in Example 1 were conducted, except that the GraspNetPC-10 dataset was used. The GraspNetPC-10 dataset contains... , , , There are a total of 4 adaptive scenarios; each training session consists of 120 epochs.

[0124] Comparative Example 1

[0125] The same experiment as in Example 1 was conducted, except that the PointDAN method was used. For the specific process of PointDAN, please refer to the paper [Qin C, You H, Wang L, et al. Pointdan: A multi-scale 3d domain adsorption network for point cloud representation[J]. Advances in Neural Information Processing Systems, 2019, 32.].

[0126] Comparative Example 2

[0127] The same experiment as in Example 1 was conducted, except that the DefRec method was used. For details of the DefRec process, please refer to the paper [Achituve I, Maron H, Chechik G. Self-supervised learning for domain adaptation on pointclouds[C] / / Proceedings of the IEEE / CVF winter conferenceon applications of computer vision. 2021: 123-133.].

[0128] Comparative Example 3

[0129] The same experiment as in Example 1 was conducted, except that the PC-Adapter method was used. For details of the PC-Adapter process, please refer to the paper [Park J, Seo H, Yang E. PC-Adapter: Topology-Aware Adapter for Efficient Domain Adaption on Point Clouds with Rectified Pseudo-label[C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2023:11530-11540.].

[0130] Comparative Example 4

[0131] The same experiment as in Example 1 was conducted, except that the GLRV method was used. For details of the GLRV process, please refer to the paper [Fan H, Chang X, Zhang W, et al. Self-supervised global-local structuremodeling for point clouddomain adaptation with reliable voted pseudo labels[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and PatternRecognition. 2022: 6377-6386.].

[0132] Comparative Example 5

[0133] The same experiment as in Example 1 was conducted, except that the GAST method was used. For details of the GAST process, please refer to the paper [Zou L, Tang H, Chen K, et al. Geometry-aware self-training for unsupervised domain adaptation on object point clouds[C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2021: 6403-6412.].

[0134] Comparative Example 6

[0135] The same experiment as in Example 1 was conducted, except that the ImplicitPCDA method was used. For details of the ImplicitPCDA process, please refer to the paper [Shen Y, Yang Y, Yan M, et al. Domain adaptation on point clouds via geometry-awareimplicits[C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition. 2022: 7223-7232.].

[0136] Comparative Example 7

[0137] The same experiment as in Example 1 was performed, except that the MLSP method was used. For details of the MLSP process, please refer to the paper [Liang H, Fan H, Fan Z, et al. Point cloud domain adaptation via masked local 3d structure prediction[C] / / European Conference on Computer Vision.Cham: Springer NatureSwitzerland, 2022: 156-172.].

[0138] Comparative Example 8

[0139] The same experiment as in Example 2 was conducted, except that the PointDAN method was used. For the specific process of PointDAN, please refer to the paper [Qin C, You H, Wang L, et al. Pointdan: A multi-scale 3d domain adsorption network for pointcloud representation[J]. Advances in Neural Information Processing Systems, 2019, 32.].

[0140] Comparative Example 9

[0141] The same experiment as in Example 2 was conducted, except that the RS method was used. For details of the RS process, please refer to the paper [Sauder J, Sievers B. Self-supervised deep learning on point clouds by reconstructing space[J]. Advances in Neural Information Processing Systems,2019, 32.].

[0142] Comparative Example 10

[0143] The same experiment as in Example 2 was conducted, except that the DefRec method was used. For details of the DefRec process, please refer to the paper [Achituve I, Maron H, Chechik G. Self-supervised learning for domain adaptation on pointclouds[C] / / Proceedings of the IEEE / CVF winter conferenceon applications of computer vision. 2021: 123-133.].

[0144] Comparative Example 11

[0145] The same experiment as in Example 2 was conducted, except that the GAST method was used. For details of the GAST process, please refer to the paper [Zou L, Tang H, Chen K, et al. Geometry-aware self-training for unsupervised domain adaptation on object point clouds[C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2021: 6403-6412.].

[0146] Comparative Example 12

[0147] The same experiment as in Example 2 was conducted, except that the ImplicitPCDA method was used. For details of the ImplicitPCDA process, please refer to the paper [Shen Y, Yang Y, Yan M, et al. Domain adaptation on point clouds via geometry-awareimplicits[C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition. 2022: 7223-7232.].

[0148] Comparative Example 13

[0149] The same experiment as in Example 2 was conducted, except that the PC-Adapter method was used. For details of the PC-Adapter process, please refer to the paper [Park J, Seo H, Yang E. PC-Adapter: Topology-Aware Adapter for Efficient Domain Adaption on Point Clouds with Rectified Pseudo-label[C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2023:11530-11540.].

[0150] Experimental Example 1

[0151] Comparing the results of Example 1 with those of Comparative Examples 1-7, the model performance was measured using recognized metrics, namely the classification accuracy on each adaptive scenario and the average classification accuracy across all adaptive scenarios. The comparison results are shown in Table 1.

[0152] Table 1

[0153]

[0154] As shown in Table 1, on the PointDA-10 dataset, the method in Example 1 achieves the highest average accuracy across the six adaptive scenarios compared to other methods, especially in... , and The method achieved the highest accuracy in adaptive scenarios, demonstrating the effectiveness of the method in Example 1.

[0155] Experiment Example 2

[0156] Comparing the results of Example 2 with those of Comparative Examples 8-13, the model performance was measured using the recognized metric of classification accuracy on each adaptive scenario and the average classification accuracy across all adaptive scenarios. The comparison results are shown in Table 2.

[0157] Table 2

[0158]

[0159] As can be seen from Table 2, on the GraspNetPC-10 dataset, the method in Example 2 has the highest average accuracy across all adaptive scenarios compared to other methods, and in... and The highest accuracy was achieved in adaptive scenarios. Combining Experiment 1 and Experiment 2, it can be seen that the methods in the embodiments can achieve higher performance under different datasets.

[0160] Various embodiments of the methods described above in this invention can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0161] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0162] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0163] To provide interaction with a user, the methods and apparatus described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0164] The methods and apparatus described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0165] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0166] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

Claims

1. A point cloud domain adaptive method based on geometric correlation prediction, characterized in that, Includes the following steps: S1. Use a feature encoder to extract features from source domain point cloud samples to obtain source domain features, and extract features from target domain point cloud samples to obtain target domain features. S2. Train the supervised task head based on source domain features, and train the self-supervised task head based on target domain features; S3. Repeat S1~S2 until convergence, to obtain the feature encoder, supervised task head and self-supervised task head after one training. S4. Use a feature encoder trained once to extract features from point cloud samples in the target domain, obtain target domain features, and use a supervised task head trained once to process the target domain features to obtain the class probability distribution of point cloud samples in the target domain. S5. Obtain the entropy of the target domain point cloud samples based on the category probability distribution of the target domain point cloud samples; S6. Divide the target domain point cloud samples into two groups according to the entropy level. Use the target domain point cloud samples of the low-entropy group as the new source domain point cloud samples, and use the target domain point cloud samples of the high-entropy group as the new target domain point cloud samples. Repeat steps S1 and S2 until convergence, and obtain the feature encoder and supervised task head after secondary training. S7. The feature encoder trained twice is used to extract features from the samples to be classified to obtain sample features. The supervised task head trained twice is used to process the samples to obtain the classification results of the samples to be classified.

2. The point cloud domain adaptive method based on geometric correlation prediction according to claim 1, characterized in that, In S1, before extraction, the process also includes: downsampling the source and target domain point cloud samples to their original values ​​using the farthest point sampling method. One point.

3. The point cloud domain adaptive method based on geometric correlation prediction according to claim 1, characterized in that, In S2, the self-supervised task head is a geometric correlation prediction network, which includes multiple convolutional layers, activation function layers, and dropout layers connected in sequence.

4. The point cloud domain adaptive method based on geometric correlation prediction according to claim 1, characterized in that, In S2, training the self-supervised task head includes the following sub-steps: S21. Use a self-supervised task head to process the features of the target domain and obtain the geometric correlation feature vector; S22. Obtain the covariance matrix of each point in the target domain point cloud sample, and select the geometric correlation feature vector corresponding to the smallest eigenvalue in the covariance matrix of each point as the predicted value of the geometric correlation feature label. S23. Use the predicted values ​​of geometric correlation feature labels as supervision to train the self-supervised task head.

5. The point cloud domain adaptive method based on geometric correlation prediction according to claim 4, characterized in that, In S23, the loss function during training is set to cosine distance loss, expressed as: ， in, This represents the loss for predicting geometric correlation features. Indicates the number of samples in the target domain. Indicates the target domain. This represents the point cloud samples selected from the target domain. This represents the points in the point cloud sample. The geometric association feature labels of points in a point cloud. The predicted value representing the geometric association feature label of points in a point cloud. This represents the vector modulo operation.

6. The point cloud domain adaptive method based on geometric correlation prediction according to claim 1, characterized in that, In S3, the loss function during training is set as follows: ， in, Indicates the overall loss. This indicates a loss of supervisory duties. This represents the loss for predicting geometric correlation features. This indicates the weight of the loss in the supervised task.

7. The point cloud domain adaptive method based on geometric correlation prediction according to claim 1, characterized in that, In S5, the entropy of the acquired target domain point cloud samples is represented as: ， in, Represents the entropy of a point cloud sample. This represents the total number of categories in the sample. c To represent different categories, This represents the probability distribution of the sample's class.

8. The point cloud domain adaptive method based on geometric correlation prediction according to claim 1, characterized in that, In S6, the target domain samples are sorted according to their entropy, from smallest to largest. The samples are regarded as the new source domain, and the pseudo-labels are confirmed based on the prediction results. The remaining samples are regarded as the new target domain.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

10. A computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

Images