Small sample point cloud classification model training method, computer equipment and computer program product

Abstract

The invention is suitable for the technical field of point cloud processing, and provides a small sample point cloud classification model training method, computer equipment and a computer program product, and the method comprises the steps: extracting a first visual feature and a position feature based on a training point cloud data set; carrying out implicit sample expansion based on the first visual feature and the position feature to obtain an expansion feature; updating parameters of the pre-training model based on the first visual features and the expansion features; returning to the first step until a preset ending condition is met; and outputting a target classification model based on the pre-training model. According to the method provided by the invention, the distinguishing ability and the understanding ability of the model for different types of point clouds are improved through the first visual features and the position features, the diversity of training data is effectively enriched through implicit expansion of the first visual features, and the model parameters are jointly updated based on the features after implicit expansion and the first visual features, so that the training efficiency is improved. And the model has better generalization ability and classification performance.

Description

Technical Field

[0001] The present application belongs to the field of point cloud processing technology, and in particular, relates to a small sample point cloud classification model training method, computer equipment and computer program product. Background Art

[0002] In the field of modern computer vision, the processing and analysis of point cloud data is becoming a hot topic. Point cloud is a dataset composed of points in three-dimensional space, which is widely used in autonomous driving, robot navigation, virtual reality and other fields. However, since obtaining high-quality point cloud data usually requires expensive sensor equipment, the training data available for training deep learning models is relatively small, and the generalization ability of the trained small sample classification model is weak, which ultimately makes it difficult to smoothly carry out the point cloud classification task in the case of small samples. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a small sample point cloud classification model training method, computer equipment and computer program product, which improve the generalization ability and classification performance of the classification model trained in small sample conditions.

[0004] A first aspect of an embodiment of the present application provides a small sample point cloud classification model training method, comprising:

[0005] Extracting first visual features and position features based on a training point cloud data set;

[0006] Performing implicit sample expansion based on the first visual feature and the position feature to obtain an expanded feature;

[0007] Updating parameters of a pre-trained model based on the first visual feature and the extended feature;

[0008] Returning to the step of extracting the first visual feature and the position feature based on the training point cloud data set until a preset end condition is met;

[0009] A target classification model is output based on the pre-trained model.

[0010] In an implementation of the first aspect, performing implicit sample expansion based on the first visual feature and the position feature to obtain the expanded feature includes:

[0011] Performing semantic feature enhancement based on the first visual feature and the position feature to obtain a semantically enhanced point cloud feature;

[0012] Based on the semantically enhanced point cloud features, implicit sample expansion is performed to obtain expanded features.

[0013] In an implementation of the first aspect, updating the parameters of the pre-trained model based on the first visual feature and the extended feature includes:

[0014] updating a category center based on the first visual feature and the extended feature;

[0015] Performing comparative learning based on the first visual feature, the extended feature, and the category center, and calculating a loss value according to a first loss function;

[0016] Update the parameters of the pre-trained model based on the loss value.

[0017] In an implementation of the first aspect, updating the parameters of the pre-trained model based on the first visual feature and the extended feature further includes:

[0018] Inputting the first visual feature into a small sample learning head and a classifier for classification, and calculating a loss value according to a second loss function;

[0019] The updating of the parameters of the pre-trained model according to the loss value includes:

[0020] Update the parameters of the pre-trained model based on the loss values ​​calculated by the first loss function and the second loss function.

[0021] In an implementation of the first aspect, the training point cloud dataset is divided into a plurality of tasks, each task including a support set and a query set;

[0022] The extracting of the first visual feature and the position feature based on the training point cloud data set includes:

[0023] Selecting a task from the multiple tasks as a target task;

[0024] A first visual feature and a position feature are extracted from a support set and a query set of the target task.

[0025] In an implementation of the first aspect, the pre-trained model includes a backbone network and a point marker;

[0026] The extracting the first visual feature and the position feature from the support set and the query set of the target task includes:

[0027] Extracting a second visual feature from the support set and the query set through the backbone network;

[0028] extracting a third visual feature and a position feature from the support set and the query set by the point marker;

[0029] The second visual feature and the third visual feature are fused to obtain a first visual feature.

[0030] In an implementation of the first aspect, updating the parameters of the pre-trained model based on the first visual feature and the extended feature includes:

[0031] Update the parameters of the backbone network and the point marker based on the first visual feature and the extended feature;

[0032] The outputting a target classification model based on the pre-trained model comprises:

[0033] The backbone network in the pre-trained model is output as a target classification model.

[0034] In an implementation of the first aspect, the preset end condition is that the classification accuracy of the backbone network reaches a preset threshold;

[0035] The step of returning to the step of extracting the first visual feature and the position feature based on the training point cloud data set until a preset end condition is met further includes:

[0036] Evaluate the classification accuracy of the backbone network.

[0037] A second aspect of an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0038] A third aspect of an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0039] The beneficial effect of the first aspect of the embodiment of the present application is: the method provided in the embodiment of the present application improves the model's ability to distinguish and understand different types of point clouds through the first visual features and position features, and effectively enriches the diversity of training data by implicitly expanding the first visual features. The model parameters are jointly updated based on the implicitly expanded features and the first visual features, so that the model has better generalization ability and classification performance.

[0040] It can be understood that the beneficial effects of the third to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 This is a schematic diagram of the implementation process of the first small sample point cloud classification model training method provided in the embodiment of the present application;

[0043] Figure 2 It is a schematic diagram of the implementation process of the second small sample point cloud classification model training method provided in the embodiment of the present application;

[0044] Figure 3 This is a schematic diagram of the implementation process of the third small sample point cloud classification model training method provided in the embodiment of the present application;

[0045] Figure 4 This is a schematic diagram of the implementation process of the fourth small sample point cloud classification model training method provided in the embodiment of the present application;

[0046] Figure 5 It is a framework diagram of a small sample point cloud classification model training method provided in an embodiment of the present application;

[0047] Figure 6 is a schematic diagram of a computer device provided in an embodiment of the present application;

[0048] Figure 7 It is a schematic diagram of a computing program product provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0050] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0051] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0052] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0053] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0054] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0055] The embodiment of the present application provides a small sample point cloud classification model training method, which is used to improve the generalization ability and classification performance of the classification model trained in the case of small samples. The method provided in the embodiment of the present application extracts the first visual features and position features based on the training point cloud data set, performs implicit sample expansion based on the first visual features and position features to obtain the expanded features, updates the parameters of the pre-trained model based on the first visual features and the expanded features, returns to the step of extracting the first visual features and position features based on the training point cloud data set until the preset end condition is met, and outputs the target classification model based on the pre-trained model, thereby improving the generalization ability and classification performance of the classification model trained in the case of small samples.

[0056] like Figure 1 A small sample point cloud classification model training method provided in an embodiment of the present application is shown, comprising:

[0057] Step S10, extracting first visual features and position features based on the training point cloud data set.

[0058] In the application, the training point cloud data set includes a high-quality annotated point cloud data set. Due to the following reasons, it is difficult to obtain high-quality annotated point cloud data, and the training point cloud data set is a small sample data set. Cost: First, equipment cost. High-quality point cloud data usually requires the use of advanced sensor equipment, such as high-resolution laser radar (LiDAR) or RGB-D camera. These devices are expensive. In addition, in order to ensure data quality, high-performance computing equipment may also be required for data processing; second, labor cost. Data annotation usually requires professionals, especially for point cloud data in complex scenes. The annotation work is not only time-consuming, but also requires certain professional knowledge. Labor costs increase with the increase of annotation complexity. In terms of quality, point cloud data is easily affected by environmental factors, such as weather conditions, lighting conditions, etc., which may lead to a decrease in data quality; there may be noise points or outliers in the original collected data, which need to be cleaned by complex algorithms to ensure the quality of the final data set. In terms of annotation, point cloud data annotation is more complicated than 2D image annotation because they contain 3D spatial information. When annotating, each point needs to be accurate, and the spatial relationship between points needs to be considered. Ensuring the consistency of annotation is also a problem, especially when multiple people participate in annotation. A set of standards is needed to unify the annotation rules to avoid deviations. In terms of application field restrictions, one is data sensitivity: in some application fields, such as military and security monitoring, point cloud data may involve sensitive information, and the acquisition and use of such data are strictly restricted; the second is data scarcity: for certain specific scenarios (such as rare natural disaster sites), it is very difficult to obtain relevant point cloud data because these situations are uncommon and it is difficult to obtain a large amount of data.

[0059] In the application, the training data set is obtained through the following steps: after collecting the original point cloud data through LiDAR or RGB-D camera, the original point cloud data is cleaned to remove noise points, outliers and duplicate data, the format is unified and the data is annotated to obtain high-quality annotated point cloud data, which is then stored in the training data set. In the application, the unlabeled point cloud data can be annotated by manual annotation, automatic annotation or semi-automatic annotation. Regardless of the annotation method, in order to ensure the accuracy and consistency of the annotation, a quality check is also required. Specifically, the quality of the annotation can be evaluated through multiple audits, cross-validation, consistency scoring, sampling inspection and other methods.

[0060] Step S20, performing implicit sample expansion based on the first visual feature and the position feature to obtain an expanded feature.

[0061] In the application, by performing implicit sample extension (ISE) based on the first visual features and position features, geometric features consistent with the original samples can be generated. The consistency of this feature ensures that the model will not be disturbed by erroneous information during the training process, thereby improving classification accuracy. In the case of a small sample data set, implicit sample extension can effectively increase the amount and diversity of training data, which is conducive to alleviating overfitting caused by sample limitations of training data, and improving the generalization ability, classification performance and robustness of the trained target classification model.

[0062] Step S30: updating the parameters of the pre-trained model based on the first visual feature and the extended feature.

[0063] In the application, forward propagation is performed based on the first visual features and the expanded features, and then the loss value is calculated according to the loss function, and the gradient is calculated according to the loss value, and the parameters of the pre-trained model are updated based on the gradient.

[0064] Step S40, returning to the step of extracting the first visual feature and the position feature based on the training point cloud data set until a preset end condition is met.

[0065] In the application, steps S10 to S30 are repeated to complete iterative training, and forward propagation and back propagation are repeated until a preset number of training batches are completed or the performance of the pre-trained model meets expectations.

[0066] Step S50: output a target classification model based on the pre-trained model.

[0067] In the application, after the iterative training is completed, the target classification model is output based on the pre-trained model after the last parameter update. The target classification model can be directly used to classify new point cloud data. For example, it is applied to the field of autonomous driving to detect obstacles. In autonomous driving vehicles, pedestrians, vehicles and other obstacles on the road can be identified by classifying point cloud data, providing a basis for vehicle obstacle avoidance decisions; lane marking, using point cloud data to identify lane boundaries, helping vehicles stay in the correct lane, and traffic sign recognition, using point cloud data to identify the location and type of traffic signs, assisting vehicles in making correct driving decisions, etc.

[0068] like Figure 2 As shown, in one embodiment, the step S20, performing implicit sample expansion based on the first visual feature and the position feature to obtain the expanded feature, includes:

[0069] Step S201: performing semantic feature enhancement based on the first visual feature and the position feature to obtain a semantically enhanced point cloud feature.

[0070] In the application, the first visual feature is used as the key vector and value vector, and the position feature is used as the query vector to input into the large language model LLAMA, and the semantic feature is enhanced through the self-attention mechanism to obtain the semantically enhanced point cloud feature. Specifically, in order to be compatible with the LLAMA model, the third visual feature can be linearly mapped by the linear projection layer and adapted to the same dimensional space as the text annotation. And an additional linear layer mapping is performed to project the semantically enhanced features back to the visual feature space.

[0071] It is understandable that the powerful reasoning ability of the large language model is integrated into the classification effect of the model through feature extraction and understanding of point cloud data. The large language model can process complex contextual information, so that the model can still maintain stable performance when facing diverse point cloud data.

[0072] Step S202: performing implicit sample expansion based on the semantically enhanced point cloud features to obtain expanded features.

[0073] In the application, implicit sample expansion is performed based on semantically enhanced point cloud features. The steps of implicit sample expansion are as follows: Determine the nearest neighbor category center: Calculate the category center. For a given set of support samples, calculate the category center of each category. The category center can be estimated by the average value of all sample features belonging to the category; Find the nearest neighbor: For each actual sample, find the K nearest neighbor category centers of its category and other categories. This can be achieved by calculating the distance between the actual sample features and all category centers, and select the K closest ones. Generate new samples by interpolation: Interpolation generation. According to the relationship between the actual sample and its K nearest neighbor category centers, new support samples are generated by interpolation (such as linear interpolation) in the feature space. Interpolation can be performed according to a certain weight factor, which can be determined based on the distance between the actual sample and the category center or some other form of correlation.

[0074] It is understandable that implicit sample expansion generates geometric features consistent with the original samples. The consistency of this feature ensures that the model will not be disturbed by erroneous information during the training process, thereby improving classification accuracy. In the case of a small sample data set, implicit sample expansion can effectively increase the amount and diversity of training data, which is conducive to alleviating overfitting caused by sample limitations of training data, and improving the generalization ability, classification performance and robustness of the trained target classification model.

[0075] like Figure 3 As shown, in one embodiment, the step S30, updating the parameters of the pre-trained model based on the first visual feature and the extended feature, includes:

[0076] Step S301: updating the category center based on the first visual feature and the extended feature.

[0077] In the application, according to the categories of the support set, the mean of the features of each category is taken as the initial center of the category. Using the expanded features, the new category center is calculated according to its category label and the old category center. The center is updated using the exponential moving average (EMA) method.

[0078] Step S302: performing comparative learning based on the first visual feature, the extended feature and the category center, and calculating a loss value according to a first loss function.

[0079] In applications, the core idea of ​​contrastive learning is to let the model learn to distinguish different samples by constructing positive pairs and negative pairs. In a specific example, samples from the same category are considered as positive pairs, while samples from different categories are considered as negative pairs. Based on the contrastive learning results, the loss value is calculated according to the first loss function.

[0080] Step S303, updating the parameters of the pre-trained model according to the loss value.

[0081] In applications, one of the commonly used contrast loss functions is InfoNCE loss (Normalized Temperature-Scaled Cross Entropy Loss). The process of updating model parameters is as follows:

[0082] Extract the first visual features and the expanded point cloud features from the support samples and query samples; Calculate the category center: Calculate the category center based on the features of the support samples; For each query sample, its corresponding positive sample pair is a support sample from the same category; Support samples from different categories are regarded as negative sample pairs; Calculate the similarity between the query sample and all support samples (including positive and negative samples). Calculate the loss value using the contrastive loss function. For each query sample, the loss function penalizes those embedded representations that cannot correctly distinguish between positive and negative samples. Backpropagate the loss value calculated by the contrastive loss function into the model and calculate the gradient of each parameter. Use optimizers such as Stochastic Gradient Descent (SGD) and Adaptive Moment Estimation (Adam) to update the parameters of the pre-trained model according to the calculated gradients to minimize the loss function.

[0083] In one embodiment, the step S30 of updating the parameters of the pre-trained model based on the first visual feature and the extended feature further includes:

[0084] Step S304, inputting the first visual feature into a small sample learning head and a classifier for classification, and calculating a loss value according to a second loss function;

[0085] The step S303, updating the parameters of the pre-trained model according to the loss value, includes:

[0086] Step S305: Update the parameters of the pre-trained model based on the loss values ​​calculated by the first loss function and the second loss function.

[0087] In the application, the first visual feature is input into the FSL Head (FewShot Learning Head) and the classifier, and forward propagation is performed through the FSL Head and the classifier to obtain a classification score or probability, and the final classification result is determined according to the classification score or probability. Based on the classification result, the loss value is calculated according to the second loss function. The gradient of each parameter is calculated according to the loss value and corresponding weight calculated by the first loss function and the loss value and corresponding loss value calculated by the second loss function, and then an optimizer such as Stochastic Gradient Descent (SGD) and Adaptive Moment Estimation (Adam) is used to update the parameters of the pre-trained model according to the calculated gradient.

[0088] In one embodiment, the training point cloud dataset is divided into a plurality of tasks, each task including a support set and a query set;

[0089] The step S10, extracting the first visual feature and the position feature based on the training point cloud data set, includes:

[0090] Step S101: selecting a task from the multiple tasks as a target task.

[0091] Step S102: extracting first visual features and position features from a support set and a query set of the target task.

[0092] In the application, each target person can be regarded as an iterative task in the model training process. Each task contains a support set and a query set. The samples of the support set are used for model training, and the samples of the query set are used for verification. The small sample learning setting is N-way K-shot. For example, in 5-way 1-shot, each task contains 5 categories, each category has 1 support sample, and several query samples.

[0093] like Figure 4 As shown, in one embodiment, the pre-trained model includes a backbone network and a point marker;

[0094] The step S102, extracting the first visual feature and the position feature from the support set and the query set of the target task, includes:

[0095] Step S103, extracting a second visual feature from the support set and the query set through the backbone network;

[0096] Step S104, extracting a third visual feature and a position feature from the support set and the query set by the point marker;

[0097] Step S105: fusing the second visual feature and the third visual feature to obtain a first visual feature.

[0098] In the application, you can choose a feature extraction network based on an image or point cloud as the backbone network. Common choices include: ResNet, VGG, Vision Transformer (ViT) and other pre-trained models to extract the first visual features; PointNet, PointNet++, DGCNN (dynamic graph convolutional neural network) and other models to extract the geometric features of 3D point clouds. In the application, the point marker is used to encode the location information of each point in the point cloud. The specific implementation can be based on the MLP (Multi-Layer Perceptron) network or self-attention mechanism to extract local and global point location features and visual features.

[0099] In applications, the second visual feature and the third visual feature can be fused through simple feature concatenation, element-wise multiplication, or weighted summation. Multimodal fusion techniques such as Cross-attention can also be used to fuse two types of different modal features.

[0100] In one embodiment, the step S30 of updating the parameters of the pre-trained model based on the first visual feature and the extended feature includes:

[0101] Step S306, updating the parameters of the backbone network and the point marker based on the first visual feature and the extended feature;

[0102] The step S50, outputting a target classification model based on the pre-trained model, includes:

[0103] Step S501, outputting the backbone network in the pre-trained model as a target classification model.

[0104] In one embodiment, the preset end condition is that the classification accuracy of the backbone network reaches a preset threshold;

[0105] The step S40, returning to the step S10, the step of extracting the first visual feature and the position feature based on the training point cloud data set, until the preset end condition is met, further includes:

[0106] Step S31, evaluating the classification accuracy of the backbone network.

[0107] In applications, classification accuracy can be evaluated based on indicators such as precision, recall, F1 score, ROC curve, and AUC (Receiver Operating Characteristic Curve and Area Under the Curve).

[0108] In the application, you can also improve the convergence speed and accuracy of the model by adjusting the dynamic learning rate, using appropriate loss functions, etc. You can also combine the automatic prompt optimization technology to systematically adjust the prompt content of the model input to further improve the model's reasoning ability and classification effect.

[0109] like Figure 5 As shown, it is a framework diagram of a small sample point cloud classification model training method provided in an embodiment of the present application. It includes a support set (Support Set) and a query set (Query Set) of the target task; the pre-trained model includes a backbone network (Backbone Network F φ ) and point tokenizer; the support set and query set are extracted through the backbone network to obtain the second visual feature E 2 , the support set and the query set are extracted by the point marker to obtain the third visual feature E 3 and position feature E p The first visual feature is used as the key vector K and the value vector V, and the position feature is used as the query vector Q, which are input into the large language model layer (LLAMA Layer), and then the extended feature E is obtained through the implicit sample extension module (ISE). 4 . Update the category center C based on the first visual feature and the expanded feature. Then perform contrastive learning based on the first visual feature, the expanded feature and the category center C, and input the first visual feature into the FSL head (Few Shot Learning) and the classifier for forward propagation. Finally, calculate the loss value based on the contrastive learning result and the corresponding loss function as well as the forward propagation result and the corresponding loss function, and update the parameters of the backbone network and the point marker in the pre-trained model based on the loss value.

[0110] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0111] Figure 6 This is a schematic diagram of the structure of a computer device provided in one embodiment of the present application. Figure 6 As shown, the computer device 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown in the figure) a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, wherein the processor 60 implements the steps of any of the above-mentioned method embodiments when executing the computer program 62.

[0112] The computer device 6 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of the computer device 6 and does not constitute a limitation on the computer device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0113] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0114] In some embodiments, the memory 61 may be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 may also be an external storage device of the computer device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the computer device 6. The memory 61 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0115] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0116] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0117] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0118] like Figure 7 As shown, an embodiment of the present application provides a computer program product 7, including a computer program 70, and when the computer program 70 is executed by a processor, the steps in the above-mentioned various method embodiments are implemented.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0120] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0121] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0122] In the embodiments provided in the present application, it should be understood that the disclosed computer devices and methods can be implemented in other ways. For example, the computer device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0123] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A small sample point cloud classification model training method, characterized in that: include: Extracting first visual features and position features based on a training point cloud data set; Performing implicit sample expansion based on the first visual feature and the position feature to obtain an expanded feature; Updating parameters of a pre-trained model based on the first visual feature and the extended feature; Returning to the step of extracting the first visual feature and the position feature based on the training point cloud data set until a preset end condition is met; A target classification model is output based on the pre-trained model.

2. The small sample point cloud classification model training method according to claim 1, characterized in that: The implicit sample expansion based on the first visual feature and the position feature to obtain the expanded feature includes: Performing semantic feature enhancement based on the first visual feature and the position feature to obtain a semantically enhanced point cloud feature; Based on the semantically enhanced point cloud features, implicit sample expansion is performed to obtain expanded features.

3. The small sample point cloud classification model training method according to claim 2, characterized in that: The updating of the parameters of the pre-trained model based on the first visual feature and the extended feature includes: updating a category center based on the first visual feature and the extended feature; Performing comparative learning based on the first visual feature, the extended feature, and the category center, and calculating a loss value according to a first loss function; Update the parameters of the pre-trained model based on the loss value.

4. The small sample point cloud classification model training method according to claim 3, characterized in that: The updating of the parameters of the pre-trained model based on the first visual feature and the extended feature further includes: Inputting the first visual feature into a small sample learning head and a classifier for classification, and calculating a loss value according to a second loss function; The updating of the parameters of the pre-trained model according to the loss value includes: Update the parameters of the pre-trained model based on the loss values ​​calculated by the first loss function and the second loss function.

5. The small sample point cloud classification model training method according to any one of claims 1 to 4, characterized in that: The training point cloud dataset is divided into multiple tasks, each task includes a support set and a query set; The extracting of the first visual feature and the position feature based on the training point cloud data set includes: Selecting a task from the multiple tasks as a target task; A first visual feature and a position feature are extracted from a support set and a query set of the target task.

6. The small sample point cloud classification model training method according to claim 5, characterized in that: The pre-trained model includes a backbone network and a point marker; The extracting the first visual feature and the position feature from the support set and the query set of the target task includes: Extracting a second visual feature from the support set and the query set through the backbone network; extracting a third visual feature and a position feature from the support set and the query set by the point marker; The second visual feature and the third visual feature are fused to obtain a first visual feature.

7. The small sample point cloud classification model training method according to claim 6, characterized in that: The updating of the parameters of the pre-trained model based on the first visual feature and the extended feature includes: Update the parameters of the backbone network and the point marker based on the first visual feature and the extended feature; The outputting a target classification model based on the pre-trained model comprises: The backbone network in the pre-trained model is output as a target classification model.

8. The small sample point cloud classification model training method according to claim 7, characterized in that: The preset end condition is that the classification accuracy of the backbone network reaches a preset threshold; The step of returning to the step of extracting the first visual feature and the position feature based on the training point cloud data set until a preset end condition is met further includes: Evaluate the classification accuracy of the backbone network.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images