Small sample point cloud object recognition method and system based on global and part matching, and medium

Abstract

The application discloses a kind of based on global and component matching small sample point cloud object identification method, system and medium, method includes constructing small sample point cloud object identification dataset;Small sample point cloud object identification data set is used to construct small sample point cloud object identification task, each task includes support set and query set;Point cloud object identification network model is constructed;Utilize base class small sample point cloud object identification task training and obtain trained point cloud object identification network model;The performance of point cloud object identification network model is verified using new class small sample point cloud object identification task, and the prediction result is obtained using the model.The application designs a small sample point cloud object identification model, by similarity comparison to point cloud on global, component two levels, effectively solve the problem of poor performance of existing method under the limitation of insufficient data, realizes small sample point cloud object accurate identification.

Description

Technical Field

[0001] This invention relates to the technical field of 3D point cloud object recognition, and in particular to a method, system, and medium for small-sample point cloud object recognition based on global and component matching. Background Technology

[0002] In recent years, artificial intelligence and related industries have been rapidly developing and expanding, becoming a focus of attention for academia, industry, and governments worldwide. AI research and industry are among the most important topics today. Point clouds have wide applications in the field of artificial intelligence, such as autonomous vehicles, intelligent medical imaging, and robotics. Point cloud object recognition, as an important task in the field of point clouds, has received widespread attention and research from academia and industry. Currently, methods based on deep neural networks have achieved certain results, but they generally rely on training with large amounts of point cloud data. In real-world applications, due to the complexity and difficulty in collecting point cloud data, obtaining a large amount of point cloud data for a specific target category requires significant costs. Therefore, how to design and train a point cloud object recognition model suitable for small sample sizes is a pressing problem that needs to be solved. Summary of the Invention

[0003] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and provide a method, system, and medium for small sample point cloud object recognition based on global and component matching. This invention designs a small sample point cloud object recognition model, which effectively solves the problem of poor performance of existing methods under the limitation of insufficient data by comparing the similarity of point clouds at both the global and component levels, and achieves accurate recognition of small sample point cloud objects.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] In a first aspect, the present invention provides a method for small-sample point cloud object recognition based on global and component matching, comprising the following steps:

[0006] A few-shot point cloud object recognition dataset is constructed, comprising a base class dataset and a new class dataset. The base class dataset is used to construct a few-shot point cloud object recognition task and update network parameters in a context-based training mode. The new class dataset is used to construct a few-shot point cloud object recognition task and verify the few-shot point cloud object recognition performance of the model.

[0007] A few-shot point cloud object recognition task is constructed using a few-shot point cloud object recognition dataset. The few-shot point cloud object recognition task includes a base class few-shot point cloud object recognition task and a new class few-shot point cloud object recognition task. Each task includes a support set and a query set. The recognition of the query set point cloud is learned using the information of the support set.

[0008] A point cloud object recognition network model is constructed, comprising a point cloud feature extractor, a point cloud global matching module, a point cloud component matching module, and a matching fusion module. The point cloud feature extractor extracts features from point clouds in the support set and the query set. The point cloud global matching module calculates the global matching probability between point clouds in the query set and the support class. The point cloud component matching module calculates the component matching probability between point clouds in the query set and the support class. The matching fusion module fuses the global matching probability and the component matching probability to obtain the matching probability between point clouds in the query set and the support class.

[0009] A point cloud object recognition network model is trained using a base class small sample point cloud object recognition task. The cross-entropy loss function is used as the objective function, and the parameters of the point cloud object recognition network model are iteratively trained using the stochastic gradient descent method until convergence, thus obtaining the trained point cloud object recognition network model.

[0010] The performance of the point cloud object recognition network model was verified using a novel few-sample point cloud object recognition task. The prediction results were obtained by inputting the trained point cloud object recognition network model.

[0011] As a preferred technical solution, the construction of the small sample point cloud object recognition task specifically includes:

[0012] The few-sample point cloud object recognition task includes a support set S and a query set Q. The support set S includes N target categories and K labeled samples for each target category. The query set Q includes T samples to be predicted. The point cloud object recognition network model learns to recognize the N target categories based on the information provided by the support set S, thereby making accurate predictions for the T samples to be predicted in the query set Q.

[0013] As a preferred technical solution, the point cloud feature extractor adopts a DGCNN network. The support set and query set share the same point cloud feature extractor. The point clouds from the support set and query set are input into the feature extractor to obtain the point cloud features of the support set. and querying centralized point cloud features

[0014] As a preferred technical solution, the calculation of the global matching probability between the point cloud in the query set and the supporting class specifically involves:

[0015] Perform global max pooling on each point cloud in the support set and query set to obtain the global feature vector of each point cloud;

[0016] Calculate the average feature vector of point clouds of all categories in the support set as the prototype feature vector of the support set;

[0017] For each point cloud in the query set, the cosine similarity between the point cloud in the query set and the point cloud in the support set is calculated based on the global feature vector and the prototype feature vector as the global similarity.

[0018] A softmax operation is performed on all global similarities to obtain the global matching probability between the point cloud in the query set and the supporting class.

[0019] As a preferred technical solution, the calculation of the matching probability between the point cloud in the query set and the components of the supported class is specifically as follows:

[0020] Each point cloud is divided into L parts. For any point cloud, the farthest point sampling method is used to select L center points. Each center point and its (H-1) nearest neighbor points of any point cloud constitute a part. The feature vector of the part is defined as the average feature vector of the center point and all neighbor points. Each point cloud is represented as a set of parts Z.

[0021] The set of support class components is defined as a set of components based on the support set. union

[0022] Calculate the component set based on the query set. With support class component set Similarity

[0023] The support class component matching probability is obtained by applying softmax operation on all component similarity values.

[0024] As a preferred technical solution, the method of fusing the global matching probability and the component matching probability to obtain the matching probability between the query set point cloud and the supporting class is as follows:

[0025] The matching probability between a query set of point clouds and its supporting classes is defined as the global matching probability of the supporting classes. Matching probability of components in the supporting class weighted value p t,n :

[0026]

[0027] Where w is the weight of the component matching probability.

[0028] As a preferred technical solution, the method of using the cross-entropy loss function as the objective function and iteratively training the point cloud object recognition network model parameters using stochastic gradient descent until convergence is as follows:

[0029] The objective function loss for training a point cloud object recognition network model is:

[0030]

[0031] in The model represents the samples The predicted value, express The true category label;

[0032] Calculate the gradient of the loss with respect to the parameters of the point cloud object recognition network model.

[0033] Secondly, the present invention provides a small sample point cloud object recognition system based on global and component matching, which is applied to the small sample point cloud object recognition method based on global and component matching, including a dataset construction module, a task construction module, a model construction module, a model training module and a task execution module;

[0034] The dataset construction module is used to construct a few-shot point cloud object recognition dataset, which includes a base class dataset and a new class dataset. The base class dataset is used to construct a few-shot point cloud object recognition task and update network parameters in a scenario training mode. The new class dataset is used to construct a few-shot point cloud object recognition task and verify the few-shot point cloud object recognition performance of the model.

[0035] The task construction module is used to construct a small sample point cloud object recognition task using a small sample point cloud object recognition dataset. The small sample point cloud object recognition task includes a base class small sample point cloud object recognition task and a new class small sample point cloud object recognition task. Each task includes a support set and a query set. The module learns to recognize the point cloud in the query set using the information in the support set.

[0036] The model building module is used to construct a point cloud object recognition network model, which includes a point cloud feature extractor, a point cloud global matching module, a point cloud component matching module, and a matching fusion module. The point cloud feature extractor is used to extract features from point clouds in the support set and the query set. The point cloud global matching module is used to calculate the global matching probability between point clouds in the query set and the support class. The point cloud component matching module is used to calculate the component matching probability between point clouds in the query set and the support class. The matching fusion module fuses the global matching probability and the component matching probability to obtain the matching probability between point clouds in the query set and the support class.

[0037] The model training module is used to train a point cloud object recognition network model using a base class small sample point cloud object recognition task. It uses the cross-entropy loss function as the objective function and uses the stochastic gradient descent method to iteratively train the parameters of the point cloud object recognition network model until convergence, thus obtaining the trained point cloud object recognition network model.

[0038] The task execution module is used to verify the performance of the point cloud object recognition network model using a new type of small sample point cloud object recognition task. It inputs the trained point cloud object recognition network model and obtains the prediction results.

[0039] Thirdly, the present invention provides an electronic device, the electronic device comprising:

[0040] At least one processor; and,

[0041] A memory communicatively connected to the at least one processor; wherein,

[0042] The memory stores computer program instructions that can be executed by the at least one processor, which enables the at least one processor to perform the small sample point cloud object recognition method based on global and component matching.

[0043] Fourthly, the present invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the small sample point cloud object recognition method based on global and component matching.

[0044] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0045] (1) In view of the problem that the existing point cloud object recognition model has poor performance under the condition of insufficient data, this invention proposes a small sample point cloud object recognition method based on global and component matching. By performing similarity matching on point clouds at both the global and component levels, the method can achieve accurate recognition of small sample point cloud objects.

[0046] (2) This invention uses the support set and query set provided by the base class small sample point cloud object recognition task as input to iteratively train the point cloud object recognition network model, and uses the support set and query set provided by the new class small sample point cloud object recognition task to verify the model performance and obtain the prediction results. This solves the problem that point cloud data is complex and difficult to collect, and that it is costly to obtain a large amount of point cloud data of the target category. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of a small-sample point cloud object recognition method based on global and component matching according to an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of global point cloud matching in an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of point cloud component matching in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram illustrating the iterative training of a small-sample point cloud object recognition model based on global and component matching, according to an embodiment of the present invention.

[0052] Figure 5 This is a schematic diagram of the structure of a small-sample point cloud object recognition system based on global and component matching according to an embodiment of the present invention;

[0053] Figure 6 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0054] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0055] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0056] Please see Figure 1 This embodiment uses a small-sample point cloud object recognition method based on global and component matching, which includes the following steps:

[0057] S1. Construct a small sample point cloud object recognition dataset, which includes a base class dataset and a new class dataset;

[0058] Optionally, the base class dataset is used to construct a few-sample point cloud object recognition task and update network parameters in a scenario training mode;

[0059] Optionally, the new class dataset is used to construct a small-sample point cloud object recognition task to verify the small-sample point cloud object recognition performance of the model.

[0060] S2. The small sample point cloud object recognition task includes a base class small sample point cloud object recognition task and a new class small sample point cloud object recognition task. Each task includes a support set and a query set. The information of the support set is used to learn the recognition of the query set point cloud.

[0061] Furthermore, the task of small sample point cloud object recognition lies in providing a support set S and a query set Q;

[0062] In an "N-way-K-shot" task, support sets Where N represents the number of point cloud categories in the support set, K represents the number of labeled samples in each category, n represents the index of the point cloud category in the support set, and k represents the index of the labeled samples in each category;

[0063] query set Where T represents the number of samples to be predicted, and t represents the index of the sample to be predicted;

[0064] The point cloud object recognition network model quickly learns to recognize N categories based on the information provided by the support set S, thereby making accurate predictions for samples in the query set Q.

[0065] S3. Construct a point cloud object recognition network model, which includes a point cloud feature extractor, a point cloud global matching module, a point cloud component matching module, and a matching fusion module;

[0066] Optionally, the feature extractor is used to extract features from the point clouds in the support set and the query set;

[0067] Furthermore, the feature extraction method for the point cloud in the support set and query set specifically includes:

[0068] S301. The point cloud feature extractor uses the DGCNN network;

[0069] S302 will support set-based queries of any point cloud within a set. Input the feature extractor to obtain the point features of the point cloud. Where x′ i Let f represent the i-th point in the point cloud X′, where i represents the index of the point in the point cloud, M is the number of points, F is the set of features of all points, and f is the index of the points in the point cloud. i ∈R 1×d Represents the i-th point x′ i d-dimensional features;

[0070] S303. Support for centralized point cloud features is obtained. and querying centralized point cloud features

[0071] Optionally, the global matching module calculates the global matching probability between the point cloud in the query set and the supporting classes;

[0072] Furthermore, the calculation of the global matching probability between the point cloud in the query set and the supporting classes, such as... Figure 2 As shown, specifically:

[0073] S304. Perform global max pooling on each point cloud in the support set and query set to obtain the global feature vector of each point cloud, denoted as: v = GMP(F);

[0074] S305. Calculate the average feature vector of the point clouds of all classes in the support set as the prototype feature vector of the support set:

[0075]

[0076]

[0077] S306. Calculate the cosine similarity between each point cloud in the query set and each point cloud in the support set to obtain the global similarity.

[0078]

[0079] in, represents the feature vector of each point cloud in the query set, and g represents the scaling factor;

[0080] S307. Perform a softmax operation on all global similarity values ​​to obtain the global matching probability between the point cloud in the query set and the supporting classes.

[0081]

[0082] in, Represents class c j The prototype feature vector, where j represents the index of the supporting class;

[0083] Optionally, the component matching module calculates the matching probability between the query set point cloud and the components of the supporting class;

[0084] Furthermore, the component matching module calculates the matching probability between the query set point cloud and components of the supporting class, such as... Figure 3 As shown, specifically:

[0085] S308. Divide any point cloud into L parts. Select L center points for any point cloud using the farthest point sampling method. Each center point and its (H-1) nearest neighbors constitute a part. Define the feature vector of any part as the average feature vector of the center point and all its neighbors, and represent any point cloud as a set of parts Z:

[0086]

[0087] Among them, z l This represents the feature vector of the l-th component;

[0088] S309. Define the support class component set as a set of components based on the support set. union

[0089]

[0090] S310. Calculate the set of components based on the query set. With support class component set Similarity

[0091]

[0092] Among them, z a and z b express and Z cn A feature of a component in a component set, where g is a scaling factor;

[0093] S311. Use the softmax operation on all component similarity values ​​to obtain the support class component matching probability.

[0094]

[0095] in, This represents the component matching probability output by the model;

[0096] Optionally, the matching fusion module fuses the global matching probability and the component matching probability to obtain the matching probability of the query set point cloud and the supporting class;

[0097] Furthermore, the fusion of global matching probability and component matching probability yields the matching probability between the query set point cloud and the supporting class, specifically as follows:

[0098] S312. The matching probability between a query set of point clouds and its supporting classes is defined as the global matching probability of the supporting classes. Matching probability of components in the supporting class weighted value p t,n :

[0099]

[0100] Where w is the weight of the component matching probability.

[0101] S4. Train the point cloud object recognition network model using the base class small sample point cloud object recognition task, use the cross-entropy loss function as the objective function, and use the stochastic gradient descent method to iteratively train the point cloud object recognition network model parameters until convergence, and obtain the trained point cloud object recognition network model.

[0102] Preferred, such as Figure 4 As shown, step S4 further includes the following steps:

[0103] S401. Input the support set and query set provided by the base class small sample point cloud object recognition task;

[0104] S402. Use a feature extractor to extract feature vectors from the point clouds in the support set and query set;

[0105] S403. Based on the feature vectors of the extracted support set and the point cloud in the query set, calculate the global matching probability between the point cloud in the query set and the support class;

[0106] S404. Calculate the matching probability between the query set point cloud and the components of the supporting class;

[0107] S405. By weighted fusion of the global matching probability of the supporting class and the component matching probability of the supporting class, the matching probability of the query set point cloud and the supporting class is obtained;

[0108] S406. Using the cross-entropy loss function as the objective function, the gradient of the point cloud object recognition network model parameters is obtained based on the calculation results, and then the stochastic gradient descent method is used to update the point cloud object recognition network model parameters.

[0109] S407. Iterate through the above steps until the parameters of the point cloud object recognition network model converge and the training is complete;

[0110] Furthermore, the objective function loss for training the point cloud object recognition network model is:

[0111]

[0112] in, The model represents the samples The predicted value, express The true category label;

[0113] Calculate the gradient of the loss with respect to the parameters of the point cloud object recognition network model;

[0114] Optionally, during iterative training, the number of points in the point cloud is 1024, the dimension of all feature vectors is 512, the scaling factor is 16, the number of parts is 64, the number of points within a part is 32, and the part matching weight is 0.8. The point cloud object recognition network model uses Adam as the optimizer, and the training is conducted for a total of 100 rounds. The learning rate is 0.001, and it decays to 50% of the current learning rate every 20 rounds.

[0115] S5. Verify the performance of the point cloud object recognition network model using a novel few-sample point cloud object recognition task. Input the trained point cloud object recognition network model to obtain the prediction results.

[0116] Preferably, step S5 specifically includes:

[0117] S501. Deploy the trained point cloud object recognition network model;

[0118] S502. Input the support set and query set provided by the new class of small sample point cloud object recognition task, and calculate the prediction result;

[0119] Furthermore, the point cloud object recognition network model can accurately identify point cloud objects with only 1 or 5 available support samples, and has achieved good performance on different datasets (ModelNet, ShapeNet).

[0120] Based on the same idea as the small-sample point cloud object recognition method based on global and component matching in the above embodiments, the present invention also provides a small-sample point cloud object recognition system based on global and component matching. This system can be used to execute the above-described small-sample point cloud object recognition method based on global and component matching. For ease of explanation, the structural diagram of the embodiment of the small-sample point cloud object recognition system based on global and component matching only shows the parts related to the embodiments of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0121] Please see Figure 5 In another embodiment of this application, a small sample point cloud object recognition system 10 based on global and component matching is provided. The system includes a dataset construction module 11, a task construction module 12, a model construction module 13, a model training module 14, and a task execution module 15.

[0122] The dataset construction module 11 is used to construct a few-sample point cloud object recognition dataset, which includes a base class dataset and a new class dataset. The base class dataset is used to construct a few-sample point cloud object recognition task to update network parameters in a scenario training mode. The new class dataset is used to construct a few-sample point cloud object recognition task to verify the few-sample point cloud object recognition performance of the model.

[0123] The task construction module 12 constructs a small sample point cloud object recognition task using a small sample point cloud object recognition dataset. The small sample point cloud object recognition task includes a base class small sample point cloud object recognition task and a new class small sample point cloud object recognition task. Each task includes a support set and a query set. The module learns to recognize the point cloud in the query set using the information in the support set.

[0124] The model building module 13 is used to build a point cloud object recognition network model, which includes a point cloud feature extractor, a point cloud global matching module, a point cloud component matching module, and a matching fusion module. The point cloud feature extractor is used to extract features of point clouds in the support set and the query set. The point cloud global matching module is used to calculate the global matching probability between point clouds in the query set and the support class. The point cloud component matching module is used to calculate the component matching probability between point clouds in the query set and the support class. The matching fusion module fuses the global matching probability and the component matching probability to obtain the matching probability between point clouds in the query set and the support class.

[0125] The model training module 14 is used to train a point cloud object recognition network model using a base class small sample point cloud object recognition task. It uses the cross-entropy loss function as the objective function and uses the stochastic gradient descent method to iteratively train the parameters of the point cloud object recognition network model until convergence, thus obtaining the trained point cloud object recognition network model.

[0126] The task execution module 15 is used to verify the performance of the point cloud object recognition network model using a new type of small sample point cloud object recognition task, by inputting the trained point cloud object recognition network model and obtaining the prediction result.

[0127] It should be noted that the small sample point cloud object recognition system based on global and component matching of the present invention corresponds one-to-one with the small sample point cloud object recognition method based on global and component matching of the present invention. The technical features and beneficial effects described in the embodiments of the small sample point cloud object recognition method based on global and component matching are applicable to the embodiments of small sample point cloud object recognition based on global and component matching. For details, please refer to the description in the embodiments of the method of the present invention, which will not be repeated here.

[0128] Furthermore, in the above embodiments of the small sample point cloud object recognition system based on global and component matching, the logical division of each program module is only an example. In actual applications, the above functions can be assigned to different program modules as needed, for example, for the sake of corresponding hardware configuration requirements or software implementation convenience. That is, the internal structure of the small sample point cloud object recognition system based on global and component matching is divided into different program modules to complete all or part of the functions described above.

[0129] Please see Figure 6 In one embodiment, an electronic device is provided for implementing a small sample point cloud object recognition method based on global and component matching. The electronic device 20 may include a first processor 21, a first memory 22 and a bus, and may also include a computer program stored in the first memory 22 and executable on the first processor 21, such as a small sample point cloud object recognition program 23 based on global and component matching.

[0130] The first memory 22 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the first memory 22 can be an internal storage unit of the electronic device 20, such as the portable hard drive of the electronic device 20. In other embodiments, the first memory 22 can also be an external storage device of the electronic device 20, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 20. Furthermore, the first memory 22 can include both internal storage units and external storage devices of the electronic device 20. The first memory 22 can be used not only to store application software and various types of data installed on the electronic device 20, such as the code of a small sample point cloud object recognition program 23 based on global and component matching, but also to temporarily store data that has been output or will be output.

[0131] In some embodiments, the first processor 21 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The first processor 21 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the first memory 22 and calls data stored in the first memory 22 to perform various functions of the electronic device 20 and process data.

[0132] Figure 6 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 6 The structure shown does not constitute a limitation on the electronic device 20, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0133] The object recognition program 23 based on global and component matching, stored in the first memory 22 of the electronic device 20, is a combination of multiple instructions. When run in the first processor 21, it can achieve the following:

[0134] A few-shot point cloud object recognition dataset is constructed, comprising a base class dataset and a new class dataset. The base class dataset is used to construct a few-shot point cloud object recognition task and update network parameters in a context-based training mode. The new class dataset is used to construct a few-shot point cloud object recognition task and verify the few-shot point cloud object recognition performance of the model.

[0135] A few-shot point cloud object recognition task is constructed using a few-shot point cloud object recognition dataset. The few-shot point cloud object recognition task includes a base class few-shot point cloud object recognition task and a new class few-shot point cloud object recognition task. Each task includes a support set and a query set. The recognition of the query set point cloud is learned using the information of the support set.

[0136] A point cloud object recognition network model is constructed, comprising a point cloud feature extractor, a point cloud global matching module, a point cloud component matching module, and a matching fusion module. The point cloud feature extractor extracts features from point clouds in the support set and the query set. The point cloud global matching module calculates the global matching probability between point clouds in the query set and the support class. The point cloud component matching module calculates the component matching probability between point clouds in the query set and the support class. The matching fusion module fuses the global matching probability and the component matching probability to obtain the matching probability between point clouds in the query set and the support class.

[0137] A point cloud object recognition network model is trained using a base class small sample point cloud object recognition task. The cross-entropy loss function is used as the objective function, and the parameters of the point cloud object recognition network model are iteratively trained using the stochastic gradient descent method until convergence, thus obtaining the trained point cloud object recognition network model.

[0138] The performance of the point cloud object recognition network model was verified using a novel few-sample point cloud object recognition task. The prediction results were obtained by inputting the trained point cloud object recognition network model.

[0139] Furthermore, if the modules / units integrated in the electronic device 20 are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0140] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0141] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for object recognition from small-sample point clouds based on global and component matching, characterized in that, Includes the following steps: A few-shot point cloud object recognition dataset is constructed, comprising a base class dataset and a new class dataset. The base class dataset is used to construct a few-shot point cloud object recognition task and update network parameters in a context-based training mode. The new class dataset is used to construct a few-shot point cloud object recognition task and verify the few-shot point cloud object recognition performance of the model. A few-shot point cloud object recognition task is constructed using a few-shot point cloud object recognition dataset. The few-shot point cloud object recognition task includes a base class few-shot point cloud object recognition task and a new class few-shot point cloud object recognition task. Each task includes a support set and a query set. The recognition of the query set point cloud is learned using the information of the support set. A point cloud object recognition network model is constructed, comprising a point cloud feature extractor, a point cloud global matching module, a point cloud component matching module, and a matching fusion module. The point cloud feature extractor extracts features from point clouds in the support set and the query set. The point cloud global matching module calculates the global matching probability between point clouds in the query set and the support class. The point cloud component matching module calculates the component matching probability between point clouds in the query set and the support class. The matching fusion module fuses the global matching probability and the component matching probability to obtain the matching probability between point clouds in the query set and the support class. A point cloud object recognition network model is trained using a base class small sample point cloud object recognition task. The cross-entropy loss function is used as the objective function, and the parameters of the point cloud object recognition network model are iteratively trained using the stochastic gradient descent method until convergence, thus obtaining the trained point cloud object recognition network model. The performance of the point cloud object recognition network model was verified using a novel few-sample point cloud object recognition task. The prediction results were obtained by inputting the trained point cloud object recognition network model. The specific steps for constructing the small sample point cloud object recognition task are as follows: The small sample point cloud object recognition task includes a support set S and a query set Q. The support set S includes N target categories and K labeled samples for each target category. The query set Q includes T samples to be predicted. The point cloud object recognition network model learns to recognize N target categories based on the information provided by the support set S, thereby making accurate predictions for the T samples to be predicted in the query set Q. The calculation of the global matching probability between the point cloud in the query set and the supporting classes is specifically as follows: Perform global max pooling on each point cloud in the support set and query set to obtain the global feature vector of each point cloud; Calculate the average feature vector of point clouds of all categories in the support set as the prototype feature vector of the support set; For each point cloud in the query set, the cosine similarity between the point cloud in the query set and the point cloud in the support set is calculated based on the global feature vector and the prototype feature vector as the global similarity. Perform softmax operation on all global similarities to obtain the global matching probability between the point cloud in the query set and the supporting class; The calculation of the matching probability between the point cloud in the query set and the components of the supported class is specifically as follows: Each point cloud is divided into L parts. For any point cloud, the farthest point sampling method is used to select L center points. Each center point and its (H-1) nearest neighbor points of any point cloud constitute a part. The feature vector of the part is defined as the average feature vector of the center point and all neighbor points. Each point cloud is represented as a set of parts Z. The set of support class components is defined as a set of components based on the support set. union Where n represents the index of the supported centralized point cloud categories, and k represents the index of the labeled samples for each category; Calculate the component set based on the query set. With support class component set Similarity t represents the index of the sample to be predicted; The support class component matching probability is obtained by applying softmax operation on all component similarity values.

2. The method for small-sample point cloud object recognition based on global and component matching according to claim 1, characterized in that, The point cloud feature extractor uses a DGCNN network. The support set and query set share the same point cloud feature extractor. Point clouds from both the support set and query set are input into the feature extractor to obtain point cloud features from the support set. and querying centralized point cloud features 3. The method for small-sample point cloud object recognition based on global and component matching according to claim 1, characterized in that, The fusion of global matching probability and component matching probability yields the matching probability between the query set point cloud and the supporting class, specifically as follows: The matching probability between a query set of point clouds and its supporting classes is defined as the global matching probability of the supporting classes. Matching probability of components in the supporting class weighted value p t,n : Where w is the weight of the component matching probability.

4. The method for small-sample point cloud object recognition based on global and component matching according to claim 1, characterized in that, The method employs the cross-entropy loss function as the objective function and uses stochastic gradient descent to iteratively train the point cloud object recognition network model parameters until convergence. Specifically: The objective function loss for training a point cloud object recognition network model is: in The model represents the samples The predicted value, express The true category label; Calculate the gradient of the loss with respect to the parameters of the point cloud object recognition network model.

5. A small-sample point cloud object recognition system based on global and component matching, characterized in that, The method for small-sample point cloud object recognition based on global and component matching, as described in any one of claims 1-4, includes a dataset construction module, a task construction module, a model construction module, a model training module, and a task execution module. The dataset construction module is used to construct a few-shot point cloud object recognition dataset, which includes a base class dataset and a new class dataset. The base class dataset is used to construct a few-shot point cloud object recognition task and update network parameters in a scenario training mode. The new class dataset is used to construct a few-shot point cloud object recognition task and verify the few-shot point cloud object recognition performance of the model. The task construction module is used to construct a small sample point cloud object recognition task using a small sample point cloud object recognition dataset. The small sample point cloud object recognition task includes a base class small sample point cloud object recognition task and a new class small sample point cloud object recognition task. Each task includes a support set and a query set. The module learns to recognize the point cloud in the query set using the information in the support set. The model building module is used to construct a point cloud object recognition network model, which includes a point cloud feature extractor, a point cloud global matching module, a point cloud component matching module, and a matching fusion module. The point cloud feature extractor is used to extract features from point clouds in the support set and the query set. The point cloud global matching module is used to calculate the global matching probability between point clouds in the query set and the support class. The point cloud component matching module is used to calculate the component matching probability between point clouds in the query set and the support class. The matching fusion module fuses the global matching probability and the component matching probability to obtain the matching probability between point clouds in the query set and the support class. The model training module is used to train a point cloud object recognition network model using a base class small sample point cloud object recognition task. It uses the cross-entropy loss function as the objective function and uses the stochastic gradient descent method to iteratively train the parameters of the point cloud object recognition network model until convergence, thus obtaining the trained point cloud object recognition network model. The task execution module is used to verify the performance of the point cloud object recognition network model using a new type of small sample point cloud object recognition task. It inputs the trained point cloud object recognition network model and obtains the prediction results.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the small sample point cloud object recognition method based on global and component matching as described in any one of claims 1-4.

7. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the small sample point cloud object recognition method based on global and component matching as described in any one of claims 1-4.

Images