Point cloud classification model construction method, point cloud classification method, device and equipment

Abstract

The application discloses a point cloud classification model construction method, a point cloud classification method, a device and equipment, relates to the technical field of data processing, and is used for solving the problems of complex point cloud model structure and low model reasoning speed in the prior art. The application comprises the following steps: acquiring point cloud sample data, starting model training, obtaining an initial training model, extracting the weight of the initial training model, fusing the weight of a one-dimensional convolution layer and an activation layer in the initial training model by using a structure reparameterization technology to obtain a target one-dimensional convolution layer, fusing the weight of a linear layer and an activation layer by using the structure reparameterization technology to obtain a target linear layer, and constructing a target point cloud classification model without branches based on the target one-dimensional convolution layer and the target linear layer. The model is fused by weight to be a simple structure network without branches and composed of the one-dimensional convolution layer and the target linear layer, so that the model complexity is reduced, and the model reasoning speed is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method for constructing a point cloud classification model, a point cloud classification method, an apparatus, and equipment. Background Technology

[0002] Deep learning-based point cloud classification models have developed rapidly and are gradually being applied to various fields, such as face recognition, object recognition in robots, and obstacle detection in autonomous driving. With the widespread availability and affordability of LiDAR, point clouds are beginning to be widely used in autonomous driving, robotics, and AR / VR. As a primary representation of information in the three-dimensional world, point clouds are currently a hot research topic in 3D vision, with research directions including point cloud classification, point cloud segmentation, and point cloud detection. Among these, point cloud classification, as a fundamental task in 3D point cloud analysis, has very wide applications in detection and recognition within relevant scenarios.

[0003] Currently, the most commonly used point cloud processing technology is based on deep learning, which has achieved significant success in fields such as classification and detection. However, compared to traditional technologies, deep learning has certain computational requirements, and its performance is limited in scenarios with insufficient computing resources. Furthermore, fields such as autonomous driving and AR have extremely stringent real-time requirements for algorithm models, posing challenges to algorithm optimization and acceleration. Complex model structures result in slower inference speeds, and existing fusion techniques cannot integrate large layers with residual connections like those in ResNet. Summary of the Invention

[0004] The purpose of this invention is to provide a point cloud classification model construction method, point cloud classification method, apparatus and equipment to solve the problems of complex point cloud model structure and low model inference speed in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] In a first aspect, the present invention provides a method for constructing a point cloud classification model, including:

[0007] Obtain point cloud sample data, build a model and start training to obtain an initial training model; the initial training model includes a residual layer, a one-dimensional convolutional layer, a linear layer and an activation layer;

[0008] Extract the weights of the initial trained model;

[0009] Using structural reparameterization, the weights of the one-dimensional convolutional layer and the activation layer in the initial training model are fused to obtain the target one-dimensional convolutional layer.

[0010] Based on the weights, the linear layer and the activation layer are fused using the structural reparameterization technique to obtain the target linear layer;

[0011] Based on the target one-dimensional convolutional layer and the target linear layer, a branchless target point cloud classification model is constructed.

[0012] Secondly, the present invention provides a point cloud classification method, including:

[0013] Obtain the point cloud data to be classified;

[0014] Extract the feature vector of the point cloud data to be classified;

[0015] The feature vector is input into the target point cloud classification model to obtain the classification result. The target point cloud classification model is a branchless network constructed based on a target one-dimensional convolutional layer and a target linear layer. The target one-dimensional convolutional layer is obtained by extracting the weights of the initial training model and fusing the weights of the one-dimensional convolutional layer and activation layer in the initial training model using the structural reparameterization technique. The target linear layer is obtained by fusing the weights of the linear layer and activation layer in the initial training model using the structural reparameterization technique based on the weights.

[0016] Thirdly, the present invention provides a point cloud classification model construction apparatus, comprising:

[0017] The initial training model building module is used to acquire point cloud sample data, build the model and start training to obtain the initial training model; the initial training model includes residual layers, one-dimensional convolutional layers, linear layers and activation layers.

[0018] The weight extraction module is used to extract the weights of the initial training model;

[0019] The first weight fusion module is used to fuse the weights of the one-dimensional convolutional layer and the activation layer in the initial training model using structural reparameterization technology to obtain the target one-dimensional convolutional layer.

[0020] The second weight fusion module is used to fuse the linear layer and the activation layer according to the weights and using the structure reparameterization technique to obtain the target linear layer.

[0021] The target point cloud classification model construction module is used to construct a branchless target point cloud classification model based on the target one-dimensional convolutional layer and the target linear layer.

[0022] Fourthly, the present invention provides a point cloud classification device, comprising:

[0023] The point cloud data acquisition module is used to acquire point cloud data to be classified.

[0024] The feature vector extraction module is used to extract the feature vectors of the point cloud data to be classified.

[0025] The classification module is used to input the feature vector into the target point cloud classification model to obtain the classification result. The target point cloud classification model is a branchless network constructed based on a target one-dimensional convolutional layer and a target linear layer. The target one-dimensional convolutional layer is obtained by extracting the weights of the initial training model and fusing the weights of the one-dimensional convolutional layer and activation layer in the initial training model using the structural reparameterization technique. The target linear layer is obtained by fusing the weights of the linear layer and activation layer in the initial training model using the structural reparameterization technique, based on the weights.

[0026] Fifthly, the present invention provides a point cloud classification model construction method and apparatus, comprising:

[0027] A communication unit / interface is used to acquire point cloud sample data, build a model to start training, and obtain an initial training model; the initial training model includes a residual layer, a one-dimensional convolutional layer, a linear layer, and an activation layer.

[0028] A processing unit / processor for extracting the weights of the initial trained model;

[0029] Using structural reparameterization, the weights of the one-dimensional convolutional layer and the activation layer in the initial training model are fused to obtain the target one-dimensional convolutional layer.

[0030] Based on the weights, the linear layer and the activation layer are fused using the structural reparameterization technique to obtain the target linear layer;

[0031] Based on the target one-dimensional convolutional layer and the target linear layer, a branchless target point cloud classification model is constructed.

[0032] Sixthly, the present invention provides a point cloud classification device, comprising:

[0033] Communication unit / communication interface, used to acquire point cloud data to be classified;

[0034] A processing unit / processor is used to extract the feature vector of the point cloud data to be classified;

[0035] The feature vector is input into the target point cloud classification model to obtain the classification result. The target point cloud classification model is a branchless network constructed based on a target one-dimensional convolutional layer and a target linear layer. The target one-dimensional convolutional layer is obtained by extracting the weights of the initial training model and fusing the weights of the one-dimensional convolutional layer and activation layer in the initial training model using the structural reparameterization technique. The target linear layer is obtained by fusing the weights of the linear layer and activation layer in the initial training model using the structural reparameterization technique based on the weights.

[0036] Compared with existing technologies, this invention provides a method for constructing a point cloud classification model, a point cloud classification method, an apparatus, and a device. The point cloud classification model construction method involves acquiring point cloud sample data, building a model for training, and obtaining an initial training model including residual layers, one-dimensional convolutional layers, linear layers, and activation layers. Weights of the initial training model are extracted; and using structural reparameterization (SRP) technology, the weights of the one-dimensional convolutional layers and activation layers in the initial training model are fused to obtain a target one-dimensional convolutional layer. Based on the weights, the weights of the linear layers and activation layers are fused using SRP technology to obtain a target linear layer. Based on the target one-dimensional convolutional layer and the target linear layer, a branchless target point cloud classification model is constructed. During the training phase, the model has a multi-branch residual structure and a large number of batch normalization (BN) layers, resulting in good representation capabilities and easily achieving competitive performance after training. During the inference phase, the model undergoes weight fusion, transforming it into a simple branchless network structure primarily composed of one-dimensional convolutional layers and the target linear layer, thereby reducing model complexity and improving inference speed.

[0037] The point cloud classification method uses a constructed target point cloud classification model to classify point clouds, which can improve classification efficiency. Attached Figure Description

[0038] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0039] Figure 1 A flowchart illustrating the point cloud classification model construction method provided by this invention;

[0040] Figure 2 A schematic diagram illustrating the principle of weight fusion based on structural reparameterization in the point cloud classification model construction method provided by this invention;

[0041] Figure 3 A flowchart illustrating the point cloud classification method provided by this invention;

[0042] Figure 4 This is a schematic diagram of the structure of the point cloud classification model construction device provided by the present invention;

[0043] Figure 5 This is a schematic diagram of the structure of the point cloud classification device provided by the present invention;

[0044] Figure 6 This is a schematic diagram of the electronic device structure provided by the present invention. Detailed Implementation

[0045] To facilitate a clear description of the technical solutions in the embodiments of the present invention, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, the first threshold and the second threshold are merely used to distinguish different thresholds and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0046] It should be noted that in this invention, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0047] In this invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a combination of a and b, a combination of a and c, a combination of b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0048] Currently, the most commonly used point cloud processing technology is based on deep learning, which has achieved significant success in fields such as classification and detection. However, compared to traditional technologies, deep learning has certain computational requirements, and its performance is limited in scenarios with insufficient computing resources. Furthermore, fields such as autonomous driving and AR have extremely stringent real-time requirements for algorithm models, posing challenges to algorithm optimization and acceleration. Complex model structures result in slower inference speeds, and existing fusion techniques cannot integrate large layers with residual connections like those in ResNet.

[0049] In the prior art, Chinese patent (CN114611700A) proposes a structure reparameterization based on Conv2D operators. Its basic idea is to fuse Conv2D_1x1 branches and Conv2D_3x3 branches, solving the problem that existing fusion techniques cannot fuse large layers with residual connections like those in ResNet. However, it is limited to Conv2D operators, while the classic point cloud processing algorithm PointNet uses a large number of Conv1D operators.

[0050] In response, this invention provides a structure reparameterization technique suitable for Conv1D operators, which can be easily extended to the point cloud domain.

[0051] Next, the solutions provided in the embodiments of this specification will be described in conjunction with the accompanying drawings: Example

[0052] Figure 1 This is a flowchart illustrating the point cloud classification model construction method provided by the present invention, as shown below. Figure 1 As shown, the process may include the following steps:

[0053] Step 110: Obtain point cloud sample data, build a model and start training to obtain an initial training model; the initial training model includes a residual layer, a one-dimensional convolutional layer, a linear layer and an activation layer.

[0054] Point cloud data refers to a collection of vectors in a three-dimensional coordinate system. Besides geometric location, point cloud data sometimes also contains color information. Color information is typically obtained by acquiring color images from a camera and then assigning the color information (RGB) of the corresponding pixels to the corresponding points in the point cloud. Intensity information is obtained from the echo intensity collected by a laser scanner receiver. This intensity information is related to the target's surface material, roughness, incident angle, and the instrument's emission energy and laser wavelength.

[0055] The initial training model can include at least residual layers, one-dimensional convolutional layers, linear layers, and activation layers. One-dimensional convolution (Conv1D) involves sliding a window along the width or height direction and multiplying the results. Since a sentence sequence in natural language processing is one-dimensional, Conv1D is used; in this case, the convolution kernel, excluding the channel, is also one-dimensional.

[0056] Convolutional neural networks (CNNs) are generally viewed as having two parts: a feature extraction layer and a task-related layer. The feature extraction layer consists of a series of interconnected or parallel network layers such as Conv, ReLU, and Pool, ultimately producing a feature map. Task-related layers, for example, use fully connected layers to perform regression tasks on the obtained feature maps, fitting distributions, etc. In image classification, fully connected layers are often used to output the probability of each class, but a fully connected layer is also described as a linear transformation layer + activation function + linear transformation layer + ..., or a multilayer perceptron. When the input features are received by a linear layer, they are received as a flattened one-dimensional tensor, and then multiplied by a weight matrix. This matrix multiplication produces the output features.

[0057] Like activation function layers, convolutional layers, fully connected layers, and pooling layers, the activation layer (Batch Normalization, BN) is also a layer in the network. At the input of each layer, a normalization layer is inserted, which performs a normalization process (normalizing to a mean of 0 and a variance of 1) before the data enters the next layer. BN normalizes the same feature across different samples. The main functions of the BN layer are: accelerating network training and convergence, controlling gradient explosion to prevent gradient vanishing, and preventing overfitting.

[0058] While the initial trained model obtained in step 110 exhibits good performance, its complexity and computational cost limit the model's inference speed to some extent. Therefore, the solution described in steps 120-150 is adopted to address this issue.

[0059] Step 120: Extract the weights of the initial training model.

[0060] The model weights of the initial trained model are extracted and used for subsequent weight fusion.

[0061] Step 130: Using structural reparameterization technology, the weights of the one-dimensional convolutional layer and the activation layer in the initial training model are fused to obtain the target one-dimensional convolutional layer.

[0062] Structural re-parameterization refers to first constructing a set of structures for training, and then equivalently converting their parameters into another set of parameters for inference or deployment, thus transforming this set of structures into another set of structures. The training structure corresponds to one set of parameters, and the inference structure corresponds to another set of parameters; as long as the parameters of the former can be equivalently converted to the latter, the structure of the former can also be equivalently converted to the latter. Structure A corresponds to a set of parameters X, and structure B corresponds to a set of parameters Y. If we can equivalently convert X to Y, then structure A can be equivalently converted to B.

[0063] Step 140: Based on the weights, use the structure reparameterization technique to fuse the weights of the linear layer and the activation layer to obtain the target linear layer.

[0064] Step 150: Based on the target one-dimensional convolutional layer and the target linear layer, construct a branchless target point cloud classification model.

[0065] Figure 1 The proposed method for constructing a point cloud classification model involves acquiring point cloud sample data, building a model for training, and obtaining an initial training model including residual layers, one-dimensional convolutional layers, linear layers, and activation layers. Weights of the initial training model are extracted. Using structural reparameterization (SRP), the weights of the one-dimensional convolutional layers and activation layers in the initial training model are fused to obtain a target one-dimensional convolutional layer. Based on the weights, the weights of the linear layers and activation layers are fused using SRP again to obtain a target linear layer. Based on the target one-dimensional convolutional layer and the target linear layer, a branchless target point cloud classification model is constructed. During the training phase, the model features a multi-branch residual structure and numerous batch normalization (BN) layers, resulting in good representation capabilities and competitive performance after training. During the inference phase, the model undergoes weight fusion, transforming it into a simple branchless network primarily composed of one-dimensional convolutional layers and the target linear layer, thereby reducing model complexity and improving inference speed.

[0066] In other words, the method provided in this scheme has a multi-branch residual structure and a large number of BN layers during the training phase, and the model has good representation ability. After training, it is easy to achieve competitive performance. During the inference phase, the model is weighted and transformed into a simple structure network composed mainly of Conv1D, thereby reducing the model complexity and improving the model inference speed.

[0067] Figure 1 The implementation principle of the method in can be combined with Figure 2 Please provide an explanation. Figure 2 This is a schematic diagram illustrating the principle of weight fusion based on structure reparameterization in the point cloud classification model construction method provided by this invention. Figure 2 As shown, a point cloud classification model is built based on Conv1D layers, Linear layers, and BN layers. The model is trained using point cloud classification data, and the weights of the model are obtained after training. The weights of the Conv1D layer and the BN layer are fused and reparameterized into a single Conv1D layer. Similarly, the weights of the Linear layer and the BN layer are fused and reparameterized into a single Linear layer. After weight fusion, the model structure is simplified to a simple network structure mainly composed of Conv1D layers.

[0068] based on Figure 1 In addition to the method described herein, this specification also provides some specific implementation methods of this method, which will be described below.

[0069] Optionally, step 130 may specifically include:

[0070] Using structural reparameterization, the weights of the one-dimensional convolutional layer and the activation layer in the initial training model are fused; the branches of the one-dimensional convolutional layer and the activation layer are fused; wherein, the fusion residual layer of the initial training model is a Conv1D operator, which is reparameterized into a one-dimensional convolutional layer.

[0071] Optionally, step 140 may specifically include:

[0072] Based on the weights, the linear layer and the activation layer are weighted and fused using the structure reparameterization technique, and the BN operator is removed to obtain the target linear layer; wherein, the reparameterization is performed to form a single linear layer.

[0073] Furthermore, when performing weight fusion on the one-dimensional convolutional layer and the activation layer, the following implementation method can be adopted:

[0074] Formula (1) is used:

[0075] (1);

[0076] Perform Conv1D operation to fuse the weights of the one-dimensional convolutional layer and the activation layer in the initial trained model, wherein the input is... , Indicates Batch Size. Indicates the number of input feature channels. Indicates the number of input features. Indicates weight, Indicates bias;

[0077] Formula (2) is used:

[0078] (2);

[0079] Perform BN operations, where, This represents the scaling factor of the BN layer. This represents the mean of the BN layer. Indicates the variance of the BN layer. Let represent the offset coefficient of the BN layer, and then the calculation process is as shown in formula (3):

[0080]

[0081] (3);

[0082] make:

[0083] The expression for the target one-dimensional convolutional layer obtained by fusion is formula (4):

[0084] (4);

[0085] The weight fusion is completed through the above steps.

[0086] Furthermore, the one-dimensional convolutional layer branches and activation layer branches are fused, specifically as follows:

[0087] The calculation process for multi-branch residual layer operation is as follows:

[0088] ;

[0089]

[0090] (5);

[0091] make:

[0092] ;

[0093] The expression for the target one-dimensional convolutional layer obtained by fusion is formula (6):

[0094] (6);

[0095] The residual layer is fused using the Conv1D operator to complete the weight fusion.

[0096] After fusing the Conv1D layer weights and the BN layer weights, it is also necessary to fuse the weights of the linear layer and the activation layer, remove the BN operator, and obtain the target linear layer. Specifically, this may include:

[0097] Weight fusion is performed using formula (7) for the linear layer and the activation layer:

[0098]

[0099] ;

[0100] (7);

[0101] The input is , Indicates Batch Size. Indicates the number of categories. Represents the weights of the linear layer. Indicates linear layer bias; let:

[0102] ;

[0103] The target linear layer obtained by fusion is shown in formula (8):

[0104] (8);

[0105] Finally, the model weights are reparameterized using the above structure, significantly reducing model complexity. The model is transformed into a simple, branchless, cylindrical network composed of Conv1D, ReLU, and Linear algorithms, expressed as:

[0106] .

[0107] This approach was trained on the ModelNet40 dataset, and the final model achieved an accuracy of 89.3%. After structural reparameterization, the inference speed of the model was [missing information]. Although the accuracy performance still lags slightly behind current cutting-edge point cloud classification algorithms, its training convergence speed and inference speed are highly competitive compared to these algorithms, and it can achieve better performance under conditions of limited computing resources.

[0108] Through specific application comparisons, taking the V100 32G configuration as an example: the throughput of the existing model is 105 samples / s, and the accuracy is 89.2%; the throughput of the target classification model in this solution is 201 samples / s, and the accuracy is 89.3%.

[0109] Compared with existing technologies, the advantages of this invention are: the introduction of residual modules accelerates network convergence during training, with a convergence speed significantly superior to networks such as PointNet and PointNet++. A Conv1D-based structural reparameterization method is proposed to decouple the training and inference models, allowing complex training models to be transformed into simpler inference models through weight fusion while maintaining performance. The simplified model structure not only accelerates model inference speed but also greatly facilitates the underlying deployment of the model on various hardware platforms.

[0110] The method for constructing a point cloud classification model proposed in Embodiment 1 above can be applied to specific application scenarios for classification after training. This can be further explained in Embodiment 2: Example

[0111] Figure 3 This is a flowchart illustrating the point cloud classification method provided by the present invention, as shown below. Figure 3As shown, the process may include the following steps:

[0112] Step 310: Obtain the point cloud data to be classified.

[0113] Step 320: Extract the feature vector of the point cloud data to be classified.

[0114] Step 330: Input the feature vector into the target point cloud classification model to obtain the classification result; the target point cloud classification model is a branchless network constructed based on a target one-dimensional convolutional layer and a target linear layer; the target one-dimensional convolutional layer is obtained by extracting the weights of the initial training model and using structural reparameterization technology to fuse the weights of the one-dimensional convolutional layer and activation layer in the initial training model; the target linear layer is obtained by fusing the weights of the linear layer and activation layer in the initial training model according to the weights and using the structural reparameterization technology.

[0115] Example 2 mainly uses the target classification model from Example 1 to complete the identification and classification. The corresponding explanations are the same as in Example 1, and will not be repeated here.

[0116] Based on the same idea, corresponding to Embodiment 1, the present invention also provides a point cloud classification model construction device. Figure 4 This is a schematic diagram of the point cloud classification model construction device provided by the present invention. Figure 4 As shown, the device may include:

[0117] The initial training model building module 410 is used to acquire point cloud sample data, build the model and start training to obtain the initial training model; the initial training model includes a residual layer, a one-dimensional convolutional layer, a linear layer and an activation layer.

[0118] The weight extraction module 420 is used to extract the weights of the initial training model;

[0119] The first weight fusion module 430 is used to fuse the weights of the one-dimensional convolutional layer and the activation layer in the initial training model using structural reparameterization technology to obtain the target one-dimensional convolutional layer.

[0120] The second weight fusion module 440 is used to fuse the linear layer and the activation layer according to the weights and using the structure reparameterization technique to obtain the target linear layer.

[0121] The target point cloud classification model construction module 450 is used to construct a branchless target point cloud classification model based on the target one-dimensional convolutional layer and the target linear layer.

[0122] based on Figure 4 The device may also include specific implementation units:

[0123] Optionally, the first weight fusion module 430 may specifically include:

[0124] The first weight fusion unit is used to fuse the weights of the one-dimensional convolutional layer and the activation layer in the initial training model using structural reparameterization technology; and to fuse the one-dimensional convolutional layer branch and the activation layer branch; wherein the fusion residual layer of the initial training model is a Conv1D operator, which is reparameterized into a one-dimensional convolutional layer.

[0125] Optionally, the second weight fusion module 440 may specifically include:

[0126] The second weight fusion unit is used to perform weight fusion on the linear layer and the activation layer according to the weights and using the structure reparameterization technique, remove the BN operator, and obtain the target linear layer; wherein, the reparameterization is performed as a linear layer.

[0127] Optionally, the first weight fusion module 430 can be specifically used for:

[0128] Formula used:

[0129] ;

[0130] This indicates that a Conv1D operation is performed, which fuses the weights of the one-dimensional convolutional layer and the activation layer in the initial trained model, wherein the input is... , Indicates Batch Size. Indicates the number of input feature channels. Indicates the number of input features. Represents the weights of a one-dimensional convolutional layer. This indicates the bias of a one-dimensional convolutional layer;

[0131] Formula used:

[0132] ;

[0133] This indicates that a BN operation is performed, where, This represents the scaling factor of the activation layer. This represents the mean of the activation layer. Indicates the variance of the activation layer. Let represent the offset coefficient of the activation layer, then:

[0134]

[0135] ;

[0136] make:

[0137] The expression for obtaining the target one-dimensional convolutional layer by fusion is:

[0138] , and complete the weight fusion.

[0139] Optionally, the second weight fusion module 440 can be specifically used for:

[0140]

[0141]

[0142] ,

[0143] The input is , Indicates Batch Size. Indicates the number of categories. Represents the weights of the linear layer. Indicates linear layer bias; let:

[0144] ;

[0145] The target linear layer is obtained by fusion:

[0146] .

[0147] Based on the same idea, corresponding to Embodiment 2, the present invention also provides a point cloud classification model construction device. Figure 5 This is a schematic diagram of the point cloud classification device provided by the present invention. Figure 5 As shown, the device may include:

[0148] The point cloud data acquisition module 510 is used to acquire point cloud data to be classified.

[0149] Feature vector extraction module 520 is used to extract feature vectors from the point cloud data to be classified;

[0150] The classification module 530 is used to input the feature vector into the target point cloud classification model to obtain the classification result; the target point cloud classification model is a branchless network constructed based on a target one-dimensional convolutional layer and a target linear layer; the target one-dimensional convolutional layer is obtained by extracting the weights of the initial training model and using structural reparameterization technology to fuse the weights of the one-dimensional convolutional layer and activation layer in the initial training model; the target linear layer is obtained by fusing the weights of the linear layer and activation layer in the initial training model according to the weights and using the structural reparameterization technology.

[0151] Based on the same idea, this specification also provides a point cloud classification model construction method and device, as well as a point cloud classification device. Figure 6 A schematic diagram of the electronic device structure provided by the present invention. It may include:

[0152] Corresponding to Example 1, the communication unit / communication interface is used to acquire point cloud sample data, build a model to start training, and obtain an initial training model; the initial training model includes a residual layer, a one-dimensional convolutional layer, a linear layer, and an activation layer.

[0153] A processing unit / processor for extracting the weights of the initial trained model;

[0154] Using structural reparameterization, the weights of the one-dimensional convolutional layer and the activation layer in the initial training model are fused to obtain the target one-dimensional convolutional layer.

[0155] Based on the weights, the linear layer and the activation layer are fused using the structural reparameterization technique to obtain the target linear layer;

[0156] Based on the target one-dimensional convolutional layer and the target linear layer, a branchless target point cloud classification model is constructed.

[0157] Corresponding to Embodiment 2, the communication unit / communication interface is used to acquire point cloud data to be classified;

[0158] A processing unit / processor is used to extract the feature vector of the point cloud data to be classified;

[0159] The feature vector is input into the target point cloud classification model to obtain the classification result. The target point cloud classification model is a branchless network constructed based on a target one-dimensional convolutional layer and a target linear layer. The target one-dimensional convolutional layer is obtained by extracting the weights of the initial training model and fusing the weights of the one-dimensional convolutional layer and activation layer in the initial training model using the structural reparameterization technique. The target linear layer is obtained by fusing the weights of the linear layer and activation layer in the initial training model using the structural reparameterization technique based on the weights.

[0160] like Figure 6 As shown, the terminal device described above may also include a communication line. The communication line may include a path for transmitting information between the components described above.

[0161] Optional, such as Figure 6 As shown, the terminal device may further include a memory. The memory stores computer execution instructions for implementing the present invention, and the execution is controlled by a processor. The processor executes the computer execution instructions stored in the memory, thereby implementing the method provided in the embodiments of the present invention.

[0162] like Figure 6 As shown, the memory can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited to these. The memory can exist independently and be connected to the processor via communication lines. The memory can also be integrated with the processor.

[0163] Optionally, the computer execution instructions in the embodiments of the present invention may also be referred to as application code, and the embodiments of the present invention do not specifically limit this.

[0164] In a specific implementation, as one example, such as Figure 6 As shown, a processor may include one or more CPUs, such as Figure 6 CPU0 and CPU1 in the CPU.

[0165] In a specific implementation, as one example, such as Figure 6 As shown, the terminal device may include multiple processors, such as Figure 6 The processors in the system. Each of these processors can be a single-core processor or a multi-core processor.

[0166] The foregoing mainly describes the solutions provided by the embodiments of the present invention from the perspective of the interaction between various modules. It is understood that each module, in order to achieve the above functions, includes corresponding hardware structures and / or software units for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0167] The embodiments of the present invention can divide functional modules according to the above method examples. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in the embodiments of the present invention is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0168] The processor described in this specification may also function as a memory. The memory stores computer execution instructions for carrying out the present invention, and its execution is controlled by the processor. The processor executes the computer execution instructions stored in the memory, thereby implementing the method provided in the embodiments of the present invention.

[0169] The memory can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited to these. The memory can exist independently and be connected to the processor via communication lines. The memory can also be integrated with the processor.

[0170] Optionally, the computer execution instructions in the embodiments of the present invention may also be referred to as application code, and the embodiments of the present invention do not specifically limit this.

[0171] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a terminal, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD).

[0172] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, the disclosure, and the appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0173] Although the invention has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made therein without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely exemplary descriptions of the invention as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if such modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include such modifications and modifications.

Claims

1. A method for constructing a point cloud classification model, characterized in that, include: Acquire point cloud sample data, build a model and start training to obtain the initial training model; The initial training model includes residual layers, one-dimensional convolutional layers, linear layers, and activation layers; Extract the weights of the initial trained model; Using structural reparameterization, the one-dimensional convolutional layer and the activation layer in the initial training model are weighted and fused to obtain a target one-dimensional convolutional layer. This includes: using structural reparameterization to fuse the weights of the one-dimensional convolutional layer and the activation layer in the initial training model; and fusing the one-dimensional convolutional layer branches and the activation layer branches. The fusion residual layer of the initial training model is a Conv1D operator, reparameterized into a one-dimensional convolutional layer. Based on the weights, the linear layer and the activation layer are fused using the structure reparameterization technique, and the BN operator is removed to obtain the target linear layer; wherein, the reparameterization is performed to form a single linear layer. Based on the target one-dimensional convolutional layer and the target linear layer, a branchless target point cloud classification model is constructed.

2. The method according to claim 1, characterized in that, Using structural reparameterization techniques, the weights of the one-dimensional convolutional layer and the activation layer in the initial training model are fused, specifically including: Formula used: ； in, This indicates that a Conv1D operation is performed, which fuses the weights of the one-dimensional convolutional layer and the activation layer in the initial trained model, wherein the input is... , Indicates Batch Size. Indicates the number of input feature channels. Indicates the number of input features. This represents the weights of a one-dimensional convolutional layer. This indicates the bias of a one-dimensional convolutional layer; Formula used: ； in, This indicates that a BN operation is being performed. This represents the scaling factor of the activation layer. This represents the mean of the activation layer. Indicates the variance of the activation layer. Let represent the offset coefficient of the activation layer, then: ； ； make: The expression for obtaining the target one-dimensional convolutional layer by fusion is: , and complete the weight fusion.

3. The method according to claim 2, characterized in that, Based on the weights, the linear layer and the activation layer are weighted and fused using the structure reparameterization technique, and the BN operator is removed to obtain the target linear layer, specifically including: ； ； The input is , Indicates Batch Size. Indicates the number of categories. Represents the linear layer weights. Indicates linear layer bias; let: ； The target linear layer is obtained by fusion. .

4. A point cloud classification method, characterized in that, include: Obtain the point cloud data to be classified; Extract the feature vector of the point cloud data to be classified; The feature vector is input into the target point cloud classification model to obtain the classification result; The target point cloud classification model is a branchless network constructed based on a one-dimensional convolutional layer and a linear layer of the target. The target one-dimensional convolutional layer is obtained by extracting the weights of the initial training model and using structural reparameterization technology to fuse the weights of the one-dimensional convolutional layer and activation layer in the initial training model, as well as fusing the branches of the one-dimensional convolutional layer and activation layer; wherein, the fusion residual layer of the initial training model is the Conv1D operator. The target linear layer is a linear layer obtained by fusing the weights of the linear layer and activation layer in the initial training model and removing the BN operator, based on the weights and using the structure reparameterization technique.

5. A point cloud classification model construction device, characterized in that, include: The initial training model building module is used to acquire point cloud sample data, build the model and start training to obtain the initial training model; The initial training model includes residual layers, one-dimensional convolutional layers, linear layers, and activation layers; The weight extraction module is used to extract the weights of the initial training model; The first weight fusion module is used to fuse the weights of the one-dimensional convolutional layer and the activation layer in the initial training model using structural reparameterization technology to obtain a target one-dimensional convolutional layer. This includes: fusing the weights of the one-dimensional convolutional layer and the activation layer in the initial training model using structural reparameterization technology; and fusing the one-dimensional convolutional layer branch and the activation layer branch. The fusion residual layer of the initial training model is a Conv1D operator, reparameterized into a one-dimensional convolutional layer. The second weight fusion module is used to fuse the linear layer and the activation layer according to the weights and using the structure reparameterization technique, remove the BN operator, and obtain the target linear layer; wherein, the reparameterization is performed as a single linear layer. The target point cloud classification model construction module is used to construct a branchless target point cloud classification model based on the target one-dimensional convolutional layer and the target linear layer.

6. A point cloud classification device, characterized in that, include: The point cloud data acquisition module is used to acquire point cloud data to be classified. The feature vector extraction module is used to extract the feature vectors of the point cloud data to be classified. The classification module is used to input the feature vector into the target point cloud classification model to obtain the classification result; the target point cloud classification model is a branchless network constructed based on a one-dimensional convolutional layer and a linear layer of the target. The target one-dimensional convolutional layer is obtained by extracting the weights of the initial training model and using structural reparameterization technology to fuse the weights of the one-dimensional convolutional layer and activation layer in the initial training model, as well as fusing the branches of the one-dimensional convolutional layer and activation layer; wherein, the fusion residual layer of the initial training model is the Conv1D operator. The target linear layer is a linear layer obtained by fusing the weights of the linear layer and activation layer in the initial training model and removing the BN operator, based on the weights and using the structure reparameterization technique.

7. A point cloud classification model construction method and device, characterized in that, include: The communication unit / communication interface is used to acquire point cloud sample data, build a model, start training, and obtain the initial training model. The initial training model includes residual layers, one-dimensional convolutional layers, linear layers, and activation layers; A processing unit / processor for extracting the weights of the initial trained model; Using structural reparameterization, the one-dimensional convolutional layer and the activation layer in the initial training model are weighted and fused to obtain a target one-dimensional convolutional layer. This includes: using structural reparameterization to fuse the weights of the one-dimensional convolutional layer and the activation layer in the initial training model; and fusing the one-dimensional convolutional layer branches and the activation layer branches. The fusion residual layer of the initial training model is a Conv1D operator, reparameterized into a one-dimensional convolutional layer. Based on the weights, the linear layer and the activation layer are fused using the structural reparameterization technique, and the BN operator is removed to obtain the target linear layer; wherein, the reparameterization is performed as a single linear layer; based on the target one-dimensional convolutional layer and the target linear layer, a branchless target point cloud classification model is constructed.

8. A point cloud classification device, characterized in that, include: Communication unit / communication interface, used to acquire point cloud data to be classified; A processing unit / processor is used to extract the feature vector of the point cloud data to be classified; The feature vector is input into the target point cloud classification model to obtain the classification result; the target point cloud classification model is a branchless network constructed based on a one-dimensional convolutional layer and a linear layer of the target. The target one-dimensional convolutional layer is obtained by extracting the weights of the initial training model and using structural reparameterization technology to fuse the weights of the one-dimensional convolutional layer and activation layer in the initial training model, as well as fusing the branches of the one-dimensional convolutional layer and activation layer; wherein, the fusion residual layer of the initial training model is the Conv1D operator. The target linear layer is a linear layer obtained by fusing the weights of the linear layer and activation layer in the initial training model and removing the BN operator, based on the weights and using the structure reparameterization technique.

Images