Semantic segmentation method and training method of semantic segmentation model and related device

Abstract

The application discloses a semantic segmentation method and a training method of a semantic segmentation model and related devices, wherein the semantic segmentation method comprises: voxelizing point cloud data to obtain grid indexes of points in the point cloud data, extracting features of the point cloud data to obtain first data features of the points, and performing dimension reduction processing on the first data features of points in the same grid on the same straight line based on the grid indexes of the points in the point cloud data to obtain second data features; then performing inverse voxelization based on the first data features and the second data features to obtain third data features of the points; and on this basis, predicting semantic segmentation results of the points based on the first data features and the third data features of the same points, wherein the semantic segmentation results at least comprise categories to which the points belong. The above scheme can improve the accuracy of semantic segmentation.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a semantic segmentation method, a training method for a semantic segmentation model, and related apparatus. Background Technology

[0002] Semantic segmentation is one of the most important topics in computer vision. With the rapid development of computer vision algorithms, laser point cloud-based semantic segmentation methods have become one of the most important and popular research areas in computer vision in recent years.

[0003] Currently, semantic segmentation of laser point clouds involves extracting features based on prior knowledge from the point cloud data and then clustering the points using clustering algorithms; alternatively, it involves learning from a large amount of labeled data to obtain a mapping model from point cloud data to labels, which is then used for inference of new data. However, these methods are difficult to apply to large-scale outdoor point cloud data, and projection- and voxel-based methods suffer from discrete errors during projection or voxelization, leading to semantic ambiguity. Therefore, improving the accuracy of semantic segmentation has become an urgent problem to be solved. Summary of the Invention

[0004] The main technical problem addressed in this application is to provide a semantic segmentation method, a training method for a semantic segmentation model, and related apparatus, which can improve the accuracy of semantic segmentation.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a semantic segmentation method, comprising: voxelizing point cloud data to obtain grid indices for each point in the point cloud data, and extracting features from the point cloud data to obtain first data features for each point; based on the grid indices for each point in the point cloud data, performing dimensionality reduction processing on the first data features of points within the grid on the same straight line to obtain second data features; performing inverse voxelization based on the first and second data features to obtain third data features for the points; and predicting the semantic segmentation result of the points based on the first and third data features of the same points; wherein the semantic segmentation result includes at least the category to which the point belongs.

[0006] To address the aforementioned technical problems, a second aspect of this application provides a training method for a semantic segmentation model, comprising: voxelizing sample point cloud data to obtain sample grid indices for each sample point in the sample point cloud data; extracting features from the sample point cloud data to obtain first sample data features for each sample point; wherein the sample point cloud data is labeled with the sample category to which the sample point belongs; and based on the grid indices of each sample point in the sample point cloud data, performing dimensionality reduction processing on the first sample data features of sample points within the sample grid on the same straight line to obtain second sample data features; then performing inverse voxelization based on the first and second sample data features to obtain third sample data features for the sample point; predicting the predicted category to which the sample point belongs based on the first and third sample data features of the same sample point; and adjusting the network parameters of the semantic segmentation model based on the difference between the sample category to which the sample point belongs and the predicted category.

[0007] To address the aforementioned technical problems, a third aspect of this application provides a semantic segmentation apparatus, comprising a voxelization module, a feature extraction module, a feature dimensionality reduction module, an inverse voxelization module, and a result prediction module. The voxelization module performs voxelization on point cloud data to obtain grid indices for each point in the point cloud data. The feature extraction module extracts features from the point cloud data to obtain first data features for each point. The feature dimensionality reduction module performs dimensionality reduction on the first data features of points within the same grid on the same straight line based on the grid indices of each point in the point cloud data to obtain second data features. The inverse voxelization module performs inverse voxelization based on the first and second data features to obtain third data features for the points. The result prediction module predicts the semantic segmentation result of the points based on the first and third data features of the same points. The semantic segmentation result includes at least the category to which the point belongs.

[0008] To address the aforementioned technical problems, a fourth aspect of this application provides a training apparatus for a semantic segmentation model, comprising a voxelization module, a feature extraction module, a feature dimensionality reduction module, an inverse voxelization module, a result prediction module, and a parameter adjustment module. The voxelization module voxelizes sample point cloud data to obtain sample grid indices for each sample point in the sample point cloud data; the sample point cloud data is labeled with the sample category to which each sample point belongs. The feature extraction module extracts features from the sample point cloud data to obtain first sample data features for each sample point. The feature dimensionality reduction module performs dimensionality reduction on the first sample data features of sample points within the same straight line grid based on the grid indices of each sample point in the sample point cloud data to obtain second sample data features. The inverse voxelization module performs inverse voxelization based on the first and second sample data features to obtain third sample data features for the sample point. The result prediction module predicts the predicted category to which a sample point belongs based on the first and third sample data features of the same sample point. The parameter adjustment module adjusts the network parameters of the semantic segmentation model based on the difference between the sample category to which the sample point belongs and the predicted category.

[0009] To address the aforementioned technical problems, the fifth aspect of this application provides an electronic device, including a memory and a processor coupled to each other. The memory stores program instructions, and the processor executes the program instructions to implement the semantic segmentation method in the first aspect or the training method for the semantic segmentation model in the second aspect.

[0010] To address the aforementioned technical problems, a sixth aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor. These program instructions are used to implement the semantic segmentation method in the first aspect or the training method for the semantic segmentation model in the second aspect.

[0011] The above scheme obtains the grid index of each point in the point cloud data by voxelizing the data, extracts features from the point cloud data to obtain the first data feature of each point, performs dimensionality reduction on the first data feature of points within the grid on the same straight line based on the grid index of each point in the point cloud data to obtain the second data feature, and performs inverse voxelization on the first and second data features to obtain the third data feature of the point. Then, based on the first and third data features of the same point, the semantic segmentation result of the point is predicted. The semantic segmentation result includes at least the category to which the point belongs. On the one hand, performing dimensionality reduction on the first data feature of points within the grid on the same straight line based on the grid index of each point in the point cloud data to obtain the second data feature helps to increase the voxel information in the second data feature. On the other hand, performing inverse voxelization on the first and second data features to obtain the third data feature of the point helps to improve the correlation between voxels and point cloud data, thereby improving the effectiveness of the third data feature. Based on this, the semantic segmentation result of the point is predicted based on the first and third data features of the same point. The semantic segmentation result includes at least the category to which the point belongs, thereby improving the accuracy of predicting the category to which the point belongs. Therefore, it can improve the accuracy of semantic segmentation.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0014] Figure 1 This is a flowchart illustrating an embodiment of the semantic segmentation method of this application;

[0015] Figure 2 This is a schematic diagram of an embodiment of pooling processing;

[0016] Figure 3 This is a flowchart illustrating an embodiment of the training method for the semantic segmentation model of this application;

[0017] Figure 4 This is a schematic diagram of an embodiment of the predicted category;

[0018] Figure 5 This is a schematic diagram of the framework of an embodiment of the semantic segmentation apparatus of this application;

[0019] Figure 6 This is a schematic diagram of the framework of an embodiment of the training device for the semantic segmentation model of this application;

[0020] Figure 7This is a schematic diagram of the framework of an embodiment of the electronic device of this application;

[0021] Figure 8 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0022] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0023] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0024] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. "Several" means at least one. The terms "first," "second," etc., in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0025] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the semantic segmentation method of this application.

[0026] Specifically, this may include the following steps:

[0027] Step S11: Voxelize the point cloud data to obtain the grid index of each point in the point cloud data, and extract features from the point cloud data to obtain the first data feature of each point.

[0028] In one implementation scenario, point cloud data can be acquired through LiDAR (Light Detection and Ranging) systems, such as lidar, through-beam laser intrusion detectors, reflective laser intrusion detectors, and precision rangefinder systems. Furthermore, the acquired point cloud data can be three-dimensional, such as (x, y, z), or four-dimensional, such as (x, y, z, i), where x, y, and z represent the spatial coordinates of each point in the point cloud data, and i represents the reflection intensity of each point. The acquisition method and dimensions of the point cloud data can be determined based on the actual situation and are not specifically limited here.

[0029] In one implementation scenario, before voxelizing the point cloud data, preprocessing can be performed to remove outliers. For example, outliers can be represented as (nan,nan,nan,0).

[0030] In one implementation scenario, to determine the grid index of each point in a point cloud data, the boundary values ​​of the point cloud data can be obtained first. That is, the boundary values ​​(x, y, x) of the point cloud data can be determined using the points in the point cloud data. min ,y min ,z min ,x max ,y max ,z max As one possible implementation, a preset percentage of points in the point cloud data can be selected to determine the boundary values ​​of the point cloud data. The preset percentage can be set to 60%, 70%, 80%, etc., and can be determined according to the actual situation; no specific limitation is made here. After determining the boundary values ​​of the point cloud data, points exceeding the boundary values ​​can be confined to the boundary. Unlike the aforementioned implementation, the boundary values ​​of the point cloud data can be determined based on each point in the point cloud data. The boundary values ​​of the point cloud data can be determined according to the actual situation; no specific limitation is made here. Then, based on the boundary values, the preset voxel threshold, and the position information of each point in the point cloud data, the grid index of each point in the point cloud data is calculated. Specifically, the preset voxel threshold is (x... v ,y v ,z v The location information of each point in the point cloud data is (x i ,y i ,z i i iBased on boundary values, a preset voxel threshold, and the positional information of each point in the point cloud data, the voxel values ​​in each dimension are determined. One possible implementation is to average the sum of the maximum and minimum values ​​in each dimension of the point cloud data's boundary values, and then use the ratio of this average value in the same dimension to the size value of the preset voxel threshold as the voxel value in each dimension. Alternatively, unlike the aforementioned implementation, the difference can be obtained by comparing the maximum and minimum values ​​in each dimension of the point cloud data's boundary values, and the ratio of this difference in the same dimension to the size value of the preset voxel threshold is used as the voxel value in each dimension. The specific expression can be as follows:

[0031]

[0032] Where W, L, and H represent the voxel values ​​in each dimension, and W*L*H represents the total voxel value. Furthermore, the corresponding grid index can be determined from each point in the point cloud data, i.e., the i-th point (x... i ,y i ,z i i i The grid index is The grid index of each point can be obtained by retrieving the location information (x) of each point in the point cloud data. i ,y i ,z i i i The difference between the minimum value in the same dimension of the point cloud data boundary values ​​and the minimum value in the same dimension is used as the grid index for the corresponding dimension. The ratio of this difference to the value in the same dimension within the preset voxel threshold is then rounded down.

[0033]

[0034] Here, floor represents rounding down. The above method, by obtaining the boundary values ​​of the point cloud data and calculating the grid index of each point in the point cloud data based on the boundary values, the preset voxel threshold, and the position information of each point in the point cloud data, helps to increase the efficiency of data processing and thus improve the efficiency of extracting multi-scale and multi-level local feature information.

[0035] In one implementation scenario, as a possible approach, features can be directly extracted from point cloud data to obtain the first data features.

[0036] In another implementation scenario, unlike the aforementioned implementation, the first distance from each point in the point cloud data to the voxel center point can be obtained first. For example, the point cloud data is P∈R. N×4 N is the number of points in the point cloud data, and the location information of each point in the point cloud data is (x i ,y i ,zi i i A voxel has K points, and the voxel center point formed by these K points can be represented by the following expression:

[0037]

[0038] in, This represents the voxel center point. After obtaining the voxel center point, the first distance from each point in the point cloud data to the voxel center point is obtained. The specific expression can be represented as follows:

[0039]

[0040] Among them, (x c ,y c ,z c The distance (x) represents the first distance from each point in the point cloud data to the center point of the voxel. The position information of each point is then concatenated with this first distance to obtain the concatenated data, which is the position information (x) of each point. i ,y i ,z i i i ) and the first distance (x) c ,y c ,z c The points are then concatenated to obtain the concatenated features. For example, if the first distance is (0.5, 0.6, 0.7) and the point location information in the point cloud data is (1, 0, 2, 100), then the concatenated feature P is... ′ For (1, 0, 2, 100, 0.5, 0.6, 0.7), P ′ ∈R N×7 Then, high-dimensional features are extracted from the spliced ​​data to obtain the first data feature. That is, a multi-layer perceptron (MLP) is used to extract high-dimensional features from the spliced ​​features to obtain the first data feature F. p If the input dimension of a multilayer perceptron is 7 and the output dimension is C1, then the first data feature... The above method, by concatenating the position information of each point in the point cloud data with the first distance, obtains concatenated data, and extracts high-dimensional features from the concatenated data to obtain the first data feature. This helps to increase the amount of information in the voxels of the first data feature and each point in the point cloud data, thereby improving the accuracy of the semantic classification results.

[0041] Step S12: Based on the grid index of each point in the point cloud data, perform dimensionality reduction processing on the first data features of points within the grid on the same straight line to obtain the second data features.

[0042] In one implementation scenario, pooling can be performed on the first data features of points within the same grid line based on the grid index of each point in the point cloud data. Specifically, pooling can be performed on the first data features of points within the grid line perpendicular to the xoz plane, the xoy plane, and the yoz plane, thus obtaining pooled features. Pooling methods can include, but are not limited to, max pooling, random pooling, average pooling, etc.

[0043] For example, please refer to Figure 2 , Figure 2 This is a schematic diagram of an embodiment of pooling processing, which performs max pooling on points with the same x and y grid indices to obtain pooled features. Specifically, a bird's-eye view would be... If point (,l) in the bird's-eye view corresponds to M points in the point cloud data, then the c-th dimension feature at point (,l) in the bird's-eye view... Furthermore, after obtaining the pooling features, the pooling features are encoded to obtain encoded features, and based on the encoded features, the second data features are decoded. For example, the feature encoding part can consist of a 2D convolutional layer, a batch normalization layer, and an activation function; the pooling features are encoded through the feature encoding part to obtain the encoded features. The feature decoding part can consist of an upsampling layer, a 2D convolutional layer, a batch normalization layer, and an activation function; the encoded features are decoded through the feature decoding part to obtain the second data features. The above method uses the grid index of each point in the point cloud data to perform pooling processing on the first data features of points within the grid on the same straight line to obtain pooled features, which further increases semantic information and avoids aggregating local features on the point cloud data as much as possible, thereby improving the accuracy of semantic classification.

[0044] Step S13: Perform inverse voxelization based on the first and second data features to obtain the third data features of the points.

[0045] In one implementation scenario, the first and second data features can be concatenated to obtain concatenated data features. These concatenated data features are then passed through a multilayer perceptron to output a transformation matrix. The transformation matrix is ​​then mapped using a sigmoid activation function to obtain a weight matrix. Finally, the second data features are weighted using the weight matrix to obtain the third data feature.

[0046] In another implementation scenario, the second data feature is transformed to obtain a new second data feature. Then, the new second data feature and the first data feature are fused to obtain a weight matrix. This weight matrix includes the weights of voxels at each point during devoxification. Based on this weight matrix and the second data feature, a third data feature is obtained. This approach, by fusing the new second and first data features to obtain the weight matrix, and then using the weight matrix and the second data feature, helps improve the correlation between the weight matrix and the second and first data features, thereby enhancing the effectiveness of the third data feature.

[0047] In a specific implementation scenario, as one possible approach, the second data feature can be transposed to obtain a new second data feature. Unlike the previously disclosed implementation, the second data feature can first be reshaped, i.e., its features can be rearranged to obtain a reshaped feature. Then, based on the mapping relationship between voxels and points, the reshaped feature is transformed to obtain a new second data feature. For example, the second data feature... First, the second data feature is reshaped, that is, the second data feature is rearranged to obtain the first rearranged feature. Then, transpose the first rearrangement feature to obtain the transposed feature. The transposed features are then rearranged to obtain the second rearranged features. Based on this, a new second data feature is obtained by using a multilayer perceptron with input dimension W*L*H and output dimension N for the second rearranged feature. The above method reshapes the second data features to obtain reshaped features, and then transforms the reshaped features based on the mapping relationship between voxels and points to obtain new second data features. In this way, the mapping relationship between voxels and points is integrated into the new second data features, which helps to improve the effectiveness of the new second data features.

[0048] In a specific implementation scenario, to obtain the weight matrix, the second data feature and the first data feature can be concatenated to obtain the first concatenated feature, and then the first concatenated feature can be mapped using the sigmoid activation function to obtain the weight matrix. Unlike the aforementioned disclosed implementation, the new second data feature and the first data feature can be concatenated first to obtain the first concatenated feature, and then the first concatenated feature can be weighted based on a multilayer perceptron to obtain a transformation matrix. Finally, the weight matrix can be mapped based on the transformation matrix. For example, the first data feature... New second data features First, the first data feature and the second data feature are concatenated to obtain the concatenated feature. The first concatenated features are then weighted based on a multilayer perceptron with input dimensions C1+C2 and output dimensions W*L*H, resulting in a weighting matrix f.a ∈R N×(W*L*H) Then, transpose the weighted matrix to obtain the transformation matrix f∈R. (W*L*H)×N Then, the transformation matrix is ​​mapped using the sigmoid activation function to obtain the weight matrix w∈R. W*L*H×N The above method concatenates the new second data feature and the first data feature to obtain the first concatenated feature, and then weights the first concatenated feature based on a multilayer perceptron to obtain a transformation matrix. Based on the transformation matrix, a weight matrix is ​​obtained by mapping. This method increases the voxel information in the first concatenated feature and helps to improve the effectiveness of the weight matrix.

[0049] In a specific implementation scenario, as one possible approach, the second data feature can be weighted based on a weight matrix to obtain the third data feature. Unlike the aforementioned implementation, the second data feature can first be weighted based on the weight matrix to obtain a weighted feature, and then the weighted feature can be transposed to obtain the third data feature. For example, the weight matrix is ​​w∈R. (W*L*H)×N The second data feature is First, the second data feature is weighted using a weight matrix to obtain the weighted feature F″. p =F v *w, The weighted features are then transposed to obtain the third data feature F′. p , The above method, by weighting the second data features based on the weight matrix to obtain weighted features, and then transposing the weighted features to obtain the third data features, helps to improve the effectiveness of the third features, thereby avoiding ambiguity in semantic segmentation results as much as possible, and further improving the accuracy of semantic segmentation results.

[0050] Step S14: Based on the first and third data features of the same point, predict the semantic segmentation result of the point.

[0051] In this embodiment of the disclosure, the semantic segmentation result includes at least the category to which the point belongs. Furthermore, the semantic segmentation result may also include the proportion of points belonging to the same category in the point cloud data.

[0052] In one implementation scenario, as a possible approach, the first data features and the third data features of the same point can be input into the fully connected layer to obtain a first prediction result. The first prediction result can include the first category to which the point belongs and the prediction score. Then, the point with the higher prediction score among the same points is selected as the semantic segmentation result.

[0053] In another implementation scenario, unlike the aforementioned implementation, the first and third data features of the same point are concatenated to obtain a second concatenated feature. Before predicting the second concatenated feature, a multilayer perceptron is used to transform the second concatenated feature to obtain a new second concatenated feature. Based on this, prediction is performed based on the new second concatenated feature to obtain the semantic segmentation result of the point. That is, the new second concatenated feature is input into a fully connected layer to obtain the predicted category and prediction score of the point. The predicted category corresponding to the higher prediction score is selected as the category to which the point belongs. For example, the first data feature F p The third data feature is F p ′ ,and First, the first and third data features are concatenated to obtain the second concatenated feature. Using a multilayer perceptron with input dimension C1+2 and output dimension C3, a new third concatenated feature is obtained. Then, a fully connected layer with input dimension C2 and output dimension C is applied to the new second concatenated feature to obtain P∈R. N×C C represents the number of categories to be segmented. For example, if the number of categories is 3, including cars, ground, and pedestrians, P can include 0.3, 0.4, and 0.5, where 0.3 corresponds to cars, 0.4 to ground, and 0.5 to pedestrians, then the corresponding point is a pedestrian. This method concatenates the first and third data features of identical points to obtain a second concatenated feature. Prediction is then performed based on this second concatenated feature to obtain the semantic segmentation result of the point, thereby improving the accuracy of the semantic segmentation result.

[0054] The above scheme obtains the grid index of each point in the point cloud data by voxelizing the data, extracts features from the point cloud data to obtain the first data feature of each point, performs dimensionality reduction on the first data feature of points within the grid on the same straight line based on the grid index of each point in the point cloud data to obtain the second data feature, and performs inverse voxelization on the first and second data features to obtain the third data feature of the point. Then, based on the first and third data features of the same point, the semantic segmentation result of the point is predicted. The semantic segmentation result includes at least the category to which the point belongs. On the one hand, performing dimensionality reduction on the first data feature of points within the grid on the same straight line based on the grid index of each point in the point cloud data to obtain the second data feature helps to increase the voxel information in the second data feature. On the other hand, performing inverse voxelization on the first and second data features to obtain the third data feature of the point helps to improve the correlation between voxels and point cloud data, thereby improving the effectiveness of the third data feature. Based on this, the semantic segmentation result of the point is predicted based on the first and third data features of the same point. The semantic segmentation result includes at least the category to which the point belongs, thereby improving the accuracy of predicting the category to which the point belongs. Therefore, it can improve the accuracy of semantic segmentation.

[0055] Please see Figure 3 , Figure 3 This is a flowchart illustrating an embodiment of the training method for the semantic segmentation model of this application. Specifically, it may include the following steps:

[0056] Step S31: Voxelize the sample point cloud data to obtain the sample grid index of each sample point in the sample point cloud data, and extract features from the sample point cloud data to obtain the first sample data feature of each sample point.

[0057] Specifically, the method for obtaining the sample grid index of each sample point in the sample point cloud data, or the method for obtaining the first sample data feature, can refer to the method for obtaining the grid index of each point in the point cloud data or the method for obtaining the first data feature in the aforementioned public embodiments, and will not be repeated here.

[0058] In this embodiment of the disclosure, the sample point cloud data is labeled with the sample category to which each sample point belongs. Specifically, the sample category of each point in the sample point cloud data can be labeled manually, or the sample category of each point in the sample point cloud data can be labeled using Cloud Compare. The labeling method of the sample points in the sample point cloud data can be selected according to the actual situation, and no specific limitation is made here.

[0059] Step S32: Based on the grid index of each sample point in the sample point cloud data, perform dimensionality reduction processing on the first sample data features of sample points within the sample grid on the same straight line to obtain the second sample data features.

[0060] Specifically, the method for obtaining the second sample data features can refer to the method for determining the second data features in the aforementioned disclosed embodiments, and will not be repeated here.

[0061] Step S33: Perform inverse voxelization based on the first sample data features and the second sample data features to obtain the third sample data features of the sample points.

[0062] Specifically, the method for obtaining the features of the third sample data can refer to the method for determining the third data features in the aforementioned public embodiments, and will not be repeated here.

[0063] Step S34: Based on the first and third sample data features of the same sample points, predict the predicted category to which the sample points belong.

[0064] Specifically, the prediction method for the prediction category to which the sample point belongs can refer to the prediction method for the semantic segmentation results of the points in the aforementioned public embodiments, and will not be repeated here.

[0065] Step S35: Adjust the network parameters of the semantic segmentation model based on the difference between the sample category to which the sample point belongs and the predicted category.

[0066] In one implementation scenario, the network parameters of the semantic segmentation model are adjusted based on the difference between the sample category to which the sample point belongs and the predicted category. Specifically, the specific methods for measuring the difference can be found in references such as the cross-entropy loss function, and the methods for adjusting the network parameters can be found in optimization methods such as gradient descent, which will not be elaborated here.

[0067] Please see Figure 4 , Figure 4 This is a schematic diagram of an embodiment of the predicted category, as shown below. Figure 4 As shown, the sample point cloud data can first be voxelized to obtain the sample grid index of each sample point in the sample point cloud data, and then data feature extraction can be performed, that is, feature extraction can be performed on the sample point cloud data to obtain the first sample data feature of each sample point, based on the grid index of each sample point in the sample point cloud data; then bird's-eye view mapping can be performed, that is, the first sample data feature of the sample points in the sample grid on the same straight line can be dimensionality reduced and then encoded and decoded to obtain the second sample data feature; based on the first sample data feature and the second sample data feature, inverse voxelization can be performed to obtain the third sample data feature of the sample point; on this basis, feature fusion can be performed based on the first sample data feature and the third sample data feature of the same sample point to predict the predicted category to which the sample point belongs; further, based on the difference between the sample category to which the sample point belongs and the predicted category, the network parameters of the semantic segmentation model can be adjusted.

[0068] The above scheme obtains the sample grid index of each sample point in the sample point cloud data by voxelizing the sample point cloud data, and extracts features from the sample point cloud data to obtain the first sample data feature of each sample point. The sample category to which the sample point belongs is marked in the sample point cloud data. Then, based on the grid index of each sample point in the sample point cloud data, the first sample data feature of the sample points in the sample grid on the same straight line is subjected to dimensionality reduction to obtain the second sample data feature. Then, based on the first sample data feature and the second sample data feature, inverse voxelization is performed to obtain the third sample data feature of the sample point. Based on the first sample data feature and the third sample data feature of the same sample point, the predicted category to which the sample point belongs is predicted. On this basis, based on the difference between the sample category to which the sample point belongs and the predicted category, the network parameters of the semantic segmentation model are adjusted. On the one hand, based on the sample The grid index of each sample point in the point cloud data is used to perform dimensionality reduction on the first sample data features of sample points within the same grid on the same straight line, resulting in second sample data features. This helps to increase the voxel information in the second sample data features. On the other hand, inverse voxelization is performed based on the first and second sample data features to obtain the third sample data features of the sample points. This helps to improve the correlation between voxels and the sample point cloud data, thereby improving the effectiveness of the third sample data features. Based on this, the predicted category of the sample point is predicted based on the first and third sample data features of the same sample point, thereby improving the accuracy of the predicted category. Furthermore, the network parameters of the semantic segmentation model are adjusted based on the difference between the sample category and the predicted category, which helps to improve the robustness of the trained semantic segmentation model. Therefore, the accuracy of semantic segmentation can be improved.

[0069] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0070] Please see Figure 5 , Figure 5This is a schematic diagram of the framework of an embodiment of the semantic segmentation device of this application. The semantic segmentation device 50 includes a voxelization module 51, a feature extraction module 52, a feature dimensionality reduction module 53, an inverse voxelization module 54, and a result prediction module 55. Specifically, the voxelization module 51 is used to voxelize the point cloud data to obtain the grid index of each point in the point cloud data; the feature extraction module 52 is used to extract features from the point cloud data to obtain the first data feature of each point; the feature dimensionality reduction module 53 is used to perform dimensionality reduction processing on the first data feature of points within the grid on the same straight line based on the grid index of each point in the point cloud data to obtain the second data feature; the inverse voxelization module 54 is used to perform inverse voxelization based on the first and second data features to obtain the third data feature of the point; and the result prediction module 55 is used to predict the semantic segmentation result of the point based on the first and third data features of the same point, and the semantic segmentation result includes at least the category to which the point belongs.

[0071] The above scheme, on the one hand, performs dimensionality reduction on the first data features of points within the same grid on the same straight line based on the grid index of each point in the point cloud data, obtaining the second data features. This helps to increase the voxel information in the second data features. On the other hand, it performs devoxification based on the first and second data features to obtain the third data features of the points, which helps to improve the correlation between voxels and point cloud data, thereby improving the effectiveness of the third data features. Based on this, the semantic segmentation result of the points is predicted based on the first and third data features of the same points. The semantic segmentation result includes at least the category to which the point belongs, thereby improving the accuracy of predicting the category to which the point belongs. Therefore, it can improve the accuracy of semantic segmentation.

[0072] In some disclosed embodiments, the devoxization module 54 includes a transformation submodule, a fusion submodule, and a weighting submodule. The transformation submodule transforms the second data feature to obtain a new second data feature; the fusion submodule fuses the new second data feature and the first data feature to obtain a weight matrix; the weight matrix includes the weights of voxels at each point during devoxization; and the weighting submodule obtains a third data feature based on the weight matrix and the second data feature.

[0073] Therefore, by fusing the new second data feature and the first data feature to obtain the weight matrix, and then using the weight matrix and the second data feature, it is helpful to improve the correlation between the weight matrix and the second and first data features, thereby improving the effectiveness of the third data feature.

[0074] In some disclosed embodiments, the transformation submodule includes a reshaping unit and a transformation unit. The reshaping unit is used to reshape the second data feature to obtain a reshaped feature; the transformation unit is used to transform the reshaped feature based on the mapping relationship between voxels and points to obtain a new second data feature.

[0075] Therefore, by reshaping the second data features to obtain reshaped features, and then transforming the reshaped features based on the mapping relationship between voxels and points to obtain new second data features, the mapping relationship between voxels and points can be integrated into the new second data features, which helps to improve the effectiveness of the new second data features.

[0076] In some disclosed embodiments, the fusion submodule includes a splicing unit, a weighting unit, and a mapping unit. The splicing unit splices the new second data feature and the first data feature to obtain a first spliced ​​feature; the weighting unit weights the first spliced ​​feature based on a multilayer perceptron to obtain a transformation matrix; and the mapping unit maps the transformation matrix to obtain a weight matrix.

[0077] Therefore, by concatenating the new second data feature and the first data feature, a first concatenated feature is obtained. The first concatenated feature is then weighted based on a multilayer perceptron to obtain a transformation matrix. Based on the transformation matrix, a weight matrix is ​​obtained by mapping. This not only increases the voxel information in the first concatenated feature but also helps to improve the effectiveness of the weight matrix.

[0078] In some disclosed embodiments, the weighting submodule includes a weighting unit and a transpose unit. The weighting unit is used to perform weighting processing on the second data feature based on the weight matrix to obtain a weighted feature; the transpose unit is used to transpose the weighted feature to obtain a third data feature.

[0079] Therefore, by weighting the second data features based on the weight matrix to obtain weighted features, and then transposing the weighted features to obtain the third data features, the effectiveness of the third features can be improved, thereby avoiding ambiguity in semantic segmentation results as much as possible and further improving the accuracy of semantic segmentation results.

[0080] In some disclosed embodiments, the result prediction module 55 includes a splicing submodule and a prediction submodule. The splicing submodule is used to splice the first data feature and the third data feature of the same point to obtain a second spliced ​​feature; the prediction submodule is used to make a prediction based on the second spliced ​​feature to obtain the semantic segmentation result of the point.

[0081] Therefore, by concatenating the first and third data features of the same point, a second concatenated feature is obtained. Then, prediction is performed based on the second concatenated feature to obtain the semantic segmentation result of the point, thereby improving the accuracy of the semantic segmentation result.

[0082] In some disclosed embodiments, the feature dimensionality reduction module 53 includes a pooling submodule, an encoding submodule, and a decoding submodule. The pooling submodule is used to perform pooling processing on the first data features of points within the same grid on the same straight line based on the grid index of each point in the point cloud data, to obtain pooled features; the encoding submodule is used to encode the pooled features to obtain encoded features; and the decoding submodule is used to decode the encoded features to obtain second data features.

[0083] Therefore, by pooling the first data features of points within the grid on the same straight line based on the grid index of each point in the point cloud data, pooled features are obtained, which further increases semantic information, provides spatial neighborhood features as much as possible, and thus improves the accuracy of semantic classification.

[0084] In some disclosed embodiments, the voxelization module 51 includes an acquisition submodule and a calculation submodule. The acquisition submodule is used to acquire the boundary values ​​of the point cloud data; the calculation submodule is used to calculate the grid index of each point in the point cloud data based on the boundary values, a preset voxel threshold, and the position information of each point in the point cloud data.

[0085] Therefore, by obtaining the boundary values ​​of point cloud data and calculating the grid index of each point in the point cloud data based on the boundary values, preset voxel thresholds, and the position information of each point in the point cloud data, it is helpful to increase the efficiency of data processing and thus improve the efficiency of extracting multi-scale and multi-level local feature information.

[0086] In some disclosed embodiments, the feature extraction module 52 includes an acquisition submodule, a stitching submodule, and an extraction submodule. The acquisition submodule acquires the first distance from each point in the point cloud data to the center point of the voxel; the stitching submodule stitches the position information of each point in the point cloud data with the first distance to obtain stitched data; and the extraction submodule extracts high-dimensional features from the stitched data to obtain first data features.

[0087] Therefore, by concatenating the positional information of each point in the point cloud data with the first distance, we obtain concatenated data. Then, we extract high-dimensional features from the concatenated data to obtain the first data features. This helps to increase the amount of information in the voxels and points in the point cloud data in the first data features, thereby improving the accuracy of semantic classification results.

[0088] Please see Figure 6 , Figure 6This is a schematic diagram of the framework of an embodiment of the training device for the semantic segmentation model of this application. The training device 60 for the semantic segmentation model includes a voxelization module 61, a feature extraction module 62, a feature dimensionality reduction module 63, an inverse voxelization module 64, a result prediction module 65, and a parameter adjustment module 66. The voxelization module 61 is used to voxelize the sample point cloud data to obtain the sample grid index of each sample point in the sample point cloud data, and the sample category to which the sample point belongs is marked in the sample point cloud data; the feature extraction module 62 is used to extract features from the sample point cloud data to obtain the first sample data feature of each sample point; the feature dimensionality reduction module 63 is used to perform dimensionality reduction processing on the first sample data feature of the sample points in the sample grid on the same straight line based on the grid index of each sample point in the sample point cloud data to obtain the second sample data feature; the inverse voxelization module 64 is used to perform inverse voxelization based on the first sample data feature and the second sample data feature to obtain the third sample data feature of the sample point; the result prediction module 65 is used to predict the predicted category to which the sample point belongs based on the first sample data feature and the third sample data feature of the same sample point; and the parameter adjustment module 66 is used to adjust the network parameters of the semantic segmentation model based on the difference between the sample category to which the sample point belongs and the predicted category.

[0089] The above scheme, on the one hand, performs dimensionality reduction on the first sample data features of sample points within the same grid on the same straight line based on the grid index of each sample point in the sample point cloud data, obtaining the second sample data features. This helps to increase the voxel information in the second sample data features. On the other hand, it performs devoxification based on the first and second sample data features to obtain the third sample data features of the sample points, which helps to improve the correlation between voxels and the sample point cloud data, thereby improving the effectiveness of the third sample data features. Based on this, the predicted category of the sample point is predicted based on the first and third sample data features of the same sample point, thereby improving the accuracy of the predicted category. Furthermore, the network parameters of the semantic segmentation model are adjusted based on the difference between the sample category and the predicted category, which helps to improve the robustness of the trained semantic segmentation model. Therefore, it can improve the accuracy of semantic segmentation.

[0090] Please see Figure 7 , Figure 7 This is a schematic diagram of a framework of an embodiment of the electronic device of this application. The electronic device 70 includes a memory 71 and a processor 72 coupled to each other. The memory 71 stores program instructions, and the processor 72 is used to execute the program instructions to implement the steps in any of the semantic segmentation method embodiments described above, or the steps in any semantic segmentation model training method embodiment. Specifically, the electronic device 70 may include, but is not limited to, desktop computers, laptops, servers, mobile phones, tablet computers, etc., and is not limited thereto.

[0091] Specifically, processor 72 controls itself and memory 71 to implement the steps in any of the semantic segmentation method embodiments described above, or the steps in any semantic segmentation model training method embodiment. Processor 72 can also be referred to as a CPU (Central Processing Unit). Processor 72 may be an integrated circuit chip with signal processing capabilities. Processor 72 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 72 can be implemented using integrated circuit chips.

[0092] In the above scheme, the electronic device 70 can be used to implement the steps in any of the above semantic segmentation method embodiments, or the steps in any semantic segmentation model training method embodiments. On the one hand, based on the grid index of each point in the point cloud data, the first data features of points within the grid on the same straight line are subjected to dimensionality reduction processing to obtain the second data features, which helps to increase the voxel information in the second data features. On the other hand, based on the first data features and the second data features, inverse voxelization is performed to obtain the third data features of the points, which helps to improve the correlation between voxels and point cloud data, thereby improving the effectiveness of the third data features. On this basis, based on the first data features and the third data features of the same point, the semantic segmentation result of the point is predicted. The semantic segmentation result includes at least the category to which the point belongs, thereby improving the accuracy of predicting the category to which the point belongs. Therefore, the accuracy of semantic segmentation can be improved.

[0093] Please see Figure 8 , Figure 8 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 80 stores program instructions 81 that can be executed by a processor. The program instructions 81 are used to implement the steps in any of the above-described semantic segmentation method embodiments, or the steps in any of the semantic segmentation model training method embodiments.

[0094] In the above scheme, the computer-readable storage medium 80 can be used to implement the steps in any of the above semantic segmentation method embodiments, or the steps in any semantic segmentation model training method embodiments. On the one hand, based on the grid index of each point in the point cloud data, the first data features of points within the grid on the same straight line are subjected to dimensionality reduction processing to obtain the second data features, which helps to increase the voxel information in the second data features. On the other hand, based on the first data features and the second data features, inverse voxelization is performed to obtain the third data features of the points, which helps to improve the correlation between voxels and point cloud data, thereby improving the effectiveness of the third data features. On this basis, based on the first data features and the third data features of the same point, the semantic segmentation result of the point is predicted. The semantic segmentation result includes at least the category to which the point belongs, thereby improving the accuracy of predicting the category to which the point belongs. Therefore, the accuracy of semantic segmentation can be improved.

[0095] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0096] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0098] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0099] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0100] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A method of semantic segmentation, characterized in that, The method comprises the following steps: voxelizing point cloud data to obtain grid indexes of points in the point cloud data, and extracting features of the point cloud data to obtain first data features of the points; based on the grid indexes of the points in the point cloud data, performing dimension reduction processing on the first data features of the points in the same grid on the same straight line to obtain second data features; based on the first data features and the second data features, performing de-voxelization to obtain third data features of the points; based on the first data features and the third data features of the same points, predicting semantic segmentation results of the points; wherein the semantic segmentation results at least include categories to which the points belong; the de-voxelization based on the first data features and the second data features to obtain the third data features of the points comprises: converting the second data features to obtain new second data features; based on the new second data features and the first data features, fusing to obtain a weight matrix; wherein the weight matrix comprises weights of voxels to the points during de-voxelization; based on the weight matrix and the second data features, obtaining the third data features; the conversion of the second data features to obtain new second data features comprises: reshaping the second data features to obtain reshaped features; based on the mapping relationship between voxels and the points, converting the reshaped features to obtain the new second data features; the fusion based on the new second data features and the first data features to obtain a weight matrix comprises: splicing the new second data features and the first data features to obtain first spliced features; based on a multi-layer perception, weighting the first spliced features to obtain a transformation matrix; based on the transformation matrix, mapping to obtain the weight matrix; the obtaining of the third data features based on the weight matrix and the second data features comprises: based on the weight matrix, weighting the second data features to obtain weighted features; transposing the weighted features to obtain the third data features; the dimension reduction processing on the first data features of the points in the same grid on the same straight line based on the grid indexes of the points in the point cloud data comprises: based on the grid indexes of the points in the point cloud data, performing pooling processing on the first data features of the points in the same grid on the same straight line to obtain pooled features; encoding the pooled features to obtain encoded features; based on the encoded features, decoding to obtain the second data features.

2. The method of claim 1, wherein, the prediction of the semantic segmentation results of the points based on the first data features and the third data features of the same points comprises: splicing the first data features and the third data features of the same points to obtain second spliced features; based on the second spliced features, predicting to obtain the semantic segmentation results of the points.

3. The method of claim 1, wherein, the voxelization of the point cloud data to obtain the grid indexes of the points in the point cloud data comprises: obtaining boundary values of the point cloud data; Based on the boundary value, a preset voxel threshold value and position information of each point in the point cloud data, a grid index of each point in the point cloud data is calculated.

4. The method of claim 1, wherein, The feature extraction on the point cloud data obtains a first data feature, including: A first distance from each point in the point cloud data to a voxel center point is obtained. The position information of each point in the point cloud data and the first distance are spliced to obtain spliced data. A high-dimensional feature is extracted from the spliced data to obtain the first data feature.

5. A method for training a semantic segmentation model, the method comprising: including: The sample point cloud data is voxelized to obtain a sample grid index of each sample point in the sample point cloud data, and a feature is extracted from the sample point cloud data to obtain a first sample data feature of each sample point; wherein the sample point cloud data is labeled with a sample category to which a sample point belongs; Based on the grid index of each sample point in the sample point cloud data, a dimension reduction processing is performed on the first sample data feature of the sample points in the sample grid on the same straight line to obtain a second sample data feature; Based on the first sample data feature and the second sample data feature, an inverse voxelization is performed to obtain a third sample data feature of the sample points; Based on the first sample data feature and the third sample data feature of the same sample point, a predicted category to which the sample point belongs is predicted; Based on the difference between the sample category and the predicted category to which the sample point belongs, the network parameters of the semantic segmentation model are adjusted; The inverse voxelization based on the first sample data feature and the second sample data feature to obtain the third sample data feature of the sample points includes: The second sample data feature is converted to obtain a new second sample data feature; Based on the new second sample data feature and the first sample data feature, a weight matrix is obtained; wherein the weight matrix includes the weight of the voxel to each sample point during inverse voxelization; Based on the weight matrix and the second sample data feature, the third sample data feature is obtained; The conversion of the second sample data feature to obtain a new second sample data feature includes: The second sample data feature is reshaped to obtain a reshaped feature; Based on the mapping relationship between the voxel and the sample points, the reshaped feature is converted to obtain the new second sample data feature; The fusion based on the new second sample data feature and the first sample data feature to obtain a weight matrix includes: The new second sample data feature and the first sample data feature are spliced to obtain a first spliced feature; Based on a multi-layer perception, the first spliced feature is weighted to obtain a transformation matrix; Based on the transformation matrix, the weight matrix is mapped; The obtaining of the third sample data feature based on the weight matrix and the second sample data feature includes: The second sample data feature is weighted based on the weight matrix to obtain a weighted feature; The weighted feature is transposed to obtain the third sample data feature; Based on the grid index of each sample point in the sample point cloud data, the first sample data feature of the sample points in the sample grid on the same straight line is processed by dimension reduction to obtain a second sample data feature, including: Based on the grid index of each sample point in the sample point cloud data, the first sample data feature of the sample points in the sample grid on the same straight line is processed by dimension reduction to obtain a second sample data feature, including: The pooling feature is encoded to obtain an encoded feature; Based on the encoded feature, the second sample data feature is decoded.

6. A semantic segmentation apparatus, characterized by, Including: The voxelization module is configured to voxelize the point cloud data to obtain a grid index of each point in the point cloud data; The feature extraction module is configured to extract features from the point cloud data to obtain a first data feature of each point; The feature dimension reduction module is configured to process the first data feature of the points in the grid on the same straight line by dimension reduction based on the grid index of each point in the point cloud data to obtain a second data feature; The inverse voxelization module is configured to perform inverse voxelization based on the first data feature and the second data feature to obtain a third data feature of the points; The result prediction module is configured to predict a semantic segmentation result of the points based on the first data feature and the third data feature of the same points, wherein the semantic segmentation result at least includes a category to which the points belong; The inverse voxelization module includes: The conversion submodule is configured to convert the second data feature to obtain a new second data feature; The fusion submodule is configured to fuse the new second data feature and the first data feature to obtain a weight matrix, wherein the weight matrix includes a weight of a voxel to each point during inverse voxelization; The weighting submodule is configured to obtain the third data feature based on the weight matrix and the second data feature; The conversion submodule includes: The reshaping unit is configured to reshape the second data feature to obtain a reshaped feature; The conversion unit is configured to convert the reshaped feature based on a mapping relationship between a voxel and the points to obtain the new second data feature; The fusion submodule includes: The splicing unit is configured to splice the new second data feature and the first data feature to obtain a first spliced feature; The weighting unit is configured to weight the first spliced feature based on a multi-layer perception machine to obtain a transformation matrix; The mapping unit is configured to map the weight matrix based on the transformation matrix; The weighting submodule includes: The weighting unit is configured to weight the second data feature based on the weight matrix to obtain a weighted feature; The transposition unit is configured to transpose the weighted feature to obtain the third data feature; The feature dimension reduction module includes: The pooling submodule is configured to process the first data feature of the points in the grid on the same straight line by pooling based on the grid index of each point in the point cloud data to obtain a pooling feature; The encoding submodule is configured to encode the pooling feature to obtain an encoded feature; The decoding submodule is configured to decode the second data feature based on the encoded feature. 7.A device for training a semantic segmentation model, the device comprising: Including: The voxelization module is configured to voxelize the sample point cloud data to obtain a sample grid index of each sample point in the sample point cloud data; wherein a sample category to which each sample point belongs is labeled in the sample point cloud data; The feature extraction module is configured to extract features of the sample point cloud data to obtain a first sample data feature of each sample point; The feature dimension reduction module is configured to perform dimension reduction processing on the first sample data feature of each sample point in a sample grid on a same straight line based on the grid index of each sample point in the sample point cloud data to obtain a second sample data feature; The de-voxelization module is configured to perform de-voxelization based on the first sample data feature and the second sample data feature to obtain a third sample data feature of each sample point; The result prediction module is configured to predict a predicted category to which each sample point belongs based on the first sample data feature and the third sample data feature of each sample point; The parameter adjustment module is configured to adjust a network parameter of the semantic segmentation model based on a difference between the sample category and the predicted category to which each sample point belongs; The de-voxelization module includes: The conversion submodule is configured to convert the second sample data feature to obtain a new second sample data feature; The fusion submodule is configured to fuse the new second sample data feature and the first sample data feature to obtain a weight matrix; wherein the weight matrix includes a weight of each sample point to voxels during de-voxelization; The weighting submodule is configured to obtain the third sample data feature based on the weight matrix and the second sample data feature; The conversion submodule includes: The reshaping unit is configured to reshape the second sample data feature to obtain reshaped features; The conversion unit is configured to convert the reshaped features based on a mapping relationship between voxels and each sample point to obtain the new second sample data feature; The fusion submodule includes: The splicing unit is configured to splice the new second sample data feature and the first sample data feature to obtain first spliced features; The weighting unit is configured to weight the first spliced features based on a multi-layer perception machine to obtain a transformation matrix; The mapping unit is configured to map the weight matrix based on the transformation matrix; The weighting submodule includes: The weighting unit is configured to weight the second sample data feature based on the weight matrix to obtain weighted features; The transposition unit is configured to transpose the weighted features to obtain the third sample data feature; The feature dimension reduction module includes: The pooling submodule is configured to pool the first sample data feature of each sample point in the sample grid on the same straight line based on the grid index of each sample point in the sample point cloud data to obtain pooled features; The encoding submodule is configured to encode the pooled features to obtain encoded features; The decoding submodule is configured to decode the encoded features to obtain the second sample data feature.

8. An electronic device, comprising: The computer program product comprises a memory and a processor coupled to each other, the memory stores program instructions, and the processor is configured to execute the program instructions to implement the semantic segmentation method according to any one of claims 1 to 4 or the training method of the semantic segmentation model according to claim 5.

9. A computer-readable storage medium, characterized in that, The computer program product comprises a memory and a processor coupled to each other, the memory stores program instructions, and the processor is configured to execute the program instructions to implement the semantic segmentation method according to any one of claims 1 to 4 or the training method of the semantic segmentation model according to claim 5.

Images