A method and system for segmentation and target detection of offshore wind power underwater point cloud data
By employing multi-scale hierarchical partitioning and a self-attention mechanism, combined with a non-local fusion module and rendering branch, the problem of missing global and contextual information in point cloud data was solved, achieving efficient segmentation and target detection of underwater point cloud data for offshore wind power, and improving detection accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAN THERMAL POWER RES INST CO LTD
- Filing Date
- 2024-04-25
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies for underwater point cloud data segmentation and target detection in offshore wind power neglect global and contextual information between different scales and regions in the point cloud, making it difficult to apply traditional methods effectively.
We employ a multi-scale hierarchical partitioning and self-attention mechanism, generating semantic labels and instance offsets through multi-layer non-local fusion modules and rendering branches. We combine an optimizer and a learning rate scheduler for network training and use data augmentation techniques to improve model robustness.
It achieves efficient semantic and instance segmentation of point cloud data, improves the accuracy and stability of target detection, enhances the feature representation and context understanding capabilities of point cloud data, and improves the generalization ability and robustness of the model.
Smart Images
Figure CN118334502B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of underwater detection technology for offshore wind power, and relates to a method and system for segmenting and target detection of underwater point cloud data for offshore wind power. Background Technology
[0002] A point cloud is a data structure that represents the surface of an object in three-dimensional space. It consists of a set of discrete points, each containing its three-dimensional coordinates and other attribute information, such as color, normal vector, and reflectivity. The three-dimensional data of point clouds can be acquired by devices such as LiDAR, depth cameras, and stereo vision.
[0003] Point cloud segmentation and object detection are two important 3D vision tasks, used to classify points in a point cloud into different semantic categories and instances, and to detect target objects in the point cloud, along with their positions and poses. In the underwater inspection of offshore wind power projects, 3D point cloud data of underwater structures can be obtained using shipborne multibeam sonar and 3D mechanical scanning sonar mounted on underwater robots.
[0004] However, point cloud segmentation and object detection also face some challenges. For example, underwater sonar point cloud detection data is incomplete and subject to significant interference. The sparsity, irregularity, and noise of point cloud data make it difficult to directly apply traditional image- or voxel-based deep learning methods to point cloud data. Currently, some methods have attempted to use point-based deep learning methods to process point cloud data, such as PointNet, PointNet++, and PointCNN. These methods can directly extract features from the original point cloud and perform classification or segmentation. However, these methods typically only consider the local information of each point in the point cloud, ignoring the global and contextual information between different scales and regions within the point cloud. Furthermore, these methods do not fully utilize instance information in the point cloud to assist semantic segmentation or use semantic information to assist object detection. Therefore, these methods still have significant room for improvement in point cloud segmentation and object detection. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for segmenting and target detection of underwater point cloud data for offshore wind power, so as to solve the problem that the existing point cloud data segmentation and target detection process ignores the global information and contextual information between different scales and regions in the point cloud.
[0006] To achieve the above objectives, the present invention employs the following technical solution:
[0007] A method for segmentation and target detection of underwater point cloud data for offshore wind power includes the following steps:
[0008] S1, transform the point cloud data into a sparse point cloud, perform multi-scale hierarchical division of the sparse point cloud to obtain multiple sub-regions of different sizes, and extract low-dimensional feature vectors from the sub-regions.
[0009] S2 updates the low-dimensional feature vector of each point in the sub-region through a multi-layer self-attention mechanism; it fuses the updated low-dimensional feature vectors in each sub-region through a multi-layer non-local fusion module to obtain a fused feature vector; and it processes the fused feature vectors in the finest-grained sub-regions through a rendering branch to generate semantic labels and instance offsets, thus obtaining the segmentation and object detection network.
[0010] S3, the network parameters of the segmentation and object detection network are updated iteratively through the optimizer, and then the group loss function is updated. The semantic label and instance offset are optimized through the group loss function to minimize the semantic loss of the semantic label and the instance offset, thus obtaining the final segmentation and object detection network. The final segmentation and object detection network is used to segment and detect the target.
[0011] A further improvement of the present invention is that:
[0012] Preferably, in S1, the point cloud data is voxelized according to a fixed voxel size to obtain a three-dimensional mesh; points in each voxel are randomly sampled, and one point is retained in each voxel to obtain a sparse point cloud.
[0013] Preferably, in S1, a low-dimensional feature vector is extracted from the sub-region using a one-dimensional convolutional neural network.
[0014] Preferably, in S2, the self-attention mechanism is a self-attention mechanism based on [PointTransformer].
[0015] Preferably, in S2, the nonlocal fusion module is a nonlocal module based on [PointASNL].
[0016] Preferably, in S2, the rendering branch is the [PointRend] rendering branch.
[0017] Preferably, in S3, the network parameters of the segmentation and object detection network obtained in S2 are iteratively updated through the [AdamW] optimizer. During the iterative update process, the learning rate of the [AdamW] optimizer is adjusted through learning rate scheduling.
[0018] Preferably, in S3, the grouping function includes semantic loss and offset loss. The semantic loss is measured by cross-entropy loss to measure the difference between the semantic label and the true label of each point, and the offset loss is measured by smoothed L1 loss to measure the offset between each point and the center of its instance.
[0019] Preferably, before S1, the point cloud data is processed by a data augmentation method, which includes random rotation, translation, scaling, cropping, and noise addition.
[0020] A system for segmentation and target detection of underwater point cloud data from offshore wind power, comprising:
[0021] The preprocessing module is used to transform point cloud data into sparse point cloud, perform multi-scale hierarchical division of sparse point cloud to obtain sub-regions of multiple sizes, and extract low-dimensional feature vectors from the sub-regions.
[0022] The network structure module is used to update the low-dimensional feature vector of each point in the sub-region through a multi-layer self-attention mechanism; to fuse the updated low-dimensional feature vectors in each sub-region through a multi-layer non-local fusion module to obtain a fused feature vector; and to process the fused feature vectors in the finest-grained sub-regions through a rendering branch to generate semantic labels and instance offsets, thus obtaining a segmentation and object detection network.
[0023] The loss optimization module is used to iteratively update the parameters of the segmentation and object detection network through the optimizer, update the group loss function, and optimize the semantic label and instance offset through the group loss function to minimize the semantic loss of the semantic label and the instance offset, thereby obtaining the final segmentation and object detection network, and segmenting and detecting the target through the final segmentation and object detection.
[0024] Compared with the prior art, the present invention has the following beneficial effects:
[0025] This invention relates to a method for segmentation and target detection of underwater point cloud data from offshore wind power projects. First, data preprocessing is performed, including voxelization, uniform sampling, multi-scale hierarchical partitioning, random sampling, and local feature aggregation to extract low-dimensional feature vectors. In the network structure, points in each sub-region undergo multi-layer self-attention mechanisms and feature vector fusion with non-local modules; simultaneously, a rendering branch is used to output semantic labels and instance offsets. The loss function is optimized by combining semantic loss and offset loss. During training, the AdamW optimizer, learning rate scheduler, and data augmentation techniques are employed to improve the model's robustness and generalization ability. This invention can effectively perform semantic and instance segmentation of point clouds and improve the accuracy and stability of target detection.
[0026] Furthermore, this method can simultaneously perform semantic segmentation, instance segmentation, and object detection, enabling multi-task processing of point cloud data and improving the utilization efficiency and information content of point cloud data.
[0027] Furthermore, this invention uses a self-attention mechanism and nonlocal modules to process point cloud data, enabling each point to pay attention to other related points, as well as semantic information between different scales and regions, thereby enhancing the feature representation and contextual understanding capabilities of point cloud data.
[0028] Furthermore, this invention uses a rendering branch to generate semantic labels and instance offsets for each point, enabling detailed segmentation and detection results for each point, thereby improving the accuracy and robustness of point cloud segmentation and object detection.
[0029] Furthermore, this invention uses a grouping loss function to optimize the performance of semantic segmentation and instance segmentation, enabling each point to simultaneously consider its semantic category and instance affiliation, thereby achieving collaborative learning of semantic segmentation and instance segmentation and improving the consistency and accuracy of point cloud segmentation and object detection.
[0030] Furthermore, this invention uses optimizers, learning rate schedulers, and data augmentation techniques to train the network, enabling the network to converge quickly and adapt to different data scenarios, thereby improving the generalization ability and stability of point cloud segmentation and object detection. Attached Figure Description
[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0032] Figure 1 This is a flowchart of point cloud segmentation and target detection methods;
[0033] Figure 2 This is a schematic diagram of the data preprocessing steps;
[0034] Figure 3 This is a schematic diagram of the network structure steps;
[0035] Figure 4 This is a schematic diagram of the loss function steps;
[0036] Figure 5 This is a schematic diagram of the optimization method steps. Detailed Implementation
[0037] The present invention will now be described in further detail with reference to the accompanying drawings:
[0038] The first aspect of the present invention provides a point cloud segmentation and target detection method, comprising the following steps:
[0039] S1. Perform data preprocessing. The input point cloud data is voxelized to a fixed voxel size to obtain a 3D mesh. This 3D mesh is a structure that divides the entire point cloud space into several fixed-size voxels. Each voxel is a basic unit within the 3D mesh, containing several point cloud data points within that space. Points in each voxel are uniformly sampled to obtain a sparse point cloud. The sparse point cloud is then divided into multi-scale hierarchical regions of different sizes. Points in each sub-region are randomly sampled and local feature aggregated to obtain a low-dimensional feature vector.
[0040] In some embodiments of the present invention, the voxelization step voxels the point cloud to a size of 0.05 meters.
[0041] In some embodiments of the present invention, the uniform sampling step involves randomly sampling points in each voxel and retaining only one representative point.
[0042] In some embodiments of the present invention, the multi-scale hierarchical partitioning step involves partitioning the sparse point cloud according to scales of 0.2 meters, 0.4 meters, 0.8 meters, and 1.6 meters.
[0043] In some embodiments of the present invention, the random sampling and local feature aggregation steps are performed by using a one-dimensional convolutional neural network to extract feature vectors.
[0044] S2 updates the low-dimensional feature vectors of points in each sub-region using a multi-layer self-attention mechanism. It also updates the low-dimensional feature vectors of sub-regions at different scales using multi-layer non-local module fusion. Each sub-region obtains its own final fused feature vector. For the finest-grained sub-regions, it uses a rendering branch to generate and output their semantic labels and instance offsets.
[0045] In some embodiments of the present invention, the self-attention mechanism is based on the [PointTransformer] self-attention mechanism.
[0046] In some embodiments of the present invention, the nonlocal module is based on the [PointASNL] nonlocal module. By using the nonlocal module, the similarity weights between sub-regions of different scales are calculated, thereby effectively fusing these updated feature vectors to obtain the final fused feature vector.
[0047] Specifically, each sub-region R_1, R_2, R_3, and R_4 at different scales will undergo the following processing:
[0048] 1) Update the low-dimensional feature vector of each point in the sub-region using a multi-layer self-attention mechanism;
[0049] 2) Using a multi-layer nonlocal module, the updated low-dimensional feature vectors of sub-regions at the same scale are fused to obtain a fused feature vector for that sub-region. At the end of this step, the four sub-regions R_1, R_2, R_3, and R_4 at different scales each obtain their own fused feature vector, resulting in four distinct fused feature vectors corresponding to four sub-regions at different granularities. These fused feature vectors contain the global semantics and contextual information of points within their respective sub-regions.
[0050] In some embodiments of the present invention, the rendering branch is a rendering branch based on [PointRend].
[0051] The above process constructs the entire point cloud segmentation and object detection network, and then optimizes the entire network using a group loss function and an optimizer.
[0052] In some embodiments of the present invention, a grouping loss function is used to perform semantic segmentation on semantic labels and instance segmentation on instance offsets.
[0053] The grouping loss function consists of two parts: one part is semantic loss, which uses cross-entropy loss to measure the difference between the semantic label and the true label of each point; the other part is offset loss, which uses smooth L1 loss to measure the offset between each point and the center of its instance.
[0054] In some embodiments of the present invention, an optimizer and a learning rate scheduler are used to train the network, and data augmentation techniques are used to transform the point cloud data to increase the diversity and robustness of the data.
[0055] Specifically, the network parameters are updated through the [AdamW] optimizer to minimize the difference between semantic labels and real labels, and to minimize the offset between each point and the center of its instance; thus, the entire network has better segmentation and detection performance.
[0056] During the process of updating network parameters, the [AdamW] optimizer adjusts the learning rate in the [AdamW] optimizer through the learning rate scheduler.
[0057] In some embodiments of the present invention, the point cloud data of S1 is processed by data augmentation techniques before use, including random rotation, translation, scaling, cropping, and noise addition.
[0058] The second aspect of this invention discloses a system for segmenting and detecting targets in underwater point cloud data of offshore wind power, comprising:
[0059] The preprocessing module is used to transform point cloud data into sparse point cloud, perform multi-scale hierarchical division of sparse point cloud to obtain sub-regions of multiple sizes, and extract low-dimensional feature vectors from the sub-regions.
[0060] The network structure module is used to update the low-dimensional feature vector of each point in the sub-region through a multi-layer self-attention mechanism; to fuse the updated low-dimensional feature vectors in each sub-region through a multi-layer non-local fusion module to obtain a fused feature vector; and to process the fused feature vectors in the finest-grained sub-regions through a rendering branch to generate semantic labels and instance offsets, thus obtaining a segmentation and object detection network.
[0061] The loss optimization module is used to iteratively update the parameters of the segmentation and object detection network through the optimizer, update the group loss function, and optimize the semantic label and instance offset through the group loss function to minimize the semantic loss of the semantic label and the instance offset, thereby obtaining the final segmentation and object detection network, and segmenting and detecting the target through the final segmentation and object detection.
[0062] The following description, in conjunction with specific embodiments, provides further details.
[0063] This embodiment provides a point cloud segmentation and target detection method; see [link to relevant documentation]. Figure 1 This includes the following steps:
[0064] Step 1, Data Preprocessing
[0065] First, data acquisition is performed using devices such as LiDAR or depth cameras to collect point cloud data of the 3D scene, resulting in a point cloud set P consisting of N points, where each point p_i contains its 3D coordinates (x_i, y_i, z_i) and other attribute information, such as color, normal vector, reflectivity, etc.
[0066] See Figure 2 The point cloud data P is voxelized to a size of 0.05 meters to obtain a 3D mesh G. Points in each voxel are randomly sampled, and only one representative point is retained to obtain a sparse point cloud S. The sparse point cloud S is then divided into multi-scale hierarchical regions at four scales: 0.2 meters, 0.4 meters, 0.8 meters, and 1.6 meters, resulting in four sub-region sets R_1, R_2, R_3, and R_4 with different granularities. Points in each sub-region are randomly sampled, and their local feature vectors are extracted using a one-dimensional convolutional neural network, resulting in four feature vector sets F_1, F_2, F_3, and F_4 with different granularities. (See also...) Figure 2 .
[0067] Step 2, see Figure 3 For each sub-region, a multi-layer self-attention mechanism is used to update its feature vector; for sub-regions of different scales, a multi-layer non-local module is used to fuse their feature vectors; and for the finest-grained sub-regions, a rendering branch is used to generate their semantic labels and instance offsets. Specifically:
[0068] 2.1 Self-Attention Mechanism: The self-attention mechanism proposed in [PointTransformer] is used to update the feature vector of each point by calculating the similarity weights between each point and other points. Specifically, for each granularity of feature vector set F_k (k = 1, 2, 3, 4), the self-attention output is calculated using the following formula:
[0069] Q_k=W_k^Q F_k
[0070] K_k=W_k^K F_k
[0071] V_k=W_k^V F_k
[0072] A_k=softmax(Q_k K_k^T / sqrt(d))
[0073] O_k=A_k V_k
[0074] in,
[0075] Q_k is the Query matrix, which is obtained by multiplying the feature vector set F_k by the learnable weight matrix W_k^Q.
[0076] K_k is the key matrix, which is obtained by multiplying the set of feature vectors F_k by the learnable weight matrix W_k^K.
[0077] V_k is the Value matrix, which is obtained by multiplying the set of feature vectors F_k by the learnable weight matrix W_k^V.
[0078] W_k^Q, W_k^K, and W_k^V are learnable linear transformation matrices, d is the feature dimension, A_k is the similarity weight matrix, and O_k is the self-attention output. See also Figure 3 .
[0079] W_k^Q is a learnable weight matrix, and F_k is the set of input feature vectors. Multiplying them yields the query matrix Q_k. The calculation of the other matrixes, Key K_k and Value V_k, is similar.
[0080] K_k^T represents the transpose of the Key matrix K_k. K_k needs to be transposed when calculating attention weights.
[0081] 2.2 Nonlocal Module: Using the nonlocal module proposed in [PointASNL], feature vectors are fused by calculating similarity weights between sub-regions of different scales. Specifically, for each set of feature vectors O_k and O_(k+1) (k = 1, 2, 3) of two adjacent granularities, the nonlocal output is calculated using the following formula:
[0082] M_(k+1) = maxpool(O_(k+1))
[0083] B_(k+1)=softmax(O_(k+1)M_(k+1)^T / sqrt(d))
[0084] C_(k+1)=B_(k+1)O_(k+1)
[0085] E_(k+1)=concat(O_k, C_(k+1))
[0086] Where M_(k+1) is the max pooling operation, B_(k+1) is the cross-scale similarity weight matrix, C_(k+1) is the cross-scale feature fusion result, and E_(k+1) is the cross-scale feature concatenation result.
[0087] 2.3 Rendering Branch: Using the rendering branch proposed in [PointRend], the semantic label Y and instance offset X for each point are generated through bilinear interpolation and pointwise classification. Specifically, for the finest-grained feature vector set E_4, the rendering output is calculated using the following formula:
[0088] L = linear(E_4)
[0089] I = bilinear(L, P)
[0090] Y = softmax(I)
[0091] Z = smooth_l1(I, C) where L is the linear transformation layer, I is the bilinear interpolation layer, Y is the semantic label output, Z is the instance offset output, and C is the instance center coordinates.
[0092] Loss function: see Figure 4 The performance of semantic segmentation and instance segmentation is optimized using a grouping loss function, which consists of two parts: a semantic loss, which uses cross-entropy loss to measure the difference between the semantic label and the ground truth label for each point; and an offset loss, which uses smoothed L1 loss to measure the offset between each point and its instance center. Specifically, the grouping loss function is calculated using the following formula:
[0093] L_group = L_sem + L_off
[0094] L_sem = -sum(y_i*log(Y_i)) / N
[0095] L_off = sum(|Z_i-C_i|) / N
[0096] Where y_i is the true semantic label of the i-th point, Y_i is the predicted semantic label of the i-th point, Z_i is the predicted instance offset of the i-th point, and C_i is the true instance center coordinate of the i-th point.
[0097] Optimization method: See Figure 5 The network was trained using the [AdamW] optimizer and cosine annealing learning rate scheduler, and data augmentation techniques were used to transform the point cloud data to increase the diversity and robustness of the data.
[0098] Specifically, the optimizer: The network parameters are updated using the [AdamW] optimizer, and the gradients and weights are calculated using the following formulas:
[0099] g_t=(grad(L_group,w_t)+lambda*w_t) / (1-beta_1^t)
[0100] m_t=beta_1*m_(t-1)+(1-beta_1)*g_t
[0101] v_t=beta_2*v_(t-1)+(1-beta_2)*g_t^2
[0102] w_(t+1)=w_t-alpha*m_t / (sqrt(v_t)+epsilon)
[0103] Where w_t is the network weight in the t-th iteration, g_t is the gradient in the t-th iteration, m_t is the first moment estimate in the t-th iteration, v_t is the second moment estimate in the t-th iteration, alpha is the learning rate, beta_1 and beta_2 are momentum coefficients, epsilon is the smoothing term, and lambda is the weight decay coefficient.
[0104] Specifically, the learning rate scheduler: The cosine annealing learning rate scheduler adjusts the learning rate in the optimizer described above, and the learning rate is calculated using the following formula:
[0105] alpha_t = alpha_min + 0.5 * (alpha_max - alpha_min) * (1 + cos(pi * t / T)) where alpha_t is the learning rate in the t-th iteration, alpha_min and alpha_max are the minimum and maximum values of the learning rate, and T is the total number of iterations.
[0106] Specifically, data augmentation techniques use methods such as random rotation, translation, scaling, cropping, and noise addition to transform the initial input point cloud data. This expands the diversity and complexity of training data, enhances the model's generalization ability, simulates various changes and disturbances that may occur in real-world scenarios, and improves the model's robustness. These include:
[0107] Random rotation: Rotate the point cloud data at random angles along any axis.
[0108] Translation: Transform point cloud data by translating it by a random distance along any direction.
[0109] Scaling: Performs a random scaling transformation on the point cloud data.
[0110] Crop: Perform a random range cropping transformation on the point cloud data to remove some irrelevant background or noise points.
[0111] Noise addition: Add some random noise points to the point cloud data to simulate incomplete or inaccurate data in real-world scenarios.
[0112] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for segmentation and target detection of underwater point cloud data from offshore wind power, characterized in that, Includes the following steps: S1. Transform the point cloud data into a sparse point cloud, perform multi-scale hierarchical division of the sparse point cloud to obtain multiple sub-regions of different sizes, and extract low-dimensional feature vectors from the sub-regions; voxelize the point cloud data according to a fixed voxel size to obtain a three-dimensional mesh; randomly sample the points in each voxel, retain one point in each voxel, and obtain a sparse point cloud. S2 updates the low-dimensional feature vector of each point in the sub-region through a multi-layer self-attention mechanism; it fuses the updated low-dimensional feature vectors in each sub-region through a multi-layer non-local fusion module to obtain a fused feature vector; and it processes the fused feature vectors in the finest-grained sub-regions through a rendering branch to generate semantic labels and instance offsets, thus obtaining the segmentation and object detection network. S3, the network parameters of the segmentation and object detection network are updated iteratively through the optimizer, and then the group loss function is updated. The semantic label and instance offset are optimized through the group loss function to minimize the semantic loss of the semantic label and the instance offset, thus obtaining the final segmentation and object detection network. The final segmentation and object detection network is used to segment and detect the target.
2. The method for segmentation and target detection of underwater point cloud data for offshore wind power according to claim 1, characterized in that, In S1, a low-dimensional feature vector is extracted from the sub-region using a one-dimensional convolutional neural network.
3. The method for segmentation and target detection of underwater point cloud data for offshore wind power according to claim 1, characterized in that, In S2, the self-attention mechanism is a PointTransformer-based self-attention mechanism.
4. The method for segmentation and target detection of underwater point cloud data for offshore wind power according to claim 1, characterized in that, In S2, the nonlocal fusion module is a PointASNL-based nonlocal module.
5. The method for segmentation and target detection of underwater point cloud data for offshore wind power according to claim 1, characterized in that, In S2, the rendering branch is the rendering branch of PointRend.
6. The method for segmentation and target detection of underwater point cloud data for offshore wind power according to claim 1, characterized in that, In S3, the network parameters of the segmentation and object detection network obtained in S2 are updated iteratively through the AdamW optimizer. During the iterative update process, the learning rate of the AdamW optimizer is adjusted through learning rate scheduling.
7. The method for segmentation and target detection of underwater point cloud data for offshore wind power according to claim 1, characterized in that, In S3, the grouping loss function includes semantic loss and offset loss. The semantic loss is measured by cross-entropy loss to measure the difference between the semantic label and the true label of each point. The offset loss is measured by smoothed L1 loss to measure the offset between each point and the center of its instance.
8. The method for segmentation and target detection of underwater point cloud data for offshore wind power according to claim 1, characterized in that, Before S1, the point cloud data is processed by data augmentation methods, including random rotation, translation, scaling, cropping, and noise addition.
9. A system for segmenting and detecting targets in underwater point cloud data of offshore wind power, characterized in that, include: The preprocessing module transforms point cloud data into sparse point cloud, performs multi-scale hierarchical division of the sparse point cloud to obtain sub-regions of multiple sizes, and extracts low-dimensional feature vectors from the sub-regions; it also voxels the point cloud data according to a fixed voxel size to obtain a 3D mesh; and it randomly samples the points in each voxel, retaining one point in each voxel to obtain the sparse point cloud. The network structure module is used to update the low-dimensional feature vector of each point in the sub-region through a multi-layer self-attention mechanism; to fuse the updated low-dimensional feature vectors in each sub-region through a multi-layer non-local fusion module to obtain a fused feature vector; and to process the fused feature vectors in the finest-grained sub-regions through a rendering branch to generate semantic labels and instance offsets, thus obtaining a segmentation and object detection network. The loss optimization module is used to iteratively update the parameters of the segmentation and object detection network through the optimizer, update the group loss function, and optimize the semantic label and instance offset through the group loss function to minimize the semantic loss of the semantic label and the instance offset, thereby obtaining the final segmentation and object detection network, and segmenting and detecting the target through the final segmentation and object detection.