Battery replacement robot target point cloud segmentation method based on multi-scale attention aggregation

Abstract

The invention discloses a multi-scale attention aggregation-based target point cloud segmentation method for a battery replacement robot, and the method comprises the steps: 1, collecting an RGB image, a depth image and three-dimensional point cloud data of a target fastener through a binocular structured light depth camera, and carrying out the fusion to generate a FastSeg3D data set; 2, a two-stage preprocessing method is provided, noise points are removed through radius filtering, and background point clusters far away from a target are removed through DBSCAN density clustering; 3, the network encoder uses a local feature aggregation module to extract geometric features, and the calculation complexity is reduced in combination with a random sampling strategy; 4, embedding a multi-scale attention aggregation module into the jump connection of the encoder and the decoder, fusing the features through a channel and a space attention unit, and achieving the self-adaptive weight weighting of the features of each layer of the encoder; and 5, recovering the resolution of the original point cloud by adopting nearest neighbor interpolation up-sampling, outputting a segmentation semantic tag, and obtaining a high-quality point cloud target. According to the method, the operation time of the battery replacement robot is shortened, and the balance problem of large-scale target point cloud segmentation speed and precision is solved.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent battery replacement for electric vehicles, and specifically discloses a target point cloud segmentation method for a battery replacement robot based on multi-scale attention aggregation. Background Art

[0002] During the battery swapping process in new energy electric vehicles, accelerating battery swaps and reducing operation time are key research issues in automated battery swapping technology. Target fasteners are key mechanical components connecting the battery pack to the vehicle body. Therefore, their rapid removal and installation directly impact the efficiency of battery swapping robots in accurately disassembling the battery panels. LiDAR or depth cameras are commonly used as sensors for target fastener segmentation and extraction. While LiDAR excels at capturing large-scale point clouds, its accuracy is limited and it is susceptible to noise when detecting small objects. In battery swapping stations, target fasteners are small and located at the edge or bottom of the vehicle's battery panels. Therefore, depth sensors are the most preferred choice for processing small objects due to their high resolution and close-range accuracy. However, due to the large size of the target point cloud captured by high-precision depth cameras and the significant point cloud differences between the battery panel and the target fastener, background noise in the point cloud leads to uneven point cloud density distribution in the target area, significantly reducing the robustness of the segmentation model. Therefore, how to effectively remove background point clouds and perform fast and accurate object segmentation is a key issue in improving battery swapping accuracy and efficiency.

[0003] Technical comparison with patent CN119671908A "A method for dynamic point cloud classification and prediction label correction of an inspection robot";

[0004] Patent CN119671908A proposes a method for dynamic point cloud segmentation and predicted label correction of an inspection robot, which is mainly used in the scenario of train undercarriage inspection robots, and aims to solve the problem of interference of dynamic objects (such as personnel) in the shared human-machine maintenance channel environment with the positioning accuracy of the 3D laser SLAM system. This method improves the positioning accuracy and operating efficiency of the inspection robot in a narrow and highly repeatable environment by identifying and separating dynamic point clouds such as moving personnel, and reduces the need for personnel to frequently enter and exit the maintenance channel. This patent proposes a target point cloud segmentation method for a battery swapping robot based on multi-scale attention aggregation, which is mainly used in the target fastener identification and segmentation scenario in the battery swapping environment of electric vehicles. By high-precision segmentation of fasteners and background point clouds, it provides support for subsequent pose estimation and precise disassembly and assembly, thereby improving the battery swap efficiency of electric vehicles. This method focuses on processing target fasteners with small volume and complex geometric details, and uses the high-resolution advantages of depth cameras to propose solutions to problems such as large-scale point cloud data and background noise interference. There are essential differences in the application scenarios of the two.

[0005] Patent CN119671908A uses a solid-state lidar to collect point cloud data, crops the point cloud within the inspection channel through a straight-through filter, and projects the 3D point cloud into a polar coordinate system to form a bird's-eye view to balance the spatial distribution unevenness of the point cloud; uses a laser odometry to determine the transformation matrix, constructs a sliding time window to integrate the current frame and the previous and next frame data, and extracts the motion characteristics of dynamic objects by examining the height difference in the bird's-eye view; dynamic objects are recognized based on the dual-branch structure and appearance-motion co-attention mechanism of PolarNet; finally, a combination of density-based spatial clustering (DBSCAN) and cloth simulation filtering (CSF) is used to correct the predicted labels to improve the accuracy of dynamic point cloud segmentation. This patent uses a binocular structured light depth camera to collect RGB images, depth maps, and point cloud data of target fasteners to construct the FastSeg3D dataset. A radius filtering algorithm is used to remove outliers and local noise, and DBSCAN density clustering is combined to remove distant background point clusters. An encoder-decoder architecture is constructed, and a local feature aggregation module is used to extract geometric features of the point cloud, combining a random sampling strategy to reduce computational complexity. A multi-scale attention aggregation module is introduced to fuse features of different resolutions through channel attention units and spatial attention units to achieve adaptive weighting. The multi-scale attention module is embedded in the jump connection between the encoder and decoder, and point cloud density is restored through nearest neighbor interpolation upsampling. Finally, the network outputs semantic labels to separate the target fastener from the background point cloud. The two technical solutions differ fundamentally.

[0006] Patent CN119671908A focuses on dynamic point cloud segmentation. It first uses a straight-through filter to limit the point cloud processing range and projects the 3D point cloud onto a bird's-eye view of the polar coordinate system. It constructs a sliding window mechanism to obtain motion features by calculating the height differences of grid cells in continuous time windows (W1 and W2), and these features are represented in the form of residuals. It adopts an appearance-motion co-attention module, including a co-attention gate and a motion-guided attention module based on motion cues, to enhance the cross-modal interaction between appearance and motion features. Finally, in the label correction stage, the DBSCAN clustering algorithm and cloth simulation filtering are used to correct mislabeling, and the segmentation accuracy is improved by identifying high-density areas and eliminating isolated noise. This patent focuses on target point cloud segmentation in static scenes. It first uses radius filtering to remove isolated noise points and outliers, then applies DBSCAN density clustering to identify and remove high-density background point clusters far from the target. In terms of network structure, it builds an encoder-decoder architecture based on RandLA-Net. The encoder extracts point cloud geometric features through a local feature aggregation module and uses a random sampling strategy to reduce computational complexity. It innovatively introduces a multi-scale attention aggregation module, which fuses features of different resolutions through channel attention (using global average pooling, maximum pooling, and shared fully connected layers) and spatial attention (using multi-scale convolutional kernels) to achieve detail enhancement. The decoder restores point cloud density through nearest neighbor interpolation upsampling and reconstructs target details using multi-scale features. The two technologies differ fundamentally in their technical approaches.

[0007] Patent CN119671908A has achieved remarkable results in dynamic point cloud segmentation and label correction. In the simulated inspection channel environment, the algorithm's mIoU (mean intersection over union) reached 89.21% and Acc (accuracy) reached 95.62%, which is a significant improvement compared to PointNet and PointNet++. This method can effectively identify and segment dynamic targets such as people, minimize the interference of dynamic point clouds on SLAM, and enable inspection robots to maintain stable positioning accuracy in environments where people coexist. The label correction technology correctly handles misidentification situations and improves the robustness of the system. The feature extraction method based on 2D BEV and the label correction strategy based on spatial clustering have improved the system's processing efficiency and accuracy for low-cost solid-state lidar data. This patent demonstrates excellent performance in the fine segmentation of target fasteners. Through the two-stage preprocessing of radius filtering and DBSCAN density clustering, the signal-to-noise ratio of the target point cloud is significantly improved, and more than 80% of background noise points are effectively removed. The multi-scale attention aggregation module significantly improves the segmentation accuracy of the model for complex geometric structures such as the edges and threads of target fasteners, and improves the edge accuracy compared to the baseline model. The random sampling and local feature aggregation strategy reduces the computational complexity of point cloud processing from O(n 2) to O(n), enabling the processing of large-scale point clouds (>1 million points) on standard computing hardware, with a 5-8x increase in processing speed. The model maintains stable segmentation performance in challenging scenarios such as viewpoint changes, varying lighting conditions, and partial occlusion, providing high-quality target point clouds for battery-swap robots and significantly improving the accuracy and speed of subsequent registration and pose estimation. The two technologies differ fundamentally in their effectiveness. Summary of the Invention

[0008] To solve the above technical problems, the present invention proposes a target point cloud segmentation method for a battery-swapping robot based on multi-scale attention aggregation, which can improve the overall replacement time of the battery-swapping robot and break the limitation of being unable to directly, quickly and accurately process large-scale target point cloud segmentation.

[0009] To achieve the above object, the technical solution adopted by the present invention is:

[0010] The target point cloud segmentation method for battery swapping robots based on multi-scale attention aggregation includes the following steps:

[0011] (1) Use a binocular structured light depth camera to shoot and obtain the RGB image, depth image and three-dimensional point cloud data of the target fastener, and construct a target fastener point cloud dataset FastSeg3D containing multi-scene annotations, denoted as point cloud P0;

[0012] (2) Preprocessing the original point cloud P0 data, specifically including: filtering the point cloud P0 using a radius filtering algorithm to remove outliers and local noise, and obtaining the preprocessed point cloud P1; clustering the point cloud P1 using a density-based DBSCAN clustering algorithm to remove irrelevant point clusters in the distant background of the solar panel, and obtaining the point cloud P2;

[0013] (3) Obtain the neighborhood feature representation F1 of each point in the point cloud through the local spatial encoding unit, and use the attention pooling mechanism to filter and aggregate to obtain the attention score Attention score Multiply element-by-element with the neighborhood feature F1 to obtain the weighted aggregated feature Features obtained using the local feature aggregation module Random sampling is performed to obtain point cloud P3 to reduce the amount of computation. The encoder is implemented by stacking multiple sets of local spatial encoding and attention pooling units;

[0014] (4) The multi-scale attention aggregation module efficiently captures long-range dependencies through channel and spatial attention mechanisms. The module receives the feature tensors from the high-resolution, medium-resolution, and low-resolution encoders and concatenates them into the feature F2, which is input into the channel attention unit and the spatial attention unit respectively. The channel attention unit generates a channel description vector through global average pooling, and dynamically adjusts the channel weight by combining the fully connected layer and the activation function to obtain the channel weight vector W. C The spatial attention unit uses parallel 3×3, 5×5, and 7×7 convolution kernels to extract local features of different receptive fields and obtain the spatial weight vector W s , the multi-scale attention aggregation module is embedded in the skip connection between the encoder and the decoder, the output features of each layer of the encoder are spliced and input into the multi-scale attention aggregation module for processing, and then spliced layer by layer with the upsampled features of the decoder;

[0015] (5) The decoder takes the feature map after downsampling and feature extraction as input, and uses nearest neighbor interpolation upsampling to restore the features layer by layer to the resolution of the original point cloud to obtain point cloud P4;

[0016] (6) The semantic labels output by the segmentation network are indexed according to the coordinate dimension and feature dimension of the point cloud P4, and the point cloud regions corresponding to different label values are filtered out. Then, based on the filtering results, the original point cloud is split into two independent point cloud subsets, representing the solar panel and the target fastener, respectively.

[0017] As a further improvement of the present invention, the setting method of the binocular structured light depth camera in step (1) is:

[0018] (1-1) Mount the camera on the end of the battery swap robot and move it along the track to 40-60 cm below the target battery pack for data collection;

[0019] (1-2) Set the camera depth range to 50-3000mm, the depth map resolution to 960×600, and the color map resolution to 1920×1080. Ensure the accuracy of the coordinate system conversion through calibration tests.

[0020] (1-3) Data collection covers locked and unlocked states, different lighting conditions, shooting distances, tilt angles, and clutter interference scenarios to ensure the generalization of the dataset.

[0021] As a further improvement of the present invention, the pretreatment in step (2) includes:

[0022] (2-1) Radius filtering: Set the search radius r = 1.0 mm and the minimum number of neighbor points n = 10, remove noise points with a neighborhood density lower than the threshold, and obtain the point cloud P1;

[0023] (2-2) DBSCAN density clustering: Based on the k-nearest neighbor distance distribution curve, the optimal neighborhood radius ε and the minimum number of points minPts are determined, low-density background point clusters are identified and eliminated, the signal-to-noise ratio of the target point cloud is improved, and the point cloud P2 is obtained.

[0024] As a further improvement of the present invention, the encoder in step (3) is specifically implemented as follows:

[0025] (3-1) The local spatial encoding unit first calculates the neighborhood index of each point in the preprocessed point cloud P2 using the K-nearest neighbor algorithm. It then calculates the local features of the points within the neighborhood based on their spatial positions and neighborhood relationships, and then extracts the local geometric information features F1 through a convolution operation. In this way, the module can capture the fine-grained geometric shapes in the point cloud, including local geometric features such as the distance between points and normal vectors, thereby enhancing the network's sensitivity and representation capabilities for local features and effectively improving the accuracy of point cloud processing.

[0026] (3-2) The attention pooling unit filters and aggregates the output features F1 of the local spatial encoding unit through global average pooling to generate an attention score These weighting coefficients are used to dynamically adjust the weight W of each point i , which enables the model to pay more attention to the parts of the point cloud containing important geometric information in the subsequent stages, and the attention score Multiply element-by-element with the neighborhood feature F1 to obtain the weighted aggregated feature

[0027] (3-3) The point cloud after local spatial feature aggregation is randomly downsampled. A random sampling strategy with linear complexity is used to select points proportionally from the point cloud. First, the sampling rate is set to retain 10%-20% of the points. Then, sampling points are selected from the point cloud in a uniformly distributed manner to ensure that the computational complexity is reduced without losing key geometric features. A weighted strategy is used to prioritize points with important geometric information, such as edges or target areas, to further improve the quality of the sampled point cloud and obtain the sampled point cloud P3.

[0028] As a further improvement of the present invention, the specific implementation of the multi-scale attention aggregation module in step (4) is:

[0029] (4-1) Receive the feature tensors of the high-resolution, medium-resolution, and low-resolution encoders output from each layer of the encoder, and perform vector concatenation to obtain the feature F2 = [B, N, 1, C fused ], firstly, the feature F2 is uniformly reduced to the shared latent space D = [C out / α], α is the channel compression factor, and the feature F′2=[B,N,1,C fused / 4], and then input F′2 into the channel attention unit and the spatial attention unit respectively;

[0030] (4-2) Channel attention unit: compress the input feature F′2 along the spatial dimension H×W to generate a channel description vector F avg 、F max And perform pooling, then pass through the fully connected layer including two layers of shared convolution, and perform two nonlinear transformations on the pooling results to obtain the feature F′ avg , F′ max , add the outputs of the two branches and pass them through the Sigmoid activation function to obtain the channel attention weight W C , and finally through the channel attention weight W C Scaling the input feature F2 channel by channel to obtain the output feature F′ C .

[0031] (4-3) Spatial attention unit: 3×3, 5×5, and 7×7 convolution kernels are applied to the input feature F′2 to extract multi-scale local features. After the three features are concatenated, 1×1 convolution is used to reduce the dimension to generate the feature group F used , F used Input spatial attention module to perform adaptive reweighting of spatial dimensions. First, the mean and maximum values are calculated along the channel dimension to generate two spatial attention basis vectors F used_avg 、F used_max , followed by 7×7 convolution to extract the spatial context W S , and finally through the spatial attention weight W S Scaling the input feature F′2 channel by channel to obtain the output feature F′ S , enhancing the response of important areas.

[0032] As a further improvement of the present invention, the specific implementation of the decoder in step (5) is:

[0033] (5-1) For the input feature matrix F and the pre-prepared nearest neighbor index of each target point, which is the interpolation index I, calculate the interpolated feature matrix F';

[0034] (5-2) The decoder optimizes the edge segmentation accuracy by fusing multi-scale features F' layer by layer, improves the ability to retain the geometric details of the target fastener thread, and finally obtains an accurate segmentation result point cloud P4.

[0035] As a further improvement of the present invention, the point cloud subset extraction method in step (6) is:

[0036] (6-1) Output the coordinates of point cloud P4 according to the semantic label value (0 / 1) index, separate the point cloud of the solar panel and the target fastener, and obtain point cloud P5;

[0037] (6-2) The extracted point cloud data is saved in PLY or PCD format, which supports direct call of subsequent registration algorithms.

[0038] Beneficial effects:

[0039] The present invention discloses a patented method for target point cloud segmentation of a battery-swapping robot based on multi-scale attention aggregation. This method collects RGB images, Depth images and point cloud data of target fasteners through a binocular structured light depth camera, constructs a FastSeg3D dataset covering a variety of complex scenes, and provides high-quality data support for model training. In the data preprocessing stage, radius filtering and DBSCAN density clustering algorithms are used to effectively remove outliers and distant background noise, significantly improving the signal-to-noise ratio of the target point cloud. In the point cloud segmentation stage, based on the improved MSA-RandLA-Net network, combined with the multi-scale attention aggregation module, random sampling and local feature aggregation are used to significantly reduce the computational complexity while retaining the geometric details of the point cloud. The multi-scale attention aggregation module enhances the model's processing ability for complex target edges and improves segmentation accuracy through channel attention weighting and spatial attention weighting. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the method disclosed in the present invention. DETAILED DESCRIPTION

[0041] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0042] The present invention provides a patented method for target point cloud segmentation of a battery-swapping robot based on multi-scale attention aggregation, which can improve the overall replacement time of the battery-swapping robot and break the limitation of being unable to directly, quickly and accurately process large-scale target point cloud segmentation.

[0043] As an embodiment of the present invention, the present invention provides a patent - a target point cloud segmentation method for a battery-swapping robot based on multi-scale attention aggregation, wherein the target fastener segmentation flow chart is as follows: Figure 1 As shown, this is achieved through the following steps:

[0044] (1) Use a binocular structured light depth camera to shoot and obtain the RGB image, depth image and three-dimensional point cloud data of the target fastener, and construct a target fastener point cloud dataset FastSeg3D containing multi-scene annotations, denoted as point cloud P0;

[0045] (2) Preprocessing the original point cloud P0 data, specifically including: filtering the point cloud P0 using a radius filtering algorithm to remove outliers and local noise, and obtaining the preprocessed point cloud P1; clustering the point cloud P1 using a density-based DBSCAN clustering algorithm to remove irrelevant point clusters in the distant background of the solar panel, and obtaining the point cloud P2;

[0046] (3) Obtain the neighborhood feature representation F1 of each point in the point cloud through the local spatial encoding unit, and use the attention pooling mechanism to filter and aggregate to obtain the attention score Attention score Multiply element-by-element with the neighborhood feature F1 to obtain the weighted aggregated feature Features obtained using the local feature aggregation module Random sampling is performed to obtain point cloud P3 to reduce the amount of computation. The encoder is implemented by stacking multiple sets of local spatial encoding and attention pooling units.

[0047] (4) The multi-scale attention aggregation module efficiently captures long-range dependencies through channel and spatial attention mechanisms. This module receives the feature tensors from the high-resolution, medium-resolution, and low-resolution encoders and concatenates them into the feature F2, which is input into the channel attention unit and the spatial attention unit respectively. The channel attention unit generates a channel description vector through global average pooling, and dynamically adjusts the channel weight by combining the fully connected layer and the activation function to obtain the channel weight vector W. C The spatial attention unit uses parallel 3×3, 5×5, and 7×7 convolution kernels to extract local features of different receptive fields and obtain the spatial weight vector W s The multi-scale attention aggregation module is embedded in the skip connection between the encoder and the decoder. The output features of each layer of the encoder are spliced and input into the multi-scale attention aggregation module for processing, and then spliced layer by layer with the upsampled features of the decoder.

[0048] (5) The decoder takes the feature map after downsampling and feature extraction as input, and uses nearest neighbor interpolation upsampling to restore the features layer by layer to the resolution of the original point cloud to obtain point cloud P4;

[0049] (6) The semantic labels output by the segmentation network are indexed according to the coordinate and feature dimensions of point cloud P4, and the point cloud regions corresponding to different label values are filtered out. Subsequently, based on the filtering results, the original point cloud is split into two independent point cloud subsets, representing the solar panel and the target fastener, respectively.

[0050] The setting method of the binocular structured light depth camera in step (1) is:

[0051] (1-1) The camera is mounted on the end of the battery swap robot and moves along the track to 40-60 cm below the target battery pack to collect data;

[0052] (1-2) Set the camera depth range to 50-3000mm, the depth map resolution to 960×600, and the color map resolution to 1920×1080. Ensure the accuracy of the coordinate system conversion through calibration tests.

[0053] (1-3) Data collection covers locked and unlocked states, different lighting conditions, shooting distances, tilt angles, and clutter interference scenarios to ensure the generalization of the dataset.

[0054] The point cloud data preprocessing method used in step (2) includes the following steps:

[0055] (2-1) For any point P in the point cloud i , calculate the number of neighbor points N within the radius r i , if N i <k min , then the point is determined to be a noise point and is removed from the point cloud. Its mathematical expression is as follows:

[0056]

[0057] Where I(·) is an indicator function, which takes the value 1 when the condition is met and 0 otherwise. By adjusting the radius r and the minimum number of neighbor points k min , which can dynamically adapt to the point cloud data characteristics in different scenarios and effectively remove isolated points and low-density points in the point cloud.

[0058] (2-2) For any point p in the FastSeg3D dataset i , define its domain N ε (p i ) is the set of all points within the distance ε (neighborhood radius):

[0059] N ε (p i )={q∈D|dist(p i ,q)≤ε}

[0060] Where D is the point cloud dataset. dist(p,q) is the Euclidean distance between points p and q.

[0061] (2-3) If point p i Field N ε (p i ) contains fewer points than MinPts objects, but in the core object p j In the field of i Boundary point, also called p i By p j Direct density reachable. Otherwise, point p iIt is the noise point.

[0062] (2-4) If and only if there exists a path connected by the core points that satisfies: So that p1=p,p n =q, and dist(p i ,p i+1 )≤ε(1≤i≤n), point q is said to be from point p i Density reachable. Objects q and p are said to be density connected if there exists an object o∈D such that both objects q and p are density reachable from o with respect to ε and MinPts.

[0063] (2-5) Clusters are searched by examining the neighborhood of each point in the point cloud. If the neighborhood of point p contains more than MinPts points, a new cluster is created with p as the core object. DBSCAN then iteratively clusters objects that are density-reachable from these core objects, merging density-reachable clusters. This process is repeated until all core points have been visited and no new points can be added to any clusters. At this point, points that do not belong to any cluster are considered noise.

[0064] The encoder of step (3) comprises the following steps:

[0065] (3-1) Finding adjacent points: To improve efficiency, for the i-th point, its neighborhood point index is first collected using the K-nearest neighbor (KNN) algorithm based on the Euclidean distance between points.

[0066] (3-2) Relative point position encoding: For the K points closest to the center point {P i l ...P i k ...P i K}, we explicitly encode the relative point position as follows:

[0067]

[0068] where p i and P i K is the xyz coordinate of the point, is the splicing operation, ||·|| calculates the Euclidean distance between adjacent points and the center point, r i k It is obtained by encoding the adjacent point positions.

[0069] (3-3) Point feature enhancement: For each adjacent point P i K , the encoded relative point position r ik The corresponding point feature f i k Splicing is performed to obtain the enhanced feature vector

[0070] (3-3) Finally, the local spatial encoding unit outputs a new set of adjacent features It is the center point p i The local geometric structure is explicitly encoded.

[0071] (3-4) Calculate the attention score: given a set of local features A unique attention score is learned for each feature through a shared function g(). Basically, the function g() consists of a shared multi-layer perceptron (shared MLP) followed by a softmax function. Its formal definition is as follows:

[0072]

[0073] Where W is the learnable weight of the shared multilayer perceptron.

[0074] (3-5) Random downsampling: From the input point cloud P = {P1, P2, ..., P n}, K points are randomly selected from uniform distribution to form a subset S = {P σ(1) ,P σ(2) ,...,P σ(k)}, where σ(i) is a random index independently generated from the interval [1,N], satisfying σ(i) ~ Uniform(1,N).

[0075] The multi-scale attention aggregation module in step (4) includes the following steps:

[0076] (4-1) Receive the feature tensors of the high-resolution, medium-resolution, and low-resolution encoders output from each layer of the encoder, and perform vector concatenation to obtain the feature F2 = [B, N, 1, C fused ], firstly, the feature F2 is uniformly reduced to the shared latent space D = [C out / α], α is the channel compression factor, and the feature F′2=[B,N,1,C fused / 4]. F′2 is then input into the channel attention unit and the spatial attention unit respectively.

[0077] (4-2) Channel attention unit: compress the input feature F' along the spatial dimension H×W to generate a channel description vector F avg 、F max :

[0078]

[0079]

[0080] After pooling, F avg 、F max The features are transformed twice nonlinearly on the pooled results through the fully connected layer (shared two-layer convolution):

[0081] F′ avg =W2·ReLU(W1·(F avg ))

[0082] F′ max =W2·ReLU(W1·(F max ))

[0083] Where W1∈R C / r×C , W2∈R C×C / r , r is the channel compression factor.

[0084] Add the outputs of the two branches and pass them through the Sigmoid activation function to get the channel attention weight W C ∈R B ×C×1×1 :

[0085] W C =σ(F′ avg +F' max )

[0086] Where σ(·) is the Sigmoid function,

[0087] Final output feature F′ C The input feature F' is scaled channel by channel by the channel attention weight:

[0088] F′ C =F'⊙W C

[0089] Among them, ⊙ represents channel-by-channel multiplication, and the feature map of each channel is W C This operation highlights the characteristic responses of important channels and suppresses redundant information.

[0090] (4-3) Spatial attention unit: Apply 3×3, 5×5, and 7×7 convolution kernels to the input feature F' to extract multi-scale local features and generate feature groups:

[0091] F 3×3 =Conv 3×3 (F'),F 5×5 =Conv 5×5 (F'),F 7×7=Conv 7×7 (F')

[0092] Fuse by element-wise addition:

[0093] F used =F 3×3 +F 5×5 +F 7×7

[0094] F used The input spatial attention module performs adaptive reweighting of the spatial dimension. First, the mean and maximum values are calculated along the channel dimension to generate two spatial attention basis vectors:

[0095]

[0096] Output tensor F used_avg 、F used_max ∈R B×C×1×1 , characterize the global and local salient areas. Concatenate the results of average pooling and maximum pooling in the channel dimension Concat(F used_avg ,F used_max ∈R B×2×H×W ), and then extract the spatial context through 7×7 convolution:

[0097]

[0098] Where W i,j is the 7×7 convolution kernel parameter, σ(·) is the Sigmoid function, and the final output is the spatial attention weight W S ∈R B×1×H×W .

[0099] Final output feature F′ S By using the spatial attention weight W S For the input feature F used Perform channel-by-channel scaling to enhance response in important areas:

[0100] F′ S =F used ⊙W S

[0101] This operation enables the network to focus on areas with significant geometric structures such as the edges and threads of the target fasteners, suppressing background noise.

[0102] (4-4) Channel attention unit and spatial attention unit fusion: Finally, the weighted feature F′ S Feature F′ weighted with channel attention C The images are added together and restored to the target number of channels through 1×1 convolution to complete the adaptive fusion of cross-scale features.

[0103] The decoder of step (5) comprises the following steps:

[0104] (5-1) Known input feature matrix: F∈R B×N×d , where B is the batch size, N is the number of points, and d is the feature dimension; the nearest neighbor index of each target point prepared in advance is the index to be interpolated I∈R B×M , where M is the number of points after upsampling; the feature matrix after interpolation F'∈R B×M×d Calculated as follows:

[0105] F'[b,m,:]=F[b,I[b,m],:]

[0106] That is, for each batch b and target point m, the original features corresponding to index I[b,m] are directly copied to the interpolated feature matrix. This operation expands the feature tensor from [B,N,d] to [B,M,d], significantly increasing point cloud density while preserving local feature consistency. The interpolated features are then semantically enhanced using a subsequent multi-scale attention aggregation module and spatially refined using transposed convolutions, ultimately achieving feature reconstruction from sparse high-level semantics to dense low-level geometry.

[0107] The point cloud extraction in step (6) includes the following steps:

[0108] (5-1) Through the layer-by-layer feature fusion of the decoder, the final output segmentation result point cloud P seg ∈R N×2 , where the semantic label of each point indicates whether it belongs to the battery panel or the target fastener:

[0109] P seg =Softmax(FC(F output ))

[0110] Among them, FC is the fully connected layer and Softmax is the normalization operation.

[0111] The above description is merely a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent variation based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A target point cloud segmentation method for a battery swapping robot based on multi-scale attention aggregation, characterized by: The steps include: (1) Use a binocular structured light depth camera to shoot and obtain the RGB image, depth image and three-dimensional point cloud data of the target fastener, and construct a target fastener point cloud dataset FastSeg3D containing multi-scene annotations, denoted as point cloud P0; (2) Preprocessing the original point cloud P0 data, specifically including: filtering the point cloud P0 using a radius filtering algorithm to remove outliers and local noise, and obtaining the preprocessed point cloud P1; clustering the point cloud P1 using a density-based DBSCAN clustering algorithm to remove irrelevant point clusters in the distant background of the solar panel, and obtaining the point cloud P2; (3) Obtain the neighborhood feature representation F1 of each point in the point cloud through the local spatial encoding unit, and use the attention pooling mechanism to filter and aggregate to obtain the attention score Attention score Multiply element-by-element with the neighborhood feature F1 to obtain the weighted aggregated feature Features obtained using the local feature aggregation module Random sampling is performed to obtain point cloud P3 to reduce the amount of computation. The encoder is implemented by stacking multiple sets of local spatial encoding and attention pooling units; (4) The multi-scale attention aggregation module efficiently captures long-range dependencies through channel and spatial attention mechanisms. The module receives the feature tensors from the high-resolution, medium-resolution, and low-resolution encoders and concatenates them into the feature F2, which is input into the channel attention unit and the spatial attention unit respectively. The channel attention unit generates a channel description vector through global average pooling, and dynamically adjusts the channel weight by combining the fully connected layer and the activation function to obtain the channel weight vector W. C The spatial attention unit uses parallel 3×3, 5×5, and 7×7 convolution kernels to extract local features of different receptive fields and obtain the spatial weight vector W s , the multi-scale attention aggregation module is embedded in the skip connection between the encoder and the decoder, the output features of each layer of the encoder are spliced and input into the multi-scale attention aggregation module for processing, and then spliced layer by layer with the upsampled features of the decoder; (5) The decoder takes the feature map after downsampling and feature extraction as input, and uses nearest neighbor interpolation upsampling to restore the features layer by layer to the resolution of the original point cloud to obtain point cloud P4; (6) The semantic labels output by the segmentation network are indexed according to the coordinate dimension and feature dimension of the point cloud P4, and the point cloud regions corresponding to different label values are filtered out. Then, based on the filtering results, the original point cloud is split into two independent point cloud subsets, representing the solar panel and the target fastener, respectively.

2. The target point cloud segmentation method for battery swapping robots based on multi-scale attention aggregation according to claim 1 is characterized in that: The setting method of the binocular structured light depth camera in step (1) is: (1-1) Mount the camera on the end of the battery swap robot and move it along the track to 40-60 cm below the target battery pack for data collection; (1-2) Set the camera depth range to 50-3000mm, the depth map resolution to 960×600, and the color map resolution to 1920×1080. Ensure the accuracy of the coordinate system conversion through calibration tests. (1-3) Data collection covers locked and unlocked states, different lighting conditions, shooting distances, tilt angles, and clutter interference scenarios to ensure the generalization of the dataset.

3. The target point cloud segmentation method for battery swapping robots based on multi-scale attention aggregation according to claim 1 is characterized in that: The pretreatment in step (2) includes: (2-1) Radius filtering: Set the search radius r = 1.0 mm and the minimum number of neighbor points n = 10, remove noise points with a neighborhood density lower than the threshold, and obtain the point cloud P1; (2-2) DBSCAN density clustering: Based on the k-nearest neighbor distance distribution curve, the optimal neighborhood radius ε and the minimum number of points minPts are determined, low-density background point clusters are identified and eliminated, the signal-to-noise ratio of the target point cloud is improved, and the point cloud P2 is obtained.

4. The target point cloud segmentation method for battery swapping robots based on multi-scale attention aggregation according to claim 1 is characterized in that: The specific implementation of the encoder in step (3) is: (3-1) The local spatial encoding unit first calculates the neighborhood index of each point in the preprocessed point cloud P2 using the K-nearest neighbor algorithm. It then calculates the local features of the points within the neighborhood based on their spatial positions and neighborhood relationships, and then extracts the local geometric information features F1 through a convolution operation. In this way, the module can capture the fine-grained geometric shapes in the point cloud, including local geometric features such as the distance between points and normal vectors, thereby enhancing the network's sensitivity and representation capabilities for local features and effectively improving the accuracy of point cloud processing. (3-2) The attention pooling unit filters and aggregates the output features F1 of the local spatial encoding unit through global average pooling to generate an attention score These weighting coefficients are used to dynamically adjust the weight W of each point i , which enables the model to pay more attention to the parts of the point cloud containing important geometric information in the subsequent stages, and the attention score Multiply element-by-element with the neighborhood feature F1 to obtain the weighted aggregated feature (3-3) The point cloud after local spatial feature aggregation is randomly downsampled. A random sampling strategy with linear complexity is used to select points proportionally from the point cloud. First, the sampling rate is set to retain 10%-20% of the points. Then, sampling points are selected from the point cloud in a uniformly distributed manner to ensure that the computational complexity is reduced without losing key geometric features. A weighted strategy is used to prioritize points with important geometric information, such as edges or target areas, to further improve the quality of the sampled point cloud and obtain the sampled point cloud P3.

5. The target point cloud segmentation method for battery swapping robots based on multi-scale attention aggregation according to claim 1 is characterized in that: The specific implementation of the multi-scale attention aggregation module in step (4) is: (4-1) Receive the feature tensors of the high-resolution, medium-resolution, and low-resolution encoders output from each layer of the encoder, and perform vector concatenation to obtain the feature F2 = [B, N, 1, C fused ], firstly, the feature F2 is uniformly reduced to the shared latent space D = [C out / α], α is the channel compression factor, and the feature F′2=[B,N,1,C fused / 4], and then input F′2 into the channel attention unit and the spatial attention unit respectively; (4-2) Channel attention unit: compress the input feature F′2 along the spatial dimension H×W to generate a channel description vector F avg 、F max And perform pooling, then pass through the fully connected layer including two layers of shared convolution, and perform two nonlinear transformations on the pooling results to obtain the feature F′ avg , F′ max , add the outputs of the two branches and pass them through the Sigmoid activation function to obtain the channel attention weight W C , and finally through the channel attention weight W C Scaling the input feature F2 channel by channel to obtain the output feature F′ C . (4-3) Spatial attention unit: 3×3, 5×5, and 7×7 convolution kernels are applied to the input feature F′2 to extract multi-scale local features. After the three features are concatenated, 1×1 convolution is used to reduce the dimension to generate the feature group F used , F used Input spatial attention module to perform adaptive reweighting of spatial dimensions. First, the mean and maximum values are calculated along the channel dimension to generate two spatial attention basis vectors F used_avg 、F used_max , followed by 7×7 convolution to extract the spatial context W S , and finally through the spatial attention weight W S Scaling the input feature F′2 channel by channel to obtain the output feature F′ S , enhancing the response of important areas.

6. The target point cloud segmentation method for battery swapping robots based on multi-scale attention aggregation according to claim 1 is characterized in that: The specific implementation of the decoder in step (5) is: (5-1) For the input feature matrix F and the pre-prepared nearest neighbor index of each target point, which is the interpolation index I, calculate the interpolated feature matrix F'; (5-2) The decoder optimizes edge segmentation accuracy by fusing multi-scale features F' layer by layer, improves the ability to retain geometric details of the target fastener, and finally obtains an accurate segmentation result point cloud P4.

7. The target point cloud segmentation method for battery-swapping robots based on multi-scale attention aggregation according to claim 1 is characterized in that: The point cloud subset extraction method in step (6) is: (6-1) Output the coordinates of point cloud P4 according to the semantic label value (0 / 1) index, separate the point cloud of the solar panel and the target fastener, and obtain point cloud P5; (6-2) The extracted point cloud data is saved in PLY or PCD format, which supports direct call of subsequent registration algorithms.

Images