Container lifting hole positioning method based on feature probability distribution point cloud detection

Abstract

The invention relates to a container lifting hole positioning method based on feature probability distribution point cloud detection, and is mainly used for solving the problems that in the container automatic operation process, a positioning method based on RGB images has environmental sensitivity, auxiliary positioning through a depth camera is not suitable for dark or complex scenes, and the positioning accuracy is poor. And the positioning accuracy is poor. According to the method, the initial point cloud frame of the container lifting hole is acquired through the area array type laser radar; in order to enhance the uncertainty representation of the point cloud features, inputting the initial point cloud frame into a pointpillars network based on probability distribution; firstly, an initial point cloud frame is processed into a 2D pseudo image point cloud feature map, feature extraction is carried out after cooperative enhancement of space and channel dimensions, the extracted feature map is mapped to a hidden space, a hidden layer is set to be hidden representation obeying Gaussian distribution, variational approximation distribution of a hidden variable Z is obtained, then features of the hidden variable Z are reconstructed, and the reconstructed features of the hidden variable Z are obtained. And therefore, the pointpillars network has higher robustness and anti-interference performance on the noise and the change.

Description

Technical Field

[0001] This invention relates to a method for locating container lifting holes, specifically a method for locating container lifting holes based on feature probability distribution point cloud detection. Background Technology

[0002] In automated container loading and unloading and logistics warehousing systems, accurately identifying the location of the lifting holes in containers is a crucial step in achieving automated operations. Traditional lifting hole positioning methods mainly rely on RGB image processing technology, including traditional machine learning methods (SVM, OpenCV morphological processing, etc.) and deep learning methods.

[0003] RGB image-based positioning methods suffer from numerous environmental sensitivities. For instance, changes in lighting (strong light / shadow) can lead to unstable RGB image quality, and the reflective properties of container surfaces (especially metal surfaces) can cause feature distortion. Visible light imaging performance deteriorates sharply under adverse weather conditions such as rain and fog. Complex coatings, stains, and wear patterns on container surfaces can also interfere with texture-based feature recognition. Furthermore, RGB images lack depth information, making it difficult to accurately calculate the 3D spatial position of lifting holes; relying solely on RGB images makes high-precision lifting hole positioning challenging.

[0004] Depth cameras can be used to acquire depth information of a scene for auxiliary positioning. However, depth cameras have the problem of large measurement errors in dark scenes or when imaging objects, making it difficult to guarantee the presentation of complete depth images of the container loading and unloading process in complex scenes, thus affecting the accuracy of lifting hole positioning. Summary of the Invention

[0005] The purpose of this invention is to solve the problem that the method of locating lifting holes based on RGB images in the automated operation of containers has environmental sensitivity issues, and the positioning assisted by depth cameras is not suitable for dark or complex scenes, resulting in poor accuracy of lifting hole positioning. Therefore, this invention proposes a container lifting hole positioning method based on feature probability distribution point cloud detection.

[0006] To solve the above-mentioned technical problems, the technical solution provided by the present invention is as follows: This invention provides a container lifting hole positioning method based on feature probability distribution point cloud detection, characterized by the following steps: S1. Install a laser array radar on the lifting arm corresponding to the four azimuth angles of the container to collect images of the container lifting holes during the lifting process and form an initial point cloud frame. S2. Input the initial point cloud frame into the probability distribution-based pointpillars network to train the pointpillars network. When the preset number of training rounds is reached, the trained pointpillars network is obtained. S3. Input the initial point cloud frame into the trained pointpillars network to detect the position of the hanging hole: S31. Perform feature extraction and mapping on the initial point cloud frame to generate a pseudo image point cloud feature map with dimensions (B, C, H, W), where B represents the number of point cloud frames, C represents the number of feature channels, H represents the height of the pseudo image point cloud feature map, and W represents the width of the pseudo image point cloud feature map. S32. Perform joint enhancement of spatial and channel dimensions on the pseudo-image point cloud feature map, and output channel-enhanced feature map; S33. Extract features from the channel enhancement feature map and output the intermediate feature map; S34. Flatten the intermediate feature map and feed it into the fully connected layer to obtain the mean vector. Sum of variance vectors By combining noise sampling with a standard normal distribution, reparameterized sampling is performed to generate the latent variable Z; S35. After restoring the dimensions of the latent variable Z, perform feature reconstruction and output the reconstructed feature map. S36. Detect the reconstructed feature map and select the detection box with the highest confidence score and the category of hanging hole as the target detection box output; S4. Extract the center coordinates of the target detection box in the yz plane as the final 3D coordinates of the container lifting hole to complete the container lifting hole positioning.

[0007] Furthermore, the pointpillars network includes a PFN module, a spatial and channel attention coordination module, a 2D CNN, and an SSD detection head connected in sequence; In step S32, the pseudo-image point cloud feature map is input to the spatial and channel attention coordination module, which includes an SMSA unit and a PCSA unit. S32.1 Input the pseudo-image point cloud feature map into the SMSA unit for spatial feature enhancement, and output the spatially enhanced feature map; S32.2 Input the spatial enhancement feature map into the PCSA unit for channel enhancement, and output the channel enhancement feature map; In step S33, a 2D CNN is used to extract features from the channel-enhanced feature map.

[0008] Further, step S32.1 specifically includes: S32.11. Perform average pooling on the pseudo-image point cloud feature map, and calculate the mean values ​​of the pseudo-image point cloud feature map in the height H direction and the width W direction respectively to obtain two feature maps x. h and x w ; S32.12, Transfer feature map xh It is divided into four parts: a local feature map lxh and three global feature maps of different sizes gxhs, gxhm and gxhl. The number of channels in all four feature maps is C / 4. feature map x w It is divided into four parts: a local feature map lxw and three global feature maps of different sizes gxws, gxwm and gxwl. The number of channels in all four feature maps is C / 4. S32.13. Perform feature extraction. Use four depth-shared 1D convolutions with kernel sizes of 3, 5, 7, and 9 to extract features from the local feature map lxh and the global feature maps gxhs, gxhm, and gxhl, respectively, to obtain information at different scales in the height H direction. Four depth-shared 1D convolutions with kernel sizes of 3, 5, 7, and 9 are used to extract features from the local feature map lxw and the global feature maps gxws, gxwm, and gxwl respectively, to obtain information at different scales in the width W direction; S32.14. First, merge information of different scales in the height H direction by splicing, then use a normalization layer for group normalization processing, and finally calculate the spatial attention weight xh{attn} in the height direction through the Sigmoid gating mechanism. Information at different scales in the width W direction is first merged by splicing, then grouped by a normalization layer, and finally the spatial attention weights xw{attn} in the width direction are calculated by the Sigmoid gate mechanism. S32.15. Multiply the spatial attention weights xh{attn} in the height H direction, xw{attn} in the width W direction, and the pseudo-image point cloud feature map element-wise to generate a spatial augmentation feature map with dimensions (B, C, H, W).

[0009] Furthermore, step 32.2 specifically includes: S32.21. Use average pooling to downsample the spatial augmentation feature map to obtain a downsampled feature map with dimensions (B, C, H', W'); normalize the downsampled feature map. S32.22. Convolve the normalized downsampled feature map using 1×1 depth convolution to generate query q, key k and value v, which are used to calculate the relationship between channels; S32.23. Perform a dot product operation on the query q and the key k, and apply a scaling factor to the result of the dot product operation to obtain the attention matrix; S32.24. The weights of the attention matrix are calculated using the Sigmoid gating mechanism, and then regularized using Dropout to obtain the attention weights. S32.25. Multiply the attention weights and the value v element-wise to obtain the channel enhancement feature map, which has dimensions (B, C, H, W).

[0010] Furthermore, the specific steps for training the SSD detection head in the pointpillars network in step S2 are as follows: S21. Predefine the prediction anchor box, which is represented by a 7-dimensional vector (x0, y0, z0, w, l, h, θ), where x0, y0, z0 represent the coordinates of the center point of the prediction anchor box; w, l, h represent the width, length, and height of the prediction anchor box, respectively; and θ represents the orientation angle of the prediction anchor box. S22. The predicted anchor box and the ground truth box are matched using 2D IOU. When the matching value is ≥0.7, the corresponding predicted anchor box is a positive sample; when the matching value is <0.40, the corresponding predicted anchor box is a negative sample; when the matching value is ≥0.40 and <0.7, the corresponding predicted anchor box does not calculate the loss. S23. Calculate the total loss between the predicted anchor box and the ground truth box. ; S24. Based on total losses Optimize the predicted anchor boxes to complete one round of training. When the preset number of training rounds is reached, the training of the SSD detection head is completed, and the optimal SSD detection head is output. In step S36, the optimal SSD detection head is used to detect the reconstructed feature map.

[0011] Furthermore, step S23 specifically includes: S23.1 Calculate the localization regression loss between the predicted anchor frame and the actual frame. : ; in, , , , , , , ; and These represent the X-axis coordinates of the center points of the ground truth bounding box and the predicted anchor box, respectively. and These represent the Y-axis coordinates of the center points of the ground truth bounding box and the predicted anchor box, respectively. and These represent the Z-axis coordinates of the center points of the ground truth bounding box and the predicted anchor box, respectively. and These represent the widths of the ground truth bounding box and the predicted anchor box, respectively. and These represent the lengths of the ground truth bounding box and the predicted anchor box, respectively. and These represent the heights of the ground truth bounding box and the predicted anchor box, respectively. and These represent the orientation angles of the ground truth bounding box and the predicted anchor box, respectively. S23.2 Calculate the classification loss between the predicted anchor box and the ground truth box using FcolLoss loss. : ; in, To predict the balance factor of the anchor frame, To predict the probability of the anchor frame, As a regulating factor; S23.3 Calculate the prior constraint loss between the predicted anchor frame and the true frame using the prior constraint KL divergence loss. : ; in, To predict the bounding box regression value of the anchor box, To predict the uncertainty of the anchor frame; S23.4. Use softmax-based orientation classification to predict the two orientations of the anchor box; calculate the angular loss between the predicted anchor box and the ground truth box. : S23.5 Calculate the total loss between the predicted anchor frame and the actual frame. : ; in, Predict the number of anchor frames for positive samples in step S22; To locate the regression loss weights, =2; For classification loss weights, =1; For angle loss weights, =0.2; As a priori constraint loss weight, =0.2.

[0012] Furthermore, in step S1, each point in the initial point cloud frame has 4 dimensions (x, y, z, r), where x, y, and z are the coordinates in the X-axis, Y-axis, and Z-axis directions, respectively, and r represents the reflectivity; Step S3 also includes step S30, which expands the dimensions of the points in the initial point cloud frame, including the following steps: S30.1. Using the PFN module in the pointpillars network, the three-dimensional space of the initial point cloud frame is voxelized, and the three-dimensional space is divided into multiple columnar regions. S30.2, Expand the dimensions of the points in the initial point cloud frame to 12 dimensions (x, y, z, r, x). c ,y c ,z c ,x p ,y p ,z p ,z rel ,r diff ), where x c ,y c ,z c x represents the centroid deviation of a point. p ,y p ,z p z represents the deviation of the physical center of a point. rel The z-axis represents the height of a point relative to the base of the pillars. rel =z−min(z) pillar ), z pillar Indicates the height of pillars; r diff The difference between the reflectance of a point and the average reflectance of the pillar is expressed as r = r - r. pillar r pillar This represents the average reflectivity of the pillar. Step S31 is as follows: S31.1 Input the initial point cloud frame after the expansion of dimensions into the MLP network of the PFN module, and extract the features of the points in each pillar; combine the features extracted from all pillars into a pseudo image; record the position of each pillar in the pseudo image; remove pillars that do not contain points, and record the number of valid pillars. S31.2 Combining the position of each pillar in the pseudo-image and the effective number of pillars, the features extracted from the points in each pillar are mapped to the pseudo-image to generate a pseudo-image point cloud feature map.

[0013] Furthermore, step S33 specifically includes: S33.1 Input the channel-enhanced feature map into the first convolutional block ConvBlock1, perform convolution using 3×3 Conv with stride=1 and padding=1; perform batch normalization on the convolutional feature map; activate the batch-normalized feature map using rectified linear units, and output the first feature map with dimensions (B,C,H,W). S33.2. Input the first feature map into the second convolutional block ConvBlock2, perform convolution using 3×3 Conv with stride=2 and padding=1; perform batch normalization on the convolutional feature map; activate the batch normalized feature map using rectified linear units, and output the second feature map with dimensions (B,C,H / 2,W / 2). S33.3. Input the second feature map into the third convolutional block ConvBlock3, perform convolution using 3×3 Conv with stride=2 and padding=1; perform batch normalization on the convolutional feature map; activate the batch normalized feature map using rectified linear units, and output the third feature map with dimensions (B,2C,H / 4,W / 4); S33.4. Input the third feature map into the fourth convolutional block ConvBlock4, perform convolution using 3×3 Conv with stride=2 and padding=1; perform batch normalization on the convolutional feature map; activate the batch normalized feature map using rectified linear units, and output an intermediate feature map with dimensions (B,4C,H / 8,W / 8).

[0014] Furthermore, step S34 specifically includes: S34.1 Flatten the dimensions of the intermediate feature map to (B, 4C×(H / 8)×(W / 8)); S34.2. Set the hidden layers of the 2D CNN to have hidden variables Z~N(μ, σ) that follow a Gaussian distribution. 2 ), and set its prior distribution to N(0,1); S34.3. Set the output dimension of the first fully connected layer FC1 to 4096, the output dimension of the second fully connected layer FC2 to 2048, and the output dimensions of the mean fully connected layer FC31 and the logarithmic fully connected layer FC32 to 512. Input the flattened intermediate feature map into the first fully connected layer FC1. Input the output feature map of the first fully connected layer FC1 into the second fully connected layer FC2. Input the feature map output by the second fully connected layer FC2 into the mean fully connected layer FC31 and the logarithmic fully connected layer FC32 respectively. S34.4. The mean vector μ is calculated from the feature map output by the mean fully connected layer FC31; the variance vector is calculated from the feature map output by the logarithmic fully connected layer FC32. ; S34.5. Use the randn_like function to randomly sample from the standard normal distribution N(0,1) to obtain the noise ε; Based on the mean vector variance vector We perform reparameterized sampling on the noise ε to obtain the variational approximation distribution q(Z) = μ + ε☉σ, ε ~ N(0,1) and the latent variable Z ~ N(μ,σ). 2 ), its dimension is (B, 512).

[0015] Furthermore, step S35 specifically includes: S35.1. Set the output dimension of the fourth fully connected layer FC4 to 2048, the output dimension of the fifth fully connected layer FC5 to 4096, and the output dimension of the sixth fully connected layer FC6 to 4C×(H / 8)×(W / 8); input the latent variable Z into the fourth fully connected layer FC4; input the feature vector output by the fourth fully connected layer FC4 into the fifth fully connected layer FC5; input the feature vector output by the fifth fully connected layer FC5 into the sixth fully connected layer FC6, and the feature vector output by the sixth fully connected layer FC6 has a dimension of (B, 4C×(H / 8)×(W / 8)); reshape it into a fourth feature map with dimensions (B, 4C, H / 8, W / 8); S35.2. After deconvolution of the second feature map, input it into the first transposed convolution block TransConvBlock1. Convolve using 3×3 TransposedConv with stride=1, padding=1, and output padding=1. Batch normalize the feature map after transposed convolution. Activate the batch normalized feature map using rectified linear units, and output the first transposed feature map with dimensions (B, 2C, H / 2, W / 2). The third feature map is deconvolved and then input into the second transposed convolutional block TransConvBlock2. A 3×3 TransposedConv is used for convolution with stride=2, padding=1, and output padding=1. The transposed convolutional feature map is then batch normalized. The batch normalized feature map is activated using a rectified linear unit, and the output is a second transposed feature map with dimensions (B, 2C, H / 2, W / 2). The fourth feature map is deconvolved and then input into the third transposed convolutional block TransConvBlock3. A 3×3 TransposedConv is used for convolution with stride=4, padding=1, and output padding=1. The transposed convolutional feature map is then batch normalized. The batch normalized feature map is activated using a rectified linear unit, and the output is the third transposed feature map with dimensions (B,2C,H / 2,W / 2). S35.3 Connect the first transposed feature map, the second transposed feature map, and the third transposed feature map to generate a reconstructed feature map with dimensions (B, 6C, H / 2, W / 2).

[0016] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention discloses a container lifting hole positioning method based on feature probability distribution point cloud detection. The method uses an area-array LiDAR to acquire initial point cloud frames of container lifting holes. To enhance the uncertainty representation of point cloud features, the initial point cloud frames are input into a probability distribution-based pointpillars network. First, the initial point cloud frames are processed into 2D pseudo-image point cloud feature maps. After co-enhancement of spatial and channel dimensions, feature extraction is performed. The extracted feature maps are mapped to the latent space, and the latent layer is set to a Gaussian distribution. The variational approximation distribution of the latent variable Z is obtained. Then, the latent variable Z features are reconstructed, making the pointpillars network more robust and resistant to noise and variations.

[0017] 2. This invention discloses a container lifting hole positioning method based on feature probability distribution point cloud detection. It employs an area-array lidar to acquire initial point cloud frames, which features high precision and robust imaging. Since lifting holes are mostly circular or elliptical geometric structures with a diameter of 20-30cm, the point cloud data directly characterizes their spatial shape through (x,y,z) coordinates, without relying on easily disturbed surface texture features. The significant difference in reflectivity between the inner wall of the lifting hole and the container surface creates a stable and distinguishable signal in the point cloud data reflection intensity channel r, making the lifting hole detection less susceptible to illumination and texture effects. The detection range is wide, meeting the 0.2-3m detection requirements for container lifting scenarios.

[0018] 3. The present invention provides a container lifting hole positioning method based on feature probability distribution point cloud detection. By expanding the dimensions of the points in the initial point cloud frame, the method can enhance the perception of material differences, so as to better separate the container area from the background area such as the ground and improve the detection accuracy.

[0019] 4. This invention provides a container lifting hole localization method based on feature probability distribution point cloud detection. The spatial and channel attention collaboration module includes SMSA units and PCSA units. The SMSA unit enhances local and global feature representations, while the PCSA unit refines channel features, effectively mitigating semantic differences and ensuring robust feature integration across channels. By guiding the PCSA unit's learning through the SMSA unit, effective collaboration between spatial and channel attention is achieved, improving the performance of the pointpillars network.

[0020] 5. The present invention provides a container lifting hole positioning method based on feature probability distribution point cloud detection. After restoring the dimension of the latent variable Z, it is input into the decoder for feature reconstruction. Its goal is to capture the core features of the data by learning the normal distribution of the latent variable Z, thereby achieving efficient feature representation and improving the robustness of the point cloud detection algorithm.

[0021] 6. The present invention provides a container lifting hole positioning method based on feature probability distribution point cloud detection. It extracts and maps features from the initial point cloud frame to generate a pseudo-image point cloud feature map. By compressing the point cloud data, the inference speed of the pointpillars network is accelerated, and real-time and high-precision container lifting hole detection and positioning functions are realized. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of an embodiment of a container lifting hole positioning method based on feature probability distribution point cloud detection according to the present invention; Figure 2 This is a structural block diagram of the pointpillars network in an embodiment of a container lifting hole positioning method based on feature probability distribution point cloud detection according to the present invention. Figure 3 This is a schematic diagram illustrating the process of processing the spatially enhanced feature map to obtain the reconstructed feature map in step 3 of an embodiment of a container lifting hole positioning method based on feature probability distribution point cloud detection according to the present invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0024] This invention discloses a container lifting hole positioning method based on feature probability distribution point cloud detection, such as... Figure 1 As shown, it includes the following steps: S1. Install a planar array LiDAR on each of the four azimuth angles of the container's lifting arm to acquire images of the container's lifting holes during the lifting process, forming an initial point cloud frame. Preprocess the initial point cloud frame to remove noise points, and then flip and translate it. Each point in the initial point cloud frame has four dimensions (x, y, z, r), where x, y, and z are the coordinates along the X, Y, and Z axes, respectively, and r represents reflectivity. Due to different lifting scenarios, the initial point cloud frame is compressed. In this embodiment, the detection height of the lifting scenario is 20cm-200cm, and the lifting holes are small to medium-sized targets. Using meter-level imaging for the point cloud frame would reduce the algorithm's detection accuracy, while centimeter-level point cloud frames would increase inference time, which is not conducive to real-time detection deployment on edge devices. Therefore, in this embodiment, the initial point cloud frame is at the decimeter level. The detection range of the lifting holes in the x, y, and z directions is (0, 20.48) dm, (-6, 6.8) dm, and (-8, 8) dm, respectively.

[0025] S2. Input the initial point cloud frame into the probability distribution-based pointpillars network to train the pointpillars network. When the preset number of training rounds is reached, the trained pointpillars network is obtained. like Figure 2 As shown, the PointPillars network consists of a PFN module, a Spatial and Channel Synergistic Attention (SCSA) module, a 2DCNN, and an SSD detection head connected in sequence. The specific steps involved in training the SSD detection head are as follows: S21. Predefine the prediction anchor frame (anchor), which is represented by a 7-dimensional vector (x0, y0, z0, w, l, h, θ), where x0, y0, z0 represent the coordinates of the center point of the prediction anchor frame; w, l, h represent the width, length, and height of the prediction anchor frame, respectively; θ represents the orientation angle of the prediction anchor frame; in this embodiment, the detection range of the hoisting hole in the x, y, and z directions is (0, 20.48) dm, (-6, 6.8) dm, and (-8, 8) dm; the length, width, and height of the prediction anchor frame are (2.0, 2.0, 1.6) dm.

[0026] S22. 2D IOU matching is used between the predicted anchor box and the ground truth box. When the matching value is ≥0.7, the corresponding predicted anchor box is a positive sample; when the matching value is <0.40, the corresponding predicted anchor box is a negative sample; when the matching value is ≥0.40 and <0.7, the corresponding predicted anchor box does not calculate the loss; matching is performed from the BEV (Bird's Eye View) perspective, without considering height information. S23. Calculate the total loss between the predicted anchor box and the ground truth box. The specific steps are as follows: S23.1 Calculate the localization regression loss between the predicted anchor frame and the actual frame. : ; in, , , , , , , ; and These represent the X-axis coordinates of the center points of the ground truth bounding box and the predicted anchor box, respectively. and These represent the Y-axis coordinates of the center points of the ground truth bounding box and the predicted anchor box, respectively. and These represent the Z-axis coordinates of the center points of the ground truth bounding box and the predicted anchor box, respectively. and These represent the widths of the ground truth bounding box and the predicted anchor box, respectively. and These represent the lengths of the ground truth bounding box and the predicted anchor box, respectively. and These represent the heights of the ground truth bounding box and the predicted anchor box, respectively. and These represent the orientation angles of the ground truth bounding box and the predicted anchor box, respectively. S23.2 Calculate the classification loss between the predicted anchor box and the ground truth box using FcolLoss loss. FcolLoss can effectively avoid the impact of imbalanced sample size, essentially giving greater weight to samples with smaller numbers; the calculation formula is: ; in, To predict the balance factor of the anchor frame, To predict the probability of the anchor frame, As a regulating factor; S23.3 In this embodiment, the hidden layer uses a distributed representation, and therefore has prior distribution constraints. The prior constraint loss between the predicted anchor box and the ground truth box is calculated using the prior constraint KL (Kullback-Leibler Divergence) divergence loss. KL loss can effectively constrain the variational approximation distribution using the prior distribution, avoiding distribution shift. The calculation formula is as follows: ; in, To predict the bounding box regression value of the anchor box, To predict the uncertainty of the anchor frame; S23.4. When predicting the angle of the predicted anchor frame, it is impossible to distinguish between two frames with opposite directions. Therefore, the SSD detection head in this embodiment also adds direction prediction for the predicted anchor frame. Softmax-based direction classification is used to predict the two orientations of the anchor frame; the angle loss between the predicted anchor frame and the ground truth frame is calculated. : S23.5 Calculate the total loss between the predicted anchor frame and the actual frame. : ; in, The number of anchor boxes predicted for positive samples in step S22. To locate the regression loss weights, =2; For classification loss weights, =1; For angle loss weights, =0.2; As a priori constraint loss weight, =0.2.

[0027] S24. Based on total losses Optimize the predicted anchor boxes to complete one round of training. When the preset number of training rounds is reached, the training of the SSD detection head is completed, and the optimal SSD detection head is output.

[0028] S3. Input the initial point cloud frame into the trained pointpillars network to detect the position of the hanging hole: S30. Perform dimensional expansion on the points in the initial point cloud frame, including the following steps: S30.1. Using the PFN module in the pointpillars network, the three-dimensional space of the initial point cloud frame is voxelized, and the three-dimensional space is divided into multiple columnar regions. S30.2 To enhance the feature description capability of point clouds and improve the spatial information representation capability, the dimension of points in the initial point cloud frame is expanded to 12 dimensions (x, y, z, r, x). c ,y c ,z c ,x p ,y p ,z p ,z rel ,r diff ), where x c ,y c ,z c x represents the centroid deviation of a point. p ,y p ,z p z represents the deviation of the physical center of a point. rel The z-axis represents the height of a point relative to the base of the pillars. rel =z−min(z) pillar ), used to enhance resolution in the height direction, z pillar Indicates the height of pillars; r diff The difference between the reflectance of a point and the average reflectance of the pillar is expressed as r = r - r. pillar r pillar This indicates the average reflectivity of the pillar, which can enhance the perception of material differences, thus better separating the container area from background areas such as the ground.

[0029] S31. Perform feature extraction and mapping on the initial point cloud frame to generate a pseudo-image point cloud feature map, specifically including the following steps: S31.1 Input the initial point cloud frame after the expansion of dimensions into the MLP network of the PFN module, and extract the features of the points in each pillar; combine the features extracted from all pillars into a pseudo image; record the position of each pillar in the pseudo image; remove pillars that do not contain points, and record the number of valid pillars. S31.2. Combining the position of each pillar in the pseudo-image and the effective number of pillars, the features extracted from the points in each pillar are mapped to the pseudo-image to generate a pseudo-image point cloud feature map with dimensions (B, C, H, W), where B represents the number of point cloud frames, C represents the number of feature channels, H represents the image height, and W represents the image width.

[0030] S32. Input the pseudo-image point cloud feature map into the spatial and channel attention collaboration module. The spatial and channel attention collaboration module includes the SMSA unit (Shareable Multi-Semantic Spatial Attention) and the PCSA unit (Progressive Channel-wise Self-Attention). Pointpillar networks typically extract features from 2D pseudo-image point cloud feature maps using 2D CNNs (two-dimensional convolutional neural networks). Compared to 3D convolution operations, this saves time, but the direct and simple 2D convolution operation results in shallow feature extraction. The inconsistent sizes of container lifting holes further increase the difficulty of feature extraction. To improve accuracy and robustness for detecting targets of different sizes, this embodiment enhances feature extraction capabilities before feature extraction from the pseudo-image point cloud feature map through a spatial and channel attention collaboration module. The SMSA unit enhances local and global feature representations, while the PCSA unit refines channel features, effectively mitigating semantic differences and ensuring robust feature integration across channels. The SMSA unit guides the PCSA unit's learning, achieving effective collaboration between spatial and channel attention and improving model performance.

[0031] S32.1 The pseudo-image point cloud feature map is input into the SMSA unit for spatial feature enhancement, and the spatially enhanced feature map is output, including the following steps: S32.11. Perform average pooling (AvgPool) on the pseudo-image point cloud feature map, and calculate the mean values ​​of the pseudo-image point cloud feature map in the height H direction and the width W direction respectively to obtain two feature maps x. h and x w ; S32.12, Transfer feature map x hThe segment is divided into four parts: a local feature map lxh and three global feature maps of different sizes gxhs, gxhm, and gxhl. The number of channels in each of the four feature maps is C / 4. feature map x w It is divided into four parts: a local feature map lxw and three global feature maps of different sizes gxws, gxwm and gxwl. The number of channels in all four feature maps is C / 4. S32.13. Perform feature extraction. Use four depth-shared 1D convolutions with kernel sizes of 3, 5, 7, and 9 to extract features from the local feature map lxh and the global feature maps gxhs, gxhm, and gxhl, respectively, to obtain information at different scales in the height H direction. Four depth-shared 1D convolutions with kernel sizes of 3, 5, 7, and 9 are used to extract features from the local feature map lxw and the global feature maps gxws, gxwm, and gxwl respectively, to obtain information at different scales in the width W direction; S32.14. First, merge information of different scales in the height H direction by concatenation, then use the normalization layer (GroupNorm) for group normalization, and finally calculate the spatial attention weight xh{attn} in the height direction through the gate mechanism Sigmoid operation. Information at different scales in the width W direction is first merged by concatenation, then normalized by a group normalization layer, and finally the spatial attention weight xw{attn} in the width direction is calculated by the Sigmoid gating mechanism. S32.15. Multiply the spatial attention weights xh{attn} in the height H direction, xw{attn} in the width W direction, and the pseudo-image point cloud feature map element-wise to generate a spatial augmentation feature map with dimensions (B, C, H, W).

[0032] S32.2 Input the spatial enhancement feature map into the PCSA unit for channel enhancement, and output the channel enhancement feature map; S32.21. Use average pooling to downsample the spatial augmentation feature map to obtain a downsampled feature map with dimensions (B, C, H', W'); normalize the downsampled feature map (BatchNorm). S32.22. Convolve the normalized downsampled feature map using 1×1 depthwise convolution (DWConv) to generate query q, key k and value v, which are used to calculate the relationship between channels; S32.23. Perform a dot product operation on the query q and the key k, and apply a scaling factor to the result of the dot product operation to prevent numerical overflow, thereby obtaining the attention matrix; S32.24. The weights of the attention matrix are calculated using the Sigmoid gating mechanism, and then regularized using Dropout to obtain the attention weights. S32.25. Multiply the attention weights and values ​​(v) element by element to obtain the channel enhancement feature map, which has dimensions (B, C, H, W).

[0033] S33. Use a 2D CNN to extract features from the channel-enhanced feature map and output an intermediate feature map, such as... Figure 3 As shown, it includes the following steps: S33.1. Input the channel-enhanced feature map into the first convolutional block ConvBlock1, and perform convolution using 3×3 Conv with stride=1 and padding=1; perform batch normalization on the convolutional feature map; activate the batch-normalized feature map using a rectified linear unit, and output the first feature map with dimensions (B,C,H,W); retain the precise positional information of the channel-enhanced feature map after passing through the first convolutional block.

[0034] Then, the features of the first feature map are extracted step by step from left to right, specifically: S33.2. Input the first feature map into the second convolutional block ConvBlock2, perform convolution using 3×3 Conv with stride=2 and padding=1; perform batch normalization on the convolutional feature map; activate the batch normalized feature map using rectified linear units, and output the second feature map with dimensions (B,C,H / 2,W / 2). S33.3. Input the second feature map into the third convolutional block ConvBlock3, perform convolution using 3×3 Conv with stride=2 and padding=1; perform batch normalization on the convolutional feature map; activate the batch normalized feature map using rectified linear units, and output the third feature map with dimensions (B,2C,H / 4,W / 4); S33.4. Input the third feature map into the fourth convolutional block ConvBlock4, perform convolution using 3×3 Conv with stride=2 and padding=1; perform batch normalization on the convolutional feature map; activate the batch normalized feature map using rectified linear units, and output an intermediate feature map with dimensions (B,4C,H / 8,W / 8).

[0035] S34. Flatten the intermediate feature map and feed it into the fully connected layer to obtain the mean vector. Sum of variance vectors Combining noise sampling with a standard normal distribution, reparameterized sampling is performed to generate the latent variable Z; this includes the following steps: S34.1 Flatten the dimensions of the intermediate feature map to (B, 4C×(H / 8)×(W / 8)), where B represents the number of channels. In this embodiment, the detection range of x, y, and z is [(0, 20.48), (-6, 6.8), (-8, 8)] dm. By calculation, we can obtain H=80 and W=128 for the pseudo image point cloud feature map. Therefore, the dimension here is 4*64*10*16=40960. S34.2 To enhance the robustness of the 2D convolutional encoder to variations, the hidden layers of the 2D CNN are set to have hidden variables Z~N(μ,σ) that follow a Gaussian distribution. 2 ), and set its prior distribution to N(0,1); S34.3 To achieve inference on the latent variable Z, the output dimension of the first fully connected layer FC1 is set to 4096, the output dimension of the second fully connected layer FC2 is set to 2048, and the output dimensions of the mean fully connected layer FC31 and the logarithmic fully connected layer FC32 are both set to 512. The flattened intermediate feature map is input to the first fully connected layer FC1. The output feature map of the first fully connected layer FC1 is input to the second fully connected layer FC2. The feature map output by the second fully connected layer FC2 is input to the mean fully connected layer FC31 and the logarithmic fully connected layer FC32, respectively. S34.4. The mean vector μ is calculated from the feature map output by the mean fully connected layer FC31; the variance vector is calculated from the feature map output by the logarithmic fully connected layer FC32. ; S34.5. Use the randn_like function to randomly sample from the standard normal distribution N(0,1) to obtain the noise ε; Based on the mean vector variance vector We perform reparameterized sampling on the noise ε to obtain the variational approximation distribution q(Z) = μ + ε☉σ, ε ~ N(0,1) and the latent variable Z ~ N(μ,σ). 2 ), its dimension is (B,512).

[0036] S35. After restoring the dimensions of the latent variable Z, the input is fed into the decoder for feature reconstruction, and the reconstructed feature map is output, specifically: S35.1. Set the output dimension of the fourth fully connected layer FC4 to 2048, the output dimension of the fifth fully connected layer FC5 to 4096, and the output dimension of the sixth fully connected layer FC6 to 4C×(H / 8)×(W / 8); input the latent variable Z into the fourth fully connected layer FC4; input the feature vector output by the fourth fully connected layer FC4 into the fifth fully connected layer FC5; input the feature vector output by the fifth fully connected layer FC5 into the sixth fully connected layer FC6, and the feature vector output by the sixth fully connected layer FC6 has a dimension of (B, 4C×(H / 8)×(W / 8)); reshape it into a fourth feature map with dimensions (B, 4C, H / 8, W / 8); S35.2. After deconvolution of the second feature map, input it into the first transposed convolution block TransConvBlock1. Convolve using 3×3 TransposedConv with stride=1, padding=1, and output padding=1. Batch normalize the feature map after transposed convolution. Activate the batch normalized feature map using rectified linear units, and output the first transposed feature map with dimensions (B, 2C, H / 2, W / 2). The third feature map is deconvolved and then input into the second transposed convolutional block TransConvBlock2. A 3×3 TransposedConv is used for convolution with stride=2, padding=1, and output padding=1. The transposed convolutional feature map is then batch normalized. The batch normalized feature map is activated using a rectified linear unit, and the output is a second transposed feature map with dimensions (B, 2C, H / 2, W / 2). The fourth feature map is deconvolved and then input into the third transposed convolutional block TransConvBlock3. A 3×3 TransposedConv is used for convolution with stride=4, padding=1, and output padding=1. The transposed convolutional feature map is then batch normalized. The batch normalized feature map is activated using a rectified linear unit, and the output is the third transposed feature map with dimensions (B,2C,H / 2,W / 2). S35.3. Concatenate the first transposed feature map, the second transposed feature map, and the third transposed feature map to generate a reconstructed feature map with dimensions (B, 6C, H / 2, W / 2).

[0037] To enhance the representation of feature uncertainty in point clouds, a probability distribution-based pointpillars network is used as an aid. The input data is mapped to a latent space (low-dimensional latent vector) through a 2D CNN. The decoder reconstructs the data from this space. The goal is to capture the core features of the data by learning the normal distribution of the latent variables, thereby achieving efficient feature representation and improving the robustness of point cloud detection algorithms.

[0038] S36. Use the optimal SSD detection head to detect the reconstructed feature map, and select the detection box with the highest confidence score and the category of hanging hole as the target detection box output. S4. Extract the center coordinates of the target detection box in the yz plane as the final 3D coordinates of the container lifting hole to complete the container lifting hole positioning.

[0039] This invention addresses the issue of manually controlled lifting systems in container loading and unloading and logistics warehousing systems, which rely on observation methods for lifting containers. It proposes a robust and interference-resistant method for container lifting hole localization based on point cloud 3D detection using feature probability distribution. The invention chooses the PointPillars model as the baseline model because its 2D convolutional nature allows for faster inference speeds. Given the time limitation of LiDAR data acquisition (10Hz / s), some inference time in PointPillars can be sacrificed to improve inference accuracy.

Claims

1. A method for locating container lifting holes based on feature probability distribution point cloud detection, characterized in that, Includes the following steps: S1. Install a laser array radar on the lifting arm corresponding to the four azimuth angles of the container to collect images of the container lifting holes during the lifting process and form an initial point cloud frame. S2. Input the initial point cloud frame into the probability distribution-based pointpillars network to train the pointpillars network. When the preset number of training rounds is reached, the trained pointpillars network is obtained. S3. Input the initial point cloud frame into the trained pointpillars network to detect the position of the hanging hole: S31. Perform feature extraction and mapping on the initial point cloud frame to generate a pseudo image point cloud feature map with dimensions (B, C, H, W), where B represents the number of point cloud frames, C represents the number of feature channels, H represents the height of the pseudo image point cloud feature map, and W represents the width of the pseudo image point cloud feature map. S32. Perform joint enhancement of spatial and channel dimensions on the pseudo-image point cloud feature map, and output channel-enhanced feature map; S33. Extract features from the channel enhancement feature map and output the intermediate feature map; S34. Flatten the intermediate feature map and feed it into the fully connected layer to obtain the mean vector. Sum of variance vectors By combining noise sampling with a standard normal distribution, reparameterized sampling is performed to generate the latent variable Z; S35. After restoring the dimensions of the latent variable Z, perform feature reconstruction and output the reconstructed feature map. S36. Detect the reconstructed feature map and select the detection box with the highest confidence score and the category of hanging hole as the target detection box output; S4. Extract the center coordinates of the target detection box in the yz plane as the final 3D coordinates of the container lifting hole to complete the container lifting hole positioning.

2. The container lifting hole positioning method based on feature probability distribution point cloud detection according to claim 1, characterized in that: The pointpillars network includes a PFN module, a spatial and channel attention coordination module, a 2D CNN, and an SSD detection head connected in sequence. In step S32, the pseudo-image point cloud feature map is input to the spatial and channel attention coordination module, which includes an SMSA unit and a PCSA unit. S32.1 Input the pseudo-image point cloud feature map into the SMSA unit for spatial feature enhancement, and output the spatially enhanced feature map; S32.2 Input the spatial enhancement feature map into the PCSA unit for channel enhancement, and output the channel enhancement feature map; In step S33, a 2D CNN is used to extract features from the channel-enhanced feature map.

3. The container lifting hole positioning method based on feature probability distribution point cloud detection according to claim 2, characterized in that, Step S32.1 is as follows: S32.

11. Perform average pooling on the pseudo-image point cloud feature map, and calculate the mean values ​​of the pseudo-image point cloud feature map in the height H direction and the width W direction respectively to obtain two feature maps x. h and x w ; S32.12, Transfer feature map x h It is divided into four parts: a local feature map lxh and three global feature maps of different sizes gxhs, gxhm and gxhl. The number of channels in all four feature maps is C / 4. feature map x w It is divided into four parts: a local feature map lxw and three global feature maps of different sizes gxws, gxwm and gxwl. The number of channels in all four feature maps is C / 4. S32.

13. Perform feature extraction. Use four depth-shared 1D convolutions with kernel sizes of 3, 5, 7, and 9 to extract features from the local feature map lxh and the global feature maps gxhs, gxhm, and gxhl, respectively, to obtain information at different scales in the height H direction. Four depth-shared 1D convolutions with kernel sizes of 3, 5, 7, and 9 are used to extract features from the local feature map lxw and the global feature maps gxws, gxwm, and gxwl respectively, to obtain information at different scales in the width W direction; S32.

14. First, merge information of different scales in the height H direction by splicing, then use a normalization layer for group normalization processing, and finally calculate the spatial attention weight xh{attn} in the height direction through the Sigmoid gating mechanism. Information at different scales in the width W direction is first merged by splicing, then grouped by a normalization layer, and finally the spatial attention weights xw{attn} in the width direction are calculated by the Sigmoid gate mechanism. S32.

15. Multiply the spatial attention weights xh{attn} in the height H direction, xw{attn} in the width W direction, and the pseudo-image point cloud feature map element-wise to generate a spatial augmentation feature map with dimensions (B, C, H, W).

4. The container lifting hole positioning method based on feature probability distribution point cloud detection according to claim 3, characterized in that, Step 32.2 specifically involves: S32.

21. Use average pooling to downsample the spatial augmentation feature map to obtain a downsampled feature map with dimensions (B, C, H', W'); normalize the downsampled feature map. S32.

22. Convolve the normalized downsampled feature map using 1×1 depth convolution to generate query q, key k and value v, which are used to calculate the relationship between channels; S32.

23. Perform a dot product operation on the query q and the key k, and apply a scaling factor to the result of the dot product operation to obtain the attention matrix; S32.

24. The weights of the attention matrix are calculated using the Sigmoid gating mechanism, and then regularized using Dropout to obtain the attention weights. S32.

25. Multiply the attention weights and the value v element-wise to obtain the channel enhancement feature map, which has dimensions (B, C, H, W).

5. The container lifting hole positioning method based on feature probability distribution point cloud detection according to claim 4, characterized in that, The specific steps in step S2 for training the SSD detection head in the PointPillars network are as follows: S21. Predefine the prediction anchor box, which is represented by a 7-dimensional vector (x0, y0, z0, w, l, h, θ), where x0, y0, z0 represent the coordinates of the center point of the prediction anchor box; w, l, h represent the width, length, and height of the prediction anchor box, respectively; and θ represents the orientation angle of the prediction anchor box. S22. The predicted anchor box and the ground truth box are matched using 2D IOU. When the matching value is ≥0.7, the corresponding predicted anchor box is a positive sample; when the matching value is <0.40, the corresponding predicted anchor box is a negative sample; when the matching value is ≥0.40 and <0.7, the corresponding predicted anchor box does not calculate the loss. S23. Calculate the total loss between the predicted anchor box and the ground truth box. ; S24. Based on total losses Optimize the predicted anchor boxes to complete one round of training. When the preset number of training rounds is reached, the training of the SSD detection head is completed, and the optimal SSD detection head is output. In step S36, the optimal SSD detection head is used to detect the reconstructed feature map.

6. The container lifting hole positioning method based on feature probability distribution point cloud detection according to claim 5, characterized in that, Step S23 is as follows: S23.1 Calculate the localization regression loss between the predicted anchor frame and the actual frame. : ； in, , , , , , , ; and These represent the X-axis coordinates of the center points of the ground truth bounding box and the predicted anchor box, respectively. and These represent the Y-axis coordinates of the center points of the ground truth bounding box and the predicted anchor box, respectively. and These represent the Z-axis coordinates of the center points of the ground truth bounding box and the predicted anchor box, respectively. and These represent the widths of the ground truth bounding box and the predicted anchor box, respectively. and These represent the lengths of the ground truth bounding box and the predicted anchor box, respectively. and These represent the heights of the ground truth bounding box and the predicted anchor box, respectively. and These represent the orientation angles of the ground truth bounding box and the predicted anchor box, respectively. S23.2 Calculate the classification loss between the predicted anchor box and the ground truth box using FcolLoss loss. : ； in, To predict the balance factor of the anchor frame, To predict the probability of the anchor frame, As a regulating factor; S23.3 Calculate the prior constraint loss between the predicted anchor frame and the true frame using the prior constraint KL divergence loss. : ； in, To predict the bounding box regression value of the anchor box, To predict the uncertainty of the anchor frame; S23.

4. Use softmax-based orientation classification to predict the two orientations of the anchor box; calculate the angular loss between the predicted anchor box and the ground truth box. : S23.5 Calculate the total loss between the predicted anchor frame and the actual frame. : ； in, Predict the number of anchor frames for positive samples in step S22; To locate the regression loss weights, =2; For classification loss weights, =1; For angle loss weights, =0.2; As a priori constraint loss weight, =0.

2.

7. The container lifting hole positioning method based on feature probability distribution point cloud detection according to claim 6, characterized in that: In step S1, each point in the initial point cloud frame has 4 dimensions (x, y, z, r), where x, y, and z are the coordinates in the X-axis, Y-axis, and Z-axis directions, respectively, and r represents the reflectivity. Step S3 also includes step S30, which expands the dimensions of the points in the initial point cloud frame, including the following steps: S30.

1. Using the PFN module in the pointpillars network, the three-dimensional space of the initial point cloud frame is voxelized, and the three-dimensional space is divided into multiple columnar regions. S30.2, Expand the dimensions of the points in the initial point cloud frame to 12 dimensions (x, y, z, r, x). c ,y c ,z c ,x p ,y p ,z p ,z rel ,r diff ), where x c ,y c ,z c x represents the centroid deviation of a point. p ,y p ,z p z represents the deviation of the physical center of a point. rel The z-axis represents the height of a point relative to the base of the pillars. rel =z−min(z) pillar ), z pillar Indicates the height of pillars; r diff The difference between the reflectance of a point and the average reflectance of the pillar is expressed as r = r - r. pillar r pillar This represents the average reflectivity of the pillar. Step S31 is as follows: S31.1 Input the initial point cloud frame after the expansion of dimensions into the MLP network of the PFN module, and extract the features of the points in each pillar; combine the features extracted from all pillars into a pseudo image; record the position of each pillar in the pseudo image; remove pillars that do not contain points, and record the number of valid pillars. S31.2 Combining the position of each pillar in the pseudo-image and the effective number of pillars, the features extracted from the points in each pillar are mapped to the pseudo-image to generate a pseudo-image point cloud feature map.

8. The container lifting hole positioning method based on feature probability distribution point cloud detection according to claim 7, characterized in that, Step S33 is as follows: S33.1 Input the channel enhancement feature map into the first convolutional block ConvBlock1, perform convolution using 3×3 Conv with stride=1 and padding=1; perform batch normalization on the convolutional feature map; The batch-normalized feature map is activated using a modified linear unit, and the first feature map with dimensions (B, C, H, W) is output. S33.

2. Input the first feature map into the second convolutional block ConvBlock2, perform convolution using 3×3 Conv with stride=2 and padding=1; perform batch normalization on the convolutional feature map. The batch-normalized feature map is activated using a modified linear unit, and a second feature map with dimensions (B, C, H / 2, W / 2) is output. S33.3 Input the second feature map into the third convolutional block ConvBlock3, perform convolution using 3×3 Conv with stride=2 and padding=1; perform batch normalization on the convolutional feature map; The batch-normalized feature map is activated using a modified linear unit, and the output is a third feature map with dimensions (B, 2C, H / 4, W / 4). S33.

4. Input the third feature map into the fourth convolutional block ConvBlock4, perform convolution using 3×3 Conv with stride=2 and padding=1; perform batch normalization on the convolutional feature map. The batch-normalized feature maps are activated using modified linear units, and the output is an intermediate feature map with dimensions (B, 4C, H / 8, W / 8).

9. The container lifting hole positioning method based on feature probability distribution point cloud detection according to claim 8, characterized in that, Step S34 is as follows: S34.1 Flatten the dimensions of the intermediate feature map to (B, 4C×(H / 8)×(W / 8)); S34.

2. Set the hidden layers of the 2D CNN to have hidden variables Z~N(μ, σ) that follow a Gaussian distribution. 2 ), and set its prior distribution to N(0,1); S34.

3. Set the output dimension of the first fully connected layer FC1 to 4096, the output dimension of the second fully connected layer FC2 to 2048, and the output dimensions of the mean fully connected layer FC31 and the logarithmic fully connected layer FC32 to 512. Input the flattened intermediate feature map into the first fully connected layer FC1. Input the output feature map of the first fully connected layer FC1 into the second fully connected layer FC2. Input the feature map output by the second fully connected layer FC2 into the mean fully connected layer FC31 and the logarithmic fully connected layer FC32 respectively. S34.

4. The mean vector μ is calculated from the feature map output by the mean fully connected layer FC31; the variance vector is calculated from the feature map output by the logarithmic fully connected layer FC32. ; S34.

5. Use the randn_like function to randomly sample from the standard normal distribution N(0,1) to obtain the noise ε; Based on the mean vector variance vector We perform reparameterized sampling on the noise ε to obtain the variational approximation distribution q(Z) = μ + ε☉σ, ε ~ N(0,1) and the latent variable Z ~ N(μ,σ). 2 ), its dimension is (B, 512).

10. A container lifting hole positioning method based on feature probability distribution point cloud detection according to claim 9, characterized in that, Step S35 is as follows: S35.

1. Set the output dimension of the fourth fully connected layer FC4 to 2048, the output dimension of the fifth fully connected layer FC5 to 4096, and the output dimension of the sixth fully connected layer FC6 to 4C×(H / 8)×(W / 8); input the latent variable Z into the fourth fully connected layer FC4; input the feature vector output by the fourth fully connected layer FC4 into the fifth fully connected layer FC5; input the feature vector output by the fifth fully connected layer FC5 into the sixth fully connected layer FC6, and the feature vector output by the sixth fully connected layer FC6 has a dimension of (B, 4C×(H / 8)×(W / 8)); reshape it into a fourth feature map with dimensions (B, 4C, H / 8, W / 8); S35.

2. After deconvolution of the second feature map, input it into the first transposed convolution block TransConvBlock1, and perform convolution using 3×3 TransposedConv with stride=1, padding=1, and output padding=1; perform batch normalization on the feature map after transposed convolution. The batch-normalized feature map is activated using a modified linear unit, and the output is the first transpose feature map with dimensions (B, 2C, H / 2, W / 2). After deconvolution of the third feature map, it is input into the second transposed convolution block TransConvBlock2. Convolution is performed using 3×3 TransposedConv with stride=2, padding=1, and output padding=1. Batch normalization is then performed on the feature map after transposed convolution. The batch-normalized feature map is activated using a modified linear unit, and the output is a second transposed feature map with dimensions (B, 2C, H / 2, W / 2). After deconvolution of the fourth feature map, it is input into the third transposed convolution block TransConvBlock3. Convolution is performed using 3×3 TransposedConv with stride=4, padding=1, and output padding=1. Batch normalization is then performed on the feature map after transposed convolution. The batch-normalized feature map is activated using a modified linear unit, and the output is a third transpose feature map with dimensions (B, 2C, H / 2, W / 2). S35.3 Connect the first transposed feature map, the second transposed feature map, and the third transposed feature map to generate a reconstructed feature map with dimensions (B, 6C, H / 2, W / 2).

Images