An industrial stacked workpiece point cloud segmentation method, system and electronic device
Patent Information
- Application Number
- CN202610810300.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-05
- Publication Date
- 2026-10-13
AI Technical Summary
1.本发明通过对采集的点云进行筛选、清洗、归一化及随机旋转、缩放、抖动等增强操作，有效消除了原始数据中的噪声与离群点，扩充了训练样本的多样性，使得模型在实际堆叠场景中对遮挡、位姿变化和密度不均具有更强的鲁棒性，为后续高精度分割奠定了基础。
Smart Images
Figure CN122905931A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent sorting technology, and more particularly to a method, system, and electronic device for point cloud segmentation of industrial stacked workpieces. It can be applied to point cloud semantic segmentation, robot grasping, pose estimation, and automated sorting tasks in industrial stacked workpiece scenarios, providing point-level perception results for 3D scene understanding and intelligent operation in complex stacked environments. Background Technology
[0002] In industrial production and intelligent sorting scenarios, workpieces are typically distributed in a random stacked manner. Due to occlusion, contact, intersection, and mutual compression among the workpieces, 3D point clouds in the same scene often exhibit problems such as blurred contact boundaries, overlapping local structures, difficulty in distinguishing similar categories, and unstable point features in boundary regions. Especially for workpieces with similar geometric structures or highly similar local morphologies, semantic confusion easily occurs in the contact areas between adjacent targets, making it difficult for segmentation models to accurately recover the workpiece boundaries and complete shapes.
[0003] Traditional point cloud segmentation methods often rely on handcrafted geometric features, such as normal vectors, curvature, Euclidean distance clustering, and region growing. However, in complex stacked scenarios, local geometric features are easily affected by scanning noise, uneven sampling density, occlusion, and the contact relationships between workpieces, leading to unstable boundary representations and making it difficult to meet the requirements of fine segmentation.
[0004] With the development of deep learning technology, point cloud segmentation methods based on PointNet, graph convolutional networks, and Transformers have been increasingly used for 3D point cloud understanding tasks. These methods can learn point-level semantic features through data-driven approaches, but in industrial stacked workpiece scenarios, problems such as adjacent workpiece adhesion, overly smooth contact boundaries, loss of local details, and missegmentation of similar categories may still occur.
[0005] In recent years, the hierarchical Transformer architecture has enhanced the long-range context modeling capability of point clouds through local window attention, window translation, and hierarchical key sampling, enabling the extraction of multi-scale semantic features at a lower computational cost. However, simply relying on the hierarchical Transformer backbone is still insufficient to fully address the fine-grained boundary representation problem in stacked workpiece point clouds. Specifically, the shallow stage lacks sufficient modeling of explicit geometric attributes and local neighborhood structures, the boundary difference information in the middle stage is easily weakened during multiple feature aggregation processes, and the traditional shared MLP in the classification stage has limited ability to express high-order nonlinear feature relationships. Therefore, it is necessary to propose a stacked workpiece point cloud segmentation method that can simultaneously enhance the shallow geometric boundary representation, the middle-level boundary context refinement, and the nonlinear discriminative ability in the classification stage. Summary of the Invention
[0006] The purpose of this invention is to overcome the technical problems existing in the prior art and to provide a method, system and electronic device for segmenting point clouds of industrial stacked workpieces.
[0007] The objective of this invention is achieved through the following technical solution: A first aspect of the present invention provides a method for segmenting point clouds of industrial stacked workpieces, comprising the following steps: S1. Collect 3D point cloud data of industrial stacked workpieces and preprocess the collected 3D point cloud data; S2. Organize the preprocessed 3D point cloud data into point-level semantic segmentation data. Each point-level semantic segmentation data contains 3D coordinates and point-level category labels, and generate corresponding point-level annotations according to the point-level category label numbers. S3. Divide the point-level semantic segmentation data into a training set, a validation set, and a test set according to a preset ratio, and perform data augmentation operations on the training set; S4. Construct a stacked workpiece point cloud segmentation network STGB-Net, wherein STGB-Net comprises sequentially connected: Point embedding layer is used to extract shallow point features from point-level semantic segmentation data; The geometric boundary embedding module (GBE) is used to perform residual fusion between the shallow point features and the geometric enhancement features it generates, and output shallow geometric boundary enhancement features. The hierarchical Transformer backbone network adopts an encoder-decoder structure. The encoder introduces a boundary context refinement module (BCR) to extract multi-scale point cloud features by downsampling the shallow geometric boundary enhancement features layer by layer. The decoder is used to restore the point-level resolution of the multi-scale point cloud features by upsampling layer by layer to obtain the decoded point-level features. The KAN classification head is used to perform point-by-point category prediction on the decoded point-level features. S5. Input the point-level semantic segmentation data in the training set into STGB-Net, output the category prediction result corresponding to each point, and use the point-level semantic segmentation loss function to supervise the training of STGB-Net; S6. Using the trained STGB-Net, perform point-by-point category prediction on the 3D point cloud data of the industrial stacked workpiece to be segmented, and obtain the point cloud semantic segmentation result of the industrial stacked workpiece to be segmented.
[0008] Preferably, the preprocessing includes: The collected 3D point cloud data is filtered, cleaned, and normalized.
[0009] Preferably, the enhancement operation includes: The training set is subjected to random rotation, random scaling, and Gaussian jitter processing.
[0010] Preferably, the geometric boundary embedding module GBE consists of a multidimensional geometry enhancement branch (MGE) and a local geometry coding branch (LGE). The Multidimensional Geometry Enhancement (MGE) branch constructs a local neighborhood through K-nearest neighbor search, calculates the covariance matrix and its eigenvalues and eigenvectors based on the spatial distribution of neighborhood points, extracts geometric attributes, performs feature mapping on each geometric attribute, and then adaptively fuses them. The Local Geometric Coding (LGE) branch is used to encode the local structural relationship between the center point and neighboring points, and concatenates and encodes the center point features, neighboring point features, center point coordinates, neighboring point coordinates, relative coordinates, Euclidean distance, and local geometric statistics. The geometric boundary embedding module GBE concatenates and fuses shallow point features, multidimensional geometric enhancement branch (MGE) output features, and local geometric coding branch (LGE) output features, and then outputs shallow geometric boundary enhancement features through residual connection.
[0011] Preferably, each layer of the hierarchical Transformer backbone network performs feature modeling on local regions of the point cloud through local window self-attention, and achieves cross-window information interaction through window translation and hierarchical key sampling.
[0012] Preferably, the Boundary Context Refinement (BCR) module is located in the middle layer of the hierarchical Transformer backbone network and is composed of a semantic difference module (SDM) and a PnP-3D local context refinement module connected in series. The semantic difference module (SDM) performs K-nearest neighbor grouping on the features of the middle layer points, calculates the feature difference between the center point and its neighboring points, and generates boundary guidance information by combining relative coordinates and Euclidean distance. The PnP-3D module constructs local coordinate context and local feature context, and refines the middle layer features through local context fusion and global bilinear response modeling.
[0013] Preferably, the KAN classification head includes a linear pre-mapping layer, a KAN nonlinear mapping layer, and a linear classification layer; wherein the KAN nonlinear mapping layer uses a learnable combination of univariate functions and a grid of basis functions to perform high-order nonlinear mapping on the decoded point-level features.
[0014] Preferably, the geometric properties include linearity, planarity, scattering, curvature, and pseudo-normal vector.
[0015] A second aspect of the present invention provides an industrial stacked workpiece point cloud segmentation system, comprising: The data acquisition module is used to acquire 3D point cloud data of industrial stacked workpieces and to preprocess the acquired 3D point cloud data. The point-level semantic segmentation data processing and annotation module is used to process the preprocessed 3D point cloud data into point-level semantic segmentation data. Each point-level semantic segmentation data contains 3D coordinates and point-level category labels, and generates corresponding point-level annotations according to the point-level category label numbers. The dataset partitioning module is used to divide point-level semantic segmentation data into training set, validation set and test set according to a preset ratio, and to perform data augmentation operation on the training set; The STGB-Net building block is used to construct the STGB-Net stacked artifact point cloud segmentation network, which includes the following sequentially connected components: Point embedding layer is used to extract shallow point features from point-level semantic segmentation data; The geometric boundary embedding module (GBE) is used to perform residual fusion between the shallow point features and the geometric enhancement features it generates, and output shallow geometric boundary enhancement features. The hierarchical Transformer backbone network adopts an encoder-decoder structure. The encoder introduces a boundary context refinement module (BCR) to extract multi-scale point cloud features by downsampling the shallow geometric boundary enhancement features layer by layer. The decoder is used to restore the point-level resolution of the multi-scale point cloud features by upsampling layer by layer to obtain the decoded point-level features. The KAN classification head is used to perform point-by-point category prediction on the decoded point-level features. The STGB-Net training module is used to input point-level semantic segmentation data from the training set into STGB-Net, output the category prediction result corresponding to each point, and perform supervised training of STGB-Net using the point-level semantic segmentation loss function. The segmentation prediction and output module is used to perform point-by-point category prediction on the 3D point cloud data of the industrial stacked workpiece to be segmented using the trained STGB-Net, and obtain the point cloud semantic segmentation result of the industrial stacked workpiece to be segmented.
[0016] A third aspect of the present invention provides an electronic device including a memory and a processor, wherein the memory stores computer instructions executable on the processor, and the processor executes the point cloud segmentation method for industrial stacked workpieces described in the first aspect when executing the computer instructions.
[0017] It should be further noted that the technical features corresponding to the above options can be combined or substituted to form new technical solutions if there is no conflict.
[0018] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention effectively eliminates noise and outliers in the original data by performing filtering, cleaning, normalization, and enhancement operations such as random rotation, scaling, and jitter on the collected point cloud, thereby expanding the diversity of training samples. This makes the model more robust to occlusion, pose changes, and density unevenness in actual stacked scenarios, laying the foundation for subsequent high-precision segmentation.
[0019] 2. This invention introduces a Geometric Boundary Embedding (GBE) module in the shallow feature extraction stage. This module explicitly calculates and adaptively fuses various local geometric properties (linearity, planarity, curvature, etc.) through the MGE branch, while simultaneously encoding the fine structural relationships between the center point and its neighbors through the LGE branch. The original point features and the geometrically enhanced features are then concatenated and fused, and the residual is output. This design ensures that shallow features not only contain spatial coordinate information but also rich local shape and boundary difference information, significantly enhancing the discriminative power of point features at contact boundaries and solving the problem of feature ambiguity in boundary regions in existing methods. Experiments show that this module can improve the mIoU in boundary regions by approximately 3% or more.
[0020] 3. This invention employs a hierarchical Transformer backbone network, which expands the receptive field layer by layer across multiple levels through local window self-attention and window translation mechanisms, taking into account both local details and global contextual information. Compared to convolutional networks with fixed receptive fields, this structure can adaptively aggregate semantic information of distantly similar objects, improving the semantic confusion between occluded regions and similar categories of objects.
[0021] 4. This invention introduces a Boundary Context Refinement (BCR) module at the mid-level features. The SDM module explicitly highlights response differences in boundary regions by calculating feature differences between the center point and neighboring points, preventing boundary information from being buried in deep networks. The PnP-3D module further integrates local coordinate context and global bilinear response, improving feature consistency in adjacent regions of similar workpieces. This two-level refinement ensures that mid-level features retain clear boundary contours while possessing good intra-class compactness and inter-class separability, directly improving the completeness and accuracy of the final segmentation boundary.
[0022] 5. This invention uses a KAN classification head instead of a traditional MLP classification head. The KAN layer performs high-order nonlinear mapping through learnable univariate functions and a grid of basis functions, enabling it to capture more complex discrimination patterns in point features, especially showing stronger fitting ability for difficult-to-classify points such as boundary points and occluded edge points. Compared to MLP, the KAN head can improve segmentation accuracy (mIoU) by 1.0 to 1.5 percentage points under the same feature dimensions, while maintaining a roughly equal number of parameters.
[0023] 6. The present invention achieves advanced performance of 88.94% average intersection-to-union ratio (mIoU) and 92.31% average accuracy (mAcc) on publicly available point cloud datasets of stacked mechanical parts, which is significantly better than existing methods such as KPConv, PointNeXt, Point TransformerV2 and Stratified Transformer. Attached Figure Description
[0024] Figure 1 This is a flowchart illustrating a point cloud segmentation method for industrial stacked workpieces according to an embodiment of the present invention; Figure 2 This is a diagram illustrating the STGB-Net stacked workpiece point cloud segmentation network structure according to an embodiment of the present invention. Figure 3 This is a structural diagram of the geometric boundary embedding module GBE shown in an embodiment of the present invention; Figure 4 This is a schematic diagram of the Transformer Block structure shown in an embodiment of the present invention; Figure 5 This is a structural diagram of the Boundary Context Refinement (BCR) module as shown in an embodiment of the present invention. Detailed Implementation
[0025] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0026] It should be noted that the defects in the solutions in the prior art are all the results of the inventors' practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of this application in the following text should be the inventors' contributions to this application in the process of invention and creation, and should not be understood as technical content known to those skilled in the art.
[0027] In view of the technical problems pointed out in the background art, the present invention provides the following embodiments: In one exemplary embodiment, a method for segmenting point clouds of industrial stacked workpieces is provided, such as... Figure 1 As shown, it includes the following steps: S1. Collect 3D point cloud data of industrial stacked workpieces and preprocess the collected 3D point cloud data; S2. Organize the preprocessed 3D point cloud data into point-level semantic segmentation data. Each point-level semantic segmentation data contains 3D coordinates (x coordinate, y coordinate, z coordinate) and point-level category labels, and generate corresponding point-level annotations according to the point-level category label numbers; S3. Divide the point-level semantic segmentation data into a training set, a validation set, and a test set according to a preset ratio, and perform data augmentation operations on the training set; S4. Construct a stacked workpiece point cloud segmentation network STGB-Net, wherein STGB-Net comprises sequentially connected: Point embedding layer is used to extract shallow point features from point-level semantic segmentation data; The geometric boundary embedding module (GBE) is used to perform residual fusion between the shallow point features and the geometric enhancement features it generates, and output shallow geometric boundary enhancement features. The hierarchical Transformer backbone network adopts an encoder-decoder structure. The encoder introduces a boundary context refinement module (BCR) to extract multi-scale point cloud features by downsampling the shallow geometric boundary enhancement features layer by layer. The decoder is used to restore the point-level resolution of the multi-scale point cloud features by upsampling layer by layer to obtain the decoded point-level features. The KAN classification head is used to perform point-by-point category prediction on the decoded point-level features. S5. Input the point-level semantic segmentation data in the training set into STGB-Net, output the category prediction result corresponding to each point, and use the point-level semantic segmentation loss function to supervise the training of STGB-Net; S6. Using the trained STGB-Net, perform point-by-point category prediction on the 3D point cloud data of the industrial stacked workpiece to be segmented, and obtain the point cloud semantic segmentation result of the industrial stacked workpiece to be segmented.
[0028] Specifically, in step S1, the collected 3D point cloud data of the stacked workpieces is filtered, cleaned, and normalized. In step S3, data augmentation operations are performed on the training set using methods such as random rotation, random scaling, and Gaussian jitter to expand the training data.
[0029] In step S4, the constructed STGB-Net network (such as...) Figure 2 It includes a point embedding layer, a geometric boundary embedding module (GBE), a hierarchical Transformer backbone network, a boundary context refinement module (BCR), and a KAN classification head, as detailed below: The input stacked workpiece point-level semantic segmentation data is first organized into a 4096×3 point coordinate matrix and a 4096×1 point-level label vector, where each point contains x, y, and z three-dimensional spatial coordinates, and there are 7 categories. The point-level semantic segmentation data first undergoes initial feature extraction through a point embedding layer, mapping the original 3D coordinates to a high-dimensional feature space. In one specific embodiment, the point embedding layer adopts a KPConv structure with 3 input channels, 48 output channels, a sphere query radius of 0.10, and a maximum number of neighborhood points of 34, thus obtaining a 4096×48 shallow point feature layer. This layer is mainly used to extract local spatial structure information of the point cloud, providing an initial feature foundation for subsequent geometric boundary enhancement and context modeling.
[0030] Subsequently, the shallow point features are input into the geometric boundary embedding module GBE (e.g., Figure 3 The Geometric Enhancement (GBE) branch consists of a Multidimensional Geometric Enhancement (MGE) branch and a Local Geometric Encoding (LGE) branch, used to enhance the expressive power of contact boundaries and local structural changes in the point cloud of stacked workpieces. The MGE branch first constructs a local neighborhood through K-nearest neighbor search, and calculates the covariance matrix and its eigenvalues and eigenvectors based on the spatial distribution of neighborhood points. It further extracts geometric attributes such as linearity, planarity, scattering, curvature, and pseudo-normal vectors. These geometric attributes are adaptively fused after feature mapping. In one specific embodiment, each geometric attribute mapping branch uses Linear(3,48) for feature dimensionality upscaling. The LGE branch encodes the local structural relationship between the center point and neighborhood points. Its input consists of center point features, neighborhood point features, center point coordinates, neighborhood point coordinates, relative coordinates, Euclidean distance, and local geometric statistics concatenated together, resulting in an input dimension of 48×2+3+3+3+1+3=109. Finally, GBE concatenates and fuses the original point features, MGE output features, and LGE output features, mapping the fusion dimension from 144 to 48, and outputs shallow geometric boundary enhancement features through residual connections.
[0031] Then, the point features output by GBE enter the hierarchical Transformer backbone network. This backbone network achieves collaborative modeling of local details and cross-regional contextual information through local window self-attention, window translation, and hierarchical key sampling, and completes multi-scale semantic feature extraction and point-level resolution restoration through an encoder-decoder structure. Specifically, the backbone network adopts an encoder-decoder structure, extracting multi-scale point cloud features layer by layer in the encoding stage, and restoring point-level resolution through layer-by-layer upsampling in the decoding stage. In a specific embodiment, the backbone network is set to 4 layers, with the number of channels in each layer being 48, 96, 192, and 384 respectively, corresponding to Transformer Blocks (such as...). Figure 4The number of multi-head attention heads is 2, 2, 4, and 2 respectively, and the number of multi-head attention heads is 3, 6, 12, and 24 respectively. The window sizes of each level are 0.16, 0.32, 0.64, and 1.28 respectively. Each level performs feature modeling on local regions of the point cloud through local window self-attention, and achieves cross-window information interaction through window translation and hierarchical key sampling. This allows the network to obtain a wider range of contextual information while reducing computational complexity, thereby improving the multi-scale semantic representation capability of stacked workpiece point clouds.
[0032] In the middle layer of the network, a Boundary Context Refinement (BCR) module is introduced (e.g., ...). Figure 5 This module further enhances the discrimination capability of contact boundary regions and adjacent regions of similar workpieces. BCR consists of a semantic difference module (SDM) and a PnP-3D local context refinement module. The SDM module explicitly enhances the boundary difference response between the center point and neighboring points, and the PnP-3D local context refinement module further integrates local coordinate context, local feature context, and global bilinear response, thereby improving the feature discrimination capability of complex contact regions and adjacent regions of similar workpieces. Specifically, the SDM first performs K-nearest neighbor grouping on the mid-level point features, calculates the feature difference between the center point and neighboring points, and generates boundary guidance information by combining relative coordinates and Euclidean distance to highlight the local difference response of the boundary region. The PnP-3D module further constructs local coordinate context and local feature context, and refines the mid-level features through local context fusion and global bilinear response modeling. In one specific embodiment, the BCR (Boundary Differentiation Context) is set after the second and third layers; the key mappings of the second-layer BCR include Linear(100,96), Linear(6,48), Linear(192,48), and Linear(96,12); the key mappings of the third-layer BCR include Linear(196,192), Linear(6,96), Linear(384,96), and Linear(192,24). Through this module, the network can explicitly enhance boundary difference information in the middle layers and improve the consistency and separability of local contextual features.
[0033] During the decoding stage, the network uses a layer-by-layer upsampling method to recover the point cloud resolution and integrates feature information from the corresponding layers in the encoding stage. This ensures that the final point-level features simultaneously contain shallow geometric boundary information, mid-level boundary context information, and deep semantic information. After decoding, 48-dimensional point-level features are output and fed into the KAN classification head for point-by-point category prediction.
[0034] The KAN classification head performs high-order mapping on point-level features using learnable univariate functions and nonlinear basis functions, and outputs the class prediction result for each point through a linear classification layer. Specifically, the decoded point-level features are first pre-mapped using Linear(48,48), and then input into the KAN nonlinear mapping layer. In one specific embodiment, the base weight size of the KAN mapping layer is 48×48, and the basis function weight size is 48×48×8, where 8 represents the number of basis function grids. Finally, the features are processed by the linear classification layer Linear(48,7), outputting the class prediction result for each point corresponding to 7 workpiece categories. During the training phase, cross-entropy loss function is used for supervised learning, AdamW is used as the optimizer, the learning rate is set to 0.003, the number of training epochs is 100, the batch size is set to 8, and training enhancements include random rotation, random scaling, and Gaussian jitter. After training, the point cloud of stacked workpieces to be segmented is input into the model to obtain the class probability of each point, and the class with the highest probability is taken as the final point-level segmentation result.
[0035] Through the above structure, this invention, while retaining the advantages of hierarchical Transformer long-range context modeling, further enhances the shallow boundary representation, mid-level context discrimination, and nonlinear mapping capabilities in the classification stage, thereby improving the point cloud segmentation accuracy and boundary recovery effect of stacked workpieces. On a stacked mechanical parts dataset, the method of this embodiment achieved the performance shown in Table 1, confirming the effectiveness of the method.
[0036] Table 1. Performance comparison of different methods on the point cloud segmentation dataset of stacked mechanical parts. In another exemplary embodiment, based on the same inventive concept as the method embodiment, an industrial stacked workpiece point cloud segmentation system is provided, comprising: The data acquisition module is used to acquire 3D point cloud data of industrial stacked workpieces and to preprocess the acquired 3D point cloud data. The point-level semantic segmentation data processing and annotation module is used to process the preprocessed 3D point cloud data into point-level semantic segmentation data. Each point-level semantic segmentation data contains 3D coordinates and point-level category labels, and generates corresponding point-level annotations according to the point-level category label numbers. The dataset partitioning module is used to divide point-level semantic segmentation data into training set, validation set and test set according to a preset ratio, and to perform data augmentation operation on the training set; The STGB-Net building block is used to construct the STGB-Net stacked artifact point cloud segmentation network, which includes the following sequentially connected components: Point embedding layer is used to extract shallow point features from point-level semantic segmentation data; The geometric boundary embedding module (GBE) is used to perform residual fusion between the shallow point features and the geometric enhancement features it generates, and output shallow geometric boundary enhancement features. The hierarchical Transformer backbone network adopts an encoder-decoder structure. The encoder introduces a boundary context refinement module (BCR) to extract multi-scale point cloud features by downsampling the shallow geometric boundary enhancement features layer by layer. The decoder is used to restore the point-level resolution of the multi-scale point cloud features by upsampling layer by layer to obtain the decoded point-level features. The KAN classification head is used to perform point-by-point category prediction on the decoded point-level features. The STGB-Net training module is used to input point-level semantic segmentation data from the training set into STGB-Net, output the category prediction result corresponding to each point, and perform supervised training of STGB-Net using the point-level semantic segmentation loss function. The segmentation prediction and output module is used to perform point-by-point category prediction on the 3D point cloud data of the industrial stacked workpiece to be segmented using the trained STGB-Net, and obtain the point cloud semantic segmentation result of the industrial stacked workpiece to be segmented.
[0037] In another exemplary embodiment, based on the same inventive concept as the method embodiment, an electronic device is provided, including a memory and a processor. The memory stores computer instructions that can be executed on the processor. When the processor executes the computer instructions, it performs an industrial stacked workpiece point cloud segmentation method provided in the embodiment of the present invention.
[0038] The processor may be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.
[0039] The embodiments of the subject matter and functional operation described in this specification can be implemented in: tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing device or for controlling the operation of a data processing device. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing device.
[0040] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.
[0041] Suitable processors for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.
[0042] It should be understood that each block in a flowchart or block diagram can represent a module, segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0043] The above detailed embodiments are a description of the present invention. It should not be considered that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, several simple deductions and substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the protection scope of the present invention.
Claims
1. A method for segmenting point clouds of stacked industrial workpieces, characterized in that, Includes the following steps: S1. Collect 3D point cloud data of industrial stacked workpieces and preprocess the collected 3D point cloud data; S2. Organize the preprocessed 3D point cloud data into point-level semantic segmentation data. Each point-level semantic segmentation data contains 3D coordinates and point-level category labels, and generate corresponding point-level annotations according to the point-level category label numbers. S3. Divide the point-level semantic segmentation data into a training set, a validation set, and a test set according to a preset ratio, and perform data augmentation operations on the training set; S4. Construct a stacked workpiece point cloud segmentation network STGB-Net, wherein STGB-Net comprises sequentially connected: Point embedding layer is used to extract shallow point features from point-level semantic segmentation data; The geometric boundary embedding module (GBE) is used to perform residual fusion between the shallow point features and the geometric enhancement features it generates, and output shallow geometric boundary enhancement features. The hierarchical Transformer backbone network adopts an encoder-decoder structure. The encoder introduces a boundary context refinement module (BCR) to extract multi-scale point cloud features by downsampling the shallow geometric boundary enhancement features layer by layer. The decoder is used to restore the point-level resolution of the multi-scale point cloud features by upsampling layer by layer to obtain the decoded point-level features. The KAN classification head is used to perform point-by-point category prediction on the decoded point-level features. S5. Input the point-level semantic segmentation data in the training set into STGB-Net, output the category prediction result corresponding to each point, and use the point-level semantic segmentation loss function to supervise the training of STGB-Net; S6. Using the trained STGB-Net, perform point-by-point category prediction on the 3D point cloud data of the industrial stacked workpiece to be segmented, and obtain the point cloud semantic segmentation result of the industrial stacked workpiece to be segmented.
2. The point cloud segmentation method for industrial stacked workpieces according to claim 1, characterized in that, The preprocessing includes: The collected 3D point cloud data is filtered, cleaned, and normalized.
3. The point cloud segmentation method for industrial stacked workpieces according to claim 1, characterized in that, The enhancement operations include: The training set is subjected to random rotation, random scaling, and Gaussian jitter processing.
4. The point cloud segmentation method for industrial stacked workpieces according to claim 1, characterized in that, The geometric boundary embedding module GBE consists of a multidimensional geometry enhancement branch (MGE) and a local geometry coding branch (LGE). The Multidimensional Geometry Enhancement (MGE) branch constructs a local neighborhood through K-nearest neighbor search, calculates the covariance matrix and its eigenvalues and eigenvectors based on the spatial distribution of neighborhood points, extracts geometric attributes, performs feature mapping on each geometric attribute, and then adaptively fuses them. The Local Geometric Coding (LGE) branch is used to encode the local structural relationship between the center point and neighboring points, and concatenates and encodes the center point features, neighboring point features, center point coordinates, neighboring point coordinates, relative coordinates, Euclidean distance, and local geometric statistics. The geometric boundary embedding module GBE concatenates and fuses shallow point features, multidimensional geometric enhancement branch (MGE) output features, and local geometric coding branch (LGE) output features, and then outputs shallow geometric boundary enhancement features through residual connection.
5. The point cloud segmentation method for industrial stacked workpieces according to claim 1, characterized in that, Each layer of the hierarchical Transformer backbone network models the features of local regions of the point cloud through local window self-attention, and realizes cross-window information interaction through window translation and hierarchical key sampling.
6. The point cloud segmentation method for industrial stacked workpieces according to claim 1, characterized in that, The Boundary Context Refinement (BCR) module is located in the middle layer of the hierarchical Transformer backbone network and is composed of the Semantic Difference (SDM) module and the PnP-3D Local Context Refinement (PnP-3D) module connected in series. The SDM module performs K-nearest neighbor grouping on the features of the middle layer points, calculates the feature difference between the center point and its neighboring points, and generates boundary guidance information by combining relative coordinates and Euclidean distance. The PnP-3D module constructs local coordinate context and local feature context, and refines the middle layer features through local context fusion and global bilinear response modeling.
7. The point cloud segmentation method for industrial stacked workpieces according to claim 1, characterized in that, The KAN classification head includes a linear pre-mapping layer, a KAN nonlinear mapping layer, and a linear classification layer; wherein the KAN nonlinear mapping layer uses a learnable combination of univariate functions and a basis function grid to perform high-order nonlinear mapping on the decoded point-level features.
8. The point cloud segmentation method for industrial stacked workpieces according to claim 4, characterized in that, The geometric properties include linearity, planarity, scattering, curvature, and pseudonormal vector.
9. A point cloud segmentation system for industrial stacked workpieces, characterized in that, include: The data acquisition module is used to acquire 3D point cloud data of industrial stacked workpieces and to preprocess the acquired 3D point cloud data. The point-level semantic segmentation data processing and annotation module is used to process the preprocessed 3D point cloud data into point-level semantic segmentation data. Each point-level semantic segmentation data contains 3D coordinates and point-level category labels, and generates corresponding point-level annotations according to the point-level category label numbers. The dataset partitioning module is used to divide point-level semantic segmentation data into training set, validation set and test set according to a preset ratio, and to perform data augmentation operation on the training set; The STGB-Net building block is used to construct the STGB-Net stacked artifact point cloud segmentation network, which includes the following sequentially connected components: Point embedding layer is used to extract shallow point features from point-level semantic segmentation data; The geometric boundary embedding module (GBE) is used to perform residual fusion between the shallow point features and the geometric enhancement features it generates, and output shallow geometric boundary enhancement features. The hierarchical Transformer backbone network adopts an encoder-decoder structure. The encoder introduces a boundary context refinement module (BCR) to extract multi-scale point cloud features by downsampling the shallow geometric boundary enhancement features layer by layer. The decoder is used to restore the point-level resolution of the multi-scale point cloud features by upsampling layer by layer to obtain the decoded point-level features. The KAN classification head is used to perform point-by-point category prediction on the decoded point-level features. The STGB-Net training module is used to input point-level semantic segmentation data from the training set into STGB-Net, output the category prediction result corresponding to each point, and use the point-level semantic segmentation loss function to supervise the training of STGB-Net. The segmentation prediction and output module is used to perform point-by-point category prediction on the 3D point cloud data of the industrial stacked workpiece to be segmented using the trained STGB-Net, and obtain the point cloud semantic segmentation result of the industrial stacked workpiece to be segmented.
10. An electronic device comprising a memory and a processor, wherein the memory stores computer instructions executable by the processor, characterized in that, When the processor executes computer instructions, it performs a point cloud segmentation method for industrial stacked workpieces as described in any one of claims 1-8.