A training method and a completion method and device of a part point cloud completion network

Abstract

This application belongs to the field of point cloud completion, specifically disclosing a training method, completion method, and apparatus for a point cloud completion network for parts. Addressing the problems of insufficient adaptability of network feature extraction, lack of constraints on the geometric characteristics of industrial parts in the loss function, and the disconnect between the simulated defect point cloud and the real scene in existing technologies, this application improves the point cloud completion network by: introducing attention coefficients in EdgeConv to achieve dynamic weight allocation; adding multi-scale feature fusion in the Transformer encoder-decoder; adding symmetry penalty terms and accuracy penalty terms to the original loss function; and generating defect clouds by randomly combining high-curvature regions, common missing regions in industrial scenes, and randomly downsampled defects. This application improves the accuracy and robustness of part point cloud completion, adapting to the needs of practical industrial applications.

Description

Technical Field

[0001] This application belongs to the field of point cloud completion, and more specifically, relates to a training method, completion method and apparatus for a point cloud completion network for parts. Background Technology

[0002] In modern industrial and technological fields such as intelligent manufacturing, aerospace equipment assembly, automotive parts production, and 3D digital modeling, 3D point cloud data of parts is the core foundation for realizing 3D reconstruction, defect detection, assembly compatibility analysis, and digital twin modeling. Point cloud data is usually acquired through non-contact sensing devices such as RGB-D (RGB-Depth) cameras and LiDAR. However, due to limitations in the acquisition environment, part structural characteristics, and the sensing technology itself, the actual acquired part point clouds often have serious quality problems: for example, depth values ​​are easily distorted or point clouds are missing in geometrically sensitive areas such as part edges and cavities; occlusion in scenes with multiple overlapping parts can create local voids; and changes in ambient lighting and differences in part surface materials can cause noise interference in the point cloud. These incomplete and noisy point cloud data directly destroy the integrity and accuracy of the part's 3D structure, failing to meet the requirements of subsequent industrial applications for high-precision point clouds. Therefore, part point cloud completion technology has become a key core technology for solving point cloud quality defects and ensuring the reliability of 3D digital applications.

[0003] With the development of deep learning technology, point cloud completion has gradually shifted from traditional geometric interpolation and model fitting methods to end-to-end generative completion schemes based on neural networks. Among them, the PoinTr network, with its powerful global feature modeling capabilities, has shown significant advantages in the field of point cloud completion. It is a geometrically perceptual network that uses point proxies as intermediate carriers to generate incomplete point clouds. Compared with traditional graph convolution-based completion models such as PointNet and DGCNN, it has achieved breakthroughs in completion efficiency and structural rationality, becoming one of the mainstream benchmark models in the field of point cloud completion.

[0004] However, the original point cloud completion network still faces many technical challenges in point cloud completion tasks for industrial parts: First, the simulation of defect point clouds is out of touch with the real industrial scenario. During training, the network uses a "random point deletion" method to convert complete point clouds into defect point clouds. This artificially constructed defect pattern is completely inconsistent with the missing point cloud logic that actually exists in industrial scenarios, thus affecting the completion accuracy and robustness. Secondly, the network's feature extraction adaptability is insufficient. The point cloud distribution density and feature correlation of different regions such as flat surfaces, curved surfaces, and edges of parts are significantly different. Hard division with a fixed threshold will lead to over- or under-extraction of local features: for high-density flat areas, redundant points will introduce invalid features; for low-density edge areas, the lack of key neighboring points will lead to incomplete feature representation, ultimately affecting the detail accuracy and structural consistency of the completed point cloud. Thirdly, the loss function design lacks constraints on the geometric characteristics of industrial parts. The loss function of typical geometric perception networks only focuses on the spatial distance error between predicted points and real points, but does not consider the key characteristics such as the geometric symmetry and dimensional accuracy that are common in industrial parts. This leads to the model "emphasizing form but not quality" during the completion process. Fourth, the production of industrial parts point cloud datasets faces two challenges: First, high-precision point cloud acquisition of real industrial parts requires specialized equipment and preprocessing such as cleaning and positioning of the parts, resulting in long acquisition cycles and high costs. Second, point cloud data of real parts is easily affected by the acquisition environment and equipment accuracy, and has problems such as noise and annotation errors, making it difficult to form standardized high-quality training samples, which in turn affects the generalization ability and adaptability of the completion model. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this application is to provide a training method, a completion method, and an apparatus for a point cloud completion network for parts, aiming to solve the problem of insufficient adaptability of the original PoinTr network as a point cloud completion network for network feature extraction.

[0006] To achieve the above objectives, in a first aspect, this application provides a training method for a part point cloud completion network, comprising: Acquire training samples, which include part defect cloud and corresponding complete point cloud; A point cloud completion network model for a part is obtained. This model is an improved PoinTr network, which includes: a mapping module that maps the input point cloud to multi-dimensional features; an FPS downsampling module that reduces the size of the input point cloud to reduce subsequent computational costs while determining the center point; an improved EdgeConv module that determines local geometric features based on multi-dimensional features and the center point; a Transformer encoder / decoder that transforms the local geometric features into global features; and a FoldingNet module that generates a complete point cloud based on the global features. Specifically, the improved EdgeConv indexes the center point features from the multi-dimensional features, determines the neighborhood for each center point using KNN, indexes the corresponding neighborhood point features for each neighborhood point from the multi-dimensional features, calculates the difference features between the neighborhood point features and the center point features, concatenates the difference features with the center point features, and then performs weighted aggregation using an attention coefficient. The aggregation result is used as a local geometric feature, where the attention coefficient measures the feature correlation between the center point and its neighborhood points. Based on the training samples, the part point cloud completion network model is trained to obtain the trained part point cloud completion network model.

[0007] Preferably, in the improved EdgeConv, the calculation formula for local geometric features is as follows:

[0008]

[0009] in, For the first The center point is at the 1st Local geometric features of the layer For the first Each center point nearest neighbor set For the first The first in the layer The center point and the first Attention coefficients of neighboring points For the first Layer MLP parameters , For the first The first in the layer The center point and the first The difference feature of each neighboring point This is a feature concatenation operation used to concatenate the attention coefficients and feature difference vectors along the channel dimension. The constant coefficients, For max pooling layer, For single-layer MLP parameters Used to output scalar attention weights. This represents the number of layers in the MLP.

[0010] Preferably, the encoder of the Transformer is composed of multiple stacked Geometry-aware Transformer Blocks. Each Geometry-aware Transformer Block includes: an Attention Block that extracts attention features from the input features; an improved Geometry-aware Block that extracts geometric features from the input features; a feature fusion module that fuses the attention features and geometric features; a residual connection module that performs residual connections between the fused features and the input features; and an MLP that maps the residual connection features to output the results. The improved Geometry-aware Block employs a multi-scale branching structure, where features at different scales are used together as geometric features. The multi-scale branching structure consists of multiple branches with different neighborhood sizes set in parallel, each branch being used to extract local geometric features at different spatial scales.

[0011] Preferably, in the improved Geometry-aware Block, the calculation formula for the geometric features is as follows:

[0012] in, For the first The point at the th Geometric features after layer fusion of features from all scales For the first Parameters of layer MLP , For feature splicing operations, For the first The point at the th Layer Geometric features at this scale The number of scales in a multi-scale branching structure.

[0013] Preferably, the total loss function during training is a weighted sum of the original PoinTr loss function, a symmetry penalty term, and an accuracy penalty term. The symmetry penalty term is used to ensure the consistency of the predicted point cloud at symmetrical positions, and the accuracy penalty term is used to control size errors.

[0014] Preferably, the formula for calculating the total loss function is as follows:

[0015]

[0016]

[0017] in, The original loss function of PoinTr is... For symmetrical penalty terms, As an accuracy penalty item, These are constant coefficients, The predicted point cloud set output by the network. The number of points contained in the set. To predict points in a point cloud set, for Symmetric points about the structural symmetry axis of the point cloud of the complete part In the complete point cloud The nearest point, This is the maximum error threshold.

[0018] Preferably, the residual cloud in the training samples is generated in the following way: Calculate the Gaussian curvature or average curvature of each point on the surface of the part based on the complete point cloud of the part. For a point whose curvature exceeds a set threshold, detect other points in its neighborhood whose curvature exceeds the set threshold, treat these points as a connected set, and recursively detect them according to this rule. The final connected set is then taken as a high curvature region. Construct missing regions corresponding to different missing patterns of parts in industrial inspection scenarios; Randomly select high curvature regions, at least one missing region, and / or defects that were originally randomly downsampled, and delete them from the complete point cloud of the part to obtain the defect cloud of the part.

[0019] To achieve the above objectives, in a second aspect, this application provides a method for completing point clouds of parts, comprising: Acquire RGB-D images of the workpiece to be inspected and preprocess the depth map; The captured RGB image is segmented using a segmentation network to obtain the region containing the part to be detected, which is used as an RGB part mask image. The RGB part mask image is mapped onto the preprocessed depth map to obtain the depth map mask of the part. The depth map mask of the part is then converted into a 3D point cloud of the part through a coordinate transformation matrix. The 3D point cloud of the part is input into the part point cloud completion network model for completion, and the complete point cloud of the part is obtained. The part point cloud completion network model is trained using the method described in the first aspect.

[0020] To achieve the above objectives, in a third aspect, this application provides a part point cloud completion device, including a memory and one or more processors; The memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions; The one or more processors invoke the computer instructions to cause the system to perform the completion method as described in the second aspect.

[0021] To achieve the above objectives, in a fourth aspect, this application provides a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the training method as described in the first aspect, or the completion method as described in the second aspect.

[0022] It is understood that the beneficial effects of the third and fourth aspects mentioned above can be found in the relevant descriptions of the first and second aspects mentioned above, and will not be repeated here.

[0023] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) To address the insufficient adaptability of network feature extraction, this application improves the network structure of PoinTr. On the one hand, in EdgeConv of DGCNN_Group, the center point is quantized through the attention coefficient. With neighboring points To achieve dynamic weight allocation, the feature correlation is used. With the introduction of dynamic attention coefficients, the weights of neighboring points relative to the center point change, thus altering the features calculated by EdgeConv. The modification to EdgeConv aims to abandon the original network's equivalence weight partitioning based on k-nearest neighbors, instead quantifying the correlation between the center point and its neighbors using attention coefficients. This strengthens the accurate representation of local geometric features, highlighting the feature contributions of key neighboring points and suppressing interference from invalid redundant points. Furthermore, multi-scale feature fusion is added to both the encoder and decoder of the Transformer. The multi-scale branching structure involves setting multiple branches with different neighborhood sizes in parallel. Each branch extracts local geometric features at different spatial scales. Multi-scale feature fusion aims to overcome the limitations of single-scale neighborhood representation. By setting branches with different neighborhood sizes in parallel, it covers multi-dimensional features from local microstructures to overall macrostructures, avoiding incomplete feature extraction caused by a single scale, and thus better supporting high-quality global modeling.

[0024] (2) To address the lack of constraints on the geometric characteristics of industrial parts in the design of the loss function, this application adds two penalty terms to the original loss function of the PointTr network. The symmetry penalty term is used to ensure the symmetry of the point cloud structure, and the accuracy penalty term is used to control dimensional errors. The improvement of the loss function aims to introduce geometric constraints for industrial parts. The original loss function design only focuses on the spatial distance between the point cloud and the real points, ignoring the geometric features of the industrial parts. The symmetry penalty term strengthens the consistency of the symmetric structure of the completed point cloud, while the accuracy penalty term controls the scale error to a certain extent to adapt to industrial precision requirements.

[0025] (3) To address the disconnect between the simulation of defect point clouds and real industrial scenarios, this application randomly selects high curvature regions, at least one missing region, and / or originally randomly downsampled defects, and removes them from the complete point cloud of the part to obtain the defect cloud of the part. Since the proportion of each of the three types of defects in the total number is also allocated by generating random numbers, this means that the proportion of the defect types contained in the defect point cloud of each round of input is different, thus making it more consistent with the defect cloud of the real scenario. Attached Figure Description

[0026] Figure 1 This is a flowchart of a part point cloud completion method provided in an embodiment of this application.

[0027] Figure 2 This is a rendering example of the industrial parts being tested, provided in an embodiment of this application.

[0028] Figure 3 This is the RGB mask image provided in the embodiments of this application.

[0029] Figure 4 This is the Depth diagram provided in the embodiments of this application.

[0030] Figure 5 This is a Depth mask image provided in the embodiments of this application.

[0031] Figure 6 This is a schematic diagram of the Geometry-aware Transformer Block provided in the embodiments of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] In this application, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. In this application, the symbol " / " indicates that the related objects are in an "or" relationship, for example, A / B means A or B.

[0034] In this application, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0035] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0036] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0037] The embodiments of this application are described below with reference to the accompanying drawings.

[0038] like Figure 1 As shown, this application provides a method for completing point clouds of parts, specifically: S1. Acquire RGB-D images of the workpiece to be inspected using a depth camera, and then preprocess the depth images.

[0039] Furthermore, in step S1, when the depth image is acquired by a depth camera (such as Kinect or RealSense), it is easily affected by ambient light, object material, and measurement distance, resulting in noise. Therefore, the preprocessing operation here is mainly aimed at the depth image. Considering that the depth camera can acquire not only depth information but also RGB information, joint bilateral filtering is selected. Combining the corresponding RGB image information, the edge of the RGB image is used to guide the filtering of the depth image, further improving the edge preservation effect. While removing noise, the edges are preserved as much as possible. Then, bilinear interpolation is used for hole filling.

[0040] S2. The RGB image captured by the camera is segmented using a deep learning segmentation network to obtain the region containing the part to be detected, i.e., the corresponding mask image.

[0041] Furthermore, in step S2, the segmentation network can employ YOLO, Mask-RCNN series networks, or other deep learning-based neural network image segmentation algorithms. Considering that existing technologies for image segmentation are already quite mature, this application focuses on point cloud completion; therefore, the specific segmentation methods will not be elaborated upon further.

[0042] S3. Using the RGB part mask image generated by the segmentation network, the depth map mask of the part can be obtained. Through the coordinate transformation matrix, the corresponding depth region is converted into a 3D point cloud.

[0043] Furthermore, in step S3, the output of the segmentation network includes a series of masks and the corresponding target categories. By traversing these masks, and considering the one-to-one correspondence between pixels in the depth image and the RGB image, the depth map mask of the part can be obtained by directly mapping the mask output by the segmentation network in the RGB image onto the depth map. In one illustrated embodiment, Figure 2 For example, Figure 3 This is an RGB mask image. Figure 4 For the depth graph, Figure 5 This is a depth mask image.

[0044] Furthermore, in step S3, the depth information is transformed into point cloud information. This process requires mapping the 2D pixel coordinates and depth values ​​to 3D spatial coordinates using the camera intrinsic parameter matrix. Essentially, this is the inverse operation of the camera projection model, i.e., inferring 3D points in the camera coordinate system or world coordinate system from the image plane. Considering that actual cameras are not ideal models and may exhibit pixel distortion, the transformation matrix formula is as follows:

[0045] In the formula, These are the x and y coordinates of the pixels in the 2D image after distortion correction. These are 3D camera coordinates, i.e., point cloud coordinates. For depth value, The formula for calculating the inverse of the intrinsic parameter matrix is ​​as follows:

[0046] In the formula, The coordinates of the principal point in the image. These are the focal lengths in the horizontal and vertical directions, respectively.

[0047] Distortion correction primarily targets radial distortion:

[0048] In the formula, This is the orthodontic coefficient. The x-coordinate of the two-dimensional image pixels before distortion correction. The vertical coordinate of the two-dimensional image pixels before distortion correction. It represents the Euclidean distance from the pixel before distortion correction to the principal point of the image.

[0049] S4. Input the obtained point cloud into a deep learning point cloud completion network to complete it, and obtain a relatively complete point cloud.

[0050] The improvements to the PoinTr network completion module in this application are divided into two aspects: one is the improvement of the data module, and the other is the improvement of the network module.

[0051] I. Improvements to the data module PoinTr takes in defective point clouds as input and labels them as the corresponding complete point clouds. However, the training dataset is a complete point cloud, so a method is needed to convert the complete point cloud into a defective one. PoinTr originally used random point deletion. However, this method cannot simulate the missing point cloud patterns in real-world scenarios. Therefore, the process of converting a complete point cloud into a defective point cloud is essentially simulating the missing point cloud patterns in real-world scenarios.

[0052] To address these issues, this application replaces "a single downsampled random defect" with "a random combination of real defects appearing in actual industrial scenarios." Specifically, based on the complete point cloud of the part, the Gaussian curvature or average curvature of each point on the part surface is calculated. For points with curvature exceeding a set threshold, points in their neighborhood with curvature exceeding the set threshold are detected, and these points are treated as a connected set. Recursive detection is performed according to this rule, and the final connected set is taken as a high-curvature region. Missing regions corresponding to different missing patterns of the part in industrial inspection scenarios are constructed. High-curvature regions, at least one missing region, and / or originally randomly downsampled defects are randomly selected and deleted from the complete point cloud of the part to obtain the part's residual defect cloud. In one illustrated embodiment, assembly occlusion can be considered as a local edge defect, transportation collision can be considered as a corner defect, and wear and corrosion can be considered as a uniform surface defect. For these specific defects, only the point cloud of the corresponding region needs to be deleted. At least one of the above two types of defects is randomly selected and used together with the original randomly downsampled defects as generated defects. The proportion of each of the three types of defects in the total number is also allocated by generating random numbers. This means that the proportion of defect types contained in the defect point cloud in each round of input is different, thereby solving the problem of the disconnect between defect point cloud simulation and real industrial scene.

[0053] II. Improvements to the network module.

[0054] The PoinTr network is mainly divided into three parts: The first part is DGCNN_Group, which is a specific optimization of DGCNN for point cloud completion tasks. Its process is roughly as follows: input point cloud, the point cloud is mapped to multi-dimensional features through convolutional layers, the point cloud is downsampled for the first time using the FPS algorithm, and then the multi-dimensional features mapped from the point cloud are used as extracted point features and neighbor source features, input together with the point cloud into two EdgeConv layers for feature extraction. The feature extraction results are downsampled a second time using the FPS algorithm, and finally output through one EdgeConv layer; the second part is the Transformer encoder and decoder, where the encoder... The first part is used to perform self-attention on the Key / Value pairs to generate a global context representation. The decoder is used to perform cross-attention with the encoded Key / Value pairs using the Query as the query, and outputs deep features that fuse global information, which are then fed into FoldingNet. The third part is FoldingNet, which takes the input feature vector and generates its corresponding folding seed. The folding seed is a 2D grid that is similar in dimension to the input feature vector. Then, the feature vector and the folding seed are folded, that is, concatenated in the feature dimension. Then, the folded feature is concatenated with the original input feature vector to finally obtain the output 3D point cloud.

[0055] Furthermore, EdgeConv mainly consists of three steps. First, it uses the KNN algorithm to find the index of its neighboring points for each point in the input point cloud, thereby constructing a graph structure. Then, it extracts the corresponding features from the neighboring source features based on the corresponding index in the graph structure, performs differential calculations with the query point features, and finally concatenates the results with the query point features.

[0056] Furthermore, the Transformer's encoder is composed of multiple stacked Geometry-aware Transformer Blocks. Each Geometry-aware Transformer Block specifically includes an Attention Block, a Geometry-aware Block, a feature fusion and residual connection, and an MLP. Features input to the Geometry-aware Transformer Block first undergo attention calculation via the Attention Block, then are input into the Geometry-aware Block, and finally output by the MLP after feature fusion and residual connections. The Attention Block is responsible for capturing attention features, while the Geometry-aware Block is responsible for capturing geometric features.

[0057] The improvements to the PoinTr structure in this application are mainly in the point cloud feature extraction module in the first half, the encoder and decoder of the Transformer in the middle, and the loss function.

[0058] (1) Improvement of the loss function This application adds two penalty terms to the original loss function—symmetric penalty terms. and accuracy penalty items The calculation formula is as follows:

[0059] in, The original loss function of PoinTr is... For symmetrical penalty terms, As an accuracy penalty item, These are constant coefficients, The predicted point cloud set output by the network. The number of points contained in the set. To predict points in a point cloud set, for Symmetric points about the structural symmetry axis of the point cloud of the complete part In the complete point cloud The nearest point, This represents the maximum error threshold. When the distance exceeds τ, the penalty term increases quadratically, forcibly controlling the dimensional error. Symmetrical penalty term. Accuracy penalty term used to ensure point cloud structure symmetry. Used to control dimensional errors. This change aims to address the lack of constraints on the geometric characteristics of industrial parts in the design of the loss function.

[0060] (2) Improvement of EdgeConv structure Firstly, for EdgeConv, this application introduces a dynamic soft neighborhood with adaptive weights between KNN graph construction and neighborhood feature extraction to address the issue of EdgeConv's reliance on fixed k-nearest neighbor hard partitioning. In the original KNN algorithm, for each center point, the contribution weights of all points in its neighborhood are indistinguishable; in short, all neighborhood points are equivalent. This application quantifies the center point using an attention coefficient. With neighboring points The feature correlation is used to achieve dynamic weight allocation, thereby realizing dynamic soft domain. Therefore, after constructing the graph structure using the KNN algorithm, this application calculates the attention coefficient between each center point and its corresponding neighboring points. Finally, the result of concatenating the differential features and the query point features is aggregated and output via the attention coefficient. Specifically, the calculation of the attention coefficient between each neighboring point and its corresponding center point is as follows:

[0061] in, For the first The center point is at the 1st Local geometric features of the layer For the first Each center point nearest neighbor set For the first The first in the layer The center point and the first Attention coefficients of neighboring points For the first Layer MLP parameters , For the first The first in the layer The center point and the first The difference feature of each neighboring point This is a feature concatenation operation used to concatenate the attention coefficients and feature difference vectors along the channel dimension. The constant coefficients, For max pooling layer, For single-layer MLP parameters Used to output scalar attention weights. This refers to the number of layers in the MLP. This change primarily targets the local feature update sub-module EdgeConv in the DGCNN_Group module, where the MLP is the dedicated computational unit for this sub-module.

[0062] By introducing a dynamic attention coefficient, the weights of neighboring points relative to the center point change, thus altering the final features calculated by EgdeConv. Correspondingly, the EgdeConv features are updated as follows:

[0063] In the formula, For the first Layer MLP parameters , For the first The first in the layer The point and the first The feature difference vector of each neighboring point For feature concatenation, the attention coefficients and feature difference vectors are concatenated along the channel dimension. `max` represents the attention weight adjustment coefficient, and `max` represents the channel-wise maximum pooling value within the neighborhood. This change aims to address the insufficient adaptability of network feature extraction.

[0064] (3) Improvements to the encoder and decoder of Transformer This application adds multi-scale feature fusion to both the encoder and decoder of the Transformer to address the insufficient representation of single-scale neighborhoods. Multiple parallel branches are added to the Geometry-aware Transformer Block, and the final output is merged into the feature fusion and residual connections, as shown below. Figure 6 As shown, each Geometry-awareBlock branch corresponds to a scale, thus introducing multi-scale... Integrating features from different domain scales:

[0065] In the formula, For the first The point at the th Layer Local feature vectors at a certain scale For the first Scale-based MLP (parameters can be shared or independent), feature fusion:

[0066] In the formula, For the first The point at the th The global feature vector after fusing features from all scales. For the first The layer-fused MLP maps the spliced ​​multi-scale features to a unified dimension. This modification aims to address the insufficient adaptability of network feature extraction.

[0067] Example In this implementation, the point cloud dataset used for training the deep point cloud completion network is point cloud data based on the CAD model domain. This is synthetic virtual data, not a real dataset, and primarily consists of single parts. The dataset originates from the standard parts library in Solidworks. Virtual point cloud data for the corresponding parts is generated by performing Poisson random downsampling on the STL format (triangular mesh) of the part model. The number of target points used here ranges from 10,000 to 20,000, with a minimum sampling interval... The range is between 0.1 and 0.3 mm, aiming to address the high cost and poor adaptability of acquiring high-quality training datasets.

[0068] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0069] Based on the methods in the above embodiments, this application provides an electronic device that may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor may invoke logical instructions stored in the memory to execute the methods in the above embodiments.

[0070] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0071] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0072] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0073] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0074] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0075] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0076] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0077] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A training method for a point cloud completion network for parts, characterized in that, include: Acquire training samples, which include part defect cloud and corresponding complete point cloud; A point cloud completion network model for a part is obtained. This model is an improved PoinTr network, which includes: a mapping module that maps the input point cloud to multi-dimensional features; an FPS downsampling module that reduces the size of the input point cloud to reduce subsequent computational costs while determining the center point; an improved EdgeConv module that determines local geometric features based on multi-dimensional features and the center point; a Transformer encoder / decoder that transforms the local geometric features into global features; and a FoldingNet module that generates a complete point cloud based on the global features. Specifically, the improved EdgeConv indexes the center point features from the multi-dimensional features, determines the neighborhood for each center point using KNN, indexes the corresponding neighborhood point features for each neighborhood point from the multi-dimensional features, calculates the difference features between the neighborhood point features and the center point features, concatenates the difference features with the center point features, and then performs weighted aggregation using an attention coefficient. The aggregation result is used as a local geometric feature, where the attention coefficient measures the feature correlation between the center point and its neighborhood points. Based on the training samples, the part point cloud completion network model is trained to obtain the trained part point cloud completion network model.

2. The training method as described in claim 1, characterized in that, In the improved EdgeConv, the calculation formula for local geometric features is as follows: in, For the first The center point is at the 1st Local geometric features of the layer For the first Each center point nearest neighbor set For the first The first in the layer The center point and the first Attention coefficients of neighboring points For the first Layer MLP parameters , For the first The first in the layer The center point and the first The difference feature of each neighboring point This is a feature concatenation operation used to concatenate the attention coefficients and feature difference vectors along the channel dimension. The constant coefficients, For max pooling layer, For single-layer MLP parameters Used to output scalar attention weights. This represents the number of layers in the MLP.

3. The training method as described in claim 1, characterized in that, The encoder of the Transformer is composed of multiple stacked Geometry-aware Transformer Blocks. Each Geometry-aware Transformer Block includes: an Attention Block that extracts attention features from the input features; an improved Geometry-aware Block that extracts geometric features from the input features; a feature fusion module that fuses the attention features and geometric features; a residual connection module that performs residual connections between the fused features and the input features; and an MLP that maps the residual connection features to the output. The improved Geometry-aware Block employs a multi-scale branching structure, obtaining features at different scales that together serve as geometric features. The multi-scale branching structure consists of multiple branches with different neighborhood sizes set in parallel, each branch used to extract local geometric features at different spatial scales.

4. The training method as described in claim 3, characterized in that, In the improved Geometry-aware Block, the calculation formula for the geometric features is as follows: in, For the first The point at the th Geometric features after layer fusion of features from all scales For the first Parameters of layer MLP , For feature splicing operations, For the first The point at the th Layer Geometric features at this scale The number of scales in a multi-scale branching structure.

5. The training method according to any one of claims 1 to 4, characterized in that, The total loss function during training is a weighted sum of the original PoinTr loss function, the symmetry penalty term, and the accuracy penalty term. The symmetry penalty term is used to ensure the consistency of the predicted point cloud at symmetrical positions, and the accuracy penalty term is used to control size errors.

6. The training method as described in claim 5, characterized in that, The formula for calculating the total loss function is as follows: in, The original loss function of PoinTr is... For symmetrical penalty terms, As an accuracy penalty item, These are constant coefficients, The predicted point cloud set output by the network. The number of points contained in the set. To predict points in a point cloud set, for Symmetric points about the structural symmetry axis of the point cloud of the complete part In the complete point cloud The nearest point, This is the maximum error threshold.

7. The training method according to any one of claims 1 to 4, characterized in that, The defect cloud in the training samples is generated in the following way: Calculate the Gaussian curvature or average curvature of each point on the surface of the part based on the complete point cloud of the part. For a point whose curvature exceeds a set threshold, detect other points in its neighborhood whose curvature exceeds the set threshold, treat these points as a connected set, and recursively detect them according to this rule. The final connected set is then taken as a high curvature region. Construct missing regions corresponding to different missing patterns of parts in industrial inspection scenarios; Randomly select high curvature regions, at least one missing region, and / or defects that were originally randomly downsampled, and delete them from the complete point cloud of the part to obtain the defect cloud of the part.

8. A method for point cloud completion of a part, characterized in that, include: Acquire RGB-D images of the workpiece to be inspected and preprocess the depth map; The captured RGB image is segmented using a segmentation network to obtain the region containing the part to be detected, which is used as an RGB part mask image. The RGB part mask image is mapped onto the preprocessed depth map to obtain the depth map mask of the part. The depth map mask of the part is then converted into a 3D point cloud of the part through a coordinate transformation matrix. The 3D point cloud of the part is input into the part point cloud completion network model for completion, and the complete point cloud of the part is obtained. The part point cloud completion network model is trained using the method described in any one of claims 1 to 7.

9. A point cloud completion device for parts, characterized in that, Includes memory and one or more processors; The memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions; The one or more processors invoke the computer instructions to cause the system to perform the completion method as described in claim 8.

10. A computer-readable storage medium, characterized in that, The instruction includes instructions that, when executed on an electronic device, cause the electronic device to perform the training method as described in any one of claims 1 to 7, or the completion method as described in claim 8.

Images