Defect detection model construction method for metal 3D printing and defect detection method

Abstract

The invention discloses a defect detection model construction method for metal 3D printing. The defect detection model construction method comprises the steps that original point cloud data related to the surface of a powder bed before powder laying and after powder laying are obtained; preprocessing the original point cloud data to obtain effective preprocessed point cloud data; defect labeling including powder bed powder spreading defects and / or printing defects is carried out on the preprocessed point cloud data, and a defect detection model point cloud data training set is obtained; and training the PointNet network by using the defect detection model point cloud data training set, and constructing a defect detection model capable of detecting the powder spreading defect and / or printing defect of the powder bed. According to the method, a three-dimensional detection technology based on point cloud data and a deep learning method are combined and applied to the specific technical field of powder spreading defect detection and / or printing defect detection for metal 3D printing, real-time automatic detection of multiple defect types in the powder spreading and printing process of a powder bed is achieved, delay of manual detection is reduced, and the detection efficiency is improved. And the accuracy of defect type detection is improved.

Description

Technical Field

[0001] The present application relates to the field of additive manufacturing technology, and more specifically, to a defect detection model construction method and a defect detection method for metal 3D printing. Background Art

[0002] In the process of metal additive manufacturing, the quality of powder bed spreading and printing directly affects the performance and precision of the final product. Existing detection methods mainly rely on visual inspection and offline quality control methods, which are often affected by factors such as lighting, angle and occlusion, and have limited accuracy. They cannot achieve real-time and efficient quality monitoring, especially for defects generated during the powder bed spreading process, such as insufficient spreading, abnormal scraper, powder loss and powder surface collapse, and lack effective automatic identification methods. At the same time, during the printing process, defects such as warping and deformation that may occur are difficult to detect and predict by conventional methods. Summary of the invention

[0003] In response to at least one defect or improvement need in the prior art, the present application provides a defect detection model construction method and a defect detection method for metal 3D printing, which are used to realize real-time automatic detection of various defect types during powder bed laying and 3D printing, reduce the delay of manual detection, and improve the accuracy of defect type detection.

[0004] To achieve the above objectives, in a first aspect, the present application provides a method for constructing a defect detection model for metal 3D printing, comprising:

[0005] Obtaining original point cloud data of the powder bed surface before and after powder spreading;

[0006] Preprocessing the original point cloud data to obtain effective preprocessed point cloud data;

[0007] Perform defect annotation on the preprocessed point cloud data, including powder bed powdering defects and / or printing defects, to obtain a defect detection model point cloud data training set;

[0008] The PointNet network is trained using the defect detection model point cloud data training set to construct a defect detection model capable of detecting powder bed defects and / or printing defects.

[0009] Further, the preprocessing includes: through filtering and / or radius filtering;

[0010] The through filtering includes: filtering out valid point clouds within the powder spreading target area based on a preset spatial position range filtering condition, and removing point clouds outside the powder spreading target area;

[0011] The radius filtering includes: analyzing the distance relationship between each point in the original point cloud and its neighboring points based on a preset neighborhood radius, and filtering out isolated points whose distance parameters calculated from the distance to the neighboring points exceed a preset distance threshold.

[0012] Further, the preprocessing includes: obtaining the curvature of each point in the original point cloud data to obtain local change information of the powder bed surface;

[0013] Ways to obtain the curvature of a point include:

[0014] Find point P in the original point cloud within the set radius i (x i ,y i ,z i )’s nearest neighbor point set Q (i,j) (x (i,j) ,y (i,j) ,z (i,j) ),(j=1,2,…,i k ); i represents the number of the point, (i, j) represents the number of the nearest neighbor of the point numbered i is j;

[0015] Get point P i (x i ,y i ,z i ) and the nearest neighbor point set Q (i,j) (x (i,j) ,y (i,j) ,z (i,j) ),(j=1,2, … ,i k )’s covariance matrix M;

[0016] Perform singular value decomposition on the covariance matrix M to obtain its eigenvalues The eigenvector corresponding to its minimum eigenvalue is V = (v0, v1, v2) T ; where v n represents the value of dimension numbered n in the feature vector, λ n represents the value of the dimension numbered n in the eigenvalue, λ0 represents the minimum eigenvalue, and then the point P is obtained i (x i ,y i ,z i ) is the curvature value cure(i) = λ0 / ∑λ k ,∑λ k represents the sum of all eigenvalues.

[0017] Furthermore, the methods for training the PointNet network include:

[0018] Use T-Net to uniformly map the point cloud data training set of the defect detection model to a standardized coordinate space through rotation, scaling, or translation operations to eliminate the geometric differences between different point cloud data;

[0019] The input point cloud data is aligned and normalized through the input transformation network to ensure that the point cloud remains geometrically consistent under different observation perspectives;

[0020] Dynamically adjust and align intermediate features through feature transformation networks to optimize the distribution of multi-dimensional features in feature space;

[0021] A multi-layer perceptron is used to extract the local features of each point, and then the global features are aggregated through a maximum pooling operation to construct an overall feature representation of the printing process;

[0022] For classification tasks, the aggregated global features are sent to the fully connected layer to classify different defect types. For segmentation tasks, the global features are concatenated with the local features, and a multi-layer perceptron is used to perform point-by-point classification to achieve the positioning and labeling of complex defect areas.

[0023] Furthermore, the powder bed powder spreading defects include one or more of insufficient spreading, abnormal scraper, powder falling and powder surface collapse;

[0024] The printing defects include warping deformation.

[0025] Furthermore, the original point cloud data includes spatial position information and height information of the powder bed surface before and after powder spreading.

[0026] In a second aspect, the present application provides a defect detection method for metal 3D printing, comprising: based on a defect detection model constructed according to any of the aforementioned items, performing detection of powder bed defects and / or printing defects, and obtaining defect detection results.

[0027] Furthermore, it specifically includes:

[0028] Obtain raw point cloud data about the powder bed surface;

[0029] Performing through-filter preprocessing on the original point cloud data to obtain effective preprocessed point cloud data;

[0030] The preprocessed point cloud data is input into the defect detection model constructed by any of the above items to detect powder bed spreading defects and / or printing defects, and automatically identify and classify one or more types of defects including insufficient spreading, scraper abnormality, powder falling, powder surface collapse and warping deformation.

[0031] In a third aspect, the present application provides an electronic device comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit is enabled to perform the steps of the defect detection model building method described in any one of the preceding items and / or to perform the steps of the defect detection method described in any one of the preceding items.

[0032] In a fourth aspect, the present application provides a storage medium storing a computer program executable by an access authentication device. When the computer program runs on the access authentication device, the access authentication device is enabled to execute the steps of the defect detection model construction method described in any one of the preceding items and / or to execute the steps of the defect detection method described in any one of the preceding items.

[0033] In general, the above technical solutions conceived by the present application can achieve the following beneficial effects compared with the prior art:

[0034] (1) This application combines point cloud data-based three-dimensional detection technology with deep learning methods and applies them to the specific technical field of powder bed defect detection and / or printing defect detection for metal 3D printing, thereby realizing real-time automated detection of multiple defect types during powder bed spreading and printing, reducing the delay of manual detection and improving the accuracy of defect type detection.

[0035] (2) This application combines three-dimensional imaging technology with the PointNet network to achieve high-precision, automated detection of powder spreading and printing defects, effectively avoiding the information loss problem in traditional visual methods. It can also identify complex defects such as warping and deformation, greatly improving the accuracy and robustness of detection. The trained PointNet network can mainly detect five defects: powder falling, powder surface collapse, scraper abnormality, insufficient spreading and warping. The detection accuracy of warping can reach 100%.

[0036] (3) This application uses a curvature calculation preprocessing method when preprocessing the original point cloud data. By calculating the curvature of each point in the point cloud data, local surface change information can be obtained. This feature is very helpful for subsequent defect identification, especially defects generated during the powder spreading process, such as uneven powder surface, powder falling, etc., which usually show abnormalities in the curvature value. For the workpiece surface before and after printing, the curvature calculation is used to identify abnormal curvature values ​​in local areas. For example, small protrusions or depressions can be identified as warping defect points.

[0037] (4) This application also has the ability to classify and analyze defects. It can provide detailed classification results based on the detected defect types and assist in analyzing the causes of defects, providing a basis for process optimization. At the same time, it also supports the real-time construction of a three-dimensional visualization model of the printing process to achieve accurate monitoring of the entire printing process, allowing operators to detect abnormalities in a timely manner and make quick adjustments. By accumulating large-scale production data, it is also possible to continuously optimize deep learning models, improve overall performance, and further promote efficient and high-precision production of metal 3D printing. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A core flow chart of a method for constructing a defect detection model for metal 3D printing provided in an embodiment of the present application;

[0040] Figure 2 A core flow chart of a defect detection method for metal 3D printing provided in an embodiment of the present application;

[0041] Figure 3 The original point cloud image of the powder bed for metal 3D printing provided in the embodiment of the present application;

[0042] Figure 4 A point cloud map of the target area extracted after through filtering the original point cloud of the powder bed provided in the embodiment of the present application;

[0043] Figure 5 A comparison diagram of the powder loss defect point cloud extracted after curvature calculation and radius filtering of the target area point cloud provided in the embodiment of the present application; wherein, Figure 5 (a) shows the point cloud after straight-through filtering; Figure 5 (b) shows the point cloud of powder loss defects after straight-through filtering, curvature calculation and radius filtering;

[0044] Figure 6 A comparison diagram of the powder surface collapse defect point cloud extracted after curvature calculation and radius filtering of the target area point cloud provided in the embodiment of the present application; wherein, Figure 6 (a) shows the point cloud after straight-through filtering; Figure 6 (b) shows the point cloud image of powder surface collapse defect after straight-through filtering, curvature calculation and radius filtering;

[0045] Figure 7A comparison diagram of the scraper abnormal defect point cloud extracted after curvature calculation and radius filtering of the target area point cloud provided in the embodiment of the present application; wherein, Figure 7 (a) shows the point cloud after straight-through filtering; Figure 7 (b) shows the point cloud of abnormal defects of the scraper after straight-through filtering, curvature calculation and radius filtering;

[0046] Figure 8 A comparison diagram of the curvature calculation of the target area point cloud and the under-paving defect point cloud extracted after radius filtering provided in the embodiment of the present application; wherein, Figure 8 (a) shows the point cloud after straight-through filtering; Figure 8 (b) shows the point cloud of under-paving defects after straight-through filtering, curvature calculation and radius filtering;

[0047] Fig. 9 A comparison diagram of the curvature calculation of the target area point cloud and the warping deformation defect point cloud extracted after radius filtering provided in the embodiment of the present application; wherein, Fig. 9 (a) shows the point cloud after straight-through filtering; Fig. 9 (b) shows the warping deformation defect point cloud after straight-through filtering, curvature calculation and radius filtering;

[0048] Fig.10 A schematic diagram of the defect detection output results provided in an embodiment of the present application;

[0049] Fig.11 A block diagram of an electronic device suitable for implementing the defect detection model building method and / or defect detection method described above provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0051] The terms "including" or "having" and any variations thereof in the specification, claims or drawings of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0052] As described in the background technology section of the specification, existing detection methods mainly rely on visual inspection and offline quality control methods, which are often affected by factors such as lighting, angle and occlusion, and have limited accuracy. They cannot achieve real-time and efficient quality monitoring, especially for defects generated during the powder bed spreading process, such as insufficient spreading, abnormal scraper, powder falling and powder surface collapse. There is a lack of effective automated identification methods. At the same time, during the printing and molding process, defects such as warping and deformation that may occur are difficult to detect and predict by conventional methods. In view of this, the present application provides a defect detection model construction method and a defect detection method for metal 3D printing, which are used to realize real-time automated detection of multiple defect types in the powder bed spreading and 3D printing process, reduce the delay of manual detection, and improve the accuracy of defect type detection.

[0053] refer to Figure 1 An embodiment of the present application provides a method for constructing a defect detection model for metal 3D printing, which mainly includes the following steps.

[0054] Step 101: Obtain original point cloud data of the powder bed surface before and after powder spreading.

[0055] In some embodiments, specifically, point cloud data is collected by a three-dimensional imaging device before and after powder spreading, and the point cloud data includes spatial position information and height information of the powder bed surface before and after powder spreading.

[0056] The collection method of raw point cloud data includes the following specific steps:

[0057] (1) Equipment calibration: Before collecting point cloud data, the 3D imaging equipment is accurately calibrated to ensure that the data collected by the equipment at different positions and angles are consistent and accurate.

[0058] (2) Resolution setting: According to the size and accuracy requirements of the powder bed surface, the sampling resolution of the point cloud data is set to meet the detection requirements of tiny defects (such as those less than 0.1 mm in size).

[0059] (3) Comparison before and after powdering: Data is collected before and after powdering, and the spatial position information and height information of the two sets of point cloud data are compared to accurately identify defects that may occur during the powdering process.

[0060] As a specific embodiment, the device can use PowerScan 3D imaging equipment, whose high-precision sensor can capture point cloud data with an accuracy of 0.1mm, and its image data resolution reaches 2 million pixels, ensuring high-sensitivity detection of powder bed surface defects. At the same time, the device supports high-frequency acquisition, which can meet the real-time monitoring needs during 3D printing and provide accurate input for subsequent point cloud data processing and defect detection.

[0061] Step 102: pre-process the original point cloud data to obtain effective pre-processed point cloud data. In some embodiments, the pre-processing may include straight-through filtering, curvature calculation, radius filtering, and the like.

[0062] (1) Through filtering: Through filtering is the first crucial step in point cloud data preprocessing, which aims to effectively extract the point cloud in the target area. The purpose of through filtering is to limit the range of the point cloud based on the spatial position and retain only the valid point cloud related to the powder spreading. By setting a specific range filtering condition, the valid point cloud within the powder spreading target area is screened out and the points outside the area are removed. The point cloud data after through filtering is concentrated in the powder spreading area, laying a reliable data foundation for subsequent processing.

[0063] In terms of specific implementation, the system uses a straight-through filtering algorithm to set the filtering range according to the size of the powder bed. By removing noise points with a height exceeding ±0.006mm, the effective point cloud data within the height range of the powder bed surface is retained. The collected point cloud data includes points of the entire powder bed and its surrounding irrelevant areas, such as Figure 3 As shown in the figure, according to the size of the printing base (100mm*100mm), the spatial range limit is set to (150mm*150mm) to ensure that all point cloud data of the print part are included. Figure 4 As shown in the figure, the point cloud data only retains the base area, providing a reliable data basis for subsequent processing.

[0064] (2) Curvature calculation: Curvature describes the local curvature of a surface at different points and is an important feature that reflects the geometric changes of the powder coating surface and the workpiece surface. The calculation of curvature can help identify irregularities on the surface, such as surface protrusions or depressions caused by defects.

[0065] Compared with the normal direction or surface area, curvature can directly distinguish different types of defect morphology. For example, high curvature values ​​usually correspond to sharp protrusions or depressions, while low curvature values ​​may correspond to smoother transition areas. This feature enables curvature to better assist deep learning models in defect classification. In addition, the calculation of curvature is highly robust to surface scaling, translation, or rotation, and can maintain consistent detection results under different acquisition devices or different printing conditions, while the normal direction may be inaccurate due to local acquisition errors, and the surface area is also easily affected by the sampling resolution.

[0066] By calculating the curvature of each point in the point cloud data, local surface change information can be obtained. This feature is very helpful for subsequent defect identification, especially for defects generated during the powder spreading process, such as uneven powder surface, powder falling, etc., which usually show abnormalities in the curvature value. For the workpiece surface before and after powder spreading, the curvature calculation is used to identify abnormal curvature values ​​in local areas. For example, small protrusions or depressions can be identified as warping defect points. The curvature calculation is based on the local neighborhood of the point cloud to extract local surface change information for preliminary defect detection.

[0067] For the collected point cloud data containing n points, point P i (x i ,y i ,z i ),(i=1,2, … ,n) is calculated as follows: First, find point P within the set radius. i (x i ,y i ,z i )’s nearest neighbor point set Q (i,j) (x (i,j) ,y (i,j) ,z (i,j) ),(j=1,2, … ,i k );

[0068] Then, calculate point P i (x i ,y i ,z i ) and the nearest neighbor point set Q (i,j) (x (i,j) ,y (i,j) ,z (i,j) ),(j=1,2, … ,i k )’s covariance matrix M;

[0069] Then perform singular value decomposition on the covariance matrix M and calculate its eigenvalues The eigenvector corresponding to its minimum eigenvalue is V = (v0, v1, v2) T ; where v n represents the value of dimension numbered n in the feature vector, λ n represents the value of the dimension numbered n in the eigenvalue, and λ0 represents the minimum eigenvalue;

[0070] Finally, point P can be calculated i (x i ,y i ,z i ) has a curvature value of cure(i) = λ0 / Σλ k,∑λ k represents the sum of all eigenvalues.

[0071] The point cloud data after straight-through filtering still cannot directly reflect the location of the defect. This embodiment uses curvature calculation to identify small geometric changes on the surface. As a specific embodiment, the nearest neighbor point set is found within a radius of 1 mm, and the curvature value of each point is calculated. Figure 5-Figure 9 , the curvature changes of defects and warping areas can be intuitively seen.

[0072] (3) Radius filtering: Radius filtering is a denoising algorithm based on neighborhood search. Its main purpose is to analyze the distance relationship between each point in the point cloud and its surrounding points by setting a reasonable neighborhood radius, and filter out isolated points that are too far away from neighboring points, that is, filter out isolated points whose distance parameters calculated by a certain distance calculation method from neighboring points exceed the preset distance threshold. These outliers may be introduced due to equipment noise or mis-collection, and they cannot reflect the actual state of the powder-laying surface. If they are not removed, the subsequent defect recognition accuracy may be affected. Through radius filtering, point cloud noise in non-target areas can be further eliminated, which can further ensure the smoothness and stability of point cloud data and ensure that the data is more consistent with the actual powder bed surface.

[0073] Step 103: perform defect annotation on the preprocessed point cloud data, including powder bed powdering defects and / or printing defects, to obtain a defect detection model point cloud data training set.

[0074] In some embodiments, the pre-processed point cloud needs to be manually annotated and input into the deep learning model as training data. More specifically, the pre-processed point cloud data is manually annotated to mark the defect types such as insufficient paving, scraper abnormality, powder loss, powder surface collapse and warping deformation corresponding to the defect point cloud, and obtain the defect detection model point cloud data training set.

[0075] Step 104: Use the defect detection model point cloud data training set to train the PointNet network to build a defect detection model that can detect powder bed defects and / or printing defects.

[0076] In some embodiments, specifically, during the network training process, the model is optimized using a cross entropy loss function, and weight parameters are adjusted through back propagation. A pre-trained model can be used to accelerate convergence, and targeted training data can be provided according to different defect types. In addition, according to the usage scenario, the semantic segmentation network of PointNet is used to train the point cloud data.

[0077] PointNet is a deep learning model designed for point cloud data processing, proposed by Charles R.Qi et al. in 2017. It solves the problems of disorder and irregularity in point cloud data and performs well in tasks such as classification, segmentation, and target detection. PointNet can directly process three-dimensional point cloud data and efficiently extract features, effectively avoiding the problem of information loss in traditional visual methods. It can also identify complex defects such as warping and deformation, greatly improving the accuracy and robustness of detection. Combining three-dimensional detection technology based on point cloud data with the PointNet network is expected to achieve high-precision and automated detection of powder laying and printing defects.

[0078] PointNet receives n points of 3D point cloud data as input, each point is represented in the form of (x, y, z) coordinates, which can also be extended to (x, y, z, r, g, b) coordinates with additional attributes (such as color, normal). The order of the points is irrelevant, which reflects the disordered nature of the point cloud data. The features of all points are aggregated using a symmetric function (such as global maximum pooling) to generate a global feature vector of the point cloud. This operation ensures that the translation, rotation and disorder of the point cloud do not affect the feature representation.

[0079] In some embodiments, more specifically, point cloud data is disordered and rotationally invariant, that is, the order of the points in the point cloud or the spatial rotation does not change the properties of the object it represents. Therefore, in the deep learning process, a mechanism needs to be introduced to standardize the point cloud so that it is aligned with the input requirements of the model. To this end, PointNet introduces T-Net to learn the rotation and transformation matrices of the point cloud in response to the possible translation and rotation changes in the point cloud data.

[0080] The role of T-Net is to generate a transformation matrix that can eliminate the geometric differences between different point cloud data by rotating, scaling or translating the input point cloud data into a unified coordinate space. The point cloud corrected by T-Net can better adapt to the subsequent deep learning feature extraction module, significantly improving the robustness and accuracy of the model for different point cloud inputs. Simply put, T-Net is an automatic "corrector" that ensures that even if the arrangement or angle of the point cloud data changes, the subsequent network can still correctly extract key features.

[0081] After the n×3 point cloud is input into the n multi-layer perceptrons that process the point cloud and features, each point has a corresponding two-layer perceptron processing for dimensionality increase. Each input point is described in 1024 dimensions. Through maximum pooling, the n×3 is fused into a 1024-dimensional feature description. Max pooling is used to ensure that the extracted global variable points with a length of 1024 do not change with the order of the points, solving the problem of disorder of the point cloud.

[0082] f(x1, x2, ..., x n )≈g(h(x1),…h(x n ));

[0083] Here, h(x) is the MLP dimension-raising operation, g(x) is the pooling feature extraction operation, and f(x1,x2,…,x n ) is a global feature extraction function for a set of input data. MLP, Multilayer Perceptron, is a feedforward artificial neural network model, which is mainly used to map multiple input data sets to a single output data set. MLP consists of multiple neurons in a hierarchical structure, and each neuron is connected to all neurons in the previous layer to form a fully connected network structure.

[0084] When PointNet processes large-scale point clouds (such as scenes containing millions of points), it is difficult to apply it directly because global feature extraction will lose some detailed information and requires high computing resources. To this end, the large-scale point cloud is divided into multiple sub-regions (such as using voxel segmentation or grid segmentation), and PointNet is applied separately in each sub-region to extract local features, and then the local features are spliced ​​or further processed. A multi-scale sampling and aggregation mechanism is introduced to gradually extract local and global hierarchical features from large-scale point clouds. For large-scale sparse point clouds, sparse convolution (SparseConv) or point cloud compression methods can be used to retain only the features of key points or sparse areas for calculation, reducing time and memory consumption.

[0085] The training process of the PointNet network is mainly that the input layer receives an n×3 matrix, where n is the number of points in the point cloud and 3 represents the number of dimensions of the three-dimensional coordinates (x, y, z) of each point. PointNet introduces the spatial transformation network STN (such as T-Net), the input transformation network (Input Transform Net) and the feature transformation network (Feature Transform Net). Among them, the input transformation network is mainly used to align and normalize the input point cloud to ensure the consistency of the point cloud under different perspectives; the feature transformation network is mainly used to align the intermediate features and adjust the feature distribution in the feature space. Subsequently, MLP is used to extract the local features of each point, and then the global features are aggregated through the maximum pooling operation. For classification tasks, the global features are sent to the fully connected layer for classification. For segmentation tasks, the global features are spliced ​​with the local features, and MLP is used for point-by-point classification.

[0086] More specifically, the methods for training the PointNet network include:

[0087] Data preprocessing and coordinate alignment: Use T-Net to uniformly map the point cloud data training set of the defect detection model to a standardized coordinate space through rotation, scaling, or translation operations, thereby eliminating the geometric differences between different point cloud data and enhancing the generalization ability of the model.

[0088] Input transformation and normalization: The input point cloud data is accurately aligned and normalized through the input transformation network to ensure that the point cloud maintains geometric consistency under different observation perspectives and avoid the interference of perspective differences on the detection results.

[0089] Feature transformation and alignment: Dynamically adjust and align intermediate features through the feature transformation network to optimize the distribution of multi-dimensional features in the feature space, thereby improving the model's ability to capture complex defect features.

[0090] Local feature extraction and global feature aggregation: Use a multi-layer perceptron to extract the local features of each point and make full use of the detailed information of the point cloud; then aggregate the global features through the maximum pooling operation to construct an overall feature representation of the printing process.

[0091] Task branch processing: For classification tasks, the aggregated global features are sent to the fully connected layer for accurate classification of different defect types; for segmentation tasks, the global features are concatenated with the local features, and a multi-layer perceptron is used for point-by-point classification to achieve accurate positioning and labeling of complex defect areas.

[0092] Through the above training steps, this method can efficiently learn the spatial distribution and characteristic differences of various defects in the powder laying and printing process, significantly improve the detection accuracy of classification and segmentation tasks, and meet the real-time monitoring and quality control needs of the metal 3D printing process.

[0093] The PointNet network is trained using the produced point cloud defect dataset (defect detection model point cloud data training set), and the training accuracy can reach 96.25%, and the inter-class accuracy is 89.10%. The trained PointNet network can mainly detect five defects: powder loss, powder surface collapse, scraper abnormality, under-paving and warping deformation, among which the detection accuracy of warping deformation can reach 100%.

[0094] The innovative advantages of combining point cloud data processing with PointNet network in this invention are:

[0095] 1) Ensure the information integrity of the point cloud: avoid the information loss caused by the conversion of point cloud to voxel or image format in traditional methods, and retain the geometric and spatial characteristics of 3D data.

[0096] 2) Detection accuracy: PointNet's unique spatial transformation module and feature extraction capabilities can effectively handle complex geometric shapes and adapt to various defect modes such as powder loss, powder surface collapse, scraper abnormality, insufficient paving, and warping deformation.

[0097] 3) Real-time: Compared with the existing powder spreading defect recognition technology, this method significantly improves the detection speed and accuracy, and the detection speed reaches 0.1s.

[0098] 4) Versatility: It can detect all these defect types at once, while traditional methods can usually only detect a single defect, or require different detection technologies to implement them separately. It is suitable for more types of 3D printing processes and equipment, and is more suitable for on-site and industrialization.

[0099] refer to Figure 2 Another embodiment of the present application also provides a defect detection method for metal 3D printing (offline is the process of model training, and online is the process of model application), which may include the following steps.

[0100] Step 201: Collect point cloud data about the surface of the powder bed using a 3D sensor. Step 201 is similar to step 101, and can refer to step 101.

[0101] Step 202 : extracting the point cloud of the target detection area using the straight-through filtering preprocessing method described in step 102 .

[0102] Step 203: Input the preprocessed and feature-extracted point cloud data into the PointNet defect detection model trained in step 104. Through the previous training process, the PointNet defect detection model can determine whether there are defects in the current powder spreading or printing process, and can automatically identify and classify different types of defects. Fig.10 . The 3D printing equipment can adjust the printing parameters or powder spreading equipment in time according to the test results to prevent defects from further affecting the quality of the finished product.

[0103] This application combines point cloud data-based three-dimensional detection technology with deep learning methods and applies them to the specific technical field of powder bed defect detection and / or printing defect detection for metal 3D printing, realizing real-time automatic detection of various defect types during powder bed spreading and printing, reducing the delay of manual detection, and improving the accuracy and speed of defect type detection (the detection speed can reach 0.1s).

[0104] This application is based on the curvature calculation and filtering of point cloud data, combined with the three-dimensional feature extraction capability of the PointNet network, and can accurately detect a variety of defect types, including insufficient paving, scraper anomalies, powder loss, powder surface collapse, and warping deformation. PointNet's unique spatial transformation module and feature extraction capabilities can effectively handle complex geometric shapes and adapt to a variety of defect modes such as powder loss, powder surface collapse, scraper anomalies, insufficient paving, and warping deformation.

[0105] This application can detect all these defect types (powder loss, powder surface collapse, scraper abnormality, insufficient paving and warping deformation) at one time, while traditional methods can usually only detect a single defect, or require different detection technologies to implement them separately. Therefore, this application is applicable to more types of 3D printing processes and equipment, and is more suitable for on-site and industrialization. This application is applicable to a variety of additive manufacturing equipment and processes, not limited to specific printing materials or process parameters, and has broad industrial application prospects.

[0106] This application also has the ability to classify and analyze defects. It can provide detailed classification results based on the detected defect types and assist in analyzing the causes of defects, providing a basis for process optimization. At the same time, it also supports the real-time construction of a three-dimensional visualization model of the printing process to achieve accurate monitoring of the entire printing process, allowing operators to detect abnormalities in a timely manner and make quick adjustments. Through the accumulation of large-scale production data, it is also possible to continuously optimize deep learning models, improve overall performance, and further assist in the efficient and high-precision production of metal 3D printing.

[0107] Fig.11 A block diagram of an electronic device suitable for implementing the defect detection model building method and / or defect detection method described above according to an embodiment of the present application is schematically shown. Fig.11 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0108] like Fig.11 As shown, the electronic device 1000 described in this embodiment includes: a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 to a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), and the like. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for executing different actions of the defect detection model construction method and / or defect detection method process according to an embodiment of the present application.

[0109] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other through bus 1004. Processor 1001 performs various operations of the defect detection model construction method and / or defect detection method process according to the embodiment of the present application by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the defect detection model construction method and / or defect detection method process according to the embodiment of the present application by executing the program stored in the one or more memories.

[0110] According to an embodiment of the present application, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the I / O interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0111] According to the defect detection model construction method and / or defect detection method process of the embodiment of the present application, it can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the defect detection model construction method and / or defect detection method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present application are executed. According to the embodiment of the present application, the systems, devices, means, modules and / or units described above can be implemented by computer program modules.

[0112] The embodiments of the present application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the steps of the defect detection model construction method and / or defect detection method according to the embodiments of the present application can be implemented.

[0113] According to an embodiment of the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present application, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus, or a device. For example, according to an embodiment of the present application, the computer-readable storage medium may include one or more memories other than the ROM 1002 and / or RAM 1003 described above.

[0114] It should be noted that the functional modules in the various embodiments of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product.

[0115] The flowchart and / or block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart and / or block diagram can represent a part of a module, program segment or code, and a part of the above-mentioned module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0116] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the technical features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, and all of these combinations and / or combinations fall within the scope of the present application.

[0117] Although the present application has been shown and described with reference to specific exemplary embodiments of the present application, it should be understood by those skilled in the art that various changes in form and details may be made to the present application without departing from the spirit and scope of the present application as defined by the appended claims and their equivalents. Therefore, the scope of the present application should not be limited to the above-mentioned embodiments, but should be determined not only by the appended claims, but also by the equivalents of the appended claims.

Claims

1. A method for constructing a defect detection model for metal 3D printing, characterized in that: include: Obtaining original point cloud data of the powder bed surface before and after powder spreading; Preprocessing the original point cloud data to obtain preprocessed point cloud data; Perform defect annotation on the preprocessed point cloud data, including powder bed powdering defects and / or printing defects, to obtain a defect detection model point cloud data training set; The PointNet network is trained using the defect detection model point cloud data training set to construct a defect detection model capable of detecting powder bed defects and / or printing defects.

2. The defect detection model construction method according to claim 1, characterized in that: The preprocessing includes: through filtering and / or radius filtering; The through filtering includes: filtering out valid point clouds within the powder spreading target area based on a preset spatial position range filtering condition, and removing point clouds outside the powder spreading target area; The radius filtering includes: analyzing the distance relationship between each point in the original point cloud and its neighboring points based on a preset neighborhood radius, and filtering out isolated points whose distance parameters calculated from the distance to the neighboring points exceed a preset distance threshold.

3. The defect detection model construction method according to claim 1 or 2, characterized in that: The preprocessing includes: obtaining the curvature of each point in the point cloud data to obtain local change information of the powder bed surface; Ways to obtain the curvature of a point include: Find point P in the original point cloud within the set radius i (x i ,y i ,z i )’s nearest neighbor point set Q (i,j) (x (i,j) ,y (i,j) ,z (i,j) ),(j=1,2,…,i k ); i represents the number of the point, (i, j) represents the number of the nearest neighbor of the point numbered i is j; Get point P i (x i ,y i ,z i ) and the nearest neighbor point set Q (i,j) (x (i,j) ,y (i,j) ,z (i,j) ),(j=1,2,…,i k )’s covariance matrix M; Perform singular value decomposition on the covariance matrix M to obtain its eigenvalues The eigenvector corresponding to its minimum eigenvalue is V = (v0, v1, v2) T ; where v n represents the value of dimension numbered n in the feature vector, λ n represents the value of the dimension numbered n in the eigenvalue, and λ0 represents the minimum eigenvalue; Get point P i (x i ,y i ,z i ) is the curvature value cure(i) = λ0 / ∑λ k ,∑λ k represents the sum of all eigenvalues.

4. The defect detection model construction method according to claim 3, characterized in that: The methods for training the PointNet network include: Use T-Net to uniformly map the point cloud data training set of the defect detection model to a standardized coordinate space through rotation, scaling, or translation operations to eliminate the geometric differences between different point cloud data; The input point cloud data is aligned and normalized through the input transformation network to ensure that the point cloud remains geometrically consistent under different observation perspectives; Dynamically adjust and align intermediate features through feature transformation networks to optimize the distribution of multi-dimensional features in feature space; A multi-layer perceptron is used to extract the local features of each point, and then the global features are aggregated through a maximum pooling operation to construct an overall feature representation of the printing process; For classification tasks, the aggregated global features are sent to the fully connected layer to classify different defect types. For segmentation tasks, the global features are concatenated with the local features, and a multi-layer perceptron is used to perform point-by-point classification to achieve the positioning and labeling of complex defect areas.

5. The defect detection model construction method according to claim 1, characterized in that: The powder bed powder laying defects include one or more of insufficient laying, abnormal scraper, powder falling and powder surface collapse; The printing defects include warping deformation.

6. The defect detection model construction method according to claim 1, characterized in that: The original point cloud data includes spatial position information and height information of the powder bed surface before and after powder spreading.

7. A defect detection method for metal 3D printing, characterized in that: include: Based on the defect detection model constructed according to any one of claims 1 to 6, powder bed defects and / or printing defects are detected to obtain defect detection results.

8. The defect detection method according to claim 7, characterized in that: Specifically include: Obtain raw point cloud data about the powder bed surface; Performing through-filter preprocessing on the original point cloud data to obtain preprocessed point cloud data; The preprocessed point cloud data is input into the defect detection model constructed according to any one of claims 1 to 6 to detect powder bed spreading defects and / or printing defects, and automatically identify and classify one or more types of defects including insufficient spreading, scraper abnormality, powder falling, powder surface collapse and warping deformation.

9. An electronic device, characterized in that: It includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit is enabled to execute the steps of the defect detection model construction method described in any one of claims 1 to 6 and / or to execute the steps of the defect detection method described in any one of claims 7 to 8.

10. A storage medium, characterized in that: It stores a computer program that can be executed by an access authentication device. When the computer program is run on the access authentication device, the access authentication device can execute the steps of the defect detection model construction method described in any one of claims 1-6 and / or can execute the steps of the defect detection method described in any one of claims 7-8.

Images