Workpiece flaw detection method and system and computer program

Abstract

The invention discloses a workpiece flaw detection method which comprises the following steps: acquiring point cloud data of a to-be-detected workpiece, and constructing a sample workpiece library at least comprising flaw sample workpieces according to the point cloud data of the to-be-detected workpiece; constructing and training a YOLO neural network model based on the sample workpiece library to obtain a flaw detection model; performing flaw detection on the to-be-detected workpiece by using the flaw detection model to obtain a flaw mark; according to the defect mark of the workpiece to be detected, PointNet + + feature extraction, registration processing and difference detection are carried out on the point cloud data of the workpiece to be detected, and a defect positioning result of the workpiece to be detected is obtained. According to the technical scheme provided by the invention, the workpiece is detected by constructing the YOLO neural network model, and the workpiece flaws are roughly positioned; then, a PointNet + + model, a point cloud registration algorithm and difference detection are utilized to carry out further workpiece flaw accurate positioning on the three-dimensional point cloud data of the workpiece; the workpiece is detected twice, so that tiny flaws of the workpiece can be accurately detected.

Description

Technical Field

[0001] The present invention relates to the field of defect detection, and in particular to a workpiece defect detection method. Background Art

[0002] In the industrial manufacturing process, workpiece quality control is crucial, and the presence of workpiece defects will seriously affect the quality and performance of the product, and may even cause safety accidents. Therefore, defect detection of workpieces is an important means to ensure product quality, improve production efficiency and reduce production costs. Especially in the field of aircraft engines, engine blades are one of the core components of the engine. Their quality and performance directly affect the overall operating efficiency and safety of the engine. Therefore, it is necessary to detect tiny defects in engine blades.

[0003] Traditional visual inspection methods are difficult to meet the current requirements of high precision and high efficiency in workpiece defect detection, while deep learning-based defect detection methods have gradually become a research hotspot due to their high detection accuracy and degree of automation. At present, the detection method of two-dimensional images has achieved certain results, but how to accurately detect and analyze the location of defects in three-dimensional point cloud data is still a challenge. Summary of the invention

[0004] The present invention provides a workpiece defect detection method, which can realize the defect detection of the workpiece by constructing a workpiece defect detection model by using a neural network model.

[0005] To achieve the above object, the embodiments of the present invention adopt the following technical solutions:

[0006] A method for detecting workpiece defects comprises the following steps:

[0007] Acquire point cloud data of the workpiece to be measured, and construct a sample workpiece library including at least defective sample workpieces according to the point cloud data of the workpiece to be measured;

[0008] Build and train the YOLO neural network model based on the sample artifact library to obtain a defect detection model;

[0009] Perform defect detection on the workpiece to be tested using the defect detection model to obtain defect marks;

[0010] According to the defect mark of the workpiece to be measured, the point cloud data of the workpiece to be measured is aligned and processed to obtain the defect positioning result of the workpiece to be measured.

[0011] According to one aspect of the present invention, the step of constructing a sample workpiece library including at least defective sample workpieces according to the point cloud data of the workpiece to be tested comprises:

[0012] Generate a model image of the workpiece to be tested according to the point cloud data of the workpiece to be tested;

[0013] Defect annotation is performed on the model image of the defective sample workpiece.

[0014] According to one aspect of the present invention, generating a model image of the workpiece to be measured based on the point cloud data of the workpiece to be measured includes:

[0015] Generate a 3D model of the workpiece based on point cloud data;

[0016] Add realistic textures and materials to the workpiece 3D model;

[0017] Generate 2D images with realistic lighting effects through rendering;

[0018] Combine different image elements, including artifacts, to generate new images, and add artificial noise to the images to simulate real-world interference;

[0019] Add real-world physics to images and simulate artifacts based on physical textures and material properties;

[0020] Set up a virtual environment and adjust environment parameters to simulate different real-world conditions.

[0021] According to one aspect of the present invention, when the model image size of the workpiece to be tested exceeds the input size of the defect detection model, the image is cut into a size suitable for the model before input, and the output defect mark is restored to its position on the model image.

[0022] According to one aspect of the present invention, the step of performing registration processing on the point cloud data of the workpiece to be measured according to the defect mark of the workpiece to be measured to obtain the defect positioning result of the workpiece to be measured includes:

[0023] Obtain point cloud data of standard non-destructive workpieces;

[0024] According to the defect mark of the workpiece to be tested, extracting a point cloud data subset of the area where the defect mark of the workpiece to be tested is located and a point cloud data subset of the corresponding area of ​​the standard non-destructive workpiece;

[0025] Preprocessing the point cloud data subsets of the workpiece to be tested and the standard non-destructive workpiece;

[0026] The PointNet++ model is used to extract features from the point cloud data subsets of the workpiece to be tested and the standard non-destructive workpiece to obtain point cloud features;

[0027] The registration algorithm is used to register the point cloud features of the workpiece to be measured and the standard non-destructive workpiece;

[0028] Compare the point cloud features of the aligned workpiece to be tested and the standard non-destructive workpiece to obtain the defect positioning result.

[0029] According to one aspect of the present invention, the registration algorithm is: iterative closest point algorithm.

[0030] According to one aspect of the present invention, the step of comparing the registered point cloud features of the workpiece to be tested and the standard non-destructive workpiece to obtain the defect positioning result includes:

[0031] Calculate the difference data of the point cloud features of the workpiece to be measured and the standard non-destructive workpiece after registration;

[0032] Compare the difference data with the preset threshold to determine whether the position corresponding to the point cloud feature is a real defect.

[0033] According to one aspect of the present invention, the workpiece defect detection method further includes:

[0034] Obtain point cloud data of defective sample workpieces and standard non-destructive workpieces, and construct a point cloud sample set;

[0035] According to the point cloud sample set, a point cloud processing model based on the PointNet++ model, point cloud registration algorithm and difference detection method is constructed and trained to obtain an accurate workpiece defect detection model;

[0036] According to the defect mark of the workpiece to be tested, the workpiece defect accurate detection model is used to perform a second defect detection on the workpiece to be tested, and the defect accurate positioning result of the workpiece to be tested is obtained.

[0037] A workpiece defect detection system, based on the workpiece defect detection method as described above, comprises:

[0038] A sample construction module, used for acquiring point cloud data of the workpiece to be tested, and constructing a sample workpiece library including at least defective sample workpieces according to the point cloud data of the workpiece to be tested;

[0039] The model building module is used to build and train the YOLO neural network model based on the sample artifact library to obtain a defect detection model;

[0040] A defect detection module, used for performing defect detection on the workpiece to be tested by using a defect detection model to obtain defect marks;

[0041] The defect positioning module is used to perform registration processing on the point cloud data of the workpiece to be measured according to the defect mark of the workpiece to be measured, so as to obtain the defect positioning result of the workpiece to be measured.

[0042] A computer program is provided, which implements the steps of the workpiece defect detection method as described above when the computer program is executed.

[0043] Advantages of the present invention:

[0044] The present invention provides a workpiece defect detection method, which detects the workpiece by constructing a YOLO neural network model to roughly locate the workpiece defects; then uses the PointNet++ model, point cloud registration algorithm and difference detection to further accurately locate the workpiece defects on the three-dimensional point cloud data of the workpiece; and by performing two inspections on the workpiece, it is possible to accurately detect tiny defects of the workpiece.

[0045] The present invention combines YOLOv10 and PointNet++, uses a high-precision laser scanner to obtain the three-dimensional point cloud data of the workpiece, simulates the workpiece defects through CAD modeling, reduces the dimension to generate two-dimensional image data for deep learning training, and finally realizes high-precision detection and positioning of defects. By using the YOLOv10 model to build a rough workpiece defect detection model for two-dimensional image defect detection, and then using the PointNet++ model, point cloud registration algorithm and difference detection to build a precise workpiece defect detection model, the three-dimensional point cloud image of the workpiece is accurately detected, and the tiny defects of the workpiece can be accurately detected; by building a precise workpiece defect detection model, the relevant parameters of PointNet++, point cloud registration, etc. are trained to obtain a model that is more suitable for actual workpiece detection, thereby improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A flow chart of a workpiece defect detection method according to the present invention;

[0048] Figure 2 It is a structural diagram of the YOLOv10 model described in the present invention;

[0049] Figure 3 The present invention is a flowchart of a workpiece defect detection method. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Embodiment 1

[0052] like Figure 1 As shown, a workpiece defect detection method comprises the following steps:

[0053] S1: acquiring point cloud data of a workpiece to be measured, and constructing a sample workpiece library including at least defective sample workpieces according to the point cloud data of the workpiece to be measured.

[0054] In the actual aircraft engine blade defect detection project, we used the innovative combination of Keyence line laser sensors with micron-level precision and 6-axis collaborative robots to successfully collect high-precision 3D point cloud data of the workpiece. This combination uses the high resolution of the laser sensor and the flexible operation of the robot to achieve accurate detection and data collection of tiny defects on the blade surface, providing reliable basic data support for subsequent quality control and optimization.

[0055] The step of constructing a sample workpiece library including at least defective sample workpieces according to the point cloud data of the workpiece to be tested comprises:

[0056] Generate a model image of the workpiece to be tested according to the point cloud data of the workpiece to be tested;

[0057] Defect annotation is performed on the model image of the defective sample workpiece.

[0058] The model image of the workpiece to be measured includes a two-dimensional image and a three-dimensional point cloud image of the workpiece.

[0059] The step of generating a model image of the workpiece to be measured according to the point cloud data of the workpiece to be measured comprises:

[0060] Generate a 3D model of the workpiece based on point cloud data;

[0061] Add realistic textures and materials to the workpiece 3D model;

[0062] Generate 2D images with realistic lighting effects through rendering;

[0063] Combine different image elements, including artifacts, to generate new images, and add artificial noise to the images to simulate real-world interference;

[0064] Add real-world physics to images and simulate artifacts based on physical textures and material properties;

[0065] Set up a virtual environment and adjust environment parameters to simulate different real-world conditions.

[0066] In practical applications, after obtaining high-precision 3D point cloud data of sample workpieces, CAD software can be used to accurately model the workpieces and simulate their physical textures, material properties, and possible defect forms. The generated 3D model is converted into a 2D image with texture and lighting effects through the 3D rendering software Blender. The physics engine Unreal Engine is used to simulate physical phenomena in the real world and simulate defects on the workpiece based on the physical texture and material properties.

[0067] While generating a two-dimensional image of the workpiece, its three-dimensional point cloud map is generated synchronously using a three-dimensional modeling tool to ensure the consistency and integrity of the data.

[0068] Generating 3D point cloud images from 3D point cloud data mainly includes point cloud processing, 3D reconstruction and visualization:

[0069] (1) Point cloud processing: The acquired raw point cloud data may contain noise, redundant information or missing data, so a series of processing is required to improve the quality and usability of the data. Point cloud processing usually includes:

[0070] Point cloud filtering: remove noise points and improve the smoothness and accuracy of point cloud data. Commonly used filtering algorithms include bilateral filtering, Gaussian filtering, conditional filtering, pass filtering, and random sampling consensus filtering (RANSAC).

[0071] Key point extraction and feature description: Extract key points from the point cloud and perform feature description for subsequent point cloud registration and segmentation.

[0072] Point cloud registration: align point cloud data acquired from different perspectives or at different times into a unified coordinate system to form a complete three-dimensional model.

[0073] Segmentation and classification: According to the characteristics of point cloud data, it is segmented into different parts or areas and classified for subsequent analysis and application.

[0074] (2) 3D reconstruction: After point cloud processing is completed, 3D reconstruction is required to generate a 3D point cloud map. There are mainly the following 3D reconstruction methods:

[0075] Voxel-based method: The point cloud data is divided into small cubes (voxels) and the point cloud data within each voxel is processed to generate a 3D model.

[0076] Method based on surface reconstruction: Utilize the geometric information in the point cloud data to directly reconstruct the surface model of the object through an algorithm.

[0077] Deep learning-based method: Use deep learning models to process and analyze point cloud data to generate more detailed and accurate 3D models.

[0078] (3) Visualization: Finally, the generated 3D point cloud needs to be visualized for intuitive display and analysis. Visualization technology can present 3D point cloud data in the form of graphics or images, including point cloud rendering, texture mapping, lighting processing and other steps. Through visualization technology, users can more intuitively understand the shape, structure and characteristics of the 3D model.

[0079] In practical applications, image annotation tools are used to annotate the image data of defective sample workpieces. The annotated image data includes two-dimensional images and three-dimensional point clouds. During the annotation process, the specific location and type of the defects need to be recorded in detail. After the annotation is completed, the YOLOv10 model is trained using these annotated data to enable it to accurately identify and locate defects on the workpiece.

[0080] When constructing a sample data set, you can also use methods such as rotation, scaling, flipping, and noise addition to perform data enhancement to increase sample diversity and improve the robustness of the deep learning model.

[0081] S2: Build and train the YOLO neural network model based on the sample artifact library to obtain a defect detection model.

[0082] In practical applications, the sample workpiece library includes two-dimensional images and three-dimensional point cloud images of defective sample workpieces. The two-dimensional images are used to train the YOLO neural network model to obtain a workpiece defect detection model. The three-dimensional point cloud image of the workpiece has a large amount of data, and it is difficult to train the YOLO neural network model with the three-dimensional point cloud image. Therefore, in the specific application of this embodiment, the two-dimensional image is used to train the YOLO model.

[0083] The YOLO neural network model includes multiple versions. In practical application, this embodiment adopts the latest YOLOv10 model.

[0084] YOLOv10 is a real-time, end-to-end, high-performance target detection algorithm recently proposed by the Multimedia Intelligence Group (THU-MIG) of Tsinghua University. Based on the Ultralytics Python package, it introduces a new real-time target detection method that solves the deficiencies of previous versions of YOLO in post-processing and model architecture. By eliminating non-maximum suppression (NMS) and optimizing various model components, YOLOv10 achieves state-of-the-art performance while significantly reducing computational overhead. Extensive experiments have demonstrated that YOLOv10 achieves excellent accuracy-latency trade-offs at multiple model scales. The YOLO series has been at the forefront of this research because it strikes a balance between performance and efficiency, however, the reliance on NMS and architectural inefficiencies have hindered the achievement of optimal performance. YOLOv10 addresses these issues by introducing consistent dual assignments for NMS-free training and an overall model design strategy oriented toward efficiency-accuracy.

[0085] Figure 2 The following is a diagram of the YOLOv10 model. The structure of YOLOv10 is based on the previous YOLO model, while introducing several key innovations. The YOLOv10 model architecture consists of the following parts:

[0086] Backbone network: The backbone network in YOLOv10 is responsible for feature extraction. It uses an enhanced version of CSPNet (Cross-Stage Partial Network) to improve gradient flow and reduce computational redundancy.

[0087] Neck: The neck is designed to aggregate features of different scales and pass them to the head. It includes a PAN (Path Aggregation Network) layer for effective multi-scale feature fusion.

[0088] One-vs-Many Heads: Generate multiple predictions for each object during training to provide rich supervision signals and improve learning accuracy.

[0089] One-to-One Head: Generates a single best prediction for each object during inference without NMS, thus reducing latency and improving efficiency.

[0090] Features of the YOLOv10 model include:

[0091] (1) Training without NMS: A dual allocation strategy is introduced, including one-to-many allocation and one-to-one allocation. This strategy uses a one-to-many detection head to provide more positive samples to enrich the model training during training, and switches to a one-to-one detection head through gradient truncation during inference, thereby eliminating the reliance on non-maximum suppression (NMS) and reducing inference overhead.

[0092] (2) Comprehensively optimized model design: A comprehensive model design approach is adopted to optimize various components to improve efficiency and accuracy. This includes lightweight classification head, spatial-channel decoupled downsampling, and rank-guided block design. The lightweight classification head reduces computational overhead by using depthwise separable convolutions without significantly affecting performance. Spatial-channel decoupled downsampling enhances downsampling efficiency and reduces information loss by separating spatial reduction and channel increase operations. The rank-guided block design adjusts the complexity of the building blocks according to the inherent redundancy of different stages of the model to ensure optimal utilization of parameters.

[0093] (3) Enhanced model capabilities: The integration of large kernel convolution and partial self-attention (PSA) modules improves the model’s ability to capture global information while maintaining computational efficiency, improving performance without adding significant computational cost. Large kernel convolution is selectively used in deeper stages to expand the receptive field without significantly increasing I / O overhead. The PSA module introduces self-attention in a cost-effective manner, improving the model’s ability to learn global representations.

[0094] YOLOv10 outperforms previous versions of YOLO and other state-of-the-art models in terms of accuracy and efficiency. For example, on the COCO dataset, YOLOv10-S is 1.8 times faster than RT-DETR-R18, while YOLOv10-B reduces latency by 46% and has 25% fewer parameters than YOLOv9-C while achieving the same performance.

[0095] S3: Use the defect detection model to perform defect detection on the workpiece to obtain defect marks.

[0096] In the actual inspection process, the trained YOLOv10 model is used to inspect the image data of the workpiece. The model output results include the location and type of each defect.

[0097] When the model image size of the workpiece to be tested exceeds the input size of the defect detection model, the image is cut into a size that fits the model before input, and the output defect mark is restored to its position on the model image.

[0098] In actual application, the captured image is 3200*8000 pixels, but the fixed input size of the model is 640*640. If the image is compressed to the size specified by the model, the defect points will be very small, so that most of the information will be lost. In specific implementation, the image is cut into 3200*4000 pixels, and the model input size is increased to 1280*1280 pixels. The image is not cut into a smaller size to prevent the defect points from being missed during the cutting process; increasing the model size is to minimize the image compression ratio. Due to the increase in the model input size, the model parameters will also increase exponentially, and the resulting calculation time will also increase several times, but since the detection time requirement is not high during implementation, the increase in detection time is acceptable.

[0099] Using the trained YOLOv10 model, the position of the defect point on the workpiece to be inspected on the two-dimensional image is detected, and the two cut images and the original image are found according to the file name, and the detection result of the cut image is restored to the original image. The model output is (X, Y, W, H). For the defect point in the first image, the coordinate position is (XW / 2, YH / 2), (X+W / 2, Y+H / 2). For the defect point in the second image, 4000 (the original coordinate) should be added to this coordinate. Figure 1 half the height), that is (XW / 2,4000+YH / 2), (X+W / 2,4000+Y+H / 2).

[0100] S4: According to the defect mark of the workpiece to be measured, the point cloud data of the workpiece to be measured is registered to obtain the defect positioning result of the workpiece to be measured.

[0101] Step S4 includes:

[0102] S41: Acquire point cloud data of a standard non-destructive workpiece.

[0103] When registering point clouds, it is necessary to register the point cloud feature data of the workpiece to be measured and the standard non-destructive workpiece.

[0104] S42: extracting, according to the defect mark of the workpiece to be measured, a point cloud data subset of an area where the defect mark of the workpiece to be measured is located and a point cloud data subset of a corresponding area of ​​the standard non-destructive workpiece.

[0105] In steps S2 and S3, the defect detection model is trained using the two-dimensional image of the workpiece, and the rough location area of ​​the detected defect is on the two-dimensional image of the workpiece to be tested. The area needs to be mapped from the two-dimensional image of the workpiece to the three-dimensional point cloud through coordinates to extract the point cloud data of the area.

[0106] The defect mark position of the workpiece to be measured obtained in step S3 is the area where the defect is roughly located, and further point cloud registration processing is required to obtain the accurate location of the defect.

[0107] In practical applications, positioning is performed on the three-dimensional point cloud image of the workpiece according to the coordinates of the two-dimensional image obtained in step S3. Since the point cloud data and the two-dimensional image are consistent in size, the coordinates of the defect point in the two-dimensional image can be directly mapped to the three-dimensional point cloud image. Assuming that the point cloud data is also a 3200x8000 matrix, the coordinates of the box of the defect point detected in the two-dimensional image can be directly used as the corresponding position in the point cloud. Points within a certain area of ​​the detected defect point position are extracted from the three-dimensional point cloud data, and the data of these points are used as the point cloud data of the defect point.

[0108] S43: Preprocessing the point cloud data subsets of the workpiece to be measured and the standard non-destructive workpiece.

[0109] Preprocessing usually includes: noise removal, normalization, etc.

[0110] S44: Use the PointNet++ model to extract features from the point cloud data subsets of the workpiece to be tested and the standard non-destructive workpiece to obtain point cloud features.

[0111] PointNet++ is a neural network model, which mainly includes sampling layer, grouping layer and feature extraction layer.

[0112] The core idea of ​​PointNet++ is to use the set abstraction structure to extract features layer by layer. It can effectively process point cloud data of different scales and extract local and global features of point clouds through a multi-level feature extraction network.

[0113] S45: Use a registration algorithm to register the point cloud features of the workpiece to be measured and the standard non-destructive workpiece.

[0114] The registration algorithm is: iterative closest point algorithm.

[0115] The extracted point cloud features are matched and aligned using the registration algorithm. The point cloud data of the workpiece to be inspected is aligned with the point cloud data of the standard non-destructive workpiece through the iterative closest point (ICP) algorithm or other registration algorithms, and then the difference between the two is calculated.

[0116] S46: Compare the point cloud features of the workpiece to be measured and the standard non-destructive workpiece after registration to obtain a defect positioning result.

[0117] Step S46 includes:

[0118] Calculate the difference data of the point cloud features of the workpiece to be measured and the standard non-destructive workpiece after registration;

[0119] Compare the difference data with the preset threshold to determine whether the position corresponding to the point cloud feature is a real defect.

[0120] The difference of the aligned point cloud data in each axis direction is calculated, especially the difference in the Z axis direction. For example, if the difference in the Z axis direction exceeds a preset threshold (such as 5 units), it is considered that there is a defect at that position.

[0121] The beneficial effects of this embodiment are as follows: this method constructs a YOLOv10 neural network model to detect the workpiece and roughly locate the workpiece defects; then uses the PointNet++ model, point cloud registration algorithm and difference detection to further accurately locate the workpiece defects on the three-dimensional point cloud data of the workpiece; and by performing two inspections on the workpiece, it is possible to accurately detect tiny defects in the workpiece.

[0122] Embodiment 2

[0123] like Figure 2 As shown, a workpiece defect detection method comprises the following steps:

[0124] S1: acquiring point cloud data of a workpiece to be measured, and constructing a sample workpiece library including at least defective sample workpieces according to the point cloud data of the workpiece to be measured.

[0125] In the actual aircraft engine blade defect detection project, we used the innovative combination of Keyence line laser sensors with micron-level precision and 6-axis collaborative robots to successfully collect high-precision 3D point cloud data of the workpiece. This combination uses the high resolution of the laser sensor and the flexible operation of the robot to achieve accurate detection and data collection of tiny defects on the blade surface, providing reliable basic data support for subsequent quality control and optimization.

[0126] The step of constructing a sample workpiece library including at least defective sample workpieces according to the point cloud data of the workpiece to be tested comprises:

[0127] Generate a model image of the workpiece to be tested according to the point cloud data of the workpiece to be tested;

[0128] Defect annotation is performed on the model image of the defective sample workpiece.

[0129] The model image of the workpiece to be measured includes a two-dimensional image and a three-dimensional point cloud image of the workpiece.

[0130] The step of generating a model image of the workpiece to be measured according to the point cloud data of the workpiece to be measured comprises:

[0131] Generate a 3D model of the workpiece based on point cloud data;

[0132] Add realistic textures and materials to the workpiece 3D model;

[0133] Generate 2D images with realistic lighting effects through rendering;

[0134] Combine different image elements, including artifacts, to generate new images, and add artificial noise to the images to simulate real-world interference;

[0135] Add real-world physics to images and simulate artifacts based on physical textures and material properties;

[0136] Set up a virtual environment and adjust environment parameters to simulate different real-world conditions.

[0137] In practical applications, after obtaining high-precision 3D point cloud data of sample workpieces, CAD software can be used to accurately model the workpieces and simulate their physical textures, material properties, and possible defect forms. The generated 3D model is converted into a 2D image with texture and lighting effects through the 3D rendering software Blender. The physics engine Unreal Engine is used to simulate physical phenomena in the real world and simulate defects on the workpiece based on the physical texture and material properties.

[0138] While generating a two-dimensional image of the workpiece, its three-dimensional point cloud map is generated synchronously using a three-dimensional modeling tool to ensure the consistency and integrity of the data.

[0139] Generating 3D point cloud images from 3D point cloud data mainly includes point cloud processing, 3D reconstruction and visualization:

[0140] (1) Point cloud processing: The acquired raw point cloud data may contain noise, redundant information or missing data, so a series of processing is required to improve the quality and usability of the data. Point cloud processing usually includes:

[0141] Point cloud filtering: remove noise points and improve the smoothness and accuracy of point cloud data. Commonly used filtering algorithms include bilateral filtering, Gaussian filtering, conditional filtering, pass filtering, and random sampling consensus filtering (RANSAC).

[0142] Key point extraction and feature description: Extract key points from the point cloud and perform feature description for subsequent point cloud registration and segmentation.

[0143] Point cloud registration: align point cloud data acquired from different perspectives or at different times into a unified coordinate system to form a complete three-dimensional model.

[0144] Segmentation and classification: According to the characteristics of point cloud data, it is segmented into different parts or areas and classified for subsequent analysis and application.

[0145] (2) 3D reconstruction: After point cloud processing is completed, 3D reconstruction is required to generate a 3D point cloud map. There are mainly the following 3D reconstruction methods:

[0146] Voxel-based method: The point cloud data is divided into small cubes (voxels) and the point cloud data within each voxel is processed to generate a 3D model.

[0147] Method based on surface reconstruction: Utilize the geometric information in the point cloud data to directly reconstruct the surface model of the object through an algorithm.

[0148] Deep learning-based method: Use deep learning models to process and analyze point cloud data to generate more detailed and accurate 3D models.

[0149] (3) Visualization: Finally, the generated 3D point cloud needs to be visualized for intuitive display and analysis. Visualization technology can present 3D point cloud data in the form of graphics or images, including point cloud rendering, texture mapping, lighting processing and other steps. Through visualization technology, users can more intuitively understand the shape, structure and characteristics of the 3D model.

[0150] In practical applications, image annotation tools are used to annotate the image data of defective sample workpieces. The annotated image data includes two-dimensional images and three-dimensional point clouds. During the annotation process, the specific location and type of the defects need to be recorded in detail. After the annotation is completed, the YOLOv10 model is trained using these annotated data to enable it to accurately identify and locate defects on the workpiece.

[0151] When constructing a sample data set, you can also use methods such as rotation, scaling, flipping, and noise addition to perform data enhancement to increase sample diversity and improve the robustness of the deep learning model.

[0152] S2: Build and train the YOLO neural network model based on the sample artifact library to obtain a rough defect detection model.

[0153] In practical applications, the sample workpiece library includes two-dimensional images and three-dimensional point cloud images of defective sample workpieces. The two-dimensional images are used to train the YOLO neural network model to obtain a workpiece defect detection model. The three-dimensional point cloud image of the workpiece has a large amount of data, and it is difficult to train the YOLO neural network model with the three-dimensional point cloud image. Therefore, in the specific application of this embodiment, the two-dimensional image is used to train the YOLO model.

[0154] The YOLO neural network model includes multiple versions. In practical application, this embodiment adopts the latest YOLOv10 model.

[0155] YOLOv10 is a real-time, end-to-end, high-performance target detection algorithm recently proposed by the Multimedia Intelligence Group (THU-MIG) of Tsinghua University. Based on the Ultralytics Python package, it introduces a new real-time target detection method that solves the deficiencies of previous versions of YOLO in post-processing and model architecture. By eliminating non-maximum suppression (NMS) and optimizing various model components, YOLOv10 achieves state-of-the-art performance while significantly reducing computational overhead. Extensive experiments have demonstrated that YOLOv10 achieves excellent accuracy-latency trade-offs at multiple model scales. The YOLO series has been at the forefront of this research because it strikes a balance between performance and efficiency, however, the reliance on NMS and architectural inefficiencies have hindered the achievement of optimal performance. YOLOv10 addresses these issues by introducing consistent dual assignments for NMS-free training and an overall model design strategy oriented toward efficiency-accuracy.

[0156] Figure 2 The following is a diagram of the YOLOv10 model. The structure of YOLOv10 is based on the previous YOLO model, while introducing several key innovations. The YOLOv10 model architecture consists of the following parts:

[0157] Backbone network: The backbone network in YOLOv10 is responsible for feature extraction. It uses an enhanced version of CSPNet (Cross-Stage Partial Network) to improve gradient flow and reduce computational redundancy.

[0158] Neck: The neck is designed to aggregate features of different scales and pass them to the head. It includes a PAN (Path Aggregation Network) layer for effective multi-scale feature fusion.

[0159] One-vs-Many Heads: Generate multiple predictions for each object during training to provide rich supervision signals and improve learning accuracy.

[0160] One-to-One Head: Generates a single best prediction for each object during inference without NMS, thus reducing latency and improving efficiency.

[0161] Features of the YOLOv10 model include:

[0162] (1) Training without NMS: A dual allocation strategy is introduced, including one-to-many allocation and one-to-one allocation. This strategy uses a one-to-many detection head to provide more positive samples to enrich the model training during training, and switches to a one-to-one detection head through gradient truncation during inference, thereby eliminating the reliance on non-maximum suppression (NMS) and reducing inference overhead.

[0163] (2) Comprehensively optimized model design: A comprehensive model design approach is adopted to optimize various components to improve efficiency and accuracy. This includes lightweight classification head, spatial-channel decoupled downsampling, and rank-guided block design. The lightweight classification head reduces computational overhead by using depthwise separable convolutions without significantly affecting performance. Spatial-channel decoupled downsampling enhances downsampling efficiency and reduces information loss by separating spatial reduction and channel increase operations. The rank-guided block design adjusts the complexity of the building blocks according to the inherent redundancy of different stages of the model to ensure optimal utilization of parameters.

[0164] (3) Enhanced model capabilities: The integration of large kernel convolution and partial self-attention (PSA) modules improves the model’s ability to capture global information while maintaining computational efficiency, improving performance without adding significant computational cost. Large kernel convolution is selectively used in deeper stages to expand the receptive field without significantly increasing I / O overhead. The PSA module introduces self-attention in a cost-effective manner, improving the model’s ability to learn global representations.

[0165] YOLOv10 outperforms previous versions of YOLO and other state-of-the-art models in terms of accuracy and efficiency. For example, on the COCO dataset, YOLOv10-S is 1.8 times faster than RT-DETR-R18, while YOLOv10-B reduces latency by 46% and has 25% fewer parameters than YOLOv9-C while achieving the same performance.

[0166] S3: Use the defect detection model to perform defect detection on the workpiece to obtain defect marks.

[0167] In the actual inspection process, the trained YOLOv10 model is used to inspect the image data of the workpiece. The model output results include the location and type of each defect.

[0168] When the model image size of the workpiece to be tested exceeds the input size of the defect detection model, the image is cut into a size that fits the model before input, and the output defect mark is restored to its position on the model image.

[0169] In actual application, the captured image is 3200*8000 pixels, but the fixed input size of the model is 640*640. If the image is compressed to the size specified by the model, the defect points will be very small, so that most of the information will be lost. In specific implementation, the image is cut into 3200*4000 pixels, and the model input size is increased to 1280*1280 pixels. The image is not cut into a smaller size to prevent the defect points from being missed during the cutting process; increasing the model size is to minimize the image compression ratio. Due to the increase in the model input size, the model parameters will also increase exponentially, and the resulting calculation time will also increase several times, but since the detection time requirement is not high during implementation, the increase in detection time is acceptable.

[0170] Using the trained YOLOv10 model, the position of the defect point on the workpiece to be inspected on the two-dimensional image is detected, and the two cut images and the original image are found according to the file name, and the detection result of the cut image is restored to the original image. The model output is (X, Y, W, H). For the defect point in the first image, the coordinate position is (XW / 2, YH / 2), (X+W / 2, Y+H / 2). For the defect point in the second image, 4000 (the original coordinate) should be added to this coordinate. Figure 1 half the height), that is (XW / 2,4000+YH / 2), (X+W / 2,4000+Y+H / 2).

[0171] S4: Obtain point cloud data of defective sample workpieces and standard non-destructive workpieces, and construct a point cloud sample set.

[0172] When registering the point cloud, it is necessary to register the point cloud feature data of the workpiece to be tested and the standard non-destructive workpiece. The defects of the 3D point cloud image of the defective sample workpiece are annotated to construct a 3D point cloud sample data set.

[0173] S5: Based on the point cloud sample set, a point cloud processing model based on the PointNet++ model, point cloud registration algorithm and difference detection method is constructed and trained to obtain an accurate workpiece defect detection model.

[0174] The training of the point cloud processing model in step S5 includes the training of parameters in the processes of preprocessing, PointNet++ model, difference detection, etc. Through model training, parameters that are more suitable for the workpiece to be actually detected are selected.

[0175] The point cloud processing model in step S5 includes:

[0176] S51: extracting a point cloud data subset of an area where a defect mark of a defective sample workpiece is located and a point cloud data subset of a corresponding area of ​​a standard non-destructive workpiece.

[0177] S52: Preprocessing the point cloud data subsets of the defective sample workpiece and the standard non-destructive workpiece.

[0178] Preprocessing usually includes: noise removal, normalization, etc.

[0179] S53: Use the PointNet++ model to extract features from the point cloud data subsets of the workpiece to be tested and the standard non-destructive workpiece to obtain point cloud features.

[0180] PointNet++ is a neural network model, which mainly includes sampling layer, grouping layer and feature extraction layer.

[0181] The core idea of ​​PointNet++ is to use the set abstraction structure to extract features layer by layer. It can effectively process point cloud data of different scales and extract local and global features of point clouds through a multi-level feature extraction network.

[0182] S54: Use a registration algorithm to register the point cloud features of the workpiece to be measured and the standard non-destructive workpiece.

[0183] The registration algorithm is: iterative closest point algorithm.

[0184] The extracted point cloud features are matched and aligned using the registration algorithm. The point cloud data of the workpiece to be inspected is aligned with the point cloud data of the standard non-destructive workpiece through the iterative closest point (ICP) algorithm or other registration algorithms, and then the difference between the two is calculated.

[0185] S55: Compare the point cloud features of the workpiece to be measured and the standard non-destructive workpiece after registration to obtain a defect positioning result.

[0186] Step S55 includes:

[0187] Calculate the difference data of the point cloud features of the workpiece to be measured and the standard non-destructive workpiece after registration;

[0188] Compare the difference data with the preset threshold to determine whether the position corresponding to the point cloud feature is a real defect.

[0189] The difference of the aligned point cloud data in each axis direction is calculated, especially the difference in the Z axis direction. For example, if the difference in the Z axis direction exceeds a preset threshold (such as 5 units), it is considered that there is a defect at that position.

[0190] S6: According to the defect mark of the workpiece to be tested, a second defect detection is performed on the workpiece to be tested using the workpiece defect accurate detection model to obtain a defect accurate positioning result of the workpiece to be tested.

[0191] In steps S2 and S3, the defect detection model is trained using the two-dimensional image of the workpiece, and the rough location area of ​​the detected defect is on the two-dimensional image of the workpiece to be tested. The area needs to be mapped from the two-dimensional image of the workpiece to the three-dimensional point cloud through coordinates to extract the point cloud data of the area.

[0192] According to the defect mark of the workpiece to be tested in step S3, a point cloud data subset of the area where the defect mark of the workpiece to be tested is located is extracted, and a second defect detection is performed on the workpiece to be tested using the workpiece defect precision detection model to obtain a defect precision positioning result on the three-dimensional point cloud image of the workpiece to be tested.

[0193] The defect mark position of the workpiece to be measured obtained in step S3 is a rough location of the defect, and the accuracy is not very high. By further performing point cloud registration processing, the accurate location of the defect can be obtained.

[0194] In practical applications, positioning is performed on the three-dimensional point cloud image of the workpiece according to the coordinates of the two-dimensional image obtained in step S3. Since the point cloud data and the two-dimensional image are consistent in size, the coordinates of the defect point in the two-dimensional image can be directly mapped to the three-dimensional point cloud image. Assuming that the point cloud data is also a 3200x8000 matrix, the coordinates of the box of the defect point detected in the two-dimensional image can be directly used as the corresponding position in the point cloud. Points within a certain area of ​​the detected defect point position are extracted from the three-dimensional point cloud data, and the data of these points are used as the point cloud data of the defect point.

[0195] The beneficial effects of this embodiment are as follows: this method constructs a rough workpiece defect detection model based on YOLOv10 to perform rough defect detection on the two-dimensional image of the workpiece, and then performs precise defect detection on the three-dimensional point cloud of the workpiece based on the PointNet++ precise workpiece defect detection model, thereby achieving precise detection of tiny defects on the workpiece; by constructing a precise workpiece defect detection model and training relevant parameters of PointNet++, point cloud registration, etc., a model that is more suitable for actual workpiece detection is obtained, thereby improving detection accuracy.

[0196] Embodiment 3

[0197] A workpiece defect detection system, based on the workpiece defect detection method as described in Embodiment 1 or 2, comprises:

[0198] A sample construction module, used for acquiring point cloud data of the workpiece to be tested, and constructing a sample workpiece library including at least defective sample workpieces according to the point cloud data of the workpiece to be tested;

[0199] The model building module is used to build and train the YOLO neural network model based on the sample artifact library to obtain a defect detection model;

[0200] A defect detection module, used for performing defect detection on the workpiece to be tested by using a defect detection model to obtain defect marks;

[0201] The defect positioning module is used to perform registration processing on the point cloud data of the workpiece to be measured according to the defect mark of the workpiece to be measured, so as to obtain the defect positioning result of the workpiece to be measured.

[0202] Preferably, the system further comprises:

[0203] A point cloud sample set building module is used to obtain point cloud data of defective sample workpieces and standard non-destructive workpieces to build a point cloud sample set;

[0204] The precise detection model building module is used to build and train a point cloud processing model based on the PointNet++ model, point cloud registration algorithm, and difference detection method according to the point cloud sample set to obtain a precise detection model for workpiece defects;

[0205] The defect precise positioning module is used to perform a second defect detection on the workpiece to be tested according to the defect mark of the workpiece to be tested and use the workpiece defect precise detection model to obtain the defect precise positioning result of the workpiece to be tested.

[0206] Embodiment 4

[0207] A computer program, which, when executed, implements the steps of the workpiece defect detection method as described in embodiment one or two.

[0208] Embodiment 5

[0209] A readable storage medium stores a computer program as described in Example 4, and when the computer program is executed, the steps of the workpiece defect detection method as described in Example 1 or 2 are implemented.

[0210] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting workpiece defects, characterized in that: The following steps are involved: Acquire point cloud data of the workpiece to be measured, and construct a sample workpiece library including at least defective sample workpieces according to the point cloud data of the workpiece to be measured; Build and train the YOLO neural network model based on the sample artifact library to obtain a defect detection model; Perform defect detection on the workpiece to be tested using the defect detection model to obtain defect marks; According to the defect mark of the workpiece to be measured, the point cloud data of the workpiece to be measured is aligned and processed to obtain the defect positioning result of the workpiece to be measured.

2. The workpiece defect detection method according to claim 1, characterized in that: The step of constructing a sample workpiece library including at least defective sample workpieces according to the point cloud data of the workpiece to be tested comprises: Generate a model image of the workpiece to be tested according to the point cloud data of the workpiece to be tested; Defect annotation is performed on the model image of the defective sample workpiece.

3. The workpiece defect detection method according to claim 2, characterized in that: The step of generating a model image of the workpiece to be measured according to the point cloud data of the workpiece to be measured comprises: Generate a 3D model of the workpiece based on point cloud data; Add realistic textures and materials to the workpiece 3D model; Generate 2D images with realistic lighting effects through rendering; Combine different image elements, including artifacts, to generate new images, and add artificial noise to the images to simulate real-world interference; Add real-world physics to images and simulate artifacts based on physical textures and material properties; Set up a virtual environment and adjust environment parameters to simulate different real-world conditions.

4. The workpiece defect detection method according to claim 2, characterized in that: When the model image size of the workpiece to be tested exceeds the input size of the defect detection model, the image is cut into a size that fits the model before input, and the output defect mark is restored to its position on the model image.

5. The workpiece defect detection method according to claim 1, characterized in that: The step of performing registration processing on the point cloud data of the workpiece to be measured according to the defect mark of the workpiece to be measured to obtain the defect positioning result of the workpiece to be measured includes: Obtain point cloud data of standard non-destructive workpieces; According to the defect mark of the workpiece to be tested, extracting a point cloud data subset of the area where the defect mark of the workpiece to be tested is located and a point cloud data subset of the corresponding area of ​​the standard non-destructive workpiece; Preprocessing the point cloud data subsets of the workpiece to be tested and the standard non-destructive workpiece; The PointNet++ model is used to extract features from the point cloud data subsets of the workpiece to be tested and the standard non-destructive workpiece to obtain point cloud features; The registration algorithm is used to register the point cloud features of the workpiece to be measured and the standard non-destructive workpiece; Compare the point cloud features of the aligned workpiece to be tested and the standard non-destructive workpiece to obtain the defect positioning result.

6. The workpiece defect detection method according to claim 5, characterized in that: The registration algorithm is: iterative closest point algorithm.

7. The workpiece defect detection method according to claim 5, characterized in that: The step of comparing the registered point cloud features of the workpiece to be tested and the standard non-destructive workpiece to obtain the defect positioning result includes: Calculate the difference data of the point cloud features of the workpiece to be measured and the standard non-destructive workpiece after registration; Compare the difference data with the preset threshold to determine whether the position corresponding to the point cloud feature is a real defect.

8. The workpiece defect detection method according to claim 1, characterized in that: The workpiece defect detection method further comprises: Obtain point cloud data of defective sample workpieces and standard non-destructive workpieces, and construct a point cloud sample set; According to the point cloud sample set, a point cloud processing model based on the PointNet++ model, point cloud registration algorithm and difference detection method is constructed and trained to obtain an accurate workpiece defect detection model; According to the defect mark of the workpiece to be tested, the workpiece defect accurate detection model is used to perform a second defect detection on the workpiece to be tested, and the defect accurate positioning result of the workpiece to be tested is obtained.

9. A workpiece defect detection system, characterized in that: The workpiece defect detection method according to any one of claims 1 to 8 comprises: A sample construction module, used for acquiring point cloud data of the workpiece to be tested, and constructing a sample workpiece library including at least defective sample workpieces according to the point cloud data of the workpiece to be tested; The model building module is used to build and train the YOLO neural network model based on the sample artifact library to obtain a defect detection model; A defect detection module, used for performing defect detection on the workpiece to be tested by using a defect detection model to obtain defect marks; The defect positioning module is used to perform registration processing on the point cloud data of the workpiece to be measured according to the defect mark of the workpiece to be measured, so as to obtain the defect positioning result of the workpiece to be measured.

10. A computer program, characterized in that When the computer program is executed, the steps of the workpiece defect detection method according to any one of claims 1 to 8 are implemented.

Images