Deep learning defect identification system and method based on simulation generation of simulation data set

Abstract

The invention provides a deep learning defect identification system and method based on simulation generation of a simulation data set. The system comprises a defect model generation module, a virtual three-dimensional CT module and a defect identification deep learning module. The defect model generation module is used for generating a defect-containing CAD model; the virtual three-dimensional CT module is used for performing CT scanning and reconstruction on the CAD model; the defect recognition deep learning module is used for marking CT sectional images and using marked data for network training, the problem that an X-ray image data set is insufficient is solved, and the method is safe, rapid and independent of equipment; the problem of inaccurate marking in deep learning defect identification is also solved, and the defect identification precision of the deep learning module obtained by training is improved.

Description

Technical Field

[0001] The present invention belongs to the fields of computer simulation, deep learning, and three-dimensional industrial CT technology, and specifically relates to a deep learning defect recognition system and method based on simulation-based generation of simulation data sets. Background Art

[0002] Nondestructive testing (NDT) ensures the safety and reliability of parts without causing damage, and has always been an important testing method for quality assurance in various industries. Among various nondestructive testing technologies, industrial CT, as one of the most advanced methods, can perform detailed characterization and analysis of the internal structure of workpieces, and is widely used in industrial fields such as aerospace, automobile manufacturing, and electronic products.

[0003] Industrial CT works by using an X-ray beam to penetrate the object being tested and obtain a series of two-dimensional images or projections from multiple angles. These images are then reconstructed to form a three-dimensional representation of the internal structure. Industrial CT can provide detailed cross-sectional images of the object being tested, which makes it widely used to detect defects such as voids and cracks in various industrial products.

[0004] The data generated by industrial CT consists of a series of tomographic slices that can reveal key information about the internal integrity of the inspected object. With the advent of deep learning, revolutionary progress has been made in the field of image processing, especially in the field of image segmentation. Deep learning algorithms have shown excellent capabilities in automatically identifying and segmenting defects in industrial CT images, but image annotation often needs to be completed by human experts.

[0005] However, the effectiveness of deep learning models relies heavily on the availability of large amounts of annotated data. In the context of industrial CT, obtaining such data faces several challenges. The CT scanning process is complex and costly, limiting the amount of data that can be practically acquired. In addition, the task of manually annotating data is time-consuming and laborious. Summary of the invention

[0006] To address the challenge of acquiring and annotating large-scale, high-quality deep learning model datasets in industrial CT, simulation is a feasible solution. The present invention proposes a deep learning defect recognition system and method based on simulation to generate simulation datasets. This method includes creating CAD models of parts and artificially introducing defects in these models. By using these models to simulate the CT scanning process, we can generate cross-sectional image datasets that are close to real-world scenes. The main advantage of this method is that since the defects are introduced in a controlled computing environment, their characteristics and locations are precisely known. This allows labels to be automatically generated for the dataset, eliminating the time-consuming and labor-intensive manual annotation process. And because the dataset is generated by simulation, any amount of training data can be generated.

[0007] Using these synthetic datasets, deep learning models can be trained to identify and classify various types of defects with high accuracy. Once the model is fully trained on simulated data, transfer learning techniques can be used to adapt the model to real-world CT datasets. This process ensures that the model maintains its learning ability while being fine-tuned for specific features and noise present in actual industrial CT images. This approach not only alleviates the limitations of industrial CT data acquisition, but also enables more efficient and automated defect detection systems.

[0008] The present invention proposes a deep learning defect recognition system based on simulation-generated simulation data sets, and its technical solution is as follows: the system comprises: a defect model generation module, a virtual three-dimensional CT module, and a defect recognition deep learning module; wherein,

[0009] The defect model generation module is used to generate a CAD model containing defects;

[0010] The virtual 3D CT module is used to perform CT scanning and reconstruction on the CAD model containing defects to obtain CT tomographic images;

[0011] The defect recognition deep learning module is used to annotate CT tomographic images and use the annotated data for network training;

[0012] The defect model generation module includes a healthy model generation part and a defect addition part; the healthy model generation part is used to generate a healthy CAD model with a typical shape, including a cylinder and a cube, and the material properties and size of the CAD model can be specified; the defect addition part is used to add defects to the healthy CAD model to obtain a CAD model with defects. The defect types include holes, cracks, and inclusions. The size of the defect varies randomly within a certain range, and the location of the defect is randomly distributed;

[0013] The virtual 3D CT module includes a CT scanning part and a CT reconstruction part. The CT scanning part simulates the scanning process of a real 3D industrial CT system. The scanning process of simulating a real 3D industrial CT includes:

[0014] Ray source simulation: the shape and size of the simulated ray source are represented by a dot matrix. The dot matrix shape, number of dots, and dot spacing are all adjustable.

[0015] Detector simulation: The simulated detector is represented by a dot matrix, where each dot represents a detector. The distance between dots represents the size of the detector. The number of rows and columns of the dot matrix represents the number of rows and columns of the detector. The product of the number of rows and columns is the number of detectors.

[0016] Ray beam simulation, the simulated ray beam is formed by connecting each point in the simulated ray source lattice with each point in the simulated detector lattice;

[0017] Ray spectrum simulation, using a discretized energy list to define the X-ray energy spectrum distribution;

[0018] The projection process is simulated by using pipeline tracing algorithm to calculate the length of each ray passing through the industrial product to be inspected. According to the material properties of the industrial product to be inspected, the ray energy spectrum emitted by the ray source, the source-object distance, and the source-detection distance, the energy value of the ray when it reaches the detector is calculated to obtain the projection map.

[0019] The CT reconstruction part reconstructs the projection image obtained by the CT scanning part to obtain a reconstructed image.

[0020] The present invention also proposes a deep learning defect recognition method based on simulation generation of a simulation data set, the method comprising the following steps:

[0021] Step S501. Determine the parameters of the pre-generated CAD model in the defect model generation module, including the shape, size, and material of the model;

[0022] Step S502. Specify the type, size, and quantity of pre-added defects in the defect model generation module; add the defects into the healthy CAD model to obtain a CAD model with defects;

[0023] Step S503. Specify the simulated industrial CT working scene parameters in the virtual 3D CT module and perform CT scanning on the CAD model with defects to obtain CT projections;

[0024] Step S504: Perform CT reconstruction on the generated CT projection in the virtual three-dimensional CT module to obtain a CT tomographic image of the model to be tested;

[0025] Step S505. Automatically annotate the CT tomographic image in the defect recognition deep learning module to obtain a CT tomographic image with defect annotations;

[0026] Step S506. In the defect recognition deep learning module, the tomographic CT images with defect annotations are used as training sets to train the deep learning network. After the training is completed, a deep learning module with automatic defect recognition function is obtained.

[0027] The beneficial effects of the present invention are:

[0028] 1. Solved the problem of insufficient X-ray image data set. The defect recognition deep learning model needs CT images for training. Acquiring real CT images requires three-dimensional industrial CT equipment and objects to be detected that meet the requirements, which is expensive, time-consuming, and has radiation pollution. By using the deep learning defect recognition system based on simulation-generated simulation data sets of the present invention, any number of simulated CT images can be generated according to the set parameters for training the deep learning network, and finally a defect recognition deep learning model is obtained. This method is safe, fast, and independent of equipment.

[0029] 2. It solves the problem of inaccurate labeling in deep learning defect recognition and improves the defect recognition accuracy of the trained deep learning module. Deep learning models usually need to be trained in a supervised manner, which requires a large amount of labeled data. The labeling process is usually a manual process. This process is very dependent on the proficiency of the labeler and is prone to errors. When erroneous labeled data appears, it will affect the final model training. The industrial CT virtual reality interactive system of the present invention can enhance the interactivity and immersion of users by using virtual reality devices, which can improve the effect of teaching and training. Users can understand and master relevant knowledge in an intuitive and vivid way, thereby improving learning efficiency and effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the functional modules of the deep learning defect recognition system based on simulation to generate simulation data sets according to the present invention;

[0031] Figure 2 A schematic diagram of a CAD model with defects generated by a defect generation module in a deep learning defect recognition system based on simulation-generated simulation data sets according to the present invention;

[0032] Figure 3 It is a schematic diagram of a CT tomographic image obtained by a virtual three-dimensional CT module in a deep learning defect recognition system based on simulation to generate a simulation data set, after automatic annotation;

[0033] Figure 4 It is a result graph finally used for detecting defects in the deep learning defect recognition system based on simulation-generated simulation data set of the present invention;

[0034] Figure 5 This is a workflow diagram of the deep learning defect recognition method based on simulation to generate a simulation data set according to the present invention.

[0035] The reference numerals are as follows:

[0036] Defect model generation module 11, virtual three-dimensional CT module 12, defect recognition deep learning module 13, health model generation part 111, defect addition part 112, CT scanning part 121, CT reconstruction part 122, defect automatic labeling part 131, deep learning network part 132. DETAILED DESCRIPTION

[0037] The present invention will be further described below with reference to the accompanying drawings and examples.

[0038] like Figure 1 As shown, the present invention proposes a deep learning defect recognition system based on simulation generation of simulation data sets, including: a defect model generation module 11, a virtual three-dimensional CT module 12, and a defect recognition deep learning module 13.

[0039] The defect model generating module 11 includes a healthy model generating part 111 and a defect adding part 112 .

[0040] The generating healthy model part 111 is used to generate a healthy CAD sample model, wherein the shape, size, and material of the model can be specified. The shape includes two basic shapes with typical characteristics, namely, a cylinder and a cube. The size is the radius and height of a cylinder or the side length of a cube. The material properties include the material name, material density, material linear attenuation coefficient, etc.

[0041] The defect adding part 112 is used to add defects to the healthy CAD sample to obtain a CAD model with defects. The defect types include holes, cracks, and inclusions. The size of the defects varies randomly within a specified range, and the locations of the defects are randomly distributed inside the sample. The specific approach is: first, we model the three defects. The pores are round and have a smooth surface. The cracks are flat, long structures, and their width gradually decreases with increasing depth. The shape of the inclusions is similar to that of bubbles, but the difference is the substance in the inclusions. The former is other substances, and the latter is air. After modeling the defects, the next step is to introduce defects with specific characteristics into the healthy 3D model. The specific method is to perform Boolean operations between the defect model and randomly located areas on the healthy model. For defects such as holes and cracks, a Boolean subtraction operation is performed between the healthy model and the predefined defect model. For included defects, a Boolean subtraction operation is first performed, and then a Boolean addition operation is performed between the healthy model and the predefined defect model. Finally, a CAD model with defects is obtained. As shown in Figure 2 As shown in Figures a, b and c, these are schematic diagrams of a CAD model with defects in the present invention.

[0042] The virtual three-dimensional CT module 12 is a mapping of the real three-dimensional industrial CT system in the virtual reality space, and includes two parts: a CT scanning part 121 and a CT reconstruction part 122. The virtual three-dimensional CT module simulates a series of virtual components: a virtual ray source, a virtual detector, a virtual rotating platform, a virtual mechanical system, a virtual lead room protection system, and a virtual computer control system. Each virtual component has a corresponding model to represent its geometric shape and its position and posture in the virtual space. In addition, each virtual component has its own parameters and behavior attributes. The parameters of the virtual component and the scanning process and scanning parameters can be set. The behavior attributes of the virtual component describe the dynamic behavior of the virtual component during the scanning process. The virtual three-dimensional CT module simulates the scanning and reconstruction process of the real three-dimensional industrial CT system. The CT scanning part 121 simulates the scanning process to obtain a series of projection images; the CT reconstruction part 122 simulates the reconstruction process to obtain a series of tomographic images.

[0043] The CT scanning part 121 simulates the scanning process of a real three-dimensional industrial CT system and outputs a series of projection images. The imaging process of the simulated three-dimensional industrial CT includes: ray source simulation, the shape and size of the simulated ray source are represented by a dot matrix, and the dot matrix shape, number of points, and point spacing are all adjustable; detector simulation, the simulated detector is represented by a dot matrix, each point represents a probe, the distance between points represents the probe size, the number of rows and columns of the dot matrix represents the number of rows and columns of the detector, and the product of the number of rows and columns is the number of probes; ray beam simulation, the simulated ray beam is formed by connecting each point in the simulated ray source dot matrix and each point in the simulated detector dot matrix; ray energy spectrum simulation, using a discretized energy list to define the X-ray energy spectrum distribution; projection process simulation, using a pipeline tracking algorithm to calculate the length of each ray passing through the industrial product to be detected, and according to the material properties of the industrial product to be detected, the ray energy spectrum emitted by the ray source, the source-object distance, and the source-detection distance, the energy value of the ray when it reaches the probe is calculated to obtain a projection image.

[0044] The CT reconstruction part 122 simulates the reconstruction process of a real three-dimensional industrial CT system and outputs a series of tomograms. The reconstruction process of the simulated three-dimensional industrial CT is: select a reconstruction algorithm, reconstruct a series of projection images obtained by the CT scanning part 121, and obtain a series of tomographic images. In the simulation of the three-dimensional CT imaging reconstruction process, the only difference from the actual reconstruction process of three-dimensional CT imaging is the source of the input projection image data, one is simulated, and the other is generated by a real three-dimensional CT system. The subsequent reconstruction algorithm and image processing method are exactly the same. The CAD model with defects is simulated by CT scanning and reconstruction, and the final tomographic image contains defect information.

[0045] The defect recognition deep learning module 13 includes an automatic defect labeling part 131 and a deep learning network part 132 .

[0046] The automatic defect annotation part 131 automatically annotates defects using the style, size, and location information of defects in the sample, as well as the correspondence between the model slices and the reconstructed image, to obtain a tomographic image with defect annotations. Specifically, after a series of simulations to generate defect models and three-dimensional CT scanning and reconstruction processes, we simulate and obtain reconstructed CT projections. From the scanning geometry we established during the simulation process, we accurately determine the slices in the polygonal mesh that correspond to the reconstructed volume data. Therefore, we can know each voxel inside the defect mesh, thereby obtaining a labeled image with defects. Figure 3 As shown, the automatically annotated tomographic image is obtained.

[0047] The deep learning network part 132 uses the fault-annotated tomographic images obtained by the defect automatic annotation part 131 as a training set. The deep learning network used is based on the network structure of Mask R-CNN. Mask-RCNN is an advanced computer vision model that extends Faster R-CNN by adding a branch to predict the mask of the object, thereby achieving simultaneous target detection and high-quality instance segmentation. One of the core improvements of Mask-RCNN is the ROIAlign layer, which solves the pixel alignment problem in traditional ROI Pooling by avoiding quantizing the RoI boundary or box grid and using bilinear interpolation to ensure accurate alignment between the feature map and the input image. In addition, Mask-RCNN has also made improvements in the loss function. The binary mask of each category is predicted independently, without competition between categories, and the RoI classification branch of the network is relied on to predict the category. The architecture of Mask-RCNN can be summarized as RPN (Region Proposal Network) plus ROIAlign, plus Faster-RCNN and FCN (Fully Convolutional Network). This structure is not only suitable for instance segmentation, but also can complete multiple tasks such as target classification, semantic segmentation, and human posture recognition by adding different branches. After a certain number of training rounds, the loss function gradually stabilizes, and a deep learning module with defect recognition function is obtained. The deep learning module after training is used to predict defects. The prediction results are as follows: Figure 4 shown.

[0048] The Mask-RCNN network structure is divided into two stages. The first stage is the region proposal network RPN, which proposes candidate object bounding boxes. Each region proposal will go through the second stage. In the second stage, for each region proposal, the feature map proposed in the first stage is ROI pooled according to the region, and passes through the remaining network to output the category, bounding box and binary mask.

[0049] The loss function of Mask-RCNN is:

[0050] ,

[0051] in, is the classification loss, is the bounding box loss, is the binary mask loss.

[0052] For the deep learning defect recognition method based on simulation-generated simulation data set invented in this invention, the complete workflow is as follows: Figure 5 As shown:

[0053] Step S501, determining the parameters of the production CAD model in the defect model generation module, including the shape, size, and material of the model.

[0054] Step S502, specifying the type, size, and quantity of pre-added defects in the defect model generation module, adding the defects into the healthy model, and obtaining a CAD model with defects.

[0055] Step S503, setting the simulated industrial CT working parameters in the virtual 3D CT module, performing CT scanning on the CAD model with defects, and obtaining a series of CT projections.

[0056] Step S504: Perform CT reconstruction on the generated CT projection in the virtual three-dimensional CT module to obtain a tomographic CT image of the model to be tested. The tomographic CT image at this time contains defect information.

[0057] Step S505: Automatically annotate the CT image in the defect recognition deep learning module to obtain a CT image with defect annotations.

[0058] Step S506: In the defect recognition deep learning module, the generated defect-annotated tomographic CT images are used as training sets to train the deep learning network, and after the training is completed, a deep learning module with automatic defect recognition function is obtained, which can be used to recognize defects in defect-containing tomographic CT images.

Claims

1. A deep learning defect recognition system based on simulation-generated simulation data sets, characterized in that: The system includes: a defect model generation module, a virtual three-dimensional CT module, and a defect recognition deep learning module; wherein, The defect model generation module is used to generate a CAD model containing defects; The virtual 3D CT module is used to perform CT scanning and reconstruction on the CAD model containing defects to obtain CT tomographic images; The defect recognition deep learning module is used to annotate CT tomographic images and use the annotated data for network training; The defect model generation module includes a healthy model generation part and a defect addition part; the healthy model generation part is used to generate a healthy CAD model with a typical shape, including a cylinder and a cube, and the material properties and size of the CAD model can be specified; the defect addition part is used to add defects to the healthy CAD model to obtain a CAD model with defects. The defect types include holes, cracks, and inclusions. The size of the defect varies randomly within a certain range, and the location of the defect is randomly distributed; The virtual 3D CT module includes a CT scanning part and a CT reconstruction part. The CT scanning part simulates the scanning process of a real 3D industrial CT system. The scanning process of simulating a real 3D industrial CT includes: Ray source simulation: the shape and size of the simulated ray source are represented by a dot matrix. The dot matrix shape, number of dots, and dot spacing are all adjustable. Detector simulation: The simulated detector is represented by a dot matrix, where each dot represents a detector. The distance between dots represents the size of the detector. The number of rows and columns of the dot matrix represents the number of rows and columns of the detector. The product of the number of rows and columns is the number of detectors. Ray beam simulation, the simulated ray beam is formed by connecting each point in the simulated ray source lattice with each point in the simulated detector lattice; Ray spectrum simulation, using a discretized energy list to define the X-ray energy spectrum distribution; The projection process is simulated by using pipeline tracing algorithm to calculate the length of each ray passing through the industrial product to be inspected. According to the material properties of the industrial product to be inspected, the ray energy spectrum emitted by the ray source, the source-object distance, and the source-detection distance, the energy value of the ray when it reaches the detector is calculated to obtain the projection map. The CT reconstruction part reconstructs the projection image obtained by the CT scanning part to obtain a reconstructed image.

2. A deep learning defect recognition system based on simulation-generated simulation data set according to claim 1, characterized in that: The defect recognition deep learning module includes an automatic defect annotation part and a deep learning network part; the automatic defect annotation part automatically annotates defects by using the style, size, location information of the defects, as well as the correspondence between the model slices and the reconstructed images, to obtain a CT image with defects; the deep learning network part uses the CT image with defects obtained by the automatic defect annotation part as a training set to train a deep learning module with defect recognition function.

3. A deep learning defect recognition system based on simulation-generated simulation data set according to claim 2, characterized in that: The deep learning network is based on the network structure of Mask R-CNN. The Mask-RCNN network structure is divided into two stages. The first stage is the region proposal network RPN, which proposes candidate object bounding boxes. Each region proposal will go through the second stage. In the second stage, for each region proposal, the feature map proposed in the first stage is ROI pooled according to the region, and through the remaining network, the category, bounding box and binary mask are output.

4. A deep learning defect recognition system based on simulation-generated simulation data set according to claim 3, characterized in that: The loss function of Mask-RCNN is: ， in, is the classification loss, is the bounding box loss, is the binary mask loss.

5. A deep learning defect recognition method based on simulation-generated simulation data set, characterized in that: The method comprises the following steps: Step S501. Determine the parameters of the pre-generated CAD model in the defect model generation module, including the shape, size, and material of the model; Step S502. Specify the type, size, and quantity of pre-added defects in the defect model generation module; add the defects into the healthy CAD model to obtain a CAD model with defects; Step S503. Specify the simulated industrial CT working scene parameters in the virtual 3D CT module and perform CT scanning on the CAD model with defects to obtain CT projections; Step S504: Perform CT reconstruction on the generated CT projection in the virtual three-dimensional CT module to obtain a CT tomographic image of the model to be tested; Step S505. Automatically annotate the CT tomographic image in the defect recognition deep learning module to obtain a CT tomographic image with defect annotations; Step S506. In the defect recognition deep learning module, the tomographic CT images with defect annotations are used as training sets to train the deep learning network. After the training is completed, a deep learning module with automatic defect recognition function is obtained.

6. The deep learning defect recognition method based on simulation generation of simulation data set according to claim 5 is characterized in that: The parameters of the industrial CT working scene include the spatial position and spatial posture of the industrial product to be inspected, the source-object distance, the detector-object distance, the shape and spacing of the radiation source lattice, the shape and spacing of the detector lattice, the radiation source tube voltage, and the number of scan frames.

Images