Internal foreign matter efficient classification method based on small amount of CT scanning data

Abstract

The invention discloses an internal foreign matter efficient classification method based on a small amount of CT scanning data. The method comprises the following steps of S1, obtaining and preprocessing the CT scanning data, S2, performing 3D reconstruction on an object, S3, segmenting a reconstruction volume, S4, generating an artificial sample, and S5, performing model testing and segmentation visualization. According to the method, two major obstacles of acquiring industrial CT scanning data and label data are effectively eliminated, the labor and time cost is greatly reduced, data updating and labeling can be carried out in real time in the production process, volume data and categories of all artificial samples are automatically generated, and the method is used for training a deep learning network model. Compared with training of the same number of real samples, the internal foreign matter classification accuracy is greatly improved, even industrial CT scanning data of only one real sample is needed, the verification accuracy can reach 95.6% after artificial samples generated by the industrial CT scanning data are trained through the network model, the classification efficiency is high, and the reliability is good.

Description

Technical Field

[0001] The invention relates to the technical field of industrial CT image processing, and in particular to an efficient internal foreign body classification method based on a small amount of CT scanning data. Background Art

[0002] In modern industrial production, product quality control is a key link to ensure production efficiency and safety. With the increasing complexity of product structures, especially in areas requiring high precision and high reliability, accurate and efficient classification of internal foreign matter has become an indispensable part. If these internal foreign matter are not sorted out in time, they may seriously affect the performance and safety of the product, and may even cause serious production accidents or quality problems. However, traditional methods of sorting internal foreign matter, such as semi-automatic detection systems based on manual operations and manual visual inspections, are inefficient and susceptible to subjective interference, making it difficult to meet the urgent needs of modern industry for high-speed and accurate classification.

[0003] In recent years, industrial CT (computed tomography) technology has become an important tool for detecting and analyzing the internal structure of products due to its ability to provide detailed three-dimensional images quickly and non-invasively. Compared with traditional two-dimensional image detection methods, industrial CT technology can present a panoramic view of the inside of the product, thereby identifying potential internal foreign objects. However, the amount of data generated by industrial CT is huge and complex, which places higher requirements on data processing and analysis. At the same time, the rapid development of deep learning technology, especially ResNet-50, is showing great application prospects in the classification of internal foreign objects in the field of industrial CT with its unique residual structure, deep network processing capabilities and high accuracy in image classification. The ResNet-50 network effectively solves the gradient vanishing problem in deep networks through the design of residual connections, improving the expressiveness and accuracy of the model. This enables the automated detection method based on deep learning to effectively identify and classify complex internal foreign objects, surpassing the limitations of traditional unsupervised methods.

[0004] At present, although deep learning technology has significant advantages in theory, the acquisition of high-quality labeled industrial CT image data is costly and time-consuming, which has become a bottleneck for the application of technology. The collection and labeling of a large amount of labeled data requires a lot of manpower and time, and it is impossible to update and label data in real time during the production process; in addition, the automatic classification of internal foreign matter is costly, with poor accuracy, low efficiency, and poor classification reliability. Summary of the invention

[0005] The purpose of the present invention is to provide an efficient internal foreign body classification method based on a small amount of CT scan data to solve the above defects.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] An efficient internal foreign body classification method based on a small amount of CT scan data comprises the following steps:

[0008] S1. Obtain CT scan data and preprocess:

[0009] Download an open-source CT scan dataset of foreign bodies inside objects and preprocess the X-ray projection data in the dataset, including bright field and dark field correction;

[0010] S2. Object 3D reconstruction:

[0011] The 3D distribution of attenuation coefficients within the sample volume is generated through the CT reconstruction algorithm. The FDK algorithm of the ASTRA toolbox is used to convert the data collected when the X-rays pass through the object into the attenuation coefficient distribution of each voxel in three-dimensional space, and the beam hardening correction method is used to reduce artifacts.

[0012] S3. Segment the reconstructed volume:

[0013] Apply the segmentation mask, calculate the average intensity of each material in the reconstructed volume, and extract the attenuation coefficient of each material; apply the median filter to the segmented data to smooth the image without blurring the edges, and accurately classify the volume data; use the Otsu threshold segmentation method to segment the internal foreign matter, obtain the volume data and category of all segmented samples, and then save them to the segmentation result folder;

[0014] S4. Artificial sample generation:

[0015] Select a representative sample in the segmented data (the number of internal foreign bodies must be 1 or more), use the average strength of the material to convert the segmented volume back to the attenuated 3D distribution, obtain the volume data and categories of all segmented samples, add a mixed Poisson-Gaussian noise model to simulate the actual environment; then use the affine transformation method to change the object volume, the location of internal foreign bodies and increase the number of internal foreign bodies, use the region removal algorithm to reduce the number of internal foreign bodies, and generate the volume data and categories of all artificial samples; finally, use forward projection to create an artificial projection of the modified volume and save it to the output folder;

[0016] S5. Model testing and segmentation visualization:

[0017] By comparison, we selected the ResNet-50 network architecture with accelerated convergence characteristics, performed 500 iterations of training and validation on artificially generated sample data, minimized the difference in probability distribution between classification predictions and actual labels using cross entropy loss, and saved the best model for training and validation in the iteration as well as the validation accuracy, training and validation losses. We tested the best model and used PyTorch-CAM to create a class activation map for the model. We then overlaid CAM on the original image for visualization to improve and validate the model.

[0018] Preferably, in step S4, the mixed Poisson-Gaussian noise model has the formula:

[0019] I noisy =I true +Poisson(λ)+N(μ,δ 2 ),

[0020] In the formula, I true is the real image pixel value, Poisson(λ) is the Poisson noise term, N(μ,δ 2 ) is the Gaussian noise term.

[0021] Preferably, in step S5, the verification accuracy is calculated as follows:

[0022]

[0023] In the formula, y i is the sample label, Refers to the predicted category (number of internal foreign objects), is the indicator function, N val Represents the total number of images used for model validation.

[0024] Preferably, in step S5, the ratio of the number of samples in the training set to the number of samples in the validation set is tra :S val =25:5.

[0025] The beneficial effects of the present invention are:

[0026] (1) The method of the present invention can create a large amount of CT sample label data similar to real-world samples by obtaining industrial CT scan data of a small number of real samples of internal foreign matter, which can be used to train deep learning network models, effectively eliminating the two major obstacles of obtaining industrial CT scan data and label data; compared with the same number of real sample training, the method of the present invention greatly improves the accuracy of internal foreign matter classification.

[0027] (2) The method of the present invention reduces artifacts and improves reconstruction accuracy by using a beam hardening correction method during the 3D reconstruction process; uses a median filter to smooth the image without blurring the edges before volume segmentation, so that the boundaries of tissue structures can be better identified during the segmentation process, thereby improving the segmentation accuracy of Otsu threshold segmentation; in addition, mixed Poisson-Gaussian noise is added before sample deformation to simulate the actual environment, providing more realistic training data for model training.

[0028] (3) The method of the present invention further improves the accuracy of internal foreign body classification by applying artificial sample data generated by ResNet-50 network architecture training. Even only industrial CT scanning data of a real sample is needed. After the generated artificial sample is trained through the network model, the verification accuracy can reach 95.6%, and the test accuracy on the real sample is 87.6%. By applying PyTorch-CAM to create class activation map visualization for the model in the test phase, it is helpful to diagnose and improve model performance and optimize model design. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 : Sample graphs of three categories in the embodiment of the present invention;

[0030] Figure 2 : Comparison diagram before and after adding the noise model in the embodiment of the present invention;

[0031] Figure 3 : A generation flow chart based on a real sample in an embodiment of the present invention;

[0032] Figure 4 : Verification accuracy change curve diagram in the embodiment of the present invention (model trained with artificial data);

[0033] Figure 5 : Verification accuracy change curve diagram in the embodiment of the present invention (model trained with real data);

[0034] Figure 6 : Class activation map of internal foreign bodies of blocks 0, 1 and 2 in the embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention is further described below in conjunction with the embodiments. It should be noted that these are merely examples and illustrations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the claims, they should be deemed to fall within the protection scope of the present invention.

[0036] Embodiment 1:

[0037] like Figure 1-6As shown, an efficient internal foreign body classification method based on a small amount of CT scan data includes the following steps:

[0038] S1. Obtain CT scan data and preprocess:

[0039] Download the open source CT scan dataset of internal foreign bodies. The samples are plasticine with different amounts of gravel. Only the sample group with 0-3 gravel is selected for the experiment. According to the number of stones inside the plasticine, it is divided into 3 categories, such as Figure 1 The original X-ray projection data is sampled with a sampling rate of 20%, so the original 1800 scanned images of a sample are reduced to 360, and the projection data, bright field data, dark field data and geometric information of the sample are read, and then bright field and dark field corrections are applied to the projection data.

[0040] S2: Object 3D reconstruction:

[0041] Through the CT reconstruction algorithm, the 3D distribution of the attenuation coefficient within the sample volume is generated. The beam hardening correction method is applied to the CT scan image to reduce artifacts. Then, the FDK algorithm of the ASTRA toolbox is used to convert the data collected when the X-rays pass through the object into the attenuation coefficient distribution of each voxel in three-dimensional space. Finally, the reconstructed volume data is saved as a slice file.

[0042] S3: Segment the reconstructed volume:

[0043] Since each voxel represents a specific material, by applying the segmentation mask we can calculate the average intensity of each material in the reconstructed volume and thus extract the attenuation coefficient for each material.

[0044] The pixels of all slice files were normalized and converted to 8-bit unsigned integers, and then a median filter was applied to the segmented data to smooth the image without blurring the edges. This step helped to accurately classify the volume data and provide clear segmented material information for subsequent processing.

[0045] In the 3D volume, since foreign bodies and the matrix have obvious differences in attenuation characteristics, the Otsu threshold segmentation method can be used to segment internal foreign bodies. The Otsu method can automatically find the optimal threshold to maximize the inter-class variance between the foreground and background, thereby creating a mask to mark all voxels of the material of interest, obtain the volume data and categories of all segmented samples, and then save them to the segmentation result folder.

[0046] S4: Artificial sample generation:

[0047] First, select a representative sample in the segmented data (the number of internal foreign objects must be 1 or more), use the average strength of the material to convert the segmented volume back to the attenuated 3D distribution, obtain the volume data and category of all segmented samples, and then add a mixed Poisson-Gaussian noise model to simulate the actual environment. The formula of the mixed Poisson-Gaussian noise model is:

[0048] I noisy =I true +Poisson(λ)+N(μ,δ 2 ),

[0049] In the formula, I true is the real image pixel value, Poisson(λ) is the Poisson noise term, N(μ,δ 2 ) is the Gaussian noise term.

[0050] For the same segmented image, the effect of adding the mixed Poisson-Gaussian noise model to it is compared with the image without adding the noise model. Figure 2 shown.

[0051] Then, the sample shape is deformed by affine transformation, and the volume of the internal foreign matter is changed by affine transformation (including scaling, rotation, shearing and translation). At the same time, affine transformation is applied to change the position of the internal foreign matter and increase the number of internal foreign matter. At the same time, its morphological characteristics are maintained, and the number of internal foreign matter is reduced by replacing the internal foreign matter material with the segmented sample material, that is, the region removal algorithm, thereby creating a variety of new objects similar to real-world samples and automatically generating volume data and categories for all artificial samples. The generation flow chart based on a real sample is shown in the figure below: Figure 3 shown.

[0052] Finally, the new sample information is forward projected to create an artificial projection of the modified volume and saved to the output folder;

[0053] S5: Model testing and segmentation visualization:

[0054] S5.1. Network architecture selection and data training and verification:

[0055] Here, representative samples are selected from category 1 (i.e., the number of internal foreign bodies is 1), and 30 samples of volume shapes are generated. The ratio of the number of samples in the training set to the number of samples in the validation set is S tra :S val=25:5, where each volume shape sample can generate two other types of samples (i.e., the number of internal foreign bodies is 2 and 0), so a total of 30×3=90 samples are generated, and each sample has 72 X-ray virtual projection images, so a total of 90×72=6480 generated X-ray projection images are generated, of which the training image

[0056] N tra =5400, verification image N val =1080.

[0057] In addition, for comparison purposes, we selected 8 samples of category 1 (i.e., the number of internal foreign bodies is 1) for training and 2 samples for verification from the real projection data; 8 samples of category 2 (i.e., the number of internal foreign bodies is 2) for training and 2 samples for verification; and 8 samples of category 0 (i.e., the number of internal foreign bodies is 0) for training and 2 samples for verification. tra :S val =24:6, 72 projection photos are taken in each sample, a total of 24×72=1728 photos are used for real data training, and 6×72=432 photos are used for real data verification.

[0058] In both experiments, the ResNet-50 network architecture with accelerated convergence characteristics was used to train the artificially generated sample data, and the best effect was achieved. The algorithm deterministic deterministic was not enabled and the batch_size was selected as 4, which better increased the generalization ability of the model. 500 iterations were performed, and the cross-entropy loss backpropagation was used to minimize the difference in the probability distribution of the classification prediction and the actual label. The best model and verification accuracy, training and verification loss of the training verification in the iteration were saved.

[0059] Verification accuracy, calculated as:

[0060]

[0061] In the formula, y i is the sample label, Refers to the predicted category (number of internal foreign objects), is the indicator function, N val Represents the total number of images used for model validation.

[0062] The accuracy curve of the model trained with artificial data is as follows: Figure 4 The accuracy curve of the model trained with real data is shown in Figure 5 shown.

[0063] S5.2. Model testing and segmentation visualization:

[0064] Take real samples as the test set: category 1 (i.e., the number of internal foreign bodies is 1) is 10 samples, category 2 (i.e., the number of internal foreign bodies is 2) is 10 samples, category 0 (i.e., the number of internal foreign bodies is 0) is 10 samples, and 72 projection photos are taken for each sample, for a total of 30*72=2160 photos.

[0065] Use the best model obtained from artificial data training to test the overall accuracy The model trained with artificial samples generated by 1 real sample exceeds the test results of the model trained with 24 real samples (up to 73.1%). PyTorch-CAM is used to create class activation maps for the model, and CAM is superimposed on the original image for visualization to improve and verify the model. Create a class activation map for the ResNet-50 network model, such as Figure 6 shown.

[0066] (1) The present invention provides an efficient internal foreign body classification method based on a small amount of CT scan data. By obtaining industrial CT scan data of a small amount of real samples of internal foreign bodies, a large amount of CT sample label data similar to real-world samples can be created, which can be used to train deep learning network models, effectively eliminating the two major obstacles of obtaining industrial CT scan data and label data. Compared with training with the same number of real samples, this method greatly improves the accuracy of internal foreign body classification.

[0067] (2) The present invention provides an efficient internal foreign body classification method based on a small amount of CT scan data. The method reduces artifacts and improves reconstruction accuracy by using a beam hardening correction method in the 3D reconstruction process. A median filter is used to smooth the image without blurring the edges before volume segmentation, so that the boundaries of tissue structures can be better identified during the segmentation process, thereby improving the segmentation accuracy of Otsu threshold segmentation. In addition, mixed Poisson-Gaussian noise is added before sample deformation to simulate the actual environment, providing more realistic training data for model training.

[0068] (3) The present invention provides an efficient classification method for internal foreign bodies based on a small amount of CT scan data, which improves the method of generating artificial samples. The object volume is changed by affine transformation (including scaling, rotation, shearing and translation) and the like. At the same time, affine transformation is applied to change the position of internal foreign bodies and increase the number of internal foreign bodies. The number of internal foreign bodies is reduced by region removal algorithm, thereby creating a variety of new objects similar to real-world samples, and automatically generating volume data and categories of all artificial samples, which can be used to train deep learning network models.

[0069] (4) The present invention provides an efficient internal foreign body classification method based on a small amount of CT scan data. By applying the artificial sample data generated by ResNet-50 network architecture training, the accuracy of internal foreign body classification is further improved. Even only one real sample of industrial CT scan data is needed. After the generated artificial sample is trained through the network model, the verification accuracy can reach 95.6%, and the test accuracy on the real sample is 87.6%. By applying PyTorch-CAM to create a class activation map visualization for the model during the test phase, it is helpful to diagnose and improve model performance and optimize model design.

[0070] The above is an exemplary description of the invention. Obviously, the specific implementation of the present invention is not limited to the above-mentioned method. As long as such non-substantial improvements are made by adopting the method concept and technical solution of the present invention, or the concept and technical solution of the invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. An efficient internal foreign body classification method based on a small amount of CT scan data, characterized in that: The following steps are involved: S1. Obtain CT scan data and preprocess: Download an open-source CT scan dataset of foreign bodies inside objects and preprocess the X-ray projection data in the dataset, including bright field and dark field correction; S2. Object 3D reconstruction: The 3D distribution of the attenuation coefficient within the sample volume is generated through a CT reconstruction algorithm. The FDK algorithm in the ASTRA toolbox is used to convert the data collected when the X-rays pass through the object into the attenuation coefficient distribution of each voxel in three-dimensional space, and a beam hardening correction method is applied to reduce artifacts. S3. Segment the reconstructed volume: Apply a segmentation mask, calculate the average intensity of each material in the reconstructed volume, and extract the attenuation coefficient of each material. Apply a median filter to the segmented data to smooth the image without blurring the edges and accurately classify the volume data. Use the Otsu threshold segmentation method to segment internal foreign bodies, obtain the volume data and classification of all segmented samples, and save them to the segmentation result folder. S4. Artificial sample generation: Select a representative sample from the segmented data (the number of internal foreign objects must be 1 or more), use the average strength of the material to convert the segmented volume back to an attenuated 3D distribution, obtain the volume data and category of all segmented samples, and add a mixed Poisson-Gaussian noise model to simulate the actual environment. Then, use the affine transformation method to change the object volume, the location of internal foreign objects, and increase the number of internal foreign objects. Use the region removal algorithm to reduce the number of internal foreign objects, and generate the volume data and category of all artificial samples. Finally, use forward projection to create an artificial projection of the modified volume and save it to the output folder. S5. Model testing and segmentation visualization: By comparison, we selected the ResNet-50 network architecture with accelerated convergence characteristics, and performed 500 iterations of training and validation on artificially generated sample data. We used cross-entropy loss to minimize the difference in probability distribution between classification predictions and actual labels, and saved the best model for training and validation in the iteration, as well as the validation accuracy, training and validation losses. We used the best model for testing, and used PyTorch-CAM to create a class activation map for the model. The CAM was then overlaid on the original image for visualization to improve and validate the model.

2. The method for efficiently classifying internal foreign bodies based on a small amount of CT scan data according to claim 1, characterized in that: In step S4, the mixed Poisson-Gaussian noise model is formulated as follows: I noisy =I true +Poisson(λ)+N(μ,δ 2 ), Where, I true is the real image pixel value, Poisson(λ) is the Poisson noise term, N(μ,δ 2 ) is the Gaussian noise term.

3. The method for efficiently classifying internal foreign bodies based on a small amount of CT scan data according to claim 1, characterized in that: In step S5, the verification accuracy is calculated as follows: Where y i is the sample label, Refers to the predicted category (number of internal foreign objects), is the indicator function, N val Indicates the total number of images used for model validation.

4. The method for efficiently classifying internal foreign bodies based on a small amount of CT scan data according to claim 1, characterized in that: In step S5, the ratio of the number of samples in the training set to the number of samples in the validation set is S tra :S val =25:5.

Images