An efficient internal foreign body classification method based on a small amount of CT scan data

Abstract

The present invention discloses a method for efficiently classifying internal foreign bodies based on a small amount of CT scan data, comprising the following steps: S1, acquiring and preprocessing CT scan data, S2, 3D reconstruction of the object, S3, segmenting the reconstructed volume, S4, generating artificial samples, and S5, model testing and segmentation visualization. The method of the present invention effectively eliminates the two major obstacles of obtaining industrial CT scan data and label data, greatly reducing manpower and time costs. It can update and label data in real time during the production process, automatically generate volume data and categories of all artificial samples, and use them to train deep learning network models. Compared with training with the same number of real samples, the accuracy of internal foreign body classification is greatly improved. Even with only one real sample of industrial CT scan data required, the generated artificial sample can be trained through the network model, and the verification accuracy can reach 95.6%, with high classification efficiency and good reliability.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial CT image processing, and in particular to an efficient internal foreign body classification method based on a small amount of CT scanning data. Background Art

[0002] In modern industrial production, product quality control is a critical step in ensuring production efficiency and safety. With the increasing complexity of product structures, especially in areas requiring high precision and high reliability, the accurate and efficient classification of internal foreign matter has become indispensable. Failure to promptly sort these foreign matter can severely impact product performance and safety, and may even cause serious production accidents or quality issues. However, traditional methods for classifying internal foreign matter, such as manual semi-automatic inspection systems and manual visual inspection, are inefficient and susceptible to subjective interference, making them unable to meet the urgent needs of modern industry for high-speed, accurate classification.

[0003] In recent years, industrial CT (computed tomography) technology has become an important tool for inspecting and analyzing the internal structure of products due to its ability to quickly and non-invasively provide detailed three-dimensional images. Compared with traditional two-dimensional image inspection methods, industrial CT technology can present a panoramic view of the interior of a product, thereby identifying potential internal foreign objects. However, the amount of data generated by industrial CT is huge and complex, placing higher demands on data processing and analysis. At the same time, the rapid development of deep learning technology, especially ResNet-50, with its unique residual structure, deep network processing capabilities, and high accuracy in image classification, is showing great application prospects in the classification of internal foreign objects in the field of industrial CT. The ResNet-50 network effectively solves the gradient vanishing problem in deep networks through the design of residual connections, improving the model's expressiveness and accuracy. This enables automated inspection methods based on deep learning to effectively identify and classify complex internal foreign objects, surpassing the limitations of traditional unsupervised methods.

[0004] Currently, despite the significant theoretical advantages of deep learning technology, the high cost and time consumption of acquiring high-quality labeled industrial CT image data have become a bottleneck in its application. Collecting and labeling large amounts of labeled data requires significant manpower and time, and real-time data updates and labeling cannot be performed during the production process. Furthermore, automatic classification of internal foreign matter is costly, inaccurate, inefficient, and unreliable. Summary of the Invention

[0005] The purpose of the present invention is to provide an efficient internal foreign body classification method based on a small amount of CT scan data to solve the above defects.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] An efficient internal foreign body classification method based on a small amount of CT scan data comprises the following steps:

[0008] S1. Obtain CT scan data and preprocess:

[0009] Download an open-source CT scan dataset of foreign bodies inside objects and preprocess the X-ray projection data in the dataset, including bright field and dark field correction;

[0010] S2. Object 3D reconstruction:

[0011] The 3D distribution of the attenuation coefficient within the sample volume is generated through a CT reconstruction algorithm. The FDK algorithm in the ASTRA toolbox is used to convert the data collected when the X-rays pass through the object into the attenuation coefficient distribution of each voxel in three-dimensional space, and a beam hardening correction method is applied to reduce artifacts.

[0012] S3. Segment the reconstructed volume:

[0013] Apply a segmentation mask, calculate the average intensity of each material in the reconstructed volume, and extract the attenuation coefficient of each material. Apply a median filter to the segmented data to smooth the image without blurring the edges and accurately classify the volume data. Use the Otsu threshold segmentation method to segment internal foreign bodies, obtain the volume data and classification of all segmented samples, and save them to the segmentation result folder.

[0014] S4. Artificial sample generation:

[0015] Select a representative sample from the segmented data (the number of internal foreign objects must be 1 or more), use the average strength of the material to convert the segmented volume back to an attenuated 3D distribution, obtain the volume data and category of all segmented samples, and add a mixed Poisson-Gaussian noise model to simulate the actual environment. Then, use the affine transformation method to change the object volume, the location of internal foreign objects, and increase the number of internal foreign objects. Use the region removal algorithm to reduce the number of internal foreign objects, and generate the volume data and category of all artificial samples. Finally, use forward projection to create an artificial projection of the modified volume and save it to the output folder.

[0016] S5. Model testing and segmentation visualization:

[0017] By comparison, we selected the ResNet-50 network architecture with accelerated convergence characteristics, and performed 500 iterations of training and validation on artificially generated sample data. We used cross-entropy loss to minimize the difference in probability distribution between classification predictions and actual labels, and saved the best model for training and validation in the iteration, as well as the validation accuracy, training and validation losses. We used the best model for testing, and used PyTorch-CAM to create a class activation map for the model. The CAM was then overlaid on the original image for visualization to improve and validate the model.

[0018] Preferably, in step S4, the mixed Poisson-Gaussian noise model has the formula:

[0019] I noisy =I true +Poisson(λ)+N(μ,δ 2 ),

[0020] Where, I true is the real image pixel value, Poisson(λ) is the Poisson noise term, N(μ,δ 2 ) is the Gaussian noise term.

[0021] Preferably, in step S5, the verification accuracy is calculated as follows:

[0022]

[0023] Where y i is the sample label, Refers to the predicted category (number of internal foreign objects), is the indicator function, N val Indicates the total number of images used for model validation.

[0024] Preferably, in step S5, the ratio of the number of samples in the training set to the number of samples in the validation set is S tra :S val =25:5.

[0025] The beneficial effects of the present invention are:

[0026] (1) The method of the present invention can create a large amount of CT sample label data similar to real-world samples by obtaining industrial CT scan data of a small number of real samples of internal foreign matter, which can be used to train deep learning network models, effectively eliminating the two major obstacles of obtaining industrial CT scan data and label data; compared with training with the same number of real samples, the method of the present invention greatly improves the accuracy of internal foreign matter classification.

[0027] (2) The method of the present invention reduces artifacts and improves reconstruction accuracy by applying a beam hardening correction method during the 3D reconstruction process; uses a median filter to smooth the image without blurring the edges before volume segmentation, so that the boundaries of tissue structures can be better identified during the segmentation process, improving the segmentation accuracy for Otsu threshold segmentation; in addition, mixed Poisson-Gaussian noise is added before sample deformation to simulate the actual environment, providing more realistic training data for model training.

[0028] (3) The method of the present invention further improves the accuracy of internal foreign body classification by applying artificial sample data generated by training with the ResNet-50 network architecture. Even with only industrial CT scan data of a real sample, the generated artificial sample can be trained with the network model, and the verification accuracy can reach 95.6%. The test accuracy on the real sample is 87.6%. By applying PyTorch-CAM to create a class activation map visualization for the model during the test phase, it is helpful to diagnose and improve model performance and optimize model design. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 : Sample images of three categories in the embodiment of the present invention;

[0030] Figure 2 : Comparison diagram before and after adding noise model in the embodiment of the present invention;

[0031] Figure 3 : A generation flow chart based on a real sample in an embodiment of the present invention;

[0032] Figure 4 : Verification accuracy change curve diagram in the embodiment of the present invention (model trained with artificial data);

[0033] Figure 5 : Verification accuracy change curve diagram in the embodiment of the present invention (model trained with real data);

[0034] Figure 6 : Class activation maps of internal foreign objects in blocks 0, 1, and 2 in the embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention is further described below with reference to the embodiments. It should be noted that these are merely examples and illustrations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the claims, they should be deemed to fall within the scope of protection of the present invention.

[0036] Example 1:

[0037] like Figure 1-6As shown, an efficient internal foreign body classification method based on a small amount of CT scan data includes the following steps:

[0038] S1. Obtain CT scan data and preprocess:

[0039] Download the open source CT scan dataset of internal foreign bodies. The sample is plasticine with different amounts of gravel. Only the sample group with 0-3 gravel is selected for the experiment. The sample group is divided into three categories according to the number of stones inside the plasticine, such as Figure 1 As shown in the figure, the original X-ray projection data is sampled at a sampling rate of 20%, so the original 1800 scanned images of a sample are reduced to 360. The projection data, bright field data, dark field data and geometric information of the sample are read, and then bright field and dark field corrections are applied to the projection data.

[0040] S2: Object 3D reconstruction:

[0041] Through the CT reconstruction algorithm, the 3D distribution of the attenuation coefficient within the measured sample volume is generated. The beam hardening correction method is applied to the CT scan image to reduce artifacts. Then, the FDK algorithm of the ASTRA toolbox is used to convert the data collected when X-rays pass through the object into the attenuation coefficient distribution of each voxel in three-dimensional space. Finally, the reconstructed volume data is saved as a slice file.

[0042] S3: Segment the reconstructed volume:

[0043] Since each voxel represents a specific material, by applying the segmentation mask, we can calculate the average intensity of each material in the reconstructed volume and thus extract the attenuation coefficient of each material.

[0044] The pixels of all slice files are normalized and converted to 8-bit unsigned integers. Then, a median filter is applied to the segmentation data to smooth the image without blurring the edges. This step helps to accurately classify the volume data and provide clear segmentation material information for subsequent processing.

[0045] In the 3D volume, since foreign bodies and the matrix have obvious differences in attenuation characteristics, the Otsu threshold segmentation method can be used to segment internal foreign bodies. The Otsu method can automatically find the optimal threshold to maximize the inter-class variance between the foreground and background, thereby creating a mask to mark all voxels of the material of interest, obtain the volume data and category of all segmented samples, and then save them to the segmentation result folder.

[0046] S4: Artificial sample generation:

[0047] First, a representative sample is selected from the segmented data (the number of internal foreign objects must be 1 or more), and the segmented volume is converted back to the attenuated 3D distribution using the average strength of the material. The volume data and categories of all segmented samples are obtained, and then a mixed Poisson-Gaussian noise model is added to simulate the actual environment. The formula of the mixed Poisson-Gaussian noise model is:

[0048] I noisy =I true +Poisson(λ)+N(μ,δ 2 ),

[0049] Where, I true is the real image pixel value, Poisson(λ) is the Poisson noise term, N(μ,δ 2 ) is the Gaussian noise term.

[0050] For the same segmented image, the effect of adding the mixed Poisson-Gaussian noise model to it is compared with the image without adding the noise model. Figure 2 shown.

[0051] Then, the sample shape is deformed by affine transformation, and the volume of the internal foreign matter is changed by affine transformation (including scaling, rotation, shearing and translation). At the same time, affine transformation is applied to change the position of the internal foreign matter and increase the number of internal foreign matter. At the same time, its morphological characteristics are maintained, and the number of internal foreign matter is reduced by replacing the internal foreign matter material with the segmented sample material, that is, the region removal algorithm, thereby creating a variety of new objects similar to real-world samples and automatically generating volume data and categories for all artificial samples. The generation flow chart based on a real sample is as follows: Figure 3 shown.

[0052] Finally, the new sample information is forward projected to create an artificial projection of the modified volume and saved to the output folder;

[0053] S5: Model testing and segmentation visualization:

[0054] S5.1. Network architecture selection and data training and verification:

[0055] Here, the representative sample is selected from category 1 (i.e., the number of internal foreign bodies is 1), and 30 samples of volume and shape are generated. The ratio of the number of samples in the training set to the number of samples in the validation set is S tra :S val=25:5, where each volume shape sample can generate two other types of samples (i.e., the number of internal foreign matter is 2 and 0), thus generating a total of 30×3=90 samples, and each sample has 72 X-ray virtual projection images, thus generating a total of 90×72=6480 generated X-ray projection images, of which the training images

[0056] N tra =5400, verification image N val =1080.

[0057] In addition, for comparison, we selected 8 samples of category 1 (i.e., the number of internal foreign bodies is 1) for training and 2 samples for verification from the real projection data; 8 samples of category 2 (i.e., the number of internal foreign bodies is 2) for training and 2 samples for verification; 8 samples of category 0 (i.e., the number of internal foreign bodies is 0) for training and 2 samples for verification, i.e., S tra :S val =24:6, 72 projection photos are taken in each sample, a total of 24×72=1728 photos are used for real data training, and 6×72=432 photos are used for real data verification.

[0058] In both experiments, the ResNet-50 network architecture with accelerated convergence characteristics was used to train the artificially generated sample data for the best results. In both experiments, deterministic algorithm was not enabled and batch_size was selected as 4, which better improved the generalization ability of the model. 500 iterations were performed, and cross-entropy loss backpropagation was used to minimize the difference in the probability distribution of classification predictions and actual labels. The best model and verification accuracy, training and verification losses of the training and verification iterations were saved.

[0059] Verification accuracy is calculated as follows:

[0060]

[0061] Where y i is the sample label, Refers to the predicted category (number of internal foreign objects), is the indicator function, N val Indicates the total number of images used for model validation.

[0062] The accuracy curve of the model trained with artificial data is as follows Figure 4 The accuracy curve of the model trained with real data is shown in Figure 5 shown.

[0063] S5.2. Model testing and segmentation visualization:

[0064] Take real samples as the test set: category 1 (i.e., the number of internal foreign bodies is 1) is 10 samples, category 2 (i.e., the number of internal foreign bodies is 2) is 10 samples, category 0 (i.e., the number of internal foreign bodies is 0) is 10 samples, and 72 projection photos are taken for each sample, for a total of 30*72=2160 photos.

[0065] Use the best model obtained from artificial data training to test the overall accuracy The model trained with artificial samples generated from 1 real sample outperformed the model trained with 24 real samples (up to 73.1%). PyTorch-CAM was used to create class activation maps for the model, and CAM was superimposed on the original image for visualization to improve and verify the model. Class activation maps were created for the ResNet-50 network model, such as Figure 6 shown.

[0066] (1) The present invention provides an efficient internal foreign body classification method based on a small amount of CT scan data. By obtaining industrial CT scan data of a small number of real samples of internal foreign bodies, a large amount of CT sample label data similar to real-world samples can be created, which can be used to train deep learning network models, effectively eliminating the two major obstacles of obtaining industrial CT scan data and label data. Compared with training with the same number of real samples, this method greatly improves the accuracy of internal foreign body classification.

[0067] (2) The present invention provides an efficient internal foreign body classification method based on a small amount of CT scan data. The method reduces artifacts and improves reconstruction accuracy by using a beam hardening correction method during 3D reconstruction. A median filter is used to smooth the image without blurring the edges before volume segmentation, so that the boundaries of tissue structures can be better identified during the segmentation process, thereby improving the segmentation accuracy of Otsu threshold segmentation. In addition, mixed Poisson-Gaussian noise is added before sample deformation to simulate the actual environment, providing more realistic training data for model training.

[0068] (3) The present invention provides an efficient classification method for internal foreign bodies based on a small amount of CT scan data, which improves the method of generating artificial samples. The volume of the object is changed by affine transformation (including scaling, rotation, shearing and translation), and the position of the internal foreign bodies is changed by affine transformation and the number of internal foreign bodies is increased. The number of internal foreign bodies is reduced by region removal algorithm, thereby creating a variety of new objects similar to real-world samples, and automatically generating volume data and categories of all artificial samples, which can be used to train deep learning network models.

[0069] (4) The present invention provides an efficient internal foreign body classification method based on a small amount of CT scan data. By applying the ResNet-50 network architecture to train the artificial sample data generated, the accuracy of internal foreign body classification is further improved. Even with only one real sample of industrial CT scan data, the generated artificial sample can be trained with the network model, and the verification accuracy can reach 95.6%. The test accuracy on the real sample is 87.6%. By applying PyTorch-CAM to create a class activation map visualization for the model in the test phase, it is helpful to diagnose and improve model performance and optimize model design.

[0070] The above is an exemplary description of the invention. Obviously, the specific implementation of the present invention is not limited to the above-mentioned method. As long as such non-substantial improvements are made by adopting the method concept and technical solution of the present invention, or the concept and technical solution of the invention are directly applied to other occasions without improvement, they are all within the scope of protection of the present invention.

Claims

1. An efficient internal foreign body classification method based on CT scan data, characterized in that: The following steps are involved: S1. Obtain CT scan data and preprocess: Download an open-source CT scan dataset of foreign bodies inside objects and preprocess the X-ray projection data in the dataset, including bright field and dark field correction; S2. Object 3D reconstruction: The 3D distribution of the attenuation coefficient within the sample volume is generated using a CT reconstruction algorithm. The FDK algorithm in the ASTRA toolbox is used to convert the data collected when X-rays pass through the object into the attenuation coefficient distribution of each voxel in three-dimensional space, and a beam hardening correction method is applied to reduce artifacts. S3. Segment the reconstructed volume: Apply a segmentation mask, calculate the average intensity of each material in the reconstructed volume, and extract the attenuation coefficient of each material. Apply a median filter to the segmented data to smooth the image without blurring the edges and accurately classify the volume data. Use the Otsu threshold segmentation method to segment internal foreign bodies, obtain the volume data and classification of all segmented samples, and save them to the segmentation result folder. S4. Artificial sample generation: A representative sample is selected from the segmented data. The representative sample must have at least one internal foreign object. The segmented volume is converted back to an attenuated 3D distribution using the average strength of the material. The volume data and categories of all segmented samples are obtained. A mixed Poisson-Gaussian noise model is added to simulate the actual environment. An affine transformation method is then used to change the object volume, the location of internal foreign objects, and the number of internal foreign objects. The region removal algorithm is used to reduce the number of internal foreign objects. The volume data and categories of all artificial samples are generated. Finally, a forward projection is used to create an artificial projection of the modified volume and save it to the output folder. S5. Model testing and segmentation visualization: By comparison, we selected the ResNet-50 network architecture with accelerated convergence characteristics, and performed 500 iterations of training and validation on artificially generated sample data. We used cross-entropy loss to minimize the difference in probability distribution between classification predictions and actual labels, and saved the best model for training and validation in the iteration, as well as the validation accuracy, training and validation losses. We used the best model for testing, and used PyTorch-CAM to create a class activation map for the model. The CAM was then superimposed on the original image for visualization to improve and validate the model.

2. The method for efficiently classifying internal foreign bodies based on CT scan data according to claim 1, characterized in that: In step S4, the mixed Poisson-Gaussian noise model is formulated as follows: ， Where, is the real image pixel value, is the Poisson noise term, is the Gaussian noise term.

3. The efficient internal foreign body classification method based on CT scan data according to claim 1, characterized in that: In step S5, the verification accuracy is calculated as follows: Where, is the sample label, Refers to the predicted category, is the indicator function, Indicates the total number of images used for model validation.

4. The method for efficiently classifying internal foreign bodies based on CT scan data according to claim 1, characterized in that: In step S5, the ratio of the number of samples in the training set to the number of samples in the validation set is .

Images