A method for correcting rotational blurring of cl projection

Abstract

The application discloses a method for correcting CL projection rotation blur, and the steps include: 1) generating a plurality of PCB simulation models; 2) generating n equidistant angles with the rotation axis as the center; 3) respectively taking each angle as the center, simulating the projection of each model uniformly within a set angle I° range, and generating n projection images without motion blur for each model; 4) respectively superimposing the projection images within J° and K° ranges with each angle as the center, and continuously generating a plurality of corresponding J° rotation blur projection images and K° rotation blur projection images; taking the projection images corresponding to the angle center as the labels of each J° rotation blur projection image and K° rotation blur projection image, thereby generating two groups of training samples for the angle; 5) training a full convolution deblurring network by using the training samples; and 6) inputting a to-be-processed projection image into the trained full convolution deblurring network for processing, and obtaining a corrected projection image.

Description

Technical Field

[0001] This invention belongs to the field of X-ray computed tomography (CL) nondestructive testing technology, and specifically relates to a method for correcting rotational blur in CL projection data. Background Technology

[0002] With the continuous development of modern microelectronics technology and large-scale integrated circuits, the layer density and wiring complexity of printed circuit boards (PCBs), which serve as carriers for various electronic devices and chips, are increasing. This places higher demands on PCB processing accuracy and yield. Therefore, defect detection technology for PCBs, especially non-destructive testing technology, has become a crucial link in ensuring PCB processing quality and improving processing technology.

[0003] In the field of nondestructive testing, X-ray computed tomography (CT) is one of the mainstream detection technologies. Its basic principle is that X-rays penetrate the object being inspected. Due to the different absorption rates of different materials, the X-rays passing through the object leave counts of varying intensities on pixels at different locations on the detector. All the detector pixel counts are arranged together to form a projected image. CT acquires projected images at different angles by rotating the X-ray source and detector relative to the object, and then uses reconstruction algorithms (such as FDK) to reconstruct the internal information of the object.

[0004] When CT scans PCBs, due to their flat shape, the X-ray source cannot get too close to the object to avoid collisions during rotation, making it difficult to increase the magnification ratio. Therefore, the object can only be reconstructed at a lower resolution, which is insufficient to meet the precision requirements of PCBs. To solve this problem, computed tomography (CL) technology has gradually developed. Compared to CT, CL's special scanning geometry ensures that the X-ray source remains close to the object, achieving high-precision imaging and reconstruction. Among them, the X-ray three-dimensional tomography system developed by the Institute of High Energy Physics, Chinese Academy of Sciences, can achieve high-resolution three-dimensional scanning of a series of plate-shaped objects such as PCBs, IGBTs, and paleontological fossils, basically solving industrial needs. The structure of the X-ray three-dimensional tomography system is as follows: Figure 1 As shown.

[0005] Due to the limitations of the imaging physical process, detector imaging is not instantaneous; it takes time to acquire sufficient X-ray counts. Therefore, there are two scanning methods: step scanning and continuous scanning. Step scanning involves rotating the X-ray source and detector (or object) to a designated position beforehand, allowing them to remain stationary while the detector completes imaging, before moving further. Continuous scanning, on the other hand, involves the object rotating at a constant speed during the scanning process, maintaining its rotational motion even during the brief imaging time. It can be observed that continuous scanning is faster, typically taking less than half the time of step scanning, a high efficiency particularly important for industrial inspection. However, the projected image under continuous scanning exhibits significant rotational blurring as the counts accumulate, severely impacting the quality of the reconstructed image. To acquire high-quality projected images while maintaining high-efficiency scanning, removing rotational blur from the projected image is essential.

[0006] Removing rotational blur from CL projection images is essentially a process of image deblurring. With the advancements in deep learning within computer vision, numerous deep learning-based deblurring studies have yielded excellent results, outperforming traditional algorithms in both deblurring effectiveness and computational efficiency. Examples include MIMO-UNet proposed by Sung-Jin Cho et al. in 2021 and Restormer proposed by Syed Waqas Zamir et al. in 2022, both classic image deblurring networks. However, these classic networks are designed for linear motion blur in color images. For the task of removing rotational blur from CL projection images, a redesign of the network is necessary, taking into account the characteristics of the projection data itself.

[0007] Network training requires a large amount of training data, and the quantity and quality of this data directly affect the model's performance in real-world applications. However, obtaining training data with CL projection is difficult. To ensure the training data encompasses as many scenarios as possible, a single CL device needs to spend a significant amount of time scanning numerous different objects with varying parameters, which is impractical. The biggest advantage of simulation data is its unlimited availability of any type and quantity of training data, eliminating tedious manual operations, generating data extremely quickly, and requiring no specialized equipment. The main problem with simulation data is its different distribution compared to real data. Networks trained entirely on simulation data often cannot be applied to real-world data. Currently, many networks are trained using a combination of real and simulation data. Summary of the Invention

[0008] Compared to traditional motion blur, the most significant characteristic of CL projection images is their uniform and continuous image sequence. The purpose of this invention is to design and train a fully convolutional deblurring network model based on continuous frames, which can efficiently remove rotational blur from CL. The trained network model can directly process CL projection sequences without the need for intervention from other information.

[0009] The first major aspect of this invention is the network design. Based on the fully convolutional encoder-decoder network structure, this invention incorporates an input module that can utilize multi-frame projection sequences. Verification has shown that this input module significantly improves the network's training performance and generalization ability. Simultaneously, modules such as multi-scale feature extraction, residual convolution, and cross-connects are introduced into the network to enhance its training efficiency and generalization capabilities. The main structure of the network is as follows: Figure 2 As shown.

[0010] The second main aspect of this invention is the acquisition of training data. This invention directly uses fully simulated data for training. The strategy for generating simulated data directly affects the network's performance on real data, and is therefore a crucial part of this invention. Generating training data first requires generating simulated phantoms with different structures. Then, these phantoms are projected under various conditions to simulate different devices, different ray intensities, and different objects, in order to obtain training data that is as complete as possible. The rotation interval of the projection needs to be as small as possible, simulating the effect of rotational blurring by superimposing projections from adjacent angles.

[0011] The technical solution of this invention is as follows:

[0012] A method for correcting rotational blur in CL projection data, comprising the following steps:

[0013] (1) Generate a simulation model with a random shape, set different attenuation coefficients for different materials, and let the attenuation coefficients change randomly within a small range to simulate the non-uniform state of the material.

[0014] (2) Figure 1 As shown, a random starting angle is selected within a range of 0 to 360° centered on the axis of rotation. Using this starting angle as an endpoint, n equally spaced angles are generated within a range of 180°.

[0015] (3) Perform the following operation on each simulation phantom generated in step (1): Using the n angles generated in step (2) as n centers, uniformly simulate projection m times within an angle range of 6°. For each angle, generate m projection images without motion blur. A total of n×m projection images without motion blur are generated for each simulation phantom.

[0016] (4) Figure 3As shown, the projection image without motion blur generated in step (3) is processed with each angle selected in step (2) as the center x. Figure 3 As shown in (a), projected images within a 2° range are added together, centered at x-2, x, and x+2. The summed image values ​​are then divided by the number of images added to obtain projection data p1, p2, and p3 for continuous angles with 2° rotational blur. Figure 3 As shown in (b), projected images within a 1° range are added together, centered at x-1, x, and x+1. The summed image values ​​are divided by the number of images added to obtain projection data p1, p2, and p3 for consecutive angles with 1° rotational blur. The clear projected image located at the selected angle x serves as the corresponding label for the 1° blurred image and the 2° blurred image. A clear projected image and three blurred projected images at consecutive angles (i.e., p1, p2, and p3) form a data pair. In this step, each selected angle of each simulation phantom can generate two sets of data pairs: one with 1° blur and one with 2° blur. Therefore, each phantom can generate a total of n×2 sets of data pairs.

[0017] (5) Each modulus can generate n×2 sets of data in step (4). Each set of data is downsampled by 2x and 4x respectively, which expands the amount of data and improves the network's generalization ability to projected images of different resolutions. After this step, each modulus generates a total of n×2×3 sets of data.

[0018] (6) Add random types and intensities of noise to each data pair to mitigate the mismatch between simulated and real data caused by different noise distributions. To make the network focus more on the areas of rotational blurring rather than the noise, the noise in the clear image used as the label must be completely identical to the noise in the blurred image p2, i.e. Where s represents the label data, s 噪 For the labeled data with added noise, p2 is the middle image of one of the three blurred projected images. The image after adding noise to p2.

[0019] (7) Repeat steps (1)-(6) until a sufficient number of data pairs are generated, which are divided into training set and validation set.

[0020] (8) Based on the characteristics of CL projection, design a fully convolutional deblurring network model based on continuous projection.

[0021] (9) Select an appropriate loss function, train the network and validate it.

[0022] Furthermore, the continuous sequence length used here is 3, meaning that the blurred projection and the one before and after it need to be removed. Therefore, each data pair consists of one label image and three blurred images from consecutive angles. The training set and the real dataset are not the same. To simplify the data organization and facilitate fast reading during training, the three blurred images in the training set are saved as a single RGB image, and the corresponding label is saved as a grayscale image. In the real dataset, each projection is saved independently and needs to be read separately and temporarily combined into a three-channel data input when needed.

[0023] Furthermore, when simulating projection onto the phantom, the projection parameters (such as SOD, SDD, and pixel size) are randomized, reducing the network's dependence on different device parameters.

[0024] Furthermore, random noise added to the projected data includes Gaussian noise, Poisson noise, and a combination of the two.

[0025] The present invention also provides a server, characterized in that it includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the above-described method.

[0026] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the above-described method.

[0027] The advantages of this invention are as follows:

[0028] (1) This invention proposes a feasible solution for the problem of rotational blur correction of CL for the first time, and it has been verified that it can achieve a good deblurring effect.

[0029] (2) This invention utilizes three consecutive blurred projection images at different angles to directly extract the location and size of the blur in the multi-angle input module. It can also reduce the interference to the network caused by differences in the internal structure of objects and the size of the rotation angle. It can simultaneously correct the rotation blur of multiple angles and multiple objects.

[0030] (3) The present invention achieves good results in real data by using fully simulated data for training, further verifying the excellent generalization ability of the designed network. The fully simulated method can not only generate specific training data according to the application needs using specific simulated objects and scan geometry, but also eliminates the need for real CL equipment and cumbersome manual operation. Only a computer is needed, and training data can be obtained automatically in a few days or even a few hours, which greatly improves the training efficiency of the model.

[0031] (4) This invention only processes projection data and does not need to consider the influence of various variables such as projection equipment and projection parameters, and has a wide range of applicability. Attached Figure Description

[0032] Figure 1 This is the structure of an X-ray three-dimensional layered imaging system.

[0033] Figure 2 This forms the main structure of the network.

[0034] Figure 3 This is a schematic diagram illustrating the generation of a simulated rotating blurred projection.

[0035] (a) is the projection data for generating a 2° rotational blur, and (b) is the projection data for generating a 1° rotational blur.

[0036] Figure 4 This is a multi-angle input module.

[0037] Figure 5 This is the feature fusion module.

[0038] Figure 6 This is the residual module.

[0039] Figure 7 For downsampling modules;

[0040] (a) shows the structure of the first, third, and fourth downsampling modules, and (b) shows the structure of the second downsampling module.

[0041] Figure 8 This is the upsampling module. Detailed Implementation

[0042] The following describes the invention in further detail. The examples given are for illustrative purposes only and are not intended to limit the scope of the invention.

[0043] This invention provides a method for removing rotational blur from CL projection data. The steps are described below.

[0044] 1. Network Module Design

[0045] This invention uses a fully convolutional neural network. The network primarily consists of residuals and convolutional blocks, and also includes a multi-angle input module, a feature fusion module, an upsampling module, and a downsampling module. The output data dimensions of each module are already specified. Figure 2 Mark the corresponding module around it.

[0046] like Figure 2As shown, the input data dimension of the multi-angle input module is h×w×3, meaning the size of the projection data is h×w, and the input consists of three consecutive angled, rotated, blurred projection images. The output data dimension of the multi-angle input module is h×w×c, where c represents the number of basic channels in the network, which is set to 32 in this experiment. The specific structure of the multi-angle input module is as follows: Figure 4 As shown, taking the central image p2 as the reference, the difference is calculated with the adjacent images p1 and p3 respectively. The resulting difference image contains information such as the position and size of the rotation blur. Then, the difference image is inserted between p1, p2, and p3 to generate h×w×5 data. This data is then processed through three convolutional layers for feature extraction, with the feature data dimension being h×w×(c-5). Finally, the data is concatenated with the data before convolution in the channel dimension to form a whole h×w×c, obtaining the fused data as the input to the next module. The specific structure of the feature fusion module is as follows: Figure 5 As shown, this module's function is to fuse the data that has undergone feature extraction and downsampling by the multi-angle fusion module, convolution module, and residual module in the previous stage with the data directly downsampled after passing through the multi-angle fusion module, increasing the fluidity of information across different scales. The two input data of the feature fusion module have the same dimension as the output data, thus not increasing the data volume for subsequent modules and controlling the computational load of the network while fusing information. Figure 6 As shown, the residual module mainly consists of several 3×3 convolutions and ReLU activation functions, with a cross-connection every two convolution layers. Figure 7 As shown, the downsampling module is implemented through pixel unshuffle and convolution. Pixel unshuffle only changes the position of elements in the matrix, without changing the number of elements. Then, a 3×3 convolution is used to change the number of channels in the output data. Figure 8 As shown, the upsampling module is mainly implemented by deconvolution + convolution, where deconvolution is used to increase the data dimension and convolution is used to change the number of channels in the output data.

[0047] 2. Simulation training data generation

[0048] The training data for this invention is entirely generated through simulation. The specific simulation steps are as follows:

[0049] (1) Random Simulation Mold: The 3D PCB molds generated here are all randomly generated, including insulating boards, copper traces, copper pillars and vias, solder joints, and air. First, the trace layers of the mold to be generated are randomly obtained, usually 3-6 layers. According to the number of trace layers, a fixed-size area is randomly cut from the randomly obtained circuit diagram to serve as the circuit of the trace layer. Insulating layers and full copper layers of random thickness are added between the trace layers. Then, a random number of copper pillars and vias and solder joints are added at random positions. Finally, different materials are assigned their own attenuation coefficients to complete the generation of a simulation mold.

[0050] (2) Simulated Projection with Random Parameters: The parameters used in random projection are based on fixed parameters with added random fluctuations, such as the SOD and SDD of the CL, the detector tilt angle, the detector pixel size, and the reconstructed voxel size. Each phantom has its unique scanning parameters to avoid network dependence on a single parameter. After determining the projection parameters, the object is projected in steps of 0.1°. Based on the actual detector parameters, the size of each simulated projection image is uniformly 2304×2304, and the projection data at this time has no rotational blur. According to the method described above, 12 angles are selected as centers for each phantom, and two sets of data pairs with 1° blur and 2° blur are generated for each angle.

[0051] (3) Projection expansion and combination: The projected size generated after steps (1) and (2) is 2304×2304. In order to expand the training data and to ensure that the network can have a good deblurring effect on images with different resolutions, the projection is downsampled by 2 times and 4 times.

[0052] (4) Adding Random Noise: Random noise needs to be added to the simulated projections, which is a crucial step in enhancing the network's generalization ability. In reality, the noise in projection data is affected by various factors, including energy, scattering, hardening, and electronic noise. Different devices and scanning conditions can lead to different noise distributions. Therefore, to reduce the network's sensitivity to noise distribution, random noise is added to the projection data. The basic noise consists of Gaussian and Poisson noise, and a random superposition of the two noises is added to different projections. For rotational blur, this blur increases with the rotation radius. In most areas near the center, the rotational blur is almost negligible. To allow the network to focus on the rotational blur rather than the noise, the label projection needs to have noise that is exactly the same as the blurred projection.

[0053] (5) Then, perform grayscale mapping on the three projections of adjacent angles, that is, map the floating-point values ​​of the projection images to the range of 0 to 65535, and save the three projections as one image as RGB channels. The corresponding labels are saved as grayscale images with the same name. This step can simplify the data organization, transforming the original 3-to-1 data into a 1-to-1 format.

[0054] Repeat the above steps until a sufficient amount of training data has been collected.

[0055] 3. Network training and prediction

[0056] The network was trained using simulation data, with a total of 7200 training datasets. The initial learning rate was 1.6e-4, the batch size was 8, and the training lasted for 32 epochs. Random pruning and random flipping were used to augment the data during training. The loss function used was a hybrid L1+SSIM loss function. SSIM stands for Structural Similarity, and its formula is... The formula for the SSIM loss function is L SSIM =1-SSIM(img pred ,img label ). Where μ x and μ y σ is the pixel mean of the image at x and y. x and σ y σ is the pixel variance of image x and y. xy Let c1 and c2 be the pixel covariance of the image x and y, where c1 and c2 are constants, and c1 = (0.01 × L). 2 c2 = (0.03 × L) 2 L represents the grayscale range of the image pixels, and img pred It is the network output image, img label It is a labeled image. The formula for the L1 loss function is: Where N is the number of pixels in the image. The final mixture loss function formula is: Based on experience, the weighting factor here is α = 0.84.

[0057] After the network is trained, the optimal set of parameters is selected to predict the CL rotation blur projection, and the corrected projection image can be obtained.

[0058] 4. Correct rotational blur and reconstruct

[0059] For a set of real-world projected images with rotational blur, each blurred projected image to be corrected, along with two blurred projected images at adjacent angles, are input into the network for processing. The network output is the corrected projected image. Reconstruction is performed using the Filtered Back Projection (FDK) algorithm. Comparing the reconstruction results before and after deblurring with the unblurred label data reveals that the edges of the reconstructed image predicted by the network are clearer and closer to the reconstruction result without rotational blur.

[0060] Although specific embodiments of the invention have been disclosed for illustrative purposes to aid in understanding and implementing the invention, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the invention should not be limited to the content disclosed in the preferred embodiments, and the scope of protection claimed by the invention is defined by the claims.

Claims

1. A method for correcting rotational blur in CL projection, comprising the following steps: 1) Generate multiple PCB simulation models and set different attenuation coefficients for different materials in each PCB simulation model; 2) Randomly select a starting angle within the range of 0~360° centered on the axis of rotation; using the starting angle as the endpoint, generate a total of n equally spaced angles within a range of 180°; 3) Using each angle as the center, uniformly simulate and project each PCB simulation model within a set angle range of I°, generating n projection images without motion blur for each PCB simulation model. 4) Generate 2n training samples for each PCB simulation model. The method is as follows: For each angle, let the center degree of the angle be x°. Using x°-J°, x°, and x°+J° as centers, superimpose the motion-blur-free projection images within the J° range. Divide the summed image value by the number of superimposed projection images to obtain three different J° rotation-blurred projection images. Similarly, using x°-K°, x°, and x°+K° as centers, superimpose the motion-blur-free projection images within the K° range. Divide the summed image value by the number of superimposed projection images to obtain three different K° rotation-blurred projection images. The image is converted to a blurred projection image. The unblurred projection image at angle x° is used as the label for each J° and K° rotated blurred projection image. The unblurred projection image at angle x° and the corresponding three labeled J° rotated blurred projection images are used as one training sample for that angle, and the unblurred projection image at angle x° and the corresponding three labeled K° rotated blurred projection images are used as another training sample for that angle, thus generating a pair of training samples for the angle centered at x°. 2n training samples are generated for each PCB simulation model; where I>J>K. 5) Train a fully convolutional deblurring network using the training samples; the fully convolutional deblurring network includes a multi-angle input module, multiple convolutional blocks, multiple residual blocks, multiple feature fusion modules, two upsampling modules, multiple downsampling modules, two channel fusion modules, and a correction module; wherein, the multi-angle input module is used to take the rotated blurred projection image p2 corresponding to the angle center in the input training sample as a reference, and subtract it from the rotated blurred projection images p1 and p3 of the angle adjacent to the angle center in the training sample, respectively, insert the resulting difference image between p1, p2, and p3 to generate fused data, and send it to the first convolutional block, the first downsampling module, and the second downsampling module respectively. The first convolutional block extracts features from the fused data to obtain a first feature map and inputs it to the first residual block. The first residual block extracts features from the first feature map to obtain a second feature map and sends it to the third downsampling module and the first channel fusion module. The first downsampling module downsamples the fused data and outputs it to the first feature fusion module. The third downsampling module downsamples the second feature map and inputs it to the first feature fusion module. The first feature fusion module fuses the input data and sends it to the second residual block for feature extraction to obtain a third feature map, which is then sent to the fourth downsampling module and the second channel fusion module. The fourth downsampling module... The third feature map is downsampled and sent to the second feature fusion module. The second downsampling module downsamples the fused data and outputs it to the second feature fusion module. The second feature fusion module fuses the input data and sends it to the third residual block for feature extraction, obtaining a fourth feature map, which is then sent to the fourth residual block for feature extraction, obtaining a fifth feature map, which is then sent to the first upsampling module. The first upsampling module upsamples the fifth feature map and sends it to the second channel fusion module. The second channel fusion module concatenates the feature map sent by the first upsampling module and the third feature map along the channel dimension and inputs it to the second convolutional block for feature extraction, obtaining a sixth feature map. The feature map is sent to the fifth residual block; the fifth residual block extracts features from the sixth feature map to obtain the seventh feature map and sends it to the second upsampling module; the second upsampling module upsamples the seventh feature map and sends it to the first channel fusion module; the first channel fusion module concatenates the feature map sent by the second upsampling module and the second feature map in the channel dimension and then sequentially inputs them to the third convolution block, the sixth residual block and the fourth convolution block for feature extraction to obtain the eighth feature map and input it to the residual connection module; the residual connection module adds the corresponding pixels of the eighth feature map and the rotated blurred projection image p2 corresponding to the angle center in the input training sample to obtain the corrected projection image; 6) Input the projected image of the target PCB acquired at a set angle a and two blurred projected images acquired at adjacent angles of the set angle a into the trained fully convolutional deblurring network for processing to obtain the corrected projected image.

2. The method according to claim 1, characterized in that, The loss function used to train the fully convolutional deblurring network is: loss function , loss function ;in, It is the image output by a fully convolutional deblurring network. It is a label image. It is the mean of the image x. It is the mean of the image y. The pixel variance of image x, It is the pixel variance of image y. It is the pixel covariance of the image x and y. and It is a constant. , , This represents the grayscale range of the image pixels. It is the number of pixels in the image. These are the weights of the SSIM loss function.

3. The method according to claim 1 or 2, characterized in that, n=12, I=6, J=2, K=1.

4. The method according to claim 1 or 2, characterized in that, Each training sample image is downsampled to generate a new training sample, thus increasing the number of training samples for each PCB simulation model.

5. The method according to claim 4, characterized in that, For each training sample image, add noise of random type and intensity; where the noise in the label image is consistent with the noise in the rotated blurred projection image p2, i.e. , For label data, For label data with added noise, The image located in the middle of three different K° rotationally blurred projection images or three different J° rotationally blurred projection images. for Image with added noise.

6. The method according to claim 1, characterized in that, The method for downsampling the fused data in the first downsampling module is as follows: first, the position of the elements in the fused data is changed by pixel rearrangement, and then convolution is performed on it to change the number of channels of the output data.

7. A server, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing each step of the method of any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images