A method for reducing artifacts in oblique 3D X-ray CT images

Abstract

To enable three-dimensional observation to be appropriately performed when measuring a measurement object by oblique three-dimensional X-ray CT, and observing an oblique three-dimensional X-ray CT image to evaluate the measurement object.SOLUTION: An optical axis angle connecting an X-ray source 1 and a detector 4 is set to be in an oblique direction with respect to a rotation axis direction of a measurement object 2, a projected image acquired by the detector is reconstructed to generate an oblique three-dimensional X-ray image, and when the generated oblique three-dimensional X-ray CT image is observed to evaluate a measurement object, a convolution neural network is applied to the oblique three-dimensional X-ray CT image, so that artifacts appearing in the oblique three-dimensional X-ray CT images can be reduced.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method for reducing artifacts in oblique three-dimensional X-ray CT images. [Background technology]

[0002] For example, in on-board displays and electronic control devices mounted on vehicles, high density mounting is required to realize miniaturization, weight reduction, high functionality, etc., and a technology is being adopted in which electronic boards are multi-layered to create three-dimensional wiring patterns. As a method for visualizing the inside of a flat-shaped measurement object such as an electronic board, a method is provided in which the measurement object is measured by oblique three-dimensional X-ray CT (Computed Tomography) and an oblique three-dimensional X-ray CT image (hereinafter simply referred to as an X-ray CT image) is generated (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-341757 A Summary of the Invention [Problem to be solved by the invention]

[0004] According to a method of measuring a measurement object by oblique 3D X-ray CT, it is possible to observe the measurement object non-destructively and 3Dly at high magnification, and it is possible to observe cracks and voids in solder. However, when an X-ray CT image is viewed from a direction different from the normal observation cross-section direction, streak-like artifacts may appear in the X-ray irradiation direction. When artifacts appear on a different observation cross-section, it is difficult to perform 3D observation on the different observation cross-section, or structural information in the depth direction appears as noise on the normal observation cross-section, making it difficult to perform 3D observation even on the normal observation cross-section.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and has an object to provide a method for reducing artifacts in oblique three-dimensional X-ray CT images, which enables appropriate three-dimensional observation when measuring a measurement object using oblique three-dimensional X-ray CT and evaluating the measurement object by observing the oblique three-dimensional X-ray CT images. [Means for solving the problem]

[0006] According to the invention described in claim 1, the optical axis angle connecting the X-ray source and the detector is set to an oblique direction with respect to the rotation axis direction of the measurement object, and the projection image acquired by the detector is reconstructed to generate an oblique three-dimensional X-ray CT image. When the measurement object is evaluated by observing the generated oblique three-dimensional X-ray CT image, The method sequentially performs a learning phase for learning the weights of a convolutional neural network for removing artifacts, a global correction neural network learning phase for dealing with large structures, and an application phase for applying the results to actual images. In the global correction neural network learning phase, Repeated compression and division of oblique 3D X-ray CT images By ,We applied a convolutional neural network to oblique 3D X-ray CT images, Multiple models using convolutional neural networks By combining and processing oblique 3D X-ray CT images, we were able to achieve both an overall impression and detailed images, and reduce artifacts that appeared in the oblique 3D X-ray CT images. The oblique 3D X-ray CT images can be observed with artifacts reduced, allowing proper 3D observation. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a manner in which a measurement object is measured by an oblique three-dimensional X-ray CT. [Diagram 2] Diagram showing each axis relative to the measurement object [Diagram 3] Figure showing X-ray CT images of XY cross sections [Figure 4] Figure showing X-ray CT images of XZ cross sections [Diagram 5] Diagram explaining image processing using CNN [Figure 6] A diagram explaining generalization through learning using projection simulation of computer-generated 3D structures. [Figure 7] Diagram showing the learning phase of the basic flowchart [Figure 8] Diagram showing application phases of the basic flowchart [Figure 9] Diagram showing the learning phase of the application flowchart [Figure 10] A diagram showing the global correction NN learning phase of the application flowchart. [Figure 11] A diagram showing the flow of the global correction NN learning phase in the application flowchart. [Figure 12] Diagram showing the application phase of the application flow chart [Figure 13] Diagram showing the application phase of the application flow chart [Figure 14] A diagram showing the flow of the application phase of the application flowchart [Figure 15] Original X-ray CT image [Figure 16] A diagram showing an X-ray CT image after processing according to the basic flowchart. [Figure 17] Figure showing X-ray CT image after processing by application flow chart [Figure 18] Original X-ray CT image [Figure 19] A diagram showing an X-ray CT image after processing according to the basic flowchart. [Figure 20] Figure showing X-ray CT image after processing by application flow chart DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, the measurement object is a flat-plate-shaped object such as an electronic board. As shown in FIG. 1, in a method for measuring the measurement object by oblique three-dimensional X-ray CT, the measurement object 2 is placed on a mounting table 3 so as to face an X-ray source 1, and the measurement object 2 is rotated in a horizontal direction around an observation site. At this time, the optical axis angle connecting the X-ray source 1 and the detector 4 is set to an oblique direction with respect to the direction of the rotation axis A of the measurement object 2, and a projection image acquired by the detector 4 is reconstructed to generate an X-ray CT image.

[0009] In this case, when the X-ray CT image generated by reconstructing the projection image is viewed from a direction different from the normal observation cross-sectional direction, a streak-like artifact may appear in the X-ray irradiation direction. That is, in the X-ray CT image of the solder ball shown in FIG. 2, when the rotation axis direction of the measurement object 2 is set to the Z-axis direction, as shown in FIG. 3, no artifact appears in the X-ray CT image of the XY cross section, but as shown in FIG. 4, an artifact may appear in the X-ray CT image of the XZ cross section. When an artifact appears, there is a risk that visibility is reduced and three-dimensional observation cannot be performed appropriately. In this embodiment, to address such a problem, a convolutional neural network (CNN) is applied to the X-ray CT image to reduce the artifacts appearing in the X-ray CT image and avoid a reduction in visibility.

[0010] First, the procedure for applying CNN will be explained. The procedure for applying CNN involves (1) image processing using CNN, (2) generalization through learning to perform projection simulation of a three-dimensional structure generated by a computer, and (3) application to large images by multi-stage CNN. Steps (1) to (3) will be explained below.

[0011] (1) Image processing using CNN Image processing using CNN will be described with reference to Fig. 5. In the learning process, CNN is applied to learning data with noise to generate an output image, the error between the generated output image and learning data without noise is calculated, and the calculated error is fed back to the CNN to train it. In the noise removal process, the CNN weights when the error has converged are used, and the CNN weights when the error has converged are applied to the image before noise removal to generate the image after noise removal.

[0012] (2) Generalization through learning to project and simulate computer-generated 3D structures Generalization by learning using projection simulation of a computer-generated three-dimensional structure will be explained with reference to Figure 6. A ground truth image is generated by a computer, a projection image is generated by performing projection simulation on the generated ground truth image, and an X-ray CT image is generated by reconstructing the generated projection image. The CNN that has been trained using this image is used as a general-purpose autoencoder. The ground truth image is an image that contains a variety of features, including, for example, ellipsoids, hexahedrons, cubes, planes, lines, particles, etc.

[0013] (3) Application to large images by multi-stage CNN The application of CNN to large images by multi-stage processing will be discussed later.

[0014] There are two methods for reducing artifacts by applying a convolutional neural network: (a) a basic flowchart and (b) an applied flowchart. Each flowchart will be explained below. (A) Basic flow chart The basic flowchart will be described with reference to Figures 7 and 8. The basic flowchart is made up of a learning phase and an application phase. The learning phase is a phase in which the weights of the CNN for removing artifacts are learned. In learning the weights of the CNN, data obtained by projecting a three-dimensional structure generated by a computer may be used as the learning data, or data obtained by measuring an actual object by oblique three-dimensional X-ray CT or three-dimensional X-ray CT may be used as the learning data. In the learning phase, as shown in FIG. 7, a correct answer image that is the correct answer for learning is generated (S1, S2). Next, the generated correct answer image is subjected to a projection simulation that simulates an actual measurement to obtain a projection image (S3). Next, the obtained projection image is reconstructed in the same manner as the actual measurement to generate an X-ray CT image (S4), and the generated X-ray CT image is input as an image with artifacts (S5). Next, the CNN is trained using the initially generated correct answer image as the correct answer data (S6), and the weights of the network at the time of convergence are saved as a trained network (S7).

[0015] The application phase is a phase for removing artifacts that appear in an actually measured X-ray CT image. In the application phase, as shown in FIG. 8, the measurement object is measured by an oblique 3D X-ray CT, and the projection images are reconstructed to generate an X-ray CT image as an image with artifacts (S11 to S14). Next, the input size assumed by the network weights is compared with the size of the image with artifacts (S15). If it is determined that the input size assumed by the network weights is smaller than the size of the image with artifacts (S15: YES), the image with artifacts is divided by the input size assumed by the network weights (S16). Then, the image with artifacts is input to the CNN to remove the artifacts (S17). On the other hand, if it is determined that the input size assumed by the network weights is not smaller than the size of the image with artifacts (S15: NO), the image with artifacts is input as is to the CNN to remove the artifacts (S17). Next, the input size assumed by the network weights is compared with the size of the artifact-caused image (S18), and if it is determined that the input size assumed by the network weights is smaller than the size of the artifact-caused image (S18: YES), the divided artifact-caused images are recombined (S19) to generate an artifact-free image (S20).On the other hand, if it is determined that the input size assumed by the network weights is not smaller than the size of the artifact-caused image (S15: NO), the artifact-free image is generated as is (S20).

[0016] (a) Application Flowchart The application flowchart will be described with reference to Figs. 9 to 14. In the basic flowchart described above, macro information is lost by division, and it is not possible to handle artifacts originating from large structures. Since the structure of artifacts is maintained even when the image is compressed, it is possible to handle large structures by compressing it to a size that can be processed by a computer. Since fine structural information disappears if the image is left compressed, the next step is compressed more loosely than the first compression. In other words, if the size of a certain axis direction of the original image is, for example, 256 pixels, the first step is compressed to 64 pixels and CNN is applied, and the next step is compressed to 128 pixels, divided, and CNN is applied. By combining and processing these images, it is possible to process both the overall feeling and the details. By applying this process in multiple stages, it is possible to handle images of any size.

[0017] The application flowchart is composed of a learning phase, a global correction NN learning phase, and an application phase. The learning phase is the same as the learning phase described in the basic flowchart above, as shown in FIG. 9. That is, S21 to S27 are the same as S1 to S7 in the basic flowchart above.

[0018] The global correction NN learning phase is a phase of global correction to accommodate large structures. In the global correction NN learning phase, as shown in Fig. 10, an image larger than the size of the input expected image of the first NN is generated. For example, if the expected size of the first NN is 64 x 64 x 64 pixels, an image of 256 x 256 x 256 pixels, which is four times larger on each axis, is generated (S31), the generated image is set as the answer image (S32), and the answer image is subjected to a projection simulation simulating actual measurement to obtain a projection image (S33). Next, the obtained projection image is reconstructed in the same manner as the actual measurement to generate an X-ray CT image (S34), and the generated X-ray CT image is generated as an image with artifacts (S35).

[0019] Next, the generated image with artifacts is divided into a group of images with a size of 64×64×64 pixels (S36), and the image with artifacts is input to the CNN to remove the artifacts (S37). The initially generated image with a size of 256×256×256 pixels is similarly divided to obtain a correct image (S38). The divided images are then compressed to a size of 32×32×32 pixels (S39), and the CNN is trained using S37 and S39 as input data and S38 as correct data (S40), to obtain a neural network for overall correction (S41). Figure 11 shows an outline of the flow of the overall correction NN learning phase.

[0020] The application phase is a phase in which the image is applied to an actual image. In the application phase, as shown in FIG. 12 and FIG. 13, the size of the image with artifacts acquired by the first measurement is adjusted. For example, if the input of the artifact removal NN is assumed to be 64×64×64 pixels in size, the size is adjusted so that the size of all axes is 64×2^N (^ indicates a power) (S51 to S55), and the image with the adjusted size is acquired (S56). For example, if an image with a size of 500×500×200 pixels is to be processed, the missing parts are filled with pixel values ​​of "0" so that the size becomes 512×512×512 pixels. Next, the image with the adjusted size is compressed to a size of 64×64×64 pixels (S57), and the artifact removal NN is applied (S58).

[0021] Next, the image adjusted in size in step S55 is compressed to a size of 128×128×128 pixels (S59), the compressed image is divided into eight parts with a size of 64×64×64 pixels (S60), and the artifact removal NN is applied to the eight divided images (S61). Next, the image to which the artifact removal NN has been applied in S58 is divided into eight parts with a size of 32×32×32 pixels (S62), and these are input to the overall impression correction NN (S63). Next, the results obtained by the overall impression correction NN are combined (S64) to generate an image with a size of 128×128×128 pixels in which both the overall impression and the details have been restored. This operation is repeated until the size of the image is 512×512×512 pixels.

[0022] That is, the image adjusted in size in step S55 is compressed to a size of 256×256×256 pixels (S65), the compressed image is divided into 64 parts with a size of 64×64×64 pixels (S66), and the artifact removal NN is applied to the 64 divided images (S67). Next, the image combined in S64 is divided into 64 parts with a size of 32×32×32 pixels (S68), and these are input to the overall impression correction NN (S69). Next, the results obtained by the overall impression correction NN are combined (S70) to generate an image with a size of 256×256×256 pixels in which both the overall impression and the details are restored.

[0023] Next, the image adjusted in size in step S55 is divided into 512 images of 64×64×64 pixels (S71), and the artifact removal NN is applied to the 512 divided images (S72). Next, the image combined in S70 is divided into 512 images of 32×32×32 pixels (S73), and these are input to the overall feeling correction NN (S74). Next, the results obtained by the overall feeling correction NN are combined (S75), and an image of 512×512×512 pixels in size is generated in which both the overall feeling and the details are restored.

[0024] Next, the portion initially filled with "0" is removed and the size is adjusted (S76), and an image with the original size of 500x500x200 pixels from which artifacts have been removed is obtained (S77). In this embodiment, four stages are illustrated, but if the size of the original image is larger, five or more stages may be used, and if the size of the original image is small, three or less stages may be used. Figure 14 shows a schematic flow of the application phase.

[0025] FIG. 15 shows an original X-ray CT image when the measurement object is a BGA (Ball Grid Array), FIG. 16 shows an X-ray CT image after processing according to the basic flowchart, and FIG. 17 shows an X-ray CT image after processing according to the applied flowchart. In FIG. 16 and FIG. 17, the artifacts that appeared in FIG. 15 are reduced, and the area where the artifacts appeared is clear. FIG. 18 shows an original X-ray CT image when the measurement object is a QFN (Quad Flat Non-leaded package), FIG. 19 shows an X-ray CT image after processing according to the basic flowchart, and FIG. 20 shows an X-ray CT image after processing according to the applied flowchart. In FIG. 19 and FIG. 20, the artifacts that appeared in FIG. 18 are reduced, and the area where the artifacts appeared is clear. In this way, by performing the processing according to the basic flowchart described above, the artifacts can be reduced, and further, by performing the processing according to the applied flowchart, the lattice noise due to division can be reduced, and the area where the artifacts appeared can be clear.

[0026] As described above, according to this embodiment, the following advantageous effects can be obtained. When observing the X-ray CT image to evaluate the measurement target 2, CNN is applied to the X-ray CT image to reduce artifacts that appear in the X-ray CT image. The X-ray CT image can be observed with artifacts reduced, allowing proper three-dimensional observation.

[0027] In addition, in the CNN weight learning, the data obtained by projecting a computer-generated three-dimensional structure is used as learning data. By using any three-dimensional structure, versatility can be increased.

[0028] The X-ray CT image is repeatedly compressed and divided, and CNN is applied to the X-ray CT image. By repeating compression and division, it is possible to handle large-sized images. In addition, when compressing and dividing the X-ray CT image repeatedly, the X-ray CT images that have been divided by offsetting them are weighted and combined. This makes it possible to correct discontinuities in the image caused by division and combination.

[0029] Although the present disclosure has been described based on the embodiment, it is understood that the present disclosure is not limited to the embodiment or structure. The present disclosure also includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more than one element, or less than one element, are also within the scope and concept of the present disclosure. [Explanation of symbols]

[0030] In the drawing, 1 is an X-ray source, 2 is a measurement object, and 4 is a detector.

Claims

1. A method for reducing artifacts in an oblique three-dimensional X-ray CT image, comprising the steps of: setting an optical axis angle connecting an X-ray source (1) and a detector (4) in an oblique direction with respect to a rotation axis direction of a measured object (2); generating an oblique three-dimensional X-ray CT image by reconstructing a projection image acquired by the detector; and evaluating the measured object by observing the generated oblique three-dimensional X-ray CT image, sequentially performing a learning phase for learning weights of a convolutional neural network for removing artifacts, a global correction neural network learning phase for dealing with large structures, and an application phase for applying the image to an actual image, and applying a convolutional neural network to the oblique three-dimensional X-ray CT image by repeatedly compressing and dividing the oblique three-dimensional X-ray CT image in the global correction neural network learning phase, and combining and processing a plurality of the oblique three-dimensional X-ray CT images to which the convolutional neural network has been applied, thereby performing processing that achieves both an overall appearance and details, thereby reducing artifacts appearing in the oblique three-dimensional X-ray CT image.

2. 2. The method for reducing artifacts in oblique three-dimensional X-ray CT images according to claim 1, wherein data obtained by projecting and simulating a three-dimensional structure generated by a computer is used as learning data in weight learning of the convolutional neural network.

3. 2. The method for reducing artifacts in oblique three-dimensional X-ray CT images according to claim 1, wherein in the weight learning of the convolutional neural network, data obtained by measuring an actual object using oblique three-dimensional X-ray CT or three-dimensional X-ray CT is used as learning data.

4. A method for reducing artifacts in oblique three-dimensional X-ray CT images according to any one of claims 1 to 3, wherein when compressing and dividing the oblique three-dimensional X-ray CT images, the offset and divided oblique three-dimensional X-ray CT images are weighted and combined.

Images