X-ray imaging system and method for generating trained models

Abstract

To provide an X-ray imaging system with which it is possible to lighten a work load in the work of creating teaching data when generating a trained model for identifying a region of an inspection object and / or the region of an abnormal portion included in the inspection object, and method for generating a trained model.SOLUTION: This X-ray imaging system 100 acquires a center image and an edge image from an X-ray image 60 for teaching that includes a plurality of solder balls arranged in a regular arrangement. Meanwhile, the X-ray imaging system 100 acquires identification information for identifying a solder ball region in each of the center image and the edge image. Then, the X-ray imaging system 100 performs machine learning to generate a trained model, using inputted teaching data based on the center and edge images and outputted teaching data based on the identification information.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to an X-ray imaging system and a trained model generation method, and particularly relates to an X-ray imaging system that generates a trained model and a trained model generation method. [Background Art]

[0002] Conventionally, apparatuses that generate a trained model are known (see, for example, Patent Document 1).

[0003] Patent Document 1 mentioned above discloses an apparatus (image analysis apparatus) that performs estimation processing for estimating the region or position of an analysis target object in an analysis target image. In Patent Document 1 mentioned above, image analysis using machine learning is used in this estimation processing. The image analysis apparatus described in Patent Document 1 generates a trained model by performing machine learning using an analysis target image and a label image corresponding to the analysis target image. Then, analysis processing is executed using the generated trained model. The image analysis apparatus described in Patent Document 1 digitizes the boundary portion of the analysis target object in an image by performing binarization processing on the entire original analysis target image, and acquires a label image by combining the digitized processed images. [Prior Art Literature] [Patent Literature]

[0004] [Patent Document 1] International Publication No. 2022 / 070491 [Summary of Invention] [Problem to be Solved by Invention]

[0005] Although not described in Patent Document 1, there are cases where an operator manually selects and processes regions from the source image (analysis target image) in order to create training data (labeled images) used in machine learning. In such cases, as with the image analysis device described in Patent Document 1, if the entirety of multiple source images is manually processed to create training data, the training data creation process becomes burdensome for the operator. Therefore, when generating a trained model to identify at least one of the regions of the analysis target (inspection target) and the regions of abnormal parts contained in the inspection target, it is desirable to reduce the workload on the operator in the training data creation process.

[0006] This invention was made to solve the above-mentioned problems, and one objective of this invention is to provide an X-ray imaging system and a method for generating a trained model that can reduce the workload of the operator in the process of creating training data when generating a trained model for identifying at least one of the regions of an object to be inspected and the regions of abnormal parts contained in the object to be inspected. [Means for solving the problem]

[0007] To achieve the above objective, the X-ray imaging system in the first aspect of this invention comprises: an X-ray irradiation unit that irradiates X-rays onto objects to be inspected arranged in a regular manner; an X-ray detection unit that detects X-rays irradiated from the X-ray irradiation unit; an image generation unit that generates an X-ray image based on the X-rays detected by the X-ray detection unit; and a model generation unit that generates a trained model for analyzing the X-ray image generated by the image generation unit, wherein the model generation unit includes objects to be inspected arranged in a regular manner. identical From the teacher's X-ray images, Includes the central part of the area containing the object to be inspected. The first image is a cropped section of the first part, The target region includes the edge portion, and the X-rays are incident from an angular direction different from the angular direction of the X-rays incident on the first portion.An image extraction unit acquires a second image from which a second part different from the first part has been extracted, and an identification information acquisition unit acquires identification information for identifying at least one of the region of the object to be inspected and the region of the abnormal part contained in the object to be inspected in each of the first and second images, and the first and second images Both It includes a learning unit that performs machine learning to generate a trained model using input training data based on and output training data based on identification information.

[0008] The second aspect of this invention relates to a method for generating a trained model, which corresponds to an X-ray image generated by irradiating an object to be inspected, which is arranged in a regular manner, with X-rays. identical From the teacher's X-ray images, Includes the central part of the area containing the object to be inspected. The first image is a cropped section of the first part, The target region includes the edge portion, and the X-rays are incident from an angular direction different from the angular direction of the X-rays incident on the first portion. The steps include: obtaining a second image from which a second part different from the first part has been cut out; obtaining identification information to identify at least one of the regions of the object to be inspected and the regions of the abnormal parts contained in the object to be inspected in each of the first and second images; and the first and second images Both The process includes the step of performing machine learning to generate a trained model for analyzing X-ray images using input training data based on and output training data based on identification information. [Effects of the Invention]

[0009] In the X-ray imaging system in the first phase described above, and the trained model generation method in the second phase described above, a first image is obtained by extracting a first portion from a training X-ray image containing an object to be inspected arranged in a regular manner, and a second image is obtained by extracting a second portion different from the first portion. Then, identification information is obtained to identify at least one of the region of the object to be inspected and the region of the abnormal part contained in the object to be inspected in each of the first and second images. By obtaining identification information to identify the region in each of the first image (extracted from the first portion of the training X-ray image) and the second image (extracted from the second portion), the workload of the operator in obtaining identification information can be reduced compared to obtaining identification information corresponding to the entire training X-ray image. As a result, the workload of obtaining output training data for machine learning can be reduced, and therefore, when generating a trained model to identify at least one of the region of the object to be inspected and the region of the abnormal part contained in the object to be inspected, the workload of the operator in creating training data can be reduced. Furthermore, in X-ray imaging, when X-rays are irradiated from a point source, the resulting X-ray image will show images of X-rays incident from different angles to the center of irradiation and images of X-rays incident from different angles to the center. In contrast, the present invention obtains a first image, which is extracted from a first portion of a training X-ray image containing objects to be inspected arranged in a regular manner, and a second image, which is extracted from a second portion different from the first portion. As a result, since the objects to be inspected in the training X-ray image are arranged in a regular manner, the images of the objects to be inspected in the first and second images can be projected onto a structure having a common arrangement relationship. Therefore, by obtaining the first and second images, it is possible to extract from the training X-ray image the first and second portions, which include images of X-rays incident onto a structure having a common arrangement relationship from different angles to the first and second portions.Therefore, even when using an image extracted from a portion of the training X-ray image rather than the entire image, it is possible to acquire training data that reflects the projection image of the entire training X-ray image, thereby suppressing a decrease in the accuracy of the classification results produced by the trained model generated by machine learning. As a result, by using the first and second images extracted from the training X-ray image, it is possible to reduce the workload on the operator in creating training data for generating the trained model while suppressing a decrease in the accuracy of the classification results. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the overall configuration of an X-ray imaging system according to one embodiment of the present invention. [Figure 2] This is a schematic diagram showing the structure of the test object, including the solder ball. [Figure 3] This figure shows an example of an X-ray image. [Figure 4] This is a block diagram illustrating the functional configuration of the control unit. [Figure 5] This is a diagram illustrating inference using a pre-trained model. [Figure 6] This figure shows an example of a classified image output by inference using a pre-trained model. [Figure 7] This is a diagram illustrating the setting of the target area in X-ray images for teacher training. [Figure 8] This diagram illustrates the range setting for obtaining central and edge images in teacher-reviewed X-ray images. [Figure 9] This diagram illustrates how to define the background area in X-ray images for teacher training. [Figure 10] This diagram illustrates the acquisition of identification images corresponding to the central image, edge image, and background image. [Figure 11] This diagram illustrates the generation of input training data based on the central image, edge image, and background image. [Figure 12]FIG. 4 is a diagram for explaining generation of output teacher data based on an identification image. [Figure 13] FIG. 5 is a flowchart for explaining an X-ray image analysis method using a trained model. [Figure 14] FIG. 6 is a flowchart for explaining a trained model generation method. MODE FOR CARRYING OUT THE INVENTION

[0011] Hereinafter, embodiments embodying the present invention will be described with reference to the drawings.

[0012] (Overall Configuration of X-ray Imaging System) An X-ray imaging system 100 according to an embodiment of the present invention will be described with reference to FIGS. 1 to 12.

[0013] As shown in FIG. 1, the X-ray imaging system 100 according to the present embodiment is a system that images the inside of a subject 101 by detecting X-rays that have passed through the subject 101. The X-ray imaging system 100 is used, for example, for imaging the inside of the subject 101 as an object in non-destructive inspection applications.

[0014] As shown in Figure 2, the subject 101 is an electronic device including a substrate 102. Electronic components 103 are mounted on both sides of the substrate 102. The electronic components 103 are electrically connected to the substrate 102 by a plurality of solder balls 104 (bumps). The plurality of solder balls 104 are arranged in a regular manner. Specifically, the plurality of solder balls 104 are arranged in a grid pattern on the substrate 102 in a regular manner. That is, the electronic components 103 are connected to the substrate 102 by a BGA (Ball Grid Array). The plurality of solder balls 104 are arranged in a grid pattern on both the front and back surfaces of the substrate 102. For example, in the subject 101, 39 solder balls 104 are arranged in a grid pattern in 3 vertical columns and 13 horizontal columns on both the front and back surfaces of the substrate 102. Each of the multiple solder balls 104 on the surface of the substrate 102 and each of the multiple solder balls 104 on the back surface are arranged to overlap each other when viewed from a direction perpendicular to the surface of the substrate 102. The electronic component 103 includes, for example, an integrated circuit (IC). The X-ray imaging system 100 performs non-destructive testing for abnormalities such as voids and bridges in the multiple solder balls 104. In addition to the electronic component 103, the substrate 102 also has surface-mounted electronic components 105 such as resistors or capacitors. Note that the multiple solder balls 104 are examples of the "object to be inspected" and "multiple solder materials" in the claims.

[0015] As shown in Figure 1, the X-ray imaging system 100 comprises a fluoroscopy device 10 and an analysis device 20. The fluoroscopy device 10 generates an X-ray image 30 (see Figure 3) by performing X-ray imaging on a subject 101. The analysis device 20 performs analysis processing on the generated X-ray image 30. Both the fluoroscopy device 10 and the analysis device 20 have communication modules and transmit and receive information from each other via a network or the like.

[0016] The fluoroscopy device 10 comprises an X-ray irradiation unit 11, an X-ray detection unit 12, and an image generation unit 13. The X-ray irradiation unit 11 irradiates a subject 101 containing a plurality of solder balls 104 with X-rays. The X-ray irradiation unit 11 includes an X-ray tube that irradiates X-rays when power is supplied from a power supply unit (not shown). The X-ray detection unit 12 detects the X-rays irradiated from the X-ray irradiation unit 11. The X-ray detection unit 12 outputs an electrical signal corresponding to the detected X-rays. The X-ray detection unit 12 includes, for example, an FPD (Flat Panel Detector), which is an X-ray detector. The X-ray irradiation unit 11 and the X-ray detection unit 12 are located inside a housing (not shown) of the fluoroscopy device 10.

[0017] As shown in Figure 3, the image generation unit 13 generates an X-ray image 30 based on the X-rays detected by the X-ray detection unit 12. The image generation unit 13 is a computer that includes a processor such as a CPU (Central Processing Unit) and a memory device for storing information. The image generation unit 13 controls the operation of each part of the fluoroscopy apparatus 10. For example, the image generation unit 13 controls the irradiation of X-rays by the X-ray irradiation unit 11 by controlling a power supply device (not shown). The image generation unit 13 then outputs the generated X-ray image 30 to the analysis device 20.

[0018] The X-ray image 30 includes multiple solder balls 104 arranged in a grid pattern with regularity. In the X-ray image 30, since multiple solder balls 104 are provided on both sides of the substrate 102 in the specimen 101, there are areas where the solder balls 104 overlap. Note that since the X-ray irradiation unit 11 is a point light source, the degree of overlap of the solder balls 104 differs between the irradiation center of the X-ray image 30 and the edges of the X-ray image 30.

[0019] As shown in Figure 1, the analysis device 20 has a control unit 21 and a storage unit 22. The analysis device 20 is, for example, a personal computer that is communicatively connected to the fluoroscopic device 10. The control unit 21 includes a CPU, ROM (Read Only Memory), and RAM (Random Access Memory), etc. The control unit 21 may also include a processor such as a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) configured for image processing. Note that the control unit 21 is an example of the "model generation unit" in the claims.

[0020] The storage unit 22 is configured to store various programs and parameters executed by the control unit 21. The storage unit 22 includes, for example, non-volatile memory such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0021] Furthermore, a display unit 23 and an operation unit 24 are connected to the analysis device 20. The display unit 23 includes, for example, a liquid crystal monitor. The display unit 23 displays images and text information under the control of the control unit 21. The operation unit 24 receives input operations from the operator. The operation unit 24 includes, for example, a keyboard and a pointing device such as a mouse. The operation unit 24 outputs operation signals to the control unit 21 based on the received input operations.

[0022] Furthermore, as shown in Figure 4, the control unit 21 includes an analysis processing unit 41, a display processing unit 42, a cropped image acquisition unit 43, an identification information acquisition unit 44, a training data generation unit 45, and a learning unit 46. The control unit 21 functions as the analysis processing unit 41, display processing unit 42, cropped image acquisition unit 43, identification information acquisition unit 44, training data generation unit 45, and learning unit 46 by executing various programs (software) stored in the storage unit 22. In other words, in Figure 4, the analysis processing unit 41, display processing unit 42, cropped image acquisition unit 43, identification information acquisition unit 44, training data generation unit 45, and learning unit 46 are illustrated as software-like functional blocks. However, it is not limited to this, and some or all of the analysis processing unit 41, display processing unit 42, cropped image acquisition unit 43, identification information acquisition unit 44, training data generation unit 45, and learning unit 46 may be configured by dedicated hardware circuits.

[0023] (Analysis of X-ray images using an analysis device) As shown in Figure 5, the analysis processing unit 41 analyzes the X-ray image 30 using the trained model 50. Specifically, the analysis device 20 uses the trained model 50 to identify the region of the solder ball 104, and obtains an identification result image 31 from the X-ray image 30 as the identification result (inference result) by the trained model 50. The trained model 50 is generated by the control unit 21 of the analysis device 20 and stored in the storage unit 22. Details of the generation of the trained model 50 will be described later.

[0024] As shown in Figure 6, the identification result image 31 is a label image in which the solder balls 104 on both sides of the substrate 102 in the X-ray image 30 are identified into three types of regions: areas where the solder balls 104 overlap each other, areas where only one solder ball 104 is included without overlapping, and areas where no solder balls 104 are included. In the identification result image 31, the areas where the solder balls 104 overlap each other are colored gray, the areas where only one solder ball 104 is included without overlapping are colored white, and the areas where no solder balls 104 are included are colored black. Note that in Figure 6, gray is represented by hatching.

[0025] The analysis processing unit 41 then determines whether or not there are any abnormalities in the solder balls 104 included in the X-ray image 30 based on the regions identified in the identification result image 31. The analysis processing unit 41 detects the area and shape of each of the multiple solder balls 104 by detecting each region of the multiple solder balls 104 in the X-ray image 30 based on the identification result image 31. Then, based on the area and shape of each of the detected multiple solder balls 104, the analysis processing unit 41 determines whether or not there are any abnormalities such as voids (holes), poor wetting, bridging, and sputter in each of the multiple solder balls 104.

[0026] The display processing unit 42 controls the display of images and text information on the display unit 23. For example, the display processing unit 42 displays the X-ray image 30 and the teacher X-ray image 60 (see Figure 7), which will be described later, on the display unit 23. The display processing unit 42 may also display on the display unit 23 any portion of the X-ray image 30 that has been determined to be abnormal as a result of the analysis by the analysis processing unit 41, so that it can be identified.

[0027] <Generating a pre-trained model> As shown in Figure 5, in this embodiment, the control unit 21 generates a trained model 50 for analyzing the X-ray image 30 generated by the image generation unit 13. The trained model 50 is generated by machine learning using input training data 30t and output training data 31t as datasets. The input training data 30t is generated based on a training X-ray image 60 (see Figure 7). The training X-ray image 60 is generated by the fluoroscopy device 10, similar to the X-ray image 30 that is to be analyzed.

[0028] For example, the X-ray imaging system 100 inspects multiple subjects 101 that share a common configuration, including multiple solder balls 104. Therefore, the X-ray imaging system 100 acquires multiple X-ray images 30 generated by irradiating some of the multiple subjects 101 with X-rays, which are then used as multiple training X-ray images 60. A trained model 50 is then generated based on these acquired training X-ray images 60. The determination of whether or not there are abnormalities in the solder balls 104 using the trained model 50 is performed on the X-ray images 30 generated by irradiating the remaining subjects 101 with X-rays. In other words, the X-ray images 30 being analyzed and the training X-ray images 60 used to generate the trained model 50 are images that include subjects 101 (solder balls 104) that share a common structure.

[0029] As shown in Figure 7, the cropped image acquisition unit 43 sets a target region 60a containing multiple solder balls 104 from the training X-ray image 60 acquired to generate the trained model 50. Specifically, the operation unit 24 receives a region setting operation to set the target region 60a from the training X-ray image 60. The operator (worker) performs a region setting operation to set the target region 60a to the operation unit 24 while viewing the entire training X-ray image 60 displayed on the display unit 23 by the display processing unit 42. The cropped image acquisition unit 43 acquires the target region 60a in the training X-ray image 60 set by the region setting operation received by the operation unit 24. At this time, as an indication of the range of the set target region 60a, a dotted line frame superimposed on the training X-ray image 60 so as to surround the range of the target region 60a is displayed on the display unit 23.

[0030] Then, as shown in Figure 8, based on the input operation to the operation unit 24, the cropped image acquisition unit 43 acquires a central image 61b (see Figure 10) from a training X-ray image 60 containing a plurality of solder balls 104 arranged in a regular manner, which includes a portion 60b containing the central portion of the target region 60a, and two edge images 61c and 61d (see Figure 10) from which portions 60c and 60d containing different edge portions of the target region 60a are extracted. Portion 60b is an example of the "first portion" in the claims. Portions 60c and 60d are examples of the "second portion" in the claims. The central image 61b is an example of the "first image" in the claims. The edge images 61c and 61d are examples of the "second image" in the claims.

[0031] In detail, the cropped image acquisition unit 43 automatically sets the range of portion 60b, which is the cropping range of the central image 61b, and the ranges of portion 60c and portion 60d, which are the cropping ranges of the end images 61c and 61d, based on the target region 60a in the teacher X-ray image 60 set by the region setting operation received by the operation unit 24. For example, the cropped image acquisition unit 43 sets the range of portion 60b as an area where the vertical size is 120% of the target region 60a and the horizontal size is 30% of the target region 60a, starting from the center position of the set target region 60a. The cropped image acquisition unit 43 also sets the range of portion 60c as an area where the vertical size is 120% of the target region 60a and the horizontal size is 10% of the target region 60a from the left edge of the target region 60a. Furthermore, the cropped image acquisition unit 43 sets the area of ​​portion 60d as having a vertical size of 120% of the target area 60a and a horizontal size of 10% of the target area 60a from the right edge of the target area 60a. The cropped image acquisition unit 43 sets the sizes of portions 60c and 60d to be equal. For example, the size of each of the portions 60b, 60c, and 60d (as a percentage of the target area 60a) is set in advance.

[0032] Then, under the control of the display processing unit 42, the display unit 23 visually displays the range of portion 60b, which is cut out from the teacher X-ray image 60, and the ranges of portions 60c and 60d. For example, as a display showing the ranges of portions 60b, 60c, and 60d, a dotted line frame superimposed on the teacher X-ray image 60 is displayed on the display unit 23, similar to the target area 60a. Furthermore, the cut-out image acquisition unit 43 sets the ranges of portions 60b, 60c, and 60d, which are cut out from the teacher X-ray image 60, based on input operations received by the operation unit 24 to change the ranges of portions 60b, 60c, and 60d, respectively. In other words, the cut-out image acquisition unit 43 is configured to change the pre-set ranges of portions 60b, 60c, and 60d. For example, the operation unit 24 accepts a range setting operation to set (change) the ranges of each of the sections 60b, 60c, and 60d when the display unit 23 is showing the ranges of each section 60b, 60c, and 60d together with the teacher's X-ray image 60. If the range setting is changed, the display processing unit 42 displays the changed range on the display unit 23 so that it can be easily seen.

[0033] Here, the cropped image acquisition unit 43 sets the range of part 60b and the ranges of parts 60c and 60d to have a size greater than or equal to a predetermined threshold, based on input operations (range setting operations) that change the range of part 60b and the ranges of parts 60c and 60d respectively, which are received by the operation unit 24. The predetermined threshold is stored in the storage unit 22 in advance. For example, in the range of part 60b, which is the range of the central image 61b, a predetermined threshold is set so that the horizontal size is 20% or more of the target area 60a. Also, in the ranges of parts 60c and 60d, which are the ranges of the end images 61c and 61d, a predetermined threshold is set so that the horizontal size is 10% or more of the target area 60a. Furthermore, in each of the ranges of part 60b, part 60c, and part 60d, a predetermined threshold is set so that the vertical size is 120% or more of the target area 60a. In other words, the cropped image acquisition unit 43 is configured such that the cropped central image 61b, the edge images 61c and 61d are not smaller than a predetermined size.

[0034] For example, in each of the sections 60b, 60c, and 60d, a predetermined threshold for the horizontal size is greater than the size of at least one solder ball 104 in sections 60b, 60c, and 60d. That is, the horizontal sizes of sections 60b, 60c, and 60d are set to be greater than at least one period of the regularity in the arrangement of multiple solder balls 104 arranged in a regular manner.

[0035] Furthermore, as shown in Figure 9, after the ranges of parts 60b, 60c, and 60d have been set, the extracted image acquisition unit 43 sets the range of the background image 61e (see Figure 10) from which the background portion 60e that does not contain the solder balls 104 has been extracted from the teacher X-ray image 60, based on the input operation received by the operation unit 24. For example, the operation unit 24 receives a range selection operation to set the range of the background portion 60e while the entire teacher X-ray image 60 is displayed by the display processing unit 42. At this time, in the displayed teacher X-ray image 60, the ranges of parts 60b, 60c, and 60d that have already been set are displayed in an identifiable manner. For example, the ranges of parts 60b, 60c, and 60d are filled with gray. Then, as an indication of the set range of the background portion 60e, a dotted line frame superimposed on the teacher X-ray image 60 is displayed on the display unit 23 so as to surround the range of the background portion 60e. The background portion 60e does not include the solder balls 104, but does include, for example, the circuit board 102 and the electronic components 105.

[0036] The extracted image acquisition unit 43 acquires, from the teacher X-ray image 60, a central image 61b obtained by extracting a portion 60b of the set range based on the range setting operation described above, edge images 61c and 61d obtained by extracting portions 60c and 60d of the set range, and a background image 61e obtained by extracting a background portion 60e of the set range.

[0037] In this embodiment, when acquiring a central image 61b, edge image 61c, edge image 61d, and background image 61e from each of the multiple teacher X-ray images 60, the central image 61b, edge image 61c, edge image 61d, and background image 61e of a common range in each of the multiple teacher X-ray images 60 are acquired. Specifically, the operation unit 24 accepts a range setting operation to set the range of part 60b, the ranges of parts 60c and 60d, and the range of background part 60e in one of the multiple teacher X-ray images 60. The extracted image acquisition unit 43 then acquires, from each of the multiple teacher X-ray images 60, a plurality of central images 61b obtained by extracting a portion 60b of a common range set based on a range setting operation for one teacher X-ray image 60, a plurality of end images 61c and end images 61d obtained by extracting portions 60c and 60d of a common range set based on a range setting operation for one teacher X-ray image 60, and a plurality of background images 61e obtained by extracting a background portion 60e of a common range set based on a range setting operation for one teacher X-ray image 60.

[0038] As shown in Figure 10, the identification information acquisition unit 44 acquires identification information to identify the area of ​​the solder ball 104 in each of the center image 61b, edge image 61c, edge image 61d, and background image 61e, based on the input operation to the operation unit 24. Specifically, the identification information acquisition unit 44 acquires identification images 62b, 62c, 62d, and 62e as identification information, in which the area of ​​the solder ball 104 in each of the center image 61b, edge image 61c and 61d, and background image 61e is identified.

[0039] In detail, the display processing unit 42 displays the center image 61b, edge image 61c, edge image 61d, and background image 61e, each acquired by the cropped image acquisition unit 43, on the display unit 23. The operator then visually inspects each of the displayed center image 61b, edge image 61c, edge image 61d, and background image 61e, and colors-codes (labels) the parts of each image that correspond to the solder balls 104. For example, the operator can perform an operation to color-code the central image 61b, edge image 61c, edge image 61d, and background image 61e, such as coloring the areas where the solder balls 104 on both sides of the substrate 102 overlap gray, the areas where only one solder ball 104 is included without overlapping white, and the areas where no solder balls 104 are included black. In this way, the identification information acquisition unit 44 acquires identification images 62b to 62e. Note that since the background image 61e does not contain any solder balls 104, the system may automatically acquire an identification image 62e that is entirely black.

[0040] As shown in Figure 11, the training data generation unit 45 generates multiple input training data 30t based on the central image 61b, edge images 61c and 61d, and background image 61e acquired by the cropped image acquisition unit 43. Specifically, the training data generation unit 45 is configured to make the multiple input training data 30t of a common size by at least one of the following: arranging the duplicated central images 61b side by side while inverting them, and arranging the duplicated edge images 61c and edge images 61d side by side while inverting them. In addition, when the training data generation unit 45 generates input training data 30t based on the background image 61e, it also generates input training data 30t of a common size by arranging the duplicated background images 61e side by side while inverting them.

[0041] For example, the training data generation unit 45 obtains the vertical dimension of the image with the largest vertical dimension from among the central image 61b, edge images 61c and 61d, and background image 61e acquired by the cropped image acquisition unit 43, and uses this as the vertical dimension of the input training data 30t. Then, the training data generation unit 45 obtains the horizontal dimension of the image with the largest horizontal dimension from among the central image 61b, edge images 61c and 61d, and background image 61e acquired by the cropped image acquisition unit 43, and uses this as the horizontal dimension of the input training data 30t.

[0042] The training data generation unit 45 then duplicates and inverts the central image 61b, the end images 61c and 61d, and the background image 61e, and arranges them side by side, so that their vertical and horizontal dimensions match the acquired input training data 30t. In the example in Figure 11, the central image 61b has the largest vertical and horizontal dimensions among the central image 61b, the end images 61c and 61d, and the background image 61e. Therefore, the size of the input training data 30t becomes the size of the central image 61b. The training data generation unit 45 then makes the sizes of the end images 61c, 61d, and the background image 61e the same as (equal to) the size of the central image 61b. At this time, if the training data generation unit 45 increases the horizontal size of each of the end image 61c, end image 61d, and background image 61e, it places duplicated and horizontally flipped versions of each of the end image 61c, end image 61d, and background image 61e to the left and right of each of them. For example, if the difference in size between the left and right sides is not an integer multiple, duplicated images that are cut off midway are placed on both the left and right sides of the image. Also, if images that have been duplicated multiple times are placed side by side, the images are arranged so that adjacent images are flipped. That is, the unflipped end image 61c, end image 61d, and background image 61e are placed alternately with each other, and the flipped end image 61c, end image 61d, and background image 61e are placed alternately with each other. Similarly, when increasing the size in the vertical direction (up and down direction), the training data generation unit 45 duplicates and flips the images while arranging them side by side.

[0043] Furthermore, as shown in Figure 12, the training data generation unit 45 generates multiple output training data 31t based on the identified images 62b, 62c, 62d, and 62e. The training data generation unit 45 performs the same processing on the identified images 62b to 62e as it does on the input training data 30t. That is, the training data generation unit 45 generates multiple output training data 31t of a common size by arranging the duplicated identified images 62b to 62e inverted relative to each other, similar to the center image 61b, edge image 61c, edge image 61d, and background image 61e corresponding to each of the identified images 62b to 62e. Note that since the identified image 62e corresponding to the background image 61e is an image that is completely black, instead of performing the inversion and arrangement process described above, the size of the output training data 31t can be obtained to acquire output training data 31t that is completely black. In this way, the training data generation unit 45 generates multiple input training data 30t and multiple output training data 31t, which are images of a common size.

[0044] As shown in Figure 5, the learning unit 46 performs machine learning to generate a trained model 50 using input training data 30t based on the central image 61b, edge images 61c and 61d, and background image 61e, and output training data 31t based on the corresponding identification images 62b, 62c, 62d, and 62e for the central image 61b, edge images 61c and 61d, and background image 61e, respectively. In other words, the learning unit 46 performs machine learning to generate a trained model 50 using a dataset consisting of multiple input training data 30t and multiple output training data 31t generated by the training data generation unit 45. The trained model 50 is generated by machine learning using deep learning. For example, the learning unit 46 generates the trained model 50 by machine learning based on U-Net, a type of Fully Convolutional Network (FCN). The trained model 50 is generated by training it to perform an image transformation (image reconstruction) on each pixel in the input central image 61b, edge images 61c and 61d, and background image 61e, where the parts where the solder balls 104 overlap each other are colored gray, the parts where only one solder ball 104 is included without overlapping are colored white, and the parts where no solder balls 104 are included are colored black.

[0045] (X-ray image analysis method according to this embodiment) Next, the X-ray image analysis method according to this embodiment will be described with reference to Figure 13. Note that the control processing in steps 201 to 204 is performed by the execution of a program by the control unit 21.

[0046] First, in step 201, multiple training X-ray images 60 are acquired to generate a trained model 50. Next, in step 202, the trained model 50 is generated based on the multiple training X-ray images 60. The generated trained model 50 is stored in the storage unit 22. Next, in step 203, multiple X-ray images 30 to be analyzed are acquired. The multiple X-ray images 30 to be analyzed are generated by taking X-ray images of each of the multiple subjects 101 to be examined. Next, in step 204, the acquired multiple X-ray images 30 are analyzed using the trained model 50 generated in step 202. In the analysis using the trained model 50, multiple identification result images 31 are acquired from the multiple X-ray images 30 to be analyzed. Then, based on the acquired multiple identification result images 31, it is determined whether or not there are abnormalities in the multiple solder balls 104 contained in the subject 101. Note that the acquisition of multiple teacher X-ray images 60 in step 201 and the acquisition of multiple X-ray images 30 in step 203 may be performed in any order. Alternatively, multiple teacher X-ray images 60 may be selected from among the acquired multiple X-ray images 30.

[0047] (Method for generating pre-trained models) Next, the method for generating a trained model according to this embodiment will be described with reference to Figure 14. Note that the control processing in steps 301 to 309 corresponds to the control processing for generating the trained model 50 in step 202 of Figure 13. Furthermore, the control processing in steps 301 to 309 is performed by the execution of a program by the control unit 21.

[0048] First, in step 301, a target region 60a is set in one of the acquired multiple teacher X-ray images 60. The target region 60a is set by a region setting operation, which is an input operation to the operation unit 24.

[0049] Next, in step 302, the ranges of portions 60b, 60c, and 60d, which are extracted from a single teacher X-ray image 60, are automatically set based on the set target region 60a.

[0050] Next, in step 303, it is determined whether or not a range setting operation, which is an input operation to change (set) the range of each of the parts 60b, 60c, and 60d, has been accepted. If it is determined that a range setting operation has been accepted, the process proceeds to step 304. If it is determined that a range setting operation has not been accepted, the process proceeds to step 305.

[0051] In step 304, the ranges of parts 60b, 60c, and 60d are newly set based on the range setting operation. At this time, the ranges of parts 60b, 60c, and 60d are set to be greater than or equal to a predetermined threshold. In other words, operations to set a range smaller than the predetermined threshold are not accepted. Then, the process proceeds to step 305.

[0052] In step 305, the background portion 60e is set based on the input operation to the control unit 24. Note that setting the background portion 60e in step 305 may be omitted.

[0053] Next, in step 306, a central image 61b, edge images 61c and 61d, and a background image 61e are obtained. Specifically, from a teacher X-ray image 60 corresponding to an X-ray image 30 generated by irradiating a plurality of solder balls 104 arranged in a regular manner with X-rays, a central image 61b is obtained by cutting out part 60b, edge images 61c and 61d are obtained by cutting out parts 60c and 60d different from part 60b, and a background image 61e is obtained by cutting out the background part 60e. In addition, based on the respective ranges of parts 60b, 60c, and 60d set in step 302 or step 304, and the background part 60e set in step 305, a central image 61b, edge images 61c and 61d, and a background image 61e of a common range are obtained collectively from each of the plurality of teacher X-ray images 60.

[0054] Next, in step 307, identification information is acquired to identify the area of ​​the solder ball 104 in the center image 61b, the edge image 61c, the edge image 61d, and the background image 61e, based on the input operation to the operation unit 24. Specifically, identification images 62b, 62c, 62d, and 62e, corresponding to the center image 61b, the edge image 61c, the edge image 61d, and the background image 61e, are acquired based on the input operation to the operation unit 24.

[0055] Next, in step 308, input training data 30t and output training data 31t are generated. Specifically, based on the central image 61b, edge image 61c, edge image 61d, and background image 61e acquired in step 306, multiple input training data 30t images of a common size are generated by duplicating and inverting each of the central image 61b, edge image 61c, edge image 61d, and background image 61e and arranging them side by side. Also, based on the identification images 62b, 62c, 62d, and 62e acquired in step 307, multiple output training data 31t images of a common size are generated by duplicating and inverting each of the identification images 62b to 62e and arranging them side by side.

[0056] Next, in step 309, machine learning is performed to generate a trained model 50. Specifically, deep learning machine learning is performed to generate a trained model 50 for analyzing the X-ray image 30 by using multiple input training data 30t generated in step 308 based on the central image 61b, edge image 61c, edge image 61d, and background image 61e, and multiple output training data 31t generated in step 308 based on the identified images 62b to 62e.

[0057] (Effects of this embodiment) In this embodiment, the following effects can be obtained.

[0058] In this embodiment, as described above, the X-ray imaging system 100 obtains a central image 61b (first image) from a training X-ray image 60 which includes solder balls 104 (object to be inspected) arranged in a regular manner, with portion 60b (first portion) cut out, and edge images 61c and 61d (second images) from which portions 60c and 60d (second portion), which are different from portion 60b, are cut out. Then, identification images 62b to 62d (identification information) are obtained to identify the region of the solder balls 104 in the central image 61b and the edge images 61c and 61d, respectively. This allows for the acquisition of identification images 62b to 62d for identifying regions in the central image 61b, which is extracted from portion 60b of the training X-ray image 60, and in the edge images 61c and 61d, which are extracted from portions 60c and 60d, respectively. This reduces the workload for the operator in acquiring identification images 62b to 62d compared to acquiring identification images 62b to 62d corresponding to the entire training X-ray image 60. As a result, the workload for acquiring output training data 31t for machine learning is reduced, thus reducing the workload for the operator in creating training data to generate a trained model 50 for identifying regions of solder balls 104. Furthermore, when X-rays are irradiated from a point source during X-ray imaging, the image at the center of the X-ray irradiation and the image at the peripheral portion separated from the irradiation center in the generated X-ray image 30 are images of X-rays incident from different angular directions. In contrast, in this embodiment, a central image 61b is obtained by cutting out portion 60b from a training X-ray image 60 that includes solder balls 104 arranged in a regular manner, and edge images 61c and 61d are obtained by cutting out portions 60c and 60d that are different from portion 60b. As a result, since the solder balls 104 in the training X-ray image 60 are arranged in a regular manner, the images of the solder balls 104 in the central image 61b and the edge images 61c and 61d can be used as projection images onto a structure having a common arrangement relationship.Therefore, by acquiring the central image 61b and the edge images 61c and 61d, it is possible to extract portions 60b, 60c, and 60d from the training X-ray image 60 that contain images of X-rays incident on structures with a common arrangement relationship from different angular directions. Therefore, even when using images extracted from a part of the training X-ray image 60 rather than the whole, it is possible to acquire training data that reflects the overall projection image of the training X-ray image 60, thereby suppressing a decrease in the accuracy of the classification results by the trained model 50 generated by machine learning. As a result, by using the central image 61b and the edge images 61c and 61d extracted from the training X-ray image 60, it is possible to reduce the workload of the operator in creating training data for generating the trained model 50 while suppressing a decrease in the accuracy of the classification results.

[0059] Furthermore, in the above embodiment, additional effects can be obtained by configuring it as follows.

[0060] In other words, in this embodiment, as described above, the excised image acquisition unit 43 (control unit 21) acquires a central image 61b (first image) from the teacher X-ray image 60, which is an excised portion 60b (first portion) including the central portion of the target region 60a containing the solder ball 104 (object to be inspected), and edge images 61c and 61d (second image) from which the edges of the target region 60a, including portions 60c and 60d (second portion), are excised. Here, when performing non-destructive testing of the solder ball 104 by X-ray imaging, X-ray irradiation is generally performed with the center of the region where the solder ball 104 is located as the irradiation center. In that case, it is considered that the difference in images between the central portion and the edge portion of the target region 60a will be large. Taking this into consideration, in this embodiment, a central image 61b is obtained by cutting out a portion 60b including the central part of the target region 60a from the training X-ray image 60, and edge images 61c and 61d are obtained by cutting out portions 60c and 60d including the edge parts of the target region 60a. With this configuration, by obtaining the central image 61b, it is possible to obtain the image projected onto the solder ball 104 at the center of X-ray irradiation, and by obtaining the edge images 61c and 61d, it is possible to obtain the image of the X-rays projected onto the solder ball 104 in the most inclined irradiation direction in the training X-ray image 60. Therefore, by using the central image 61b and the edge images 61c and 61d, training data can be obtained that better reflects the overall projection image of the training X-ray image 60. As a result, the decrease in the accuracy of the identification results by the trained model 50 generated by machine learning can be further suppressed.

[0061] Furthermore, in this embodiment, as described above, the X-ray irradiation unit 11 irradiates X-rays onto a plurality of solder balls 104 (solder material, object to be inspected) arranged in a grid pattern on the substrate 102 in a regular manner, and the identification information acquisition unit 44 (control unit 21) acquires identification images 62b to 62d as identification information, in which the regions of the solder balls 104 in the central image 61b (first image) and edge images 61c and 61d (second image) are identified. With this configuration, even when generating a trained model 50 for analyzing the solder balls 104 arranged in a grid pattern on the substrate 102 in a regular manner using machine learning, the workload for generating the trained model 50 can be reduced.

[0062] Furthermore, in this embodiment, as described above, an operation unit 24 is provided to receive input operations from the operator. The operation unit 24 receives an area setting operation to set a target area 60a that includes the solder balls 104 (objects to be inspected) in the training X-ray image 60. The excised image acquisition unit 43 (control unit 21) automatically sets the range of part 60b (first part) and the ranges of part 60c and part 60d (second part) from the training X-ray image 60 based on the target area 60a in the training X-ray image 60 set by the area setting operation received by the operation unit 24. With this configuration, by performing an area setting operation to set the target area 60a to include the solder balls 104 which are arranged in a regular manner in the training X-ray image 60, the ranges of part 60b and part 60c and part 60d can be automatically set to reflect the regularity of the solder balls 104 in the training data. Therefore, the workload for setting the range of section 60b and the ranges of sections 60c and 60d by considering the regularity of the solder balls 104 can be reduced. As a result, the workload for the operator in creating training data to generate the trained model 50 can be further reduced.

[0063] Furthermore, in this embodiment, as described above, the X-ray imaging system 100 includes a display unit 23 that displays the teacher X-ray image 60 and an operation unit 24 that receives input operations from the operator. The display unit 23 visually displays an indication of the range of portion 60b (first portion) to be cut out from the teacher X-ray image 60, and an indication of the ranges of portion 60c and portion 60d (second portion). The cut-out image acquisition unit 43 (control unit 21) sets the range of portion 60b, portion 60c and portion 60d to be cut out from the teacher X-ray image 60 based on the input operations received by the operation unit 24. With this configuration, the operator (worker) can set the range of portion 60b and portion 60c and portion 60d while visually recognizing the indication of the range of portion 60b and portion 60c and portion 60d displayed on the display unit 23. Therefore, by visually inspecting the display unit 23, the operator can easily set the range of portion 60b and the ranges of portions 60c and 60d according to the regularity of the solder ball 104 (object to be inspected). As a result, training data can be acquired based on the central image 61b, edge image 61c and edge image 61d, which are appropriately cut out from the training X-ray image 60 according to the regularity of the solder ball 104. This further suppresses the decrease in the accuracy of the identification results by the trained model 50 generated by machine learning, even when using training data based on images that have been partially cut out.

[0064] Furthermore, in this embodiment, as described above, the cropped image acquisition unit 43 (control unit 21) sets the range of portion 60b (first portion) and the ranges of portion 60c and portion 60d (second portion) to be greater than or equal to a predetermined threshold, based on the input operation received by the operation unit 24. With this configuration, it is possible to suppress the ranges of the cropped portion 60b and portions 60c and 60d from becoming too small. Therefore, when training using training data based on the cropped central image 61b and the edge images 61c and 61d, it is possible to suppress a decrease in the accuracy of the identification result by the generated trained model 50. As a result, when setting the ranges of portion 60b and portions 60c and 60d based on the input operation, it is possible to effectively suppress a decrease in the accuracy of the identification result by the trained model 50 generated by machine learning.

[0065] Furthermore, in this embodiment, as described above, the control unit 21 (model generation unit) includes a training data generation unit 45 that generates input training data 30t based on the central image 61b (first image) and the end images 61c and 61d (second image) acquired by the cropped image acquisition unit 43. The training data generation unit 45 is configured to make the input training data 30t a common size by at least one of the following: arranging the duplicated multiple central images 61b side by side, and arranging the duplicated multiple end images 61c and 61d side by side. With this configuration, even when generating input training data 30t based on the central image 61b and the end images 61c and 61d from which different parts have been cropped, the size of the input training data 30t can be made a common size, thereby suppressing a decrease in the accuracy of the identification results by the generated trained model 50.

[0066] Furthermore, in this embodiment, as described above, the training data generation unit 45 (control unit 21) is configured to make the input training data 30t a common size by at least one of the following: arranging the central images 61b (first images) side by side while inverting them, and arranging the end images 61c and 61d (second images) side by side while inverting them. With this configuration, at least one of arranging the central images 61b side by side while inverting them, and arranging the end images 61c and 61d side by side while inverting them, makes it possible to smooth the boundaries between the arranged images when at least one of the central images 61b side by side and the end images 61c and 61d side by side. As a result, blurring of the arranged images can be suppressed, and the decrease in the accuracy of the identification result by the generated trained model 50 can be further suppressed when the input training data 30t is made a common size.

[0067] Furthermore, in this embodiment, as described above, the extracted image acquisition unit 43 (control unit 21) acquires a background image 61e from the training X-ray image 60, which is an extracted background portion 60e that does not contain the solder ball 104 (object to be inspected). The learning unit 46 (control unit 21) then performs machine learning to generate a trained model 50 using input training data 30t based on the central image 61b (first image), edge image 61c, edge image 61d (second image), and background image 61e. With this configuration, machine learning can be performed using the background image 61e, which is an extracted background portion 60e, in addition to portions 60b, 60c, and 60d of the target region 60a that contains the solder ball 104. Therefore, compared to performing machine learning using only the central image 61b, edge image 61c, and edge image 61d that contain the solder ball 104 without using the background image 61e, performing machine learning using the background image 61e can improve the accuracy of the identification results by the generated trained model 50.

[0068] Furthermore, in this embodiment, as described above, an operation unit 24 is provided to receive input operations from an operator. The operation unit 24 receives a range setting operation to set the range of part 60b (first part) and the ranges of part 60c and part 60d (second part) in one of the multiple teacher X-ray images 60. The extracted image acquisition unit 43 (control unit 21) acquires from each of the multiple teacher X-ray images 60 a plurality of central images 61b (first images) from which a common range part 60b set based on the range setting operation for one teacher X-ray image 60 is extracted, and a plurality of edge images 61c and edge images 61d (second images) from which a common range part 60c and part 60d (second part) set based on the range setting operation for one teacher X-ray image 60 is extracted. With this configuration, a common area portion 60b, and common areas portions 60c and 60d can be extracted from each of the multiple training X-ray images 60. Therefore, when performing machine learning using multiple training X-ray images 60 in which the arrangement of solder balls 104 (objects to be inspected) is common, multiple central images 61b and multiple edge images 61c and edge images 61d can be extracted all at once. As a result, when generating training data using multiple training X-ray images 60, the workload in creating the training data can be reduced.

[0069] (Effects of the pre-trained model generation method according to this embodiment) The pre-trained model generation method of this embodiment can achieve the following effects.

[0070] In the trained model generation method of this embodiment, as described above, a central image 61b (first image) is obtained by cutting out a portion 60b (first portion) from a training X-ray image 60 that includes solder balls 104 (object to be inspected) arranged in a regular manner, and edge images 61c and 61d (second images) are obtained by cutting out portions 60c and 60d (second portions) that are different from portion 60b. Then, identification images 62b to 62d (identification information) are obtained to identify the regions of the solder balls 104 in the central image 61b and the edge images 61c and 61d, respectively. This allows for the acquisition of identification images 62b to 62d for identifying regions in the central image 61b, which is extracted from portion 60b of the training X-ray image 60, and in the edge images 61c and 61d, which are extracted from portions 60c and 60d, respectively. This reduces the workload for the operator in acquiring identification images 62b to 62d compared to acquiring identification images 62b to 62d corresponding to the entire training X-ray image 60. As a result, the workload for acquiring output training data 31t for machine learning can be reduced, thus providing a trained model generation method that reduces the workload for the operator in creating training data for generating a trained model 50 for identifying regions of solder balls 104. Furthermore, when X-rays are irradiated from a point source during X-ray imaging, the image at the center of the X-ray irradiation and the image at the peripheral portion separated from the irradiation center in the generated X-ray image 30 are images of X-rays incident from different angular directions. In contrast, in this embodiment, a central image 61b is obtained by cutting out portion 60b from a training X-ray image 60 that includes solder balls 104 arranged in a regular manner, and edge images 61c and 61d are obtained by cutting out portions 60c and 60d that are different from portion 60b. As a result, since the solder balls 104 in the training X-ray image 60 are arranged in a regular manner, the images of the solder balls 104 in the central image 61b and the edge images 61c and 61d can be used as projection images onto a structure having a common arrangement relationship.Therefore, by acquiring the central image 61b and the edge images 61c and 61d, it is possible to extract portions 60b, 60c, and 60d from the training X-ray image 60 that contain images of X-rays incident on structures having a common arrangement relationship from different angular directions. Therefore, even when using images extracted from a part of the training X-ray image 60 rather than the whole, it is possible to acquire training data that reflects the overall projection image of the training X-ray image 60, thereby suppressing a decrease in the accuracy of the identification results by the trained model 50 generated by machine learning. As a result, by using the central image 61b and the edge images 61c and 61d extracted from the training X-ray image 60, it is possible to provide a trained model generation method that reduces the workload of the operator in the work of creating training data for generating the trained model 50 while suppressing a decrease in the accuracy of the identification results.

[0071] [Differentiation] It should be noted that the embodiments disclosed herein are illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims rather than by the description of the embodiments above, and further includes all modifications (exceptions) within the meaning and scope equivalent to the claims.

[0072] For example, in the above embodiment, an example was shown in which input training data 30t is generated based on a central image 61b (first image) obtained by cutting out a portion 60b (first portion) including the central part of the target region 60a, and edge images 61c and 61d (second image) obtained by cutting out portions 60c and 60d (second portion) including the edge parts of the target region 60a. However, the present invention is not limited to this. In the present invention, input training data may be generated by obtaining a first image from a portion of the target region other than the center, and obtaining a second image from a portion of the target region other than the edge.

[0073] Furthermore, in the above embodiment, an example was shown in which the range of part 60b (first part) and the ranges of parts 60c and 60d (second part) are automatically set based on the setting of the target area 60a by the area setting operation, but the present invention is not limited to this. In the present invention, the range of the first part and the range of the second part may each be set individually and directly based on an input operation.

[0074] Furthermore, while the above embodiment shows an example in which the range of part 60b (first part) and the ranges of parts 60c and 60d (second parts) are changed (set) based on input operations, the present invention is not limited thereto. In the present invention, either or both of the ranges of the first part and the second part may be kept unchanged. That is, when the target area 60a is set, the ranges of the first part and the second part may be set automatically, and the first and second images may be cropped.

[0075] Furthermore, in the above embodiment, an example was shown in which duplicated images were inverted and arranged in order to make the input training data 30t of a common size, but the present invention is not limited to this. In the present invention, images may be arranged as they are without inverting them in order to generate the input training data.

[0076] Furthermore, although the above embodiment shows an example of obtaining a background image 61e by cutting out the background portion 60e in order to generate input training data 30t, the present invention is not limited to this. In the present invention, input training data may be generated without using a background image.

[0077] Furthermore, the above embodiment shows an example in which a central image 61b (first image), edge image 61c, and edge image 61d (second image) of a common range are obtained from multiple teacher X-ray images 60 based on a range set in one teacher X-ray image 60, but the present invention is not limited thereto. In the present invention, a first image of a range different from each other and a second image of a range different from each other may be obtained in each of the multiple teacher X-ray images.

[0078] Furthermore, while the above embodiment shows an example of generating a trained model 50 that is trained to identify the region of the solder ball 104 (object to be inspected, solder material) in each of the central image 61b (first image) and edge images 61c and 61d (second image), the present invention is not limited thereto. In the present invention, a trained model may be generated that identifies the region of an abnormal part contained in the solder ball (object to be inspected, solder material). For example, the identification information acquisition unit may be configured to acquire an identification image as identification information in which the region of a void (hole) contained in the solder ball is identified as the abnormal part. Alternatively, instead of a solder ball in a BGA (Ball Grid Array), a trained model may be generated that identifies the region of multiple solder materials at the connection portion of multiple terminals in an LGA (Land Grid Array) where terminals are arranged in a grid, or the region of an abnormal part contained in multiple solder materials. Alternatively, instead of solder material, a trained model may be generated that identifies the region of multiple terminals, or the region of an abnormal part contained in multiple terminals.

[0079] Furthermore, although the above embodiment shows an example in which the cropped image acquisition unit 43 acquires two portions 60c and 60d (second portions) of equal size, the present invention is not limited to this. In the present invention, the cropped image acquisition unit may acquire one second portion. Also, when acquiring two second portions, the sizes of the two second portions may be made different.

[0080] Furthermore, although the above embodiment shows an example in which the image generation unit 13 that generates the X-ray image 30 (training X-ray image 60) is provided separately from the control unit 21 (model generation unit) that generates the trained model 50, the present invention is not limited to this. In the present invention, the generation of the X-ray image and the generation of the trained model may be performed by a common control device. Also, the analysis using the trained model may be performed by a control device provided separately from the model generation unit that generates the trained model.

[0081] [Pattern] Those skilled in the art will understand that the exemplary embodiments described above are specific examples of the following embodiments.

[0082] (Item 1) An X-ray irradiation unit that irradiates X-rays onto objects to be inspected that are arranged in a regular pattern, An X-ray detection unit for detecting X-rays emitted from the aforementioned X-ray irradiation unit, An image generation unit that generates an X-ray image based on the X-rays detected by the X-ray detection unit, The system comprises a model generation unit that generates a trained model for analyzing the X-ray images generated by the image generation unit, The aforementioned model generation unit, An image extraction unit acquires a first image obtained by extracting a first portion from a teacher X-ray image including the object to be inspected arranged in a regular manner, and a second image obtained by extracting a second portion different from the first portion. An identification information acquisition unit acquires identification information for identifying at least one of the regions of the object to be inspected in each of the first and second images, and the regions of abnormal parts included in the object to be inspected. An X-ray imaging system comprising: a learning unit that performs machine learning to generate the trained model using input training data based on the first image and the second image, and output training data based on the identification information.

[0083] (Item 2) The X-ray imaging system according to item 1, wherein the excision image acquisition unit acquires a first image from the teacher X-ray image, the first image being extracted from which the central portion of the target region containing the object to be examined is extracted, and the second image being extracted from which the second portion including the edge portion of the target region is extracted.

[0084] (Item 3) The X-ray irradiation unit irradiates the object to be inspected, which includes a plurality of solder materials arranged in a grid pattern on a substrate in a regular manner, with X-rays. The X-ray imaging system according to item 1 or 2, wherein the identification information acquisition unit acquires an identification image as the identification information in which at least one of the regions of the plurality of solder materials and the regions of the abnormal parts included in the plurality of solder materials in each of the first image and the second image is identified.

[0085] (Item 4) It further includes an operating section that accepts input operations from the operator, The operation unit receives a region setting operation to set a target region in the teacher's X-ray image that includes the object to be examined. The X-ray imaging system according to any one of items 1 to 3, wherein the extracted image acquisition unit automatically sets the range of the first part and the range of the second part from the training X-ray image based on the target area in the training X-ray image set by the area setting operation received by the operation unit.

[0086] (Item 5) A display unit for displaying the aforementioned teacher's X-ray image, It further includes an operating unit that accepts input operations from the operator, The display unit visually displays an indication of the range of the first portion extracted from the teacher's X-ray image and an indication of the range of the second portion. The X-ray imaging system according to any one of items 1 to 4, wherein the extracted image acquisition unit sets the range of the first portion and the range of the second portion to be extracted from the teacher X-ray image based on the input operation received by the operation unit.

[0087] (Item 6) The X-ray imaging system according to item 5, wherein the excised image acquisition unit sets the range of the first part and the range of the second part to have a size greater than or equal to a predetermined threshold based on the input operation received by the operation unit.

[0088] (Item 7) The model generation unit further includes a training data generation unit that generates the input training data based on the first image and the second image acquired by the cropped image acquisition unit, The X-ray imaging system according to any one of items 1 to 6, wherein the training data generation unit is configured to make the input training data a common size by at least one of arranging a plurality of duplicated first images side by side and arranging a plurality of duplicated second images side by side.

[0089] (Item 8) The X-ray imaging system according to item 7, wherein the training data generation unit is configured to make the input training data a common size by at least one of the following: arranging the first images side by side while inverting them, and arranging the second images side by side while inverting them.

[0090] (Item 9) The aforementioned extracted image acquisition unit acquires a background image from the teacher X-ray image, in which the background portion that does not contain the object to be examined is extracted. The X-ray imaging system according to any one of items 1 to 8, wherein the learning unit performs machine learning to generate the trained model using the input training data based on the first image, the second image and the background image.

[0091] (Item 10) It further includes an operating section that accepts input operations from the operator, The operation unit receives a range setting operation to set the range of the first portion and the range of the second portion in one of the multiple teacher X-ray images. The X-ray imaging system according to any one of items 1 to 9, wherein the extracted image acquisition unit acquires a plurality of first images obtained by extracting a first portion of a common range set based on a range setting operation for one of the plurality of teacher X-ray images, and a plurality of second images obtained by extracting a second portion of a common range set based on a range setting operation for one of the teacher X-ray images.

[0092] (Item 11) The steps include obtaining a first image obtained by cutting out a first portion from a training X-ray image corresponding to an X-ray image generated by irradiating an object to be inspected, which is arranged in a regular manner, and a second image obtained by cutting out a second portion different from the first portion, A step of obtaining identification information for identifying at least one of the region of the object to be inspected in each of the first and second images, and the region of the abnormal part included in the object to be inspected; A method for generating a trained model, comprising the step of performing machine learning to generate a trained model for analyzing an X-ray image using input training data based on the first image and the second image, and output training data based on the identification information. [Explanation of Symbols]

[0093] 11 X-ray irradiation section 12 X-ray detection unit 13 Image generation unit 21 Control Unit (Model Generation Unit) 23 Display section 24 Control section 30 X-ray images 30t Input training data 31t Output Training Data 43 Cutout image acquisition unit 44 Identification Information Acquisition Unit 45 Training Data Generation Unit 46 Learning Department 50 pre-trained models 60 X-ray images for teachers 60a Target area 60b part (1st part) 60c, 60d part (second part) 60e background part 61b Central image (first image) 61c, 61d End images (second image) 61e Background image 62b, 62c, 62d, 62e Identification Images 100 X-ray imaging system 102 circuit boards 104 Solder ball (object to be inspected, solder material)

Claims

1. An X-ray irradiation unit that irradiates X-rays onto objects to be inspected that are arranged in a regular pattern, An X-ray detection unit that emits X-rays irradiated from the aforementioned X-ray irradiation unit, An image generation unit that generates an X-ray image based on the X-rays detected by the X-ray detection unit, The system comprises a model generation unit that generates a trained model for analyzing the X-ray images generated by the image generation unit, The aforementioned model generation unit, An image extraction unit that extracts, from the same teacher X-ray image including the objects to be inspected arranged in a regular manner, a first image obtained by extracting a first portion including the central portion of the target area containing the objects to be inspected, and a second image obtained by extracting a second portion different from the first portion, which includes the edge portion of the target area and from which X-rays are incident from an angular direction different from the angular direction of the X-rays incident on the first portion. An identification information acquisition unit acquires identification information for identifying at least one of the regions of the object to be inspected in each of the first and second images, and the regions of abnormal parts included in the object to be inspected. An X-ray imaging system comprising: a learning unit that performs machine learning to generate the trained model using input training data based on both the first image and the second image, and output training data based on the identification information.

2. The X-ray irradiation unit irradiates the object to be inspected, which includes a plurality of solder materials arranged in a grid pattern on a substrate in a regular manner, with X-rays. The X-ray imaging system according to claim 1, wherein the identification information acquisition unit acquires an identification image as the identification information in which at least one of the regions of the plurality of solder materials and the regions of the abnormal parts included in the plurality of solder materials in each of the first image and the second image is identified.

3. It further includes an operating section that accepts input operations from the operator, The operation unit receives a region setting operation to set the target region in which the object to be examined is included in the teacher's X-ray image, The X-ray imaging system according to claim 1, wherein the extracted image acquisition unit automatically sets the range of the first portion and the range of the second portion from the training X-ray image based on the target region in the training X-ray image set by the region setting operation received by the operation unit.

4. A display unit for displaying the aforementioned X-ray image for teachers, It further includes an operating unit that accepts input operations from the operator, The display unit visually displays an indication of the range of the first portion extracted from the teacher's X-ray image and an indication of the range of the second portion. The X-ray imaging system according to claim 1, wherein the extracted image acquisition unit sets the range of the first portion to be extracted from the teacher X-ray image and the range of the second portion based on the input operation received by the operation unit.

5. The X-ray imaging system according to claim 4, wherein the excised image acquisition unit sets the range of the first portion and the range of the second portion to have a size greater than or equal to a predetermined threshold based on an input operation received by the operation unit.

6. The model generation unit further includes a training data generation unit that generates the input training data based on the first image and the second image acquired by the cropped image acquisition unit, The X-ray imaging system according to claim 1, wherein the training data generation unit is configured to make the input training data a common size by at least one of arranging a plurality of duplicated first images side by side and arranging a plurality of duplicated second images side by side.

7. The X-ray imaging system according to claim 6, wherein the training data generation unit is configured to make the input training data a common size by at least one of the following: arranging the first images side by side while inverting them, and arranging the second images side by side while inverting them.

8. The aforementioned extracted image acquisition unit acquires a background image from the teacher X-ray image, in which the background portion that does not contain the object to be inspected is extracted. The X-ray imaging system according to claim 1, wherein the learning unit performs machine learning to generate the trained model using the input training data based on the first image, the second image and the background image.

9. It further includes an operating section that accepts input operations from the operator, The operation unit receives a range setting operation to set the range of the first portion and the range of the second portion in one of the multiple teacher X-ray images. The X-ray imaging system according to claim 1, wherein the extracted image acquisition unit acquires a plurality of first images from each of the plurality of teacher X-ray images, each of which is an extracted portion of a common range set based on a range setting operation for one teacher X-ray image, and a plurality of second images from which is extracted a second portion of a common range set based on a range setting operation for one teacher X-ray image.

10. The steps include obtaining a first image obtained by cutting out a first portion including the central portion of the target area containing the object to be inspected from the same training X-ray image corresponding to an X-ray image generated by irradiating an object to be inspected arranged in a regular manner with X-rays, and a second image obtained by cutting out a second portion different from the first portion, which includes the edge portion of the target area and is incident on by X-rays from an angular direction different from the angular direction of the X-rays incident on the first portion, A step of obtaining identification information for identifying at least one of the region of the object to be inspected in each of the first and second images, and the region of the abnormal part included in the object to be inspected, A method for generating a trained model, comprising the step of performing machine learning to generate a trained model for analyzing an X-ray image using input training data based on both the first image and the second image, and output training data based on the identification information.

Images