Ai model creation method and inspection device

Abstract

To provide a AI model creation method capable of creating a AI model for performing correction for reducing noise from an image datum of an inspection object by using a low-quality image datum with noise, and an inspection device mounted with the creation method.SOLUTION: A method of creating an AI model in an inspection device (1) that performs correction for reducing noise from image data of an inspection object (12) using an AI model created by unsupervised machine learning, includes creating an AI model by learning a data set in which one of two or more pieces of image data captured at different timings in the same field of view of the inspection object (12) is set as input image data and the other piece is set as target image data.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a method for creating an AI model and an inspection device. [Background technology]

[0002] In inspection equipment, the presence of noise in inspection image data reduces the inspection accuracy. Increasing the radiation dose to reduce noise can shorten the lifespan of the components on the object being inspected. One method for reducing noise from image data is to use high-quality, noise-free image data as a training dataset and have AI learn to remove the noise (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 031984 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when noise-free, high-quality image data is used as a learning dataset and trained by AI to remove noise, image data with reduced noise is output, making it appear as if the product is good, which poses a problem. There is also the issue of having to prepare high-quality image data with minimal noise for training.

[0005] The present invention has been made in consideration of these problems, and aims to provide an AI model creation method that uses unsupervised learning to create an AI model for performing corrections to reduce noise from image data of an inspection target, using noisy, low-quality image data, and an inspection device in which this creation method is implemented. [Means for solving the problem]

[0006] In order to solve the above problem, the method for creating an AI model according to the present invention is a method for creating an AI model in an inspection device that performs correction to reduce noise from image data of an object to be inspected using an AI model created by unsupervised machine learning, and creates the AI ​​model by learning a data set in which one of two or more image data images captured at different times in the same field of view of the object to be inspected is used as input image data and one of the remaining images is used as target image data.

[0007] In addition, in the AI ​​model creation method according to the present invention, it is desirable that the AI ​​model is a learning result of information for generating the target image data from the input image data included in the dataset.

[0008] In addition, in the AI ​​model creation method according to the present invention, it is desirable that each of the data sets is image data acquired under different imaging conditions that affect noise.

[0009] In the AI ​​model creation method according to the present invention, it is preferable that the imaging conditions are conditions under which the amount of radiation irradiated onto the object to be inspected changes.

[0010] In addition, in the AI ​​model creation method according to the present invention, it is desirable that the AI ​​model is a learning result obtained by deep learning.

[0011] In addition, the inspection device of the present invention comprises a light source, a detector, a holding unit that holds an object under inspection, and a control unit that generates three-dimensional image data of the object under inspection by changing the relative positions of the light source, the detector, and the holding unit, and using at least two pieces of transmission image data of the object under inspection acquired by the detector using light emitted from the light source and transmitted through the object under inspection, and the control unit performs correction to reduce noise from the transmission image data or the three-dimensional image data using an AI model created by any of the AI ​​model creation methods described above.

[0012] Furthermore, in the inspection device according to the present invention, it is desirable that the control unit has an image storage unit that stores the transmission image data or the three-dimensional image data, a learning dataset creation unit that creates the dataset from the transmission image data or the three-dimensional image data, a noise reduction learning unit that creates the AI ​​model by learning using the AI ​​model creation method based on the dataset, and a noise reduction unit that performs correction to reduce noise from the transmission image data or the three-dimensional image data using the AI ​​model.

[0013] Furthermore, in the inspection device according to the present invention, it is desirable that the control unit has an image capturing unit that generates, for the learning, transmission image data under different imaging conditions or three-dimensional image data under different imaging conditions generated from the transmission image data, the learning dataset creation unit creates a dataset under different imaging conditions from the transmission image data under different imaging conditions or the three-dimensional image data under different imaging conditions, and the noise reduction learning unit creates the AI ​​model using the dataset under different imaging conditions.

[0014] Furthermore, in the inspection device according to the present invention, it is desirable that the control unit has an area setting unit that accepts an area in the transmission image data or the three-dimensional image data for correction to reduce noise, and the noise reduction unit performs correction to reduce noise on the area set by the area setting unit in the transmission image data or the three-dimensional image data. [Effects of the Invention]

[0015] According to the AI ​​model creation method and inspection device of the present invention, an AI model for performing correction to reduce noise from image data of the inspection target can be created using noisy, low-quality image data. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is an explanatory diagram for explaining a configuration of an inspection device according to an embodiment of the present invention. [Figure 2] FIG. 2 is an explanatory diagram for explaining each functional block of a control unit of the inspection device. [Figure 3] 4 is a flowchart for explaining an inspection process in the inspection device. [Figure 4] 10 is a flowchart for explaining a transmission image capturing and reconstruction image generating process in the inspection process. DETAILED DESCRIPTION OF THE INVENTION

[0017] A preferred embodiment of the present invention will now be described with reference to the drawings. As shown in Fig. 1, an examination device 1 according to this embodiment includes a control unit 10, which is configured as a processing device such as a personal computer (PC), a monitor 11, and an imaging unit 32. The imaging unit 32 further includes a radiation quality changing unit 14, a radiation generator driving unit 16, a holder driving unit 18, a detector driving unit 20, a radiation generator 22, a holder 24, and a detector 26.

[0018] The radiation generator 22 is a device (ray source) that generates radiation such as X-rays, and generates radiation by, for example, colliding accelerated electrons with a target such as tungsten or diamond. Note that, although the radiation in this embodiment is described as X-rays, the radiation is not limited to this. For example, the radiation may be alpha rays, beta rays, gamma rays, ultraviolet rays, visible light, or infrared rays. The radiation may also be microwaves or terahertz waves.

[0019] The holder 24 holds an electronic board, which is the object under inspection 12. The object under inspection 12 held by the holder 24 is irradiated with radiation generated by the radiation generator 22, and the radiation that has passed through the object under inspection 12 is detected by the detector 26 to capture an image (acquire as image data). Hereinafter, data of the radiographic image of the object under inspection 12 captured by the detector 26 will be referred to as "transmission image data." As will be described later, in this embodiment, the detector 26 and the holder 24 that holds the electronic board, which is the object under inspection 12, are moved relative to the radiation generator 22 to acquire multiple pieces of transmission image data, and reconstructed image data (cross-sectional image data), which is three-dimensional image data, is generated from the transmission image data.

[0020] The transmission image data captured by the detector 26 is sent to the control unit 10, where it is reconstructed into image data including the three-dimensional shape of the solder at the joint using a known technique such as the filtered backprojection (FBP) method. The reconstructed image data and transmission image data are then stored in a storage unit within the control unit 10 (for example, a storage unit 34, which will be described later) or in an external storage unit (not shown). Hereinafter, image data obtained by extracting one cross section of the three-dimensional shape calculated based on the transmission image data will be referred to as "cross-sectional image data." Furthermore, a set of one or more cross-sectional image data will be referred to as "three-dimensional image data" or "reconstructed image data." In other words, image data obtained by cutting out an arbitrary cross section from the three-dimensional image data (reconstructed image data) is cross-sectional image data. Such reconstructed image data and cross-sectional image data are output to the monitor 11. Note that the monitor 11 displays not only the reconstructed image data and cross-sectional image data, but also the inspection results of the solder joint state, which will be described later. Hereinafter, the reconstructed image data in this embodiment is also referred to as "planar CT" because it is reconstructed from planar image data (transmission image data) captured by the detector 26, as described above.

[0021] The radiation quality changing unit 14 changes the radiation quality of the radiation generated by the radiation generator 22. The radiation quality is determined by a voltage (hereinafter referred to as "tube voltage") applied to accelerate electrons to be bombarded with a target, and a current (hereinafter referred to as "tube current") that determines the number of electrons. The radiation quality changing unit 14 is a device that controls the tube voltage and tube current. This radiation quality changing unit 14 can be realized using known technology such as a transformer or a rectifier.

[0022] Here, the radiation quality is determined by the radiation brightness and wavelength (spectral distribution of the radiation). Increasing the tube current increases the number of electrons that collide with the target, and therefore the number of radiation photons generated. As a result, the radiation brightness increases. For example, some components, such as capacitors, are thicker than other components, and therefore, to capture transmission images of these components, radiation with high brightness must be irradiated. In such cases, the radiation brightness can be adjusted by adjusting the tube current. In this way, increasing the tube current generates more X-ray photons, which can improve the image quality of the transmission image data, but also increases the radiation dose irradiated to the inspection object 12. On the other hand, decreasing the tube current reduces the radiation dose irradiated to the inspection object 12, but may increase noise in the transmission image data.

[0023] Furthermore, increasing the tube voltage increases the energy of electrons colliding with the target, and the energy (spectrum) of the generated radiation increases. Generally, the higher the energy of radiation, the greater its penetration power into materials and the less likely it is to be absorbed by materials. A transmission image captured using such radiation has low contrast. For this reason, the tube voltage can be used to adjust the contrast of the transmission image. In this way, increasing the tube voltage generates high-energy X-ray photons that easily penetrate the specimen 12, reducing the amount of radiation absorbed by the tissue (subject 12), but also increasing scattered radiation. On the other hand, decreasing the tube voltage increases the amount of radiation absorbed by the tissue (subject 12), but improves the contrast of the transmission image data.

[0024] The inspection device 1 is configured to capture a transmission image of the inspected object 12 and acquire transmission image data by changing the relative positions of the radiation generator 22, which is a light source, the holder 24 that holds the inspected object 12, and the detector 26, and detecting with the detector 26 the radiation emitted from the radiation generator 22 and transmitted through the inspected object 12. Here, however, the description will be based on a configuration in which the inspected object 12 held by the holder 24 is rotationally moved within the beam of radiation emitted from the radiation generator 22 along a rotational orbit (substrate rotational orbit 28) on a substrate rotational orbit plane that is a plane perpendicular to the axis A, and the detector 26 is rotationally moved along a rotational orbit (detector rotational orbit 30) on a detector rotational orbit plane that is different from the substrate rotational orbit plane, thereby detecting the radiation that has transmitted through the inspected object 12 and capturing a transmission image of the inspected object 12 and acquiring transmission image data. However, the present invention is not limited to this configuration. For example, the transmission image data may be acquired by rotating the radiation generator 22 and the detector 26 relative to the holder 24 that holds the object under inspection 12, or by rotating the radiation generator 22 and the holder 24 that holds the object under inspection 12 relative to the detector 26. Furthermore, the transmission image data may be acquired by moving any of the radiation generator 22, the holder 24 that holds the object under inspection 12, and the detector 26 in a straight line rather than by rotating them.

[0025] The radiation generator driving unit 16 has a driving mechanism such as a motor (not shown), and can move the radiation generator 22 up and down along an axis A passing through its focal point (an axis (optical axis) passing through the center of the radiation direction of the radiation emitted from the radiation generator 22, the direction of this axis being referred to as the "Z-axis direction"). This makes it possible to change the distance between the radiation generator 22 and the object under test (electronic board) 12 held by the holder 24, thereby changing the irradiation field and changing the magnification ratio of the transmitted image captured by the detector 26. The position of the radiation generator 22 in the Z-axis direction is detected by a generator position detection unit (not shown), and output to the control unit 10.

[0026] The detector driver 20 also has a drive mechanism such as a motor (not shown) and rotates the detector 26 along the detector rotation orbit 30. The holder driver 18 also has a drive mechanism such as a motor (not shown) and rotates the holder 24 along the substrate rotation orbit 28, thereby moving the holder 24 in parallel with the detector 26. The holder 24 is configured to rotate on the substrate rotation orbit 28 in conjunction with the rotation of the detector 26. This makes it possible to acquire multiple pieces of transmission image data with different projection directions and projection angles while changing the relative positional relationship between the object under test 12 held by the holder 24 and the radiation generator 22. In the inspection apparatus 1 according to this embodiment, the area on the object under test 12 from which transmission image data can be acquired is determined by the size of the radiation detection area of ​​the detector 26 and the relative positions of the radiation generator 22, the object under test 12 (holder 24), and the detector 26. This area from which transmission image data can be acquired is called the "FOV (field of view)."

[0027] The rotation radii of the substrate rotation orbit 28 and the detector rotation orbit 30 are not fixed but can be freely changed. This makes it possible to arbitrarily change the irradiation angle of the radiation irradiated onto the electronic substrate, which is the inspected object 12, and onto the components attached to this substrate. The orbital planes of the substrate rotation orbit 28 and the detector rotation orbit 30 are perpendicular to the Z-axis direction described above. If the directions perpendicular to this orbital plane are the X-axis direction and the Y-axis direction, the positions of the holder 24 in the X-axis direction and the Y-axis direction are detected by a substrate position detection unit (not shown) and output to the control unit 10, and the positions of the detector 26 in the X-axis direction and the Y-axis direction are detected by a detector position detection unit (not shown) and output to the control unit 10.

[0028] The control unit 10 controls all operations of the above-mentioned inspection device 1. The main functions of the control unit 10 will be described below with reference to Fig. 2. Although not shown, input devices such as a keyboard and a mouse are connected to the control unit 10.

[0029] The control unit 10 includes a memory unit 34, an imaging processing unit 35, a cross-sectional image generating unit 36, a noise reduction unit 37, a substrate inspection surface detecting unit 38, a pseudo-cross-sectional image generating unit 40, and an inspection unit 42. Although not shown, the imaging processing unit 35 of the control unit 10 also has the function of an imaging control unit that controls the operation of the radiation quality changing unit 14, the radiation generator driving unit 16, the holder driving unit 18, and the detector driving unit 20. Furthermore, each of these functional blocks is realized by the cooperation of hardware, such as a CPU that performs various arithmetic processing and RAM that is used as a work area for storing data and executing programs, and software. Therefore, these functional blocks can be realized in various ways by combining hardware and software.

[0030] The storage unit 34 stores information such as imaging conditions for imaging the electronic board and acquiring transmission image data, and the design of the electronic board as the inspected object. The storage unit 34 also stores transmission image data and reconstructed image data (cross-sectional image data, pseudo-cross-sectional image data) of the electronic board, as well as inspection results from the inspection unit 42 (described later). The storage unit 34 also stores information for driving the radiation generator driving unit 16, the holder driving unit 18, and the detector driving unit 20 (e.g., the speed at which the radiation generator driving unit 16 drives the radiation generator 22, the speed at which the holder driving unit 18 drives the holder 24, and the speed at which the detector driving unit 20 drives the detector 26, the position at which the transmission image data is acquired (imaging position), etc.). The storage unit 34 also stores information for removing noise from image data (e.g., the reconstructed image data) (a model generated by AI learning (hereinafter referred to as an "AI model")).

[0031] In order to generate reconstructed image data (cross-sectional image data) by a cross-sectional image generating unit 36, which will be described later, the imaging processing unit 35 drives the radiation generator 22, the holder 24, and the detector 26 using the radiation generator driving unit 16, the holder driving unit 18, and the detector driving unit 20, and captures an image of the inspected object 12 held by the holder 24 to obtain transmission image data. A method for obtaining transmission image data by the imaging processing unit 35 will be described later.

[0032] The cross-sectional image generating unit 36 ​​generates cross-sectional image data (reconstructed image data) based on the multiple transmission image data acquired from the storage unit 34. This can be achieved using known techniques, such as the FBP method or the maximum likelihood estimation method. Different reconstruction algorithms result in different properties of the reconstructed image and different times required for reconstruction. Therefore, multiple reconstruction algorithms and parameters used for the algorithms may be prepared in advance and the user may select one. This provides the user with the freedom of choice, such as prioritizing a shorter reconstruction time or prioritizing better image quality even if it takes longer. Each of the generated cross-sectional images is output as cross-sectional image data to the storage unit 34 along with attribute information, such as information determining the position of each cross-sectional image in the Z-axis direction and the positions (coordinates) of pixels in the X-axis and Y-axis directions, and is stored in the storage unit 34. A method for generating reconstructed image data (cross-sectional image data) by the cross-sectional image generating unit 36 ​​will be described later.

[0033] The noise reduction unit 37 uses the above-mentioned AI model to perform correction to reduce noise from the reconstructed image data (cross-sectional image data) generated by the cross-sectional image generation unit 36. How to create the AI ​​model and how to reduce noise will be described later.

[0034] The substrate inspection surface detection unit 38 identifies an image (cross-sectional image data) that shows the surface to be inspected on the electronic substrate (for example, the surface of the electronic substrate) from the multiple cross-sectional image data generated by the cross-sectional image generation unit 36. Hereinafter, the cross-sectional image data that shows the inspection surface of the electronic substrate will be referred to as "inspection surface image data."

[0035] The pseudo-sectional image generating unit 40 stacks a predetermined number of consecutive cross-sectional image data from the cross-sectional image data generated by the cross-sectional image generating unit 36 ​​to image regions of the substrate that are thicker than each cross-sectional image data. The number of cross-sectional image data to be stacked is determined based on the thickness of the substrate region depicted by the cross-sectional image data (hereinafter referred to as the "slice thickness") and the slice thickness of the pseudo-sectional image data. For example, if the slice thickness of the cross-sectional image data is 50 μm and the height (e.g., 500 μm) of a BGA solder ball (hereinafter simply referred to as "solder") is to be used as the slice thickness for the pseudo-sectional image data, then 500 / 50 = 10 cross-sectional image data should be stacked. In this case, the inspection surface image data identified by the substrate inspection surface detecting unit 38 is used to identify the position of the solder.

[0036] The inspection unit 42 inspects the solder joint state based on the cross-sectional image data generated by the cross-sectional image generation unit 36, the inspection surface image data identified by the board inspection surface detection unit 38, and the pseudo cross-sectional image data generated by the pseudo cross-sectional image generation unit 40. Because the solder joining the electronic board and the component is located near the board inspection surface, it is possible to determine whether the solder is properly joining the board and the component by inspecting the inspection surface image data and the cross-sectional image data (pseudo cross-sectional image data) that shows the area on the radiation generator 22 side of the inspection surface image data.

[0037] Here, "solder joint condition" refers to whether an electronic board and a component are joined by solder and an appropriate conductive path is formed. Inspections of solder joint condition include bridge inspection, melting condition inspection, and void inspection. A "bridge" refers to an undesirable conductive path between conductors created by solder joining. Furthermore, a "melting condition" refers to whether the joint between the electronic board and the component is insufficient due to insufficient solder melting, or whether there is a so-called "floating" condition. A "void" refers to a defect in the solder joint caused by air bubbles in the solder joint. Therefore, the inspection unit 42 includes a bridge inspection unit 44, a melting condition inspection unit 46, and a void inspection unit 48.

[0038] The operations of the bridge inspection unit 44, molten state inspection unit 46, and void inspection unit 48 will be described in detail below, but the bridge inspection unit 44 and void inspection unit 48 inspect bridges and voids, respectively, based on the pseudo cross-sectional image data generated by the pseudo cross-sectional image generation unit 40, and the molten state inspection unit 46 inspects the molten state of solder based on the inspection surface image data identified by the board inspection surface detection unit 38. The inspection results of the bridge inspection unit 44, molten state inspection unit 46, and void inspection unit 48 are stored in the memory unit 34. Note that the inspection content using the reconstructed image data (cross-sectional image data) described here is an example, and the present invention is not limited to these inspection contents.

[0039] 3 and 4 are flowcharts showing the flow of the inspection process from capturing an image of the inspected object 12 to acquiring transmission image data, generating reconstructed image data (cross-sectional image data) based on the transmission image data, removing noise from the reconstructed image data (cross-sectional image data), and further specifying inspection surface image data and inspecting the solder joint state. The inspection process in this flowchart starts, for example, when the control unit 10 receives an instruction to start the inspection from an input device (not shown).

[0040] When the inspection is started, the control unit 10 carries the object under inspection 12 into the inspection space in the inspection apparatus 1, as shown in FIG. 3, and holds the object under inspection 12 with the holder 24 (step S100). Next, the imaging processing unit 35 of the control unit 10 positions the ray generator 22 in the Z-axis direction using the ray generator driving unit 16, and moves the holder 24 and the detector 26 to an imaging start position, thereby setting the irradiation field of the radiation emitted from the ray generator 22 (the imaging region to which the radiation is irradiated to obtain transmission image data of the above-mentioned field of view FOV) (step S102). Note that, when there are multiple imaging regions (fields of view FOV) on the object under inspection 12, one imaging region is selected and set from among them. Then, the transmission image capturing and reconstructed image generation process is started (step S104).

[0041] 4, when the transmission image capturing / reconstruction image generating process S104 is started, the imaging processing unit 35 starts the rotational movement of the holder 24 and the detector 26, and moves the holder 24 and the detector 26 on the rotational orbits (substrate rotation orbit 28 and detector rotation orbit 30) (step S1041). Then, while the holder 24 and the detector 26 are moving on the rotational orbits 28 and 30, the imaging processing unit 35 determines whether the holder 24 and the detector 26 are at the imaging position (S1042). If it determines that the holder 24 and the detector 26 are not at the imaging position (step S1042: N), the imaging processing unit 35 repeats the determination of step S1042. On the other hand, if it determines that the holder 24 and the detector 26 are at the imaging position (step S1042: Y), the imaging processing unit 35 images the inspected object 12 with the detector 26 to obtain transmission image data, stores the data in the storage unit 34 (step S1043), and determines whether imaging has been performed at all imaging positions (step S1044). When it is determined that there is an imaging position for which transmission image data has not yet been acquired (step S1044: N), the imaging processing unit 35 returns to step S1042 and repeats the above-described process. On the other hand, when it is determined that transmission image data has been acquired at all imaging positions (step S1044: Y), the imaging processing unit 35 moves the holder 24 and the detector 26 from the rotation orbit to the retracted position and stops them (step S1045).

[0042] In this way, once the transmission image data of the object to be inspected 12 is acquired, the cross-sectional image generating unit 36 ​​of the control unit 10 uses these transmission image data to generate reconstructed image data (cross-sectional image data) of the object to be inspected 12 (step S1046), thereby completing the transmission image capturing / reconstructed image generating process S104.

[0043] Returning to FIG. 3, the noise reduction unit 37 of the control unit 10 performs noise reduction processing on each of the cross-sectional image data constituting the reconstructed image data generated by the cross-sectional image generation unit 36 ​​(step S106). Specifically, if each of the cross-sectional image data constituting the reconstructed image data is input data to the AI ​​model described above, the output data output from this AI model will be cross-sectional image data in which noise has been removed from the input cross-sectional image data (this output will be referred to as "corrected cross-sectional image data," and the reconstructed image data composed of the corrected cross-sectional image data will be referred to as "corrected reconstructed image data"). The corrected reconstructed image data (corrected cross-sectional image data) is stored in the storage unit 34.

[0044] The board inspection surface detection unit 38 of the control unit 10 receives the transmission image data or the corrected reconstructed image data (corrected cross-sectional image data) from the noise reduction unit 37 and executes a board inspection surface detection process to identify an inspection surface image from the received transmission image data or the corrected reconstructed image data (corrected cross-sectional image data) (step S108). The storage unit 34 pre-stores cross-sectional image data (referred to as "reference image data") of the board inspection surface of a normal object to be inspected, with no abnormalities in the solder joint state or the like. The board inspection surface detection unit 38 reads the reference image data of the current field of view (FOV) from the storage unit 34, and further reads cross-sectional image data within a search range that includes the board inspection surface from the reconstructed image data of the current field of view (the corrected reconstructed image data (corrected cross-sectional image data) output in step S106). The board inspection surface detection unit 38 then compares the reference image data with each of the cross-sectional image data within the search range, identifies the corrected cross-sectional image data that best matches the reference image data as the inspection surface image data, and stores the Z-axis position of the identified cross-sectional image data (inspection surface image data) as the position of the board inspection surface in the current field of view (FOV). Here, as a method for identifying the cross-sectional image data that most closely matches the reference image data from among multiple corrected cross-sectional image data, for example, phase-only correlation can be used, which allows the matching rate to be calculated quickly regardless of positional deviation.

[0045] Then, the pseudo cross-sectional image generating unit 40 of the control unit 10 generates pseudo cross-sectional image data from the corrected reconstructed image data (corrected cross-sectional image data) based on the inspection surface image data identified in step S108 and the position of the substrate inspection surface in the Z-axis direction (step S110).

[0046] Next, the bridge inspection unit 44 of the control unit 10 acquires pseudo cross-sectional image data of a slice thickness similar to that of the solder balls from the pseudo cross-sectional image generation unit 40 (read from the storage unit 34), inspects for the presence or absence of a bridge (step S112), and determines whether a bridge is detected (step S114). If a bridge is not detected (step S114: N), the melted state inspection unit 46 of the control unit 10 acquires inspection surface image data from the substrate inspection surface detection unit 38 (read from the storage unit 34), inspects for the presence or absence of melted solder (step S116), and determines whether a melted solder portion is detected (step S118). If solder is present (step S118: Y), the void inspection unit 48 of the control unit 10 acquires pseudo cross-sectional images of a portion of the solder balls from the pseudo cross-sectional image generation unit 40 (read from the storage unit 34), inspects for the presence or absence of a void (step S120), and determines whether a void is detected (step S122). If no voids are found (step S122: N), the inspection unit 42 of the control unit 10 determines that the solder joint state is normal (step S124) and outputs a message to that effect to the storage unit 34. If a bridge is detected (step S114: Y), the solder is not melted (step S118: N), or a void is present (step S122: Y), the inspection unit 42 determines that the solder joint state is abnormal (step S126) and outputs a message to that effect to the storage unit 34.

[0047] When the state of the solder is output to the memory unit 34 in step S124 or step S126, the image capturing processing unit 35 of the control unit 10 determines whether or not there is a next image capturing area (field of view FOV) (step S128). If it is determined that there is a next image capturing area (that is, that there is an FOV that has not yet been inspected in the object 12 currently being inspected (step S128: Y), the image capturing processing unit 35 returns to step S102, sets the next image capturing area (field of view FOV), and repeats the subsequent processes. On the other hand, if it is determined that inspection of all image capturing areas (field of view FOV) has been completed (step S128: N), the control unit 10 carries the object 12 under inspection out of the inspection area of ​​the inspection device 1 (step S130), and ends the inspection process.

[0048] In the process shown in Figures 3 and 4, the case where an inspection is performed for each of the above-mentioned imaging areas (field of view FOV) has been described. However, it is also possible to configure the inspection to be performed for each imaging area (field of view FOV) after capturing transmission image data in all imaging areas (field of view FOV), or to configure the inspection to be performed in parallel with capturing images of other imaging areas (field of view FOV) in order from the imaging area (field of view FOV) for which generation of reconstructed image data (cross-sectional image data and pseudo-cross-sectional image data) has been completed.

[0049] As described above, the inspection device 1 according to this embodiment is configured to perform correction to reduce noise using an AI model on reconstructed image data (cross-sectional image data) generated from transmission image data. A method for creating this AI model and a method for removing noise using the AI ​​model will be described below.

[0050] 2, the control unit 10 has a learning processing unit 50 that creates an AI model. The learning processing unit 50 also includes a learning dataset creation unit 51 that creates a learning dataset for creating an AI model, and a noise reduction learning unit 52 that creates an AI model for reducing noise from reconstructed image data (cross-sectional image data) by learning the dataset.

[0051] The AI ​​model creation method according to this embodiment is configured to create an AI model by learning a dataset in which, of two or more image data captured at different times in the same imaging region (FOV) of the object under inspection 12, one image is used as input image data and the remaining image is used as target image data. Here, a case will be described in which an AI model is created by learning using cross-sectional image data constituting reconstructed image data. Specifically, transmission image data is captured two or more times for the same imaging region (FOV) under the same imaging conditions (as described above, the holder 24 and detector 26 that hold the object under inspection 12 relative to the radiation generator 22 are moved from the start position to the end position on the rotational orbits 28 and 30, and the process of acquiring transmission image data at a predetermined imaging position is performed two or more times), and reconstructed image data is generated from the transmission image data acquired in each process (reconstructed image data is generated according to the number of times the transmission image data acquisition process is performed). Then, from one of the reconstructed image data, cross-sectional image data is selected as input image data (for example, cross-sectional image data in which the substrate inspection surface is imaged is selected), and from one of the remaining reconstructed image data, cross-sectional image data at the same position as the input image data (the same position in the Z-axis direction, in the above example, the position of the substrate inspection surface) is selected as target image data to be used as a learning dataset. Note that the cross-sectional image data selected as the learning dataset is not limited to the substrate inspection surface, and may be cross-sectional image data at any other position in the Z-axis direction.

[0052] A plurality of training datasets are created from image data acquired by changing the imaging region (FOV) or imaging conditions, and are used for learning to create an AI model. Here, the imaging conditions changed to acquire image data for the datasets are preferably conditions that affect noise generated in the image data or conditions that change the amount of radiation irradiated to the inspected object 12. For example, imaging conditions include the above-mentioned tube voltage, tube current, shutter time (the time (irradiation time) during which radiation is irradiated to the FOV to acquire one piece of transmission image data), the number of acquired transmission image data (the number of transmission image data used to generate one piece of reconstructed image data, which is the number of imaging positions on the above-mentioned rotation orbit), the projection angle of radiation with respect to the imaging region (FOV), resolution, focus height, etc.

[0053] In the image data used in the dataset, it is better to set the pattern of imaging conditions so that it varies evenly within the dynamic range of the radiation dose (the range from the minimum to the maximum value of the radiation dose irradiated on the inspected object 12 when capturing transmission image data). In other words, if learning is performed using a dataset consisting of image data acquired by varying the imaging conditions from the maximum to the minimum value at predetermined intervals, the AI ​​model will effectively reduce noise.

[0054] The learning dataset creation unit 51 constituting the learning processing unit 50 selects two pieces of image data acquired under the same imaging conditions for the same imaging region (field of view FOV) from among the image data captured by the above-described method (for example, cross-sectional image data constituting the reconstructed image data), and generates a dataset using one as input image data and the other as target image data. As described above, the learning dataset creation unit 51 creates multiple datasets using different imaging conditions for the same imaging region (field of view FOV).

[0055] The noise reduction learning unit 52, which constitutes the learning processing unit 50, uses unsupervised machine learning to convert input image data included in a loaded dataset into target image data, thereby creating an AI model. The unsupervised machine learning method is a deep learning learning method, such as a convolutional neural network (CNN), and preferably uses a U-net, a fully convolutional network. The image data (input image data and target image data) included in the dataset created by the learning dataset creation unit 51 are image data acquired under the same imaging conditions for the same imaging region (FOV) but at different times. Therefore, the noise included in each image data changes randomly. Therefore, by learning these datasets, the noise reduction learning unit 52 determines that noise information included in the input image data is unnecessary information when converting the input image data to target image data, and generates an AI model that reduces noise from the input image data. The AI ​​model structure, known as the U-net, performs convolution processing on the input image data based on the AI ​​model, and outputs image data that has been deconvolved based on the results. Therefore, when the noise reduction unit 37 uses the above-mentioned AI model, unnecessary information (noise information) is reduced from the input image data (in the above explanation, the cross-sectional image data included in the reconstructed image data generated by the cross-sectional image generation unit 36) in the convolution process, and image data with reduced noise information (corrected reconstructed image data (corrected cross-sectional image data)) is generated in the deconvolution process.

[0056] In the above description, the reconstructed image data (cross-sectional image data) is learned and correction for reducing noise is performed on the reconstructed image data (cross-sectional image data) generated by the cross-sectional image generating unit 36, but the transmission image data may be learned using the above-described procedure, correction for reducing noise is performed on the transmission image data acquired by the imaging processing unit 35, and reconstructed image data may be generated from the corrected transmission image data. In the above description, the reconstructed image data (cross-sectional image data) generated by the cross-sectional image generating unit 36 ​​is noise-reduced, and the examination surface image data is identified and pseudo-cross-sectional image data is generated from the noise-reduced (corrected) reconstructed image data (cross-sectional image data). However, the reconstructed image data (examination surface image data) and pseudo-cross-sectional image data used for the examination may be noise-removed at the examination stage.

[0057] Furthermore, the image data (reconstructed image data) selected as the training dataset may be captured (generated) in advance and stored in the memory unit 34, which is an image storage unit, and read out from the memory unit 34 for use in training. Alternatively, the learning processing unit 50 may control the imaging processing unit 35 to change the imaging area (field of view FOV) and imaging conditions to image the object 12 to obtain transmission image data (or generate reconstructed image data (cross-sectional image data) from the transmission image data), and the image data may be converted into a dataset by the training dataset creation unit 51 using the above-described method, and training may be performed by the noise reduction training unit 52 to create an AI model. In this way, by configuring the training processing unit 50 to acquire and train image data for training, when the noise reduction effect of the noise reduction unit 37 deteriorates, the noise reduction effect can be improved again by acquiring image data again, learning from this image data, and creating and updating an AI model.

[0058] In the above description, the noise reduction unit 37 performs noise reduction processing on the entire input reconstructed image data (cross-sectional image data). However, the control unit 10 may output the reconstructed image data (cross-sectional image data) for which noise reduction is to be performed to the monitor 11, provide a region setting unit that accepts the range (region) for which noise reduction is to be performed from the operator, and perform noise reduction on image data within the range specified by the region setting unit of the reconstructed image data (cross-sectional image data). Alternatively, instead of displaying the reconstructed image data (cross-sectional image data) on the monitor 11 and having the operator specify the range for which noise reduction is to be performed, the noise reduction unit 37 may be presented with options for the target for noise reduction, such as "void portion" or "solder portion," and the operator may select the option. By limiting the range for which noise reduction is to be performed using the region setting unit in this way, the amount of noise reduction processing performed by the noise reduction unit 37 can be reduced, thereby shortening the inspection time for the object under test 12. In this case, the noise reduction unit 37 performs the function of the region setting unit that accepts the region for which noise reduction correction is to be performed on the image data.

[0059] Furthermore, even if multiple imaging regions (FOVs) are set for the object under inspection 12, the noise reduction unit 37 can remove noise from image data using a single AI model. However, an AI model may be created for each type of object under inspection 12, and the AI ​​model to be applied may be changed for each type of object under inspection 12. For example, for an object under inspection 12 equipped with a special component, an AI model dedicated to the object under inspection equipped with that component may be created, and this dedicated AI model may be used to remove noise from image data for inspection. Furthermore, a dedicated AI model may be created for special imaging conditions, and when those imaging conditions are selected, the dedicated AI model may be used to remove noise for inspection.

[0060] (Main effects) The main features and effects of the AI ​​model creation method according to this embodiment and the inspection device in which this AI model creation method is implemented are described below.

[0061] First, the AI ​​model creation method according to this embodiment is a method for creating an AI model using unsupervised machine learning to perform corrections to reduce noise from image data of an object under inspection. The AI ​​model is created by learning a dataset in which two or more image data images captured at different times in the same field of view of the object under inspection are used, one of which is used as input image data and the other is used as target image data. This allows the AI ​​model to be created using only low-quality image data with noise, eliminating the need to prepare high-quality image data without noise. Furthermore, because the AI ​​model created by this AI model creation method according to this embodiment is created from low-quality image data with noise, the image data from which noise has been removed based on this AI model will not be image data with noise reduced to make it appear as a good product. This reduces the likelihood that an object that should actually be determined to be defective will be determined to be good, thereby improving inspection accuracy.

[0062] Second, the AI ​​model created by the AI ​​model creation method according to this embodiment is the result of learning information for generating target image data from input image data contained in a dataset, and by having it learn from noisy, low-quality image data, the AI ​​model that is the result of this learning can be used to accurately remove noise from the input image data.

[0063] Third, the data set (a set of input image data and target image data) used in the AI ​​model creation method according to this embodiment is a set of image data acquired by varying imaging conditions that affect noise, so even if the imaging conditions of the image data to be corrected change, noise can be removed from the image data. In particular, by varying the imaging conditions to conditions that change the amount of radiation irradiated to the object, the noise removal effect by the AI ​​model can be improved.

[0064] Fourth, the AI ​​model created by the AI ​​model creation method according to this embodiment is the result of learning through deep learning, for example, through a Convolutional Neural Network (CNN). Therefore, by learning the above-mentioned dataset, the AI ​​model created by this learning can accurately remove noise from input image data. In this case, it is desirable to use a U-net, which is a type of fully convolutional network.

[0065] Fifth, by implementing in the inspection device a function for reducing noise from image data using the AI ​​model creation method according to this embodiment and the AI ​​model created by this creation method, noise can be accurately removed from transmission image data or three-dimensional image data (reconstructed image data or cross-sectional image data), thereby improving the inspection accuracy of the object being inspected.

[0066] In this case, the inspection device is provided with an image capturing unit that captures (or generates) image data for learning, a data set is created from the image data captured (or generated) by this image capturing unit, and an AI model is created by learning this data set, thereby making it possible to create an AI model that reduces noise in accordance with the object to be inspected, and the inspection accuracy of the object can be further improved using the image data with reduced noise.In addition, by having the operator of the inspection device specify the area in the image data where noise is to be reduced, the amount of processing required for noise reduction correction can be reduced, and the time required for inspection can be shortened. [Explanation of symbols]

[0067] 1. Inspection equipment 10 Control Unit 12 Test subject 22 Radiation generator (light source) 24 Holding part 26 detector 34 Memory unit (image storage unit) 35 Imaging processing unit (image capturing unit) 37 Noise reduction section (area setting section) 51 Training Dataset Creation Department 52 Noise reduction learning unit

Claims

1. A method for creating an AI model in an inspection device that performs correction to reduce noise from image data of an object to be inspected using an AI model created by unsupervised machine learning, comprising: A method for creating an AI model, in which the AI ​​model is created by learning a data set in which one of two or more image data images taken at different times in the same field of view of the object to be inspected is used as input image data and one of the remaining images is used as target image data.

2. The AI ​​model is a learning result of information for generating the target image data from the input image data included in the data set. The method for creating an AI model according to claim 1.

3. Each of the data sets is image data acquired under different imaging conditions that affect noise. The method for creating an AI model according to claim 1.

4. The imaging conditions are conditions under which the amount of radiation irradiated to the object to be inspected changes. The method for creating an AI model according to claim 3.

5. The AI ​​model is the result of deep learning. The method for creating an AI model according to claim 1.

6. A light source and A detector; a holder for holding the object to be inspected; a control unit that generates three-dimensional image data of the object under inspection using at least two pieces of transmission image data of the object under inspection, which are obtained by the detector using light emitted from the light source and transmitted through the object under inspection by changing the relative positions of the light source, the detector, and the holding unit, The control unit performs correction to reduce noise from the transmission image data or the three-dimensional image data using the AI ​​model created by the AI ​​model creation method according to any one of claims 1 to 5. Inspection equipment.

7. The control unit an image storage unit that stores the transmission image data or the three-dimensional image data; a learning dataset creation unit that creates the dataset from the transmission image data or the three-dimensional image data; a noise reduction learning unit that creates the AI ​​model by learning using the AI ​​model creation method based on the data set; a noise reduction unit that performs correction to reduce noise from the transmission image data or the three-dimensional image data using the AI ​​model. The inspection device according to claim 6.

8. The control unit an image capturing unit that generates, for the learning, transmission image data under different imaging conditions or three-dimensional image data under different imaging conditions generated from the transmission image data; The learning dataset creation unit creates datasets with different imaging conditions from transmission image data with different imaging conditions or three-dimensional image data with different imaging conditions, The noise reduction learning unit creates the AI ​​model using data sets with different imaging conditions. The inspection device according to claim 7.

9. The control unit an area setting unit that receives an area to be corrected to reduce noise in the transmission image data or the three-dimensional image data; The noise reduction unit performs correction to reduce noise in the region set by the region setting unit of the transmission image data or the three-dimensional image data. The inspection device according to claim 7.

Images