Inspection device and inspection method

Abstract

This inspection device comprises: an input unit that acquires temperature image data generated by imaging, by means of an imaging device in time series, an object that is joined in a prescribed joining region; and a processor that predicts the joining strength of the object on the basis of the temperature image data. The processor performs Fourier transform or discrete Fourier transform on the temperature image data to generate a phase image indicating the phase characteristics of the object. The processor calculates an effective area ratio indicating the value of the ratio of the area of an effective joint region, in which the phase value is within a prescribed range, in the phase image with respect to the area of the joint region, and predicts the joint strength on the basis of the calculated effective area ratio.

Description

Inspection device and inspection method

[0001] The present disclosure relates to an inspection device and an inspection method.

[0002] Patent Document 1 discloses a non-destructive inspection method using infrared lock-in thermography to determine the quality of a lap joint formed by friction stir welding dissimilar metal materials. This non-destructive inspection method determines the quality of the joint by comparing the amount and phase of acquired infrared rays with predetermined judgment values.

[0003] International Publication No. 2013 / 085075

[0004] The present disclosure provides an inspection device and an inspection method that can predict the bond strength of an object based on temperature image data of the object.

[0005] An inspection device according to one aspect of the present disclosure includes an input unit that acquires temperature image data generated by an imaging device photographing an object bonded at a predetermined bonding region in a time series, and a processor that predicts the bonding strength of the object based on the temperature image data, wherein the processor performs a Fourier transform or a discrete Fourier transform on the temperature image data to generate a phase image that indicates the phase characteristics of the object, calculates an effective area ratio that indicates the ratio of the area of ​​the effective bonding region in the phase image, whose phase value is within a predetermined range, to the area of ​​the bonding region, and predicts the bonding strength based on the calculated effective area ratio.

[0006] An inspection device according to one aspect of the present disclosure includes an input unit that acquires temperature image data generated by an imaging device photographing objects bonded in a predetermined bonding region in a time series; a storage device that stores a trained model that has been machine-learned using phase images of the bonded objects and measured bonding strengths as a training data set; and a processor that can access the storage device and performs a Fourier transform or a discrete Fourier transform on the temperature image data to generate a phase image that indicates the phase characteristics of the objects, wherein the processor inputs the phase image to the trained model, acquires the bonding strength from the trained model, and predicts the bonding strength.

[0007] An inspection method according to one aspect of the present disclosure is an inspection method for predicting the bonding strength of an object based on temperature image data generated by an imaging device capturing images of the object bonded at a predetermined bonding region in a time series, the inspection method including the steps of: acquiring the temperature image data by a processor; performing a Fourier transform or a discrete Fourier transform on the temperature image data to generate a phase image showing the phase characteristics of the object; calculating an effective area ratio indicating the ratio of the area of ​​the effective bonding region in the phase image, where the phase value is within a predetermined range, to the area of ​​the bonding region; and predicting the bonding strength based on the calculated effective area ratio.

[0008] An inspection method according to one aspect of the present disclosure is an inspection method in which a processor capable of accessing a storage device that stores a trained model non-destructively predicts the bonding strength of an object based on temperature image data generated by an imaging device capturing images of the object bonded at a predetermined bonding region in a time series, wherein the trained model is a model that has been machine-learned using phase images of the bonded object and measured bonding strength as a training data set, and the inspection method includes the steps of: the processor performing a Fourier transform or a discrete Fourier transform on the temperature image data to generate a phase image that shows the phase characteristics of the object; the processor inputting the phase image into the trained model; and the processor obtaining the bonding strength from the trained model and predicting the bonding strength.

[0009] According to the present disclosure, it is possible to predict the bond strength of an object based on temperature image data of the object.

[0010] 1A-1C are block diagrams showing an example of the configuration of an inspection system according to an embodiment of the present disclosure; FIG. 1B are block diagrams showing an example of the configuration of an inspection device of the inspection system shown in FIG. 1; FIG. 1C are flow charts showing an example of the operation of the inspection device; FIG. 1D are schematic diagrams showing phase images; FIG. 1E are graphs showing the distribution of phases in a phase image of a workpiece having a bonded region with a bond strength of 532 N; FIG. 1F are graphs showing the distribution of phases in a phase image of a workpiece having a bonded region with a bond strength of 712 N;

[0011] Hereinafter, embodiments will be described with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed description of well-known matters or redundant description of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the inventors provide the accompanying drawings and the following description to enable those skilled in the art to fully understand the present disclosure, and do not intend for them to limit the subject matter described in the claims.

[0012] 1 is a block diagram showing an example configuration of an inspection system 1 according to an embodiment of the present disclosure. The inspection system 1 includes an inspection device 10, an infrared camera 17, an excitation source 18, a control box 15, a power source 16, and an alarm device 19.

[0013] The inspection system 1 uses an active thermography method in which excitation energy is applied to the workpiece 90 to be inspected and temperature images are taken, thereby non-destructively predicting the bonding strength of the workpiece 90. In this embodiment, the inspection system 1 applies excitation energy to the workpiece 90 using an excitation source 18, and takes temperature images in time series using an infrared camera 17.

[0014] 1 , the workpiece 90 includes a transparent material 91 that mainly transmits light irradiated from a light source for welding or deposition, and an absorbing material 92 that mainly absorbs the light. The transparent material 91 and / or the absorbing material 92 are joined at a joining region 95 by the transparent material 91 and / or the absorbing material 92 melted by the light from the light source. In this embodiment, the inspection system 1 can non-destructively predict the joining strength between the transparent material 91 and the absorbing material 92.

[0015] The material of the workpiece 90 is, for example, a resin such as polybutylene terephthalate (PBT), polyphenylene sulfide (PPS), or polyamide (PA). Alternatively, the material of the workpiece 90 may be ceramics. The material of the workpiece 90 may also be a metal such as cold-rolled steel (SPCC steel) or aluminum.

[0016] The infrared camera 17 is an example of an imaging device that captures a time-series image of an imaging area including at least a portion of the workpiece 90 to generate multiple temperature image data (hereinafter, sometimes simply referred to as "temperature images"). The infrared camera 17 generates temperature image data by converting infrared image data into a temperature distribution, but in this specification, there may be no particular distinction between the infrared image data before conversion and the temperature image data after conversion. The infrared camera 17 includes an infrared sensor that detects infrared rays having a wavelength of, for example, 3 μm to 15 μm. The frame rate of the infrared camera 17 is, for example, 50 Hz (or 50 fps), but is not limited to this.

[0017] In this disclosure, an "image" includes one or more moving images and one or more still images.

[0018] The excitation source 18 is an example of a heating device capable of heating the workpiece 90. The excitation source 18 is, for example, a light source such as a xenon lamp, a halogen lamp, or a laser light source, but is not limited to these, and may be any energy source capable of radiating energy.

[0019] The excitation source 18 can perform flash heating (pulse heating) by emitting a flash, step heating by heating in steps, or heating by continuously repeating these. The wavelength band of light emitted by the excitation source 18 may be the same as or different from the wavelength band of infrared light detectable by the infrared camera 17. Although two excitation sources 18 are illustrated in FIG. 1 , the number of excitation sources 18 is not limited to this and may be one, or three or more.

[0020] The power supply 16 supplies power to the infrared camera 17 and the excitation source 18. The control box 15 includes a control circuit that controls the power supply 16 based on a control signal from the inspection device 10. The control box 15 may control the light emission method, light emission cycle, light emission time, etc. of the excitation source 18.

[0021] The notification device 19 notifies information to the outside. For example, the notification device 19 is controlled by the inspection device 10 and notifies a user of information indicating the inspection results of the internal condition of the workpiece 90. The notification device 19 may include a visual notification device such as a light source such as an LED, a display, or an indicator. The notification device 19 may also include an auditory notification device such as a speaker.

[0022] 2 is a block diagram showing an example of the configuration of the inspection device 10 of Fig. 1. The inspection device 10 includes a processor 11, a storage device 12, and an interface 13.

[0023] The processor 11 is composed of a CPU, an MPU, etc., and controls the entire inspection device 10 by executing various programs stored in the storage device 12. The processor 11 controls the excitation source 18 via the control box 15, thereby controlling the start and stop of the heating output of the excitation source 18. The processor 11 also controls the imaging operations of the infrared camera 17, such as the start and stop of imaging. The processor 11 also performs an analysis process to inspect the workpiece 90 based on the temperature image data stored in the storage device 12, as will be described later.

[0024] The storage device 12 is a recording medium that records various information including programs and data necessary to realize the functions of the inspection device 10. The storage device 12 is realized, for example, by a semiconductor storage device such as a flash memory or a solid-state drive (SSD), a magnetic storage device such as a hard disk drive (HDD), or other recording media, either alone or in combination. The storage device 12 is not limited to an internal storage device installed in the same housing as the processor 11, but may also be, for example, an external storage device or a NAS (network-attached storage) type storage device. The storage device 12 may include a volatile memory such as a RAM.

[0025] The interface 13 connects the inspection device 10 to external devices such as the infrared camera 17, the control box 15, the alarm device 19, and the excitation source 18. The interface 13 may be a communication circuit that performs data communication in accordance with an existing wired communication standard or wireless communication standard.

[0026] The interface 13 is an example of an input unit that connects the inspection device 10 and the infrared camera 17 to input temperature image data from the infrared camera 17 to the inspection device 10. The interface 13 is also an example of an output unit that connects the inspection device 10 to external devices such as the control box 15, the alarm device 19, and the excitation source 18 to output information such as control signals from the processor 11 to the external devices. Such input and output units may be integrated as an input / output interface 13 as shown in FIG. 2, or may be realized as multiple interface circuits.

[0027] [2. Operation] [2-1. Overall Operation] Generally, the bond strength of a workpiece is determined by destructive testing, as the force required to destroy the bonded portion of the workpiece. Destructive testing has the problem that the workpiece is destroyed during the test, making it difficult to reuse. In contrast, the inspection device 10 according to the present embodiment can non-destructively predict the bond strength of the workpiece 90, i.e., the bond strength indicating the bond strength between the transmitting material 91 and the absorbing material 92.

[0028] 3 is a flowchart illustrating the operation of the inspection device 10. Each process shown in this flow is executed by the processor 11 of the inspection device 10, for example.

[0029] 3, first, the processor 11 acquires analysis conditions (S1). The analysis conditions are input by, for example, a user and stored in advance in the storage device 12. The analysis conditions are parameters such as an analysis time and a set frequency for a phase image (described later). The analysis time includes, for example, an analysis start time and an analysis end time. Alternatively, instead of the analysis time, the processor 11 may acquire a setting regarding which of the multiple temperature images generated by the infrared camera 17 to use for analysis.

[0030] Next, the processor 11 starts heating the workpiece 90 with the excitation source 18 and capturing an image of the workpiece 90 with the infrared camera 17 (S2).

[0031] The processor 11 acquires, via the interface 13, a plurality of temperature images that are generated by the infrared camera 17 photographing the workpiece 90 in time series (S3).

[0032] Next, the processor 11 performs a discrete Fourier transform on the temperature images acquired in step S3 (S4) and extracts a phase image according to the analysis conditions (S5). The phase image indicates, for example, the phase characteristics of the data after the discrete Fourier transform at a set frequency.

[0033] Next, the processor 11 executes an effective area ratio calculation process (S6) to calculate the ratio of the area of ​​the effective bonding area to the area of ​​the bonding area of ​​the workpiece 90 in the phase image (hereinafter also referred to as the "effective area ratio"). The effective area ratio is calculated, for example, by dividing the area (A1) of the effective bonding area in the phase image by the area (A2) of the entire bonding area.

[0034] Here, the "bonding region" refers to a region that is set as a target for bonding. Generally, it is desirable for the bonding region to be bonded entirely, but depending on the bonding quality, some or all of the bonding region may not be bonded. Furthermore, the portion of the bonding region that is actually bonded may include a portion where the bonding strength is equal to or greater than a predetermined value and a portion where the bonding strength is less than the predetermined value.

[0035] The "effective bonding area" refers to an area of ​​the bonding area where the bonding strength is equal to or greater than a predetermined value. The predetermined value of the bonding strength is set depending on the performance required for the workpiece 90 or a product using the workpiece 90. Details of the effective area ratio calculation process S6 will be described later.

[0036] After step S6, the processor 11 predicts the bond strength based on the effective area ratio obtained in the effective area ratio calculation process S6 (S7). The inventors have found that there is a strong correlation between the effective area ratio and the bond strength, and have arrived at the present disclosure relating to predicting the bond strength based on the effective area ratio. Details of this finding will be described later.

[0037] The processor 11 determines whether the bond strength predicted in step S7 is less than a threshold value (S8). If the bond strength is less than the threshold value (Yes in S8), the processor 11 causes the alarm device 19 to perform an alarm operation (S9). This allows the user to know that the bond strength is less than the threshold value. The alarm operation includes, for example, emitting a warning sound from a speaker or turning on or blinking a light source such as an LED. Alternatively, the processor 11 may display correlation information indicating analysis results such as the bond strength and the determination result on a display, which is an example of the alarm device 19.

[0038] After step S9, the processor 11 stores information indicating the analysis results, such as the bond strength and the judgment result, in the storage device 12 (S10). The processor 11 also executes step S10 when it is determined in step S8 that the bond strength is equal to or greater than the threshold value (No in S8). The information stored in the storage device 12 is used, for example, to achieve traceability. For example, the information stored in the storage device 12 is used to determine whether the condition of the workpiece 90 has been changed due to damage, tampering, dirt, etc. during distribution, and, if so, to detect the degree of change.

[0039] In addition, even if the processor 11 determines in step S8 that the bond strength is equal to or greater than the threshold value (No in S8), it may display information indicating the analysis results such as the bond strength and the judgment result on a display, which is an example of an alarm device 19, before or after step S10.

[0040] 2-2. Prediction of Bond Strength Based on Effective Area Ratio The principle of step S7 for predicting bond strength based on the effective area ratio will be described below with reference to Figures 4 to 9. As described above, the effective area ratio is the proportion of the portion of the phase image whose phase values ​​are within a predetermined range.

[0041] Fig. 4 is a schematic diagram illustrating a phase image 21. In the phase image 21 of Fig. 4, the phase value of each pixel is shown in gray scale. That is, as shown in the scale on the right side of Fig. 4, in the phase image 21, pixels with a phase value of 20.0° are shown in black, pixels with a phase value of 10.0° are shown in white, and pixels with a phase value greater than 10.0° and less than 20.0° are shown in gray.

[0042] The inventors have found that in a phase image of the bonded region of the workpiece 90, there is a correlation between the phase value and the bonded state, for example, the bond strength. As an example, the inventors have found that in the phase image 21 shown in FIG. 4, the lower the phase value, the greater the bond strength of the corresponding bonded region, and the better the bonded state. The example in FIG. 4 indicates that the lower the phase value, the easier the heat is to conduct, and the higher the phase value, the harder the heat is to conduct. Therefore, the example in FIG. 4 indicates that the easier the heat is to conduct, the greater the bond strength. FIGS. 5 to 7 are graphs showing that the lower the phase value, the greater the bond strength.

[0043] As shown in FIG. 4, the phase values ​​of the phase image 21 can be expressed in gray scale (for example, values ​​from 0 to 255), so that the region through which heat is easily transferred can also be determined by the gray scale instead of the phase value.

[0044] 5 to 7 are graphs showing the distribution of phases in phase images of a workpiece 90 having bonded regions with bond strengths of 532 N, 712 N, and 950 N, respectively. The horizontal axis of the graphs in Figures 5 to 7 represents the phase value, and the vertical axis represents the normalized frequency. The bond strengths shown in Figures 5 to 7 were obtained by acquiring phase images of the workpiece 90 and then performing actual measurements on the workpiece 90, such as by conducting a known bond strength test.

[0045] 5 to 7, it can be seen that the greater the bonding strength, the higher the frequency of pixels in the phase image having a phase value of 12° or less. Based on this finding, the inventors have come up with the technical idea of ​​predicting bonding strength based on an effective area ratio, which indicates an example of the ratio of the number of pixels having a phase value of 12° or less.

[0046] Fig. 8 is a graph showing an example of the correlation between the effective area ratio and the bond strength. The graph in Fig. 8 plots the effective area ratios corresponding to phase images of a workpiece 90 having bonded regions with bond strengths of 532 N, 712 N, and 950 N. The effective area ratio in the graph in Fig. 8 is the ratio (proportion) of the number of pixels with a phase value of 12° or less to the number of pixels in the entire bonded region in the phase image. In this graph, the correlation coefficient r1 between the effective area ratio and the bond strength is 0.99, indicating a strong correlation between the effective area ratio and the bond strength.

[0047] 8, in step S7, the processor 11 can predict the bonding strength of the workpiece 90 based on the effective area ratio calculated in step S16. Correlation information showing the relationship between the effective area ratio and the bonding strength as shown in FIG. 8 is stored in the storage device 12 as, for example, a table, and is used by the processor 11 in step S7.

[0048] The above results are for the case where the upper limit of the phase value is 12°, and it was found that the correlation coefficient changes when the upper limit of the phase value is changed. Fig. 9 is a graph showing an example of the relationship between the phase range and the correlation coefficient. The horizontal axis of Fig. 9 represents the phase range. When the horizontal axis of Fig. 9 is 12°, the correlation coefficient on the vertical axis represents the correlation coefficient between the effective area ratio and the bonding strength, calculated by counting the number of pixels with a phase value of 12° or less.

[0049] Thus, as an example, the correlation coefficient between the effective area ratio and the bond strength depends on the upper limit of the phase value. More generally, the correlation coefficient between the effective area ratio and the bond strength depends on the range of the phase value, for example, the upper and lower limits of the phase value.

[0050] Therefore, when determining the effective bonding area that forms the basis of the effective area ratio (see S13 and S14 described below), if a phase range in which the correlation coefficient is large (e.g., 12° or less, see Figure 9) is applied, the bonding strength of the workpiece can be predicted with particularly high accuracy.

[0051] [2-3. Effective Area Ratio Calculation Processing] Fig. 10 is a flowchart showing details of the effective area ratio calculation processing S6 in Fig. 3. Fig. 11 is a schematic diagram including a group of images for explaining the effective area ratio calculation processing S6.

[0052] 10 , in the effective area ratio calculation process S6, first, the processor 11 acquires bonding area mask information from the storage device 12 (S11). The mask information is information that indicates a portion of the phase image that corresponds to the bonding area of ​​the workpiece 90.

[0053] If the mask information is not stored in advance in the storage device 12, the processor 11 may generate the mask information based on the phase image. An example of the mask information is a mask image M in which pixels corresponding to the junction regions of the phase image are white (pixel value 1) and other pixels are black (pixel value 0).

[0054] An example of the process for generating such a mask image M will be described with reference to Fig. 11. The processor 11 segments the portion of the phase image 21 that corresponds to the junction region to generate a segment image 22. Next, the processor 11 generates a mask image M by setting the portion of the segment image 22 that corresponds to the junction region to white (pixel value 1) and the other portion to black (pixel value 0). The processor 11 stores the generated mask image M in the storage device 12.

[0055] 10 , after step S11, the processor 11 generates a joint region image 23 by excluding regions other than the joint region from the phase image (S12). For example, as shown in Fig. 11 , the processor 11 generates the joint region image 23 by coloring the portions of the segment image 22 other than the portions corresponding to the joint region black (pixel value 0). Alternatively, the processor 11 may generate the joint region image 23 by performing a mask process on the phase image 21 using a mask image M.

[0056] 10 , after step S12, the processor 11 performs a binarization process on the bonded area image 23 to generate an effective bonded area image 24 in which pixels in the bonded area image 23 whose phase values ​​are within a predetermined range are colored white (pixel value 1) (S13). The effective bonded area image 24 is an image that shows the effective bonded area. The predetermined range of phase values ​​is set by the user or the processor 11, for example, based on the relationship between the phase range and the correlation coefficient for the target workpiece (see FIG. 9 ). In the example of FIG. 11 , the processor 11 generates the effective bonded area image 24 by coloring white pixels in the bonded area image 23 whose phase value is 12° or less.

[0057] After step S13, the processor 11 calculates the number of white pixels in the effective bonding area image 24 as the area of ​​the effective bonding area (S14).

[0058] The processor 11 calculates the area of ​​the joint region from the joint region mask information acquired in step S11 (S15). For example, the processor 11 calculates the number of white pixels in the mask image M as the area of ​​the joint region. Step S15 may be executed after step S11, and may be executed before step S12, for example, unlike the example shown in FIG. 10. Alternatively, step S11 may be executed after step S14, as long as it is executed before step S15.

[0059] The processor 11 calculates the ratio of the area of ​​the effective bonding area to the area of ​​the bonding area as the effective area ratio (S16). For example, the processor 11 calculates the effective area ratio (A1 / A2) by dividing the area of ​​the effective bonding area (A1) calculated in step S14 by the area of ​​the bonding area (A2) calculated in step S15. Alternatively, the processor 11 may calculate the effective area ratio as the ratio of the number of white pixels in the effective bonding area image 24 to the number of white pixels in the mask image M obtained in step S15.

[0060] After completing step S16, the processor 11 ends the effective area ratio calculation process S6 and proceeds to step S7 in FIG.

[0061] 3. Effects, etc. As described above, the inspection device 10 according to this embodiment includes the interface 13, which is an example of an input unit, and the processor 11. The interface 13 acquires temperature image data generated by time-series imaging of the workpiece 90 joined at the predetermined joining region 95 using the infrared camera 17, which is an example of an imaging device. The processor 11 predicts the joining strength of the workpiece 90 based on the temperature image data. The processor 11 performs a discrete Fourier transform on the temperature image data (S3) to generate a phase image 21 showing the phase characteristics of the workpiece 90 (S4). The processor 11 calculates an effective area ratio, which indicates the ratio of the area of ​​the effective joining region in the phase image 21, where the phase value is within a predetermined range, to the area of ​​the joining region 95 (S6). The processor 11 predicts the joining strength based on the calculated effective area ratio (S7).

[0062] According to this configuration, the inspection device 10 can predict the bonding strength of the workpiece 90 non-destructively based on the temperature image data.

[0063] The effective bonded region may be a region in the phase image 21 where the phase value is equal to or less than a predetermined threshold value. The processor 11 may calculate the effective area ratio by calculating the ratio of the number of pixels included in the effective bonded region to the number of pixels included in the bonded region in the phase image 21.

[0064] The processor 11 may predict the bond strength based on the calculated effective area ratio using correlation information indicating the relationship between the effective area ratio and the bond strength. This configuration also makes it possible to predict the bond strength of the workpiece 90 in a non-destructive manner.

[0065] The processor may cause the notification device 19 to notify the user of information indicating the predicted bond strength. This allows the user to know the predicted bond strength.

[0066] The inspection device 10 may further include a storage device 12 that stores information indicating the predicted results of the bonding strength.

[0067] [4. Other Embodiments] As described above, the embodiments have been described as examples of the technology in the present disclosure. However, the technology in the present disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made as appropriate. Furthermore, it is also possible to combine the components described in the above embodiments to create new embodiments. Therefore, below, examples of modified examples as other embodiments will be described.

[0068] [4-1. First Modification] In the above embodiment, an example has been described in which the processor 11 predicts the bond strength in step S7 using information indicating the relationship between the effective area ratio and the bond strength stored in the storage device 12. However, the bond strength prediction process is not limited to this. For example, the processor 11 may input the phase image 21 to the trained model, causing the trained model to output bond strength data indicating the bond strength. Because there is a correlation between the distribution of phase values ​​in the phase image 21, for example, the effective bond area ratio, and the bond strength, as shown in FIG. 8 , the trained model can output bond strength data in response to the input of the phase image.

[0069] The trained model is a model that has been trained to output bond strength data in response to the input of the phase image 21. The training of such a model is performed by, for example, a training device 30 shown in FIG.

[0070] The learning device 30 includes a processor 31, a storage device 32, and an interface 33. The processor 31 is composed of a CPU, an MPU, etc., and controls the entire learning device 30 by executing various programs stored in the storage device 12. The storage device 12 is a recording medium that records various information including programs and data necessary to realize the functions of the learning device 30. The storage device 12 stores a model 321. The processor 31, the storage device 32, and the interface 33 may have the same configurations as the processor 11, the storage device 12, and the interface 13 in FIG. 2, respectively.

[0071] The model 321 has, for example, a convolutional neural network (CNN) structure. The model 321 is trained by the processor 31 using, for example, a backpropagation algorithm. The processor 11 generates a trained model by having the model 321 train a large amount of training data sets including phase images of the workpieces and bonding strength data of the workpieces as ground truth data. The training data sets may be stored in advance in the storage device 32 or may be input from the outside via the interface 33.

[0072] The trained model generated by the learning device 30 is stored, for example, in the storage device 12 of the inspection device 10. Alternatively, the trained model may be stored in a recording medium in an external device, such as a server device, that is accessible by the inspection device 10.

[0073] Fig. 13 is a flowchart illustrating the operation of the inspection device 10 according to the first modification. The flowchart in Fig. 13 includes steps S21 and S22 instead of steps S6 and S7 in Fig. 3. Steps S1 to S5 and S8 to S10 in Fig. 13 are the same as those in Fig. 3, and therefore their explanation will be omitted.

[0074] 13, the processor 11 inputs the phase image 21 obtained in step S5 to the model 321 (S21). In response to the input, the model 321 outputs bonding strength data.

[0075] Next, the processor 11 acquires the bonding strength data output from the model 321 (S22).

[0076] As described above, the inspection device 10 according to the first modification includes an interface 13, a storage device 12, and a processor 11 that can access the storage device 12. The storage device 12 stores a trained model that has been machine-learned using a phase image of the bonded object and the measured bond strength as a training data set. The processor 11 performs a Fourier transform or a discrete Fourier transform on the temperature image data (S4) to generate a phase image 21 that indicates the phase characteristics of the workpiece 90 (S5). The trained model is a model that has been trained to output data indicating the bond strength in response to an input of a phase image. The processor 11 inputs the phase image 21 into the trained model (S21) and obtains an output from the trained model (S22). The processor 11 determines the obtained output as the bond strength of the workpiece 90, thereby non-destructively predicting the bond strength of the workpiece 90.

[0077] [4-2. Second Modification] In the above embodiment, an example has been described in which a phase image is extracted by performing a discrete Fourier transform on a temperature image, but the present disclosure is not limited to this. For example, the processor 11 may extract a phase image by performing a Fourier transform instead of a discrete Fourier transform.

[0078] [4-3. Third Modification] In the above embodiment, it has been explained that a portion of a phase image with a lower phase value indicates that heat is more easily transmitted, and a portion with a higher phase value indicates that heat is less easily transmitted. However, the phase value of each pixel in a phase image may be inverted when the reference phase value changes, and therefore, depending on the reference, a higher phase value may indicate that heat is more easily transmitted. In such a case, the effective bonding region is a region in the phase image where the phase value is equal to or greater than a predetermined threshold.

[0079] [4-4. Fourth Modification] In the above embodiment, an example has been described in which an effective area ratio indicating the ratio of the area of ​​the effective bonded region in the phase image 21 where the phase value is within a predetermined range is calculated (S6), and the bond strength is predicted based on the calculated effective area ratio (S7). In this example, the predetermined range of the phase value is 12° or less.

[0080] However, the predetermined range of phase values ​​need only be determined based on the relationship between the phase range and the correlation coefficient (the correlation coefficient between the effective area ratio and the bond strength) for the target workpiece, and is not limited to 12° or less. This is because the shape of the workpiece may differ from product to product, and the relationship between the phase range and the correlation coefficient may also change depending on the shape of the workpiece. For example, when determining the effective bond area, if a phase range (effective phase range) in which the correlation coefficient is large is applied, the bond strength of the workpiece can be predicted with particularly high accuracy. Below, we will explain the effective phase range determination process for searching for a phase range in which the correlation coefficient is equal to or greater than a threshold, for example, the largest.

[0081] Fig. 14 is a flowchart showing an example of a process for determining an effective phase range. The process in Fig. 14 is executed by, for example, the processor 11, and the determined effective phase range is stored in the storage device 12. Alternatively, the process in Fig. 14 may be executed by another information processing device. An example in which the processor 11 is the main controller will be described below.

[0082] First, the processor 11 executes a phase image acquisition process for a certain sample. The phase image acquisition process is, for example, a process of executing steps S1 to S5 in Fig. 3. The sample is of the same type as the workpiece 90 and has a similar bonding region to the workpiece 90.

[0083] The processor 11 acquires bonding region mask information (S41). The mask information is stored in advance in a storage medium such as the storage device 12. Step S41 may be the same process as step S11 in FIG.

[0084] The processor 11 calculates the area of ​​the joint region from the joint region mask information acquired in step S41 (S42). For example, the processor 11 calculates the number of white pixels in the mask image as the area of ​​the joint region.

[0085] Next, the processor 11 generates a joint region image by excluding regions other than the joint region from the phase image (S43). For example, the processor 11 generates the joint region image by coloring the portions of the segment image other than the joint region black (pixel value 0). Alternatively, the processor 11 may generate the joint region image by performing mask processing on the phase image using a mask image.

[0086] Next, the processor 11 acquires a preset phase search range. In the illustrated example, the processor 11 acquires the minimum value Ni and maximum value Ne of the phase value to be searched (S44). The minimum value Ni and maximum value Ne are stored in advance in the storage device 12, for example. In this embodiment, a phase range (effective phase range) is searched for in which the correlation coefficient between the minimum value Ni and the maximum value Ne is the largest. In this case, the processor 11 divides the phase range between the minimum value Ni and the maximum value Ne into n steps (n is an integer of 2 or greater) and calculates the correlation coefficient for each step. The number of steps n is also stored in the storage device 12 in advance and is acquired by the processor 11 in step S44.

[0087] To search for the phase, processor 11 sets the initial value of variable k (k is an integer equal to or greater than 0) to 0 (S45), and sets the current phase value N to Ni+(Ne-Ni)k / n (S46). When k is the initial value (0), N=Ni.

[0088] The processor 11 performs a binarization process on the joint area image, and generates an effective joint area image in which pixels in the joint area image whose phase values ​​are within a predetermined range (Ni or more and N or less) are set to white (pixel value 1) (S47).

[0089] Next, the processor 11 calculates the number of white pixels in the effective bonding area image as the area of ​​the effective bonding area (S48).

[0090] The processor 11 calculates the ratio of the area of ​​the effective bonding area to the area of ​​the bonding area as the effective area ratio (S49). For example, the processor 11 calculates the effective area ratio (A1 / A2) by dividing the area of ​​the effective bonding area (A1) calculated in step S48 by the area of ​​the bonding area (A2) calculated in step S42.

[0091] The processor 11 searches for a valid phase range within the search range by repeating the processes of steps S46 to S49. Specifically, for example, after executing the processes of steps S46 to S49, the processor 11 determines whether N determined in step S46 is equal to Ne (S50). If N=Ne is not true (No in S50), the processor 11 increments k (S51) and executes the processes of steps S46 to S50.

[0092] If N=Ne in step S50 (Yes in S50), the process proceeds to step S52.

[0093] In step S52, the processor 11 acquires the bonding strength of the sample (S52). The bonding strength is measured by, for example, a peel test.

[0094] The processor 11 associates the effective area ratio calculated in step S29 with the bonding strength obtained in step S52 and records them in the storage device 12.

[0095] The above process is executed until the number of data (number of samples) reaches a predetermined number (S54, S55). The predetermined number is determined in advance by the user, etc. For example, if the number of data is insufficient (No in S54), the sample is replaced (S55), and the processes of steps S1, S21 to S54 are executed for the replaced sample.

[0096] When the number of data reaches a predetermined number (Yes in S54), the processor 11 calculates the correlation coefficient between the effective area ratio and the bonding strength for each phase range based on the data recorded in step S53 (S56), thereby obtaining a graph showing the relationship between the phase range and the correlation coefficient as shown in FIG.

[0097] Based on the result of step S56, the processor 11 determines an effective phase range in which the correlation coefficient between the effective area ratio and the bonding strength is equal to or greater than a threshold value (S57).

[0098] As described above, the phase range in which the correlation coefficient is equal to or greater than the threshold value can be searched for by the effective phase range determination process shown in Fig. 14. When determining the effective bonding region, if the effective phase range determined by the effective phase range determination process is applied, the bonding strength of the workpiece can be predicted with particularly high accuracy.

[0099] 5. Example Aspects The following describes examples of the present disclosure.

[0100] <Aspect 1> An inspection device comprising: an input unit that acquires temperature image data generated by an imaging device capturing images of objects bonded at a predetermined bonding region in time series; and a processor that predicts the bonding strength of the objects based on the temperature image data, wherein the processor performs a Fourier transform or a discrete Fourier transform on the temperature image data to generate a phase image that indicates phase characteristics of the objects, calculates an effective area ratio that indicates the ratio of the area of ​​an effective bonding region in the phase image, where the phase value is within a predetermined range, to the area of ​​the bonding region, and predicts the bonding strength based on the calculated effective area ratio.

[0101] The effective bonding area is a region of the phase image through which heat is easily transmitted. An example of the region through which heat is easily transmitted is a region of the phase image in which the phase value is equal to or less than a predetermined threshold value or equal to or greater than a predetermined threshold value.

[0102] <Aspect 3> The inspection device according to Aspect 1 or 2, wherein the processor calculates the effective area ratio by calculating a value of a ratio of the number of pixels included in the effective bonded region to the number of pixels included in the bonded region in the phase image.

[0103] <Aspect 4> The inspection device according to any one of Aspects 1 to 3, wherein the processor predicts the bond strength based on the calculated effective area ratio using correlation information indicating a relationship between the effective area ratio and the bond strength.

[0104] <Aspect 5> The inspection device according to any one of aspects 1 to 4, wherein the processor causes a notification device to notify information indicating the predicted result of the bonding strength.

[0105] <Aspect 6> The inspection device according to any one of aspects 1 to 5, further comprising a storage device that stores information indicating the predicted results of the bonding strength.

[0106] <Aspect 7> An inspection device comprising: an input unit that acquires temperature image data generated by an imaging device capturing images of objects bonded in a predetermined bonding region in time series; a storage device that stores a trained model that has been machine-learned using phase images of the bonded objects and measured bonding strengths as a training data set; and a processor that can access the storage device and performs a Fourier transform or a discrete Fourier transform on the temperature image data to generate a phase image that indicates the phase characteristics of the objects, wherein the processor inputs the phase images to the trained model, obtains the bonding strength from the trained model, and predicts the bonding strength.

[0107] <Aspect 8> An inspection method for predicting a bonding strength of an object based on temperature image data generated by an imaging device capturing images of the object bonded at a predetermined bonding region in time series, the inspection method including: a processor acquiring the temperature image data; a processor performing a Fourier transform or a discrete Fourier transform on the temperature image data to generate a phase image showing phase characteristics of the object; a processor calculating an effective area ratio indicating a ratio of an area of ​​an effective bonding region in the phase image, the area of ​​which has a phase value within a predetermined range, to an area of ​​the bonding region; and a processor predicting the bonding strength based on the calculated effective area ratio.

[0108] <Aspect 9> An inspection method in which a processor that can access a storage device that stores a trained model non-destructively predicts the bonding strength of objects bonded at a predetermined bonding region based on temperature image data generated by an imaging device capturing images of the objects in a time series, wherein the trained model is a model that has been machine-learned using phase images of the bonded objects and measured bonding strengths as a training data set, and the inspection method includes the steps of: the processor performing a Fourier transform or a discrete Fourier transform on the temperature image data to generate a phase image that shows the phase characteristics of the objects; the processor inputting the phase image into the trained model; and the processor obtaining the bonding strength from the trained model and predicting the bonding strength.

[0109] The present disclosure is applicable to an inspection apparatus and an inspection method.

[0110] REFERENCE SIGNS LIST 1 Inspection system 10 Inspection device 11 Processor 12 Storage device 13 Interface 15 Control box 16 Power supply 17 Infrared camera 18 Excitation source 19 Notification device 21 Phase image 22 Segment image 23 Bonded area image 24 Effective bonded area image 30 Learning device 31 Processor 32 Storage device 33 Interface 90 Workpiece 91 Transmitting material 92 Absorbing material 95 Bonded area 321 Model

Claims

1. An inspection device comprising: an input unit that acquires temperature image data generated by an imaging device photographing an object bonded at a specified bonding area in a time series; and a processor that predicts the bonding strength of the object based on the temperature image data, wherein the processor performs a Fourier transform or a discrete Fourier transform on the temperature image data to generate a phase image that shows the phase characteristics of the object, calculates an effective area ratio that shows the ratio of the area of ​​the effective bonding area in the phase image, where the phase value is within a specified range, to the area of ​​the bonding area, and predicts the bonding strength based on the calculated effective area ratio.

2. The inspection device according to claim 1, wherein the effective bonding area is an area in the phase image where the phase value is below a predetermined threshold value or above a predetermined threshold value.

3. The inspection device according to claim 1, wherein the processor calculates the effective area ratio by calculating a ratio of the number of pixels contained in the effective bonded area to the number of pixels contained in the bonded area in the phase image.

4. The inspection device according to claim 1, wherein the processor predicts the bonding strength based on the calculated effective area ratio using correlation information indicating the relationship between the effective area ratio and the bonding strength.

5. An inspection device according to any one of claims 1 to 4, wherein the processor causes a notification device to notify information indicating the predicted bonding strength.

6. The inspection device according to claim 1, further comprising a storage device for storing information indicating the predicted results of the bonding strength.

7. An inspection device comprising: an input unit that acquires temperature image data generated by an imaging device photographing objects bonded at a specified bonding region in a time series; a storage device that stores a trained model that has been machine-learned using phase images of the bonded objects and measured bonding strengths as a learning dataset; and a processor that can access the storage device and performs a Fourier transform or a discrete Fourier transform on the temperature image data to generate a phase image that shows the phase characteristics of the object, wherein the processor inputs the phase image into the trained model, obtains the bonding strength from the trained model, and predicts the bonding strength.

8. An inspection method for predicting a bonding strength of an object based on temperature image data generated by an imaging device photographing an object bonded at a specified bonding region in a time series, the inspection method comprising: a step of a processor acquiring the temperature image data; a step of the processor applying a Fourier transform or a discrete Fourier transform to the temperature image data to generate a phase image showing phase characteristics of the object; a step of the processor calculating an effective area ratio indicating a ratio of an area of ​​an effective bonding region in the phase image, the phase value of which is within a specified range, to an area of ​​the bonding region; and a step of the processor predicting the bonding strength based on the calculated effective area ratio.

9. An inspection method in which a processor capable of accessing a storage device that stores a trained model non-destructively predicts the bonding strength of an object based on temperature image data generated by an imaging device photographing the object in a time series manner at a predetermined bonding area, wherein the trained model is a model trained by machine learning using phase images of the bonded object and actually measured bonding strength as a learning data set, the inspection method including the steps of: the processor performing a Fourier transform or a discrete Fourier transform on the temperature image data to generate a phase image showing the phase characteristics of the object; the processor inputting the phase image into the trained model; and the processor obtaining the bonding strength from the trained model and predicting the bonding strength.

Images