Information processing device, information processing system, information output method and program

Abstract

To specify an internal structure of an object without breaking the object.SOLUTION: An information processing device includes an image acquisition unit for acquiring a response image showing a time response of a change in a temperature distribution in an area including a place heated by irradiating an object with periodical light, and an output unit for outputting an image of a cross section structure of the object estimated by using the response image acquired by the image acquisition unit. Then, the information processing device can specify an internal structure of the object without breaking the object.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, an information output method, and a program. [Background technology]

[0002] Patent Document 1 discloses a method for analyzing the tendency of the orientation state of a filler in a part of a resin molded product by binarizing a slice image of the resin molded product obtained by X-ray CT and using a power spectrum image obtained by performing a Fourier transform on this binarized image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-2547 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, to identify the internal structure of an object such as carbon fiber reinforced plastic, destructive testing that requires cutting out, such as X-ray CT, may be used. However, destructive testing, for example, can take a long time and require large-scale testing equipment. The present invention aims to identify the internal structure of an object without destroying the object. [Means for solving the problem]

[0005] With this objective in mind, the technology disclosed in this specification is an information processing device characterized by comprising an image acquisition unit that acquires a response image showing the time response of changes in temperature distribution in an area including a heated portion by irradiating the object with periodic light, and an output unit that outputs an image of the cross-sectional structure of the object estimated using the response image acquired by the image acquisition unit.

[0006] From another perspective, the technology disclosed in this specification is an information processing system comprising: an acquisition unit that acquires learning data that is a combination of a structural image showing the internal structure of an object and a response image that shows the time response of changes in temperature distribution in an area including a heated area by irradiating the object with periodic light; a learning unit that trains a machine learning model using the learning data acquired by the acquisition unit; and an output unit that inputs the response image of the object to the machine learning model trained by the learning unit and outputs an estimated image of the internal structure of the object. Here, the object may have a specific material and a base material that supports the material, and the estimated image may be an image that indicates the position of the material in a cross section of the object. The learning data may include a plurality of response images obtained under different heating conditions for one object. The heating condition may be a heating frequency when heating one object. The heating condition may be a heating position on one object. The heating condition may be a surface of one object to be heated. The heating condition may be a heating area for one object. The learning data may include a plurality of response images of a single object, each of which has a different temperature sensing surface. In addition, the machine learning model has a neural network structure including an encoder unit having a plurality of convolutional layers and receiving the response image as input, and a decoder unit having a plurality of deconvolutional layers and outputting the estimated image, and the features extracted in the convolutional layers may not be sent to the deconvolutional layers corresponding to the convolutional layers.

[0007] From another perspective, the technology disclosed in this specification is an information processing device characterized by comprising: a memory unit that stores a machine learning model trained using training data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of changes in temperature distribution in an area including a location heated by irradiating the object with periodic light; and an output unit that inputs the response image of an object to the machine learning model and outputs information related to the internal structure of the object. Here, it is preferable to include a heating unit that periodically irradiates light onto an object to heat the object, a detection unit that detects the time response of changes in temperature distribution in an area of ​​the object that includes the portion heated by the heating unit, another output unit that outputs information indicating the orientation of at least one material that constitutes the object based on the time response of the changes in temperature distribution detected by the detection unit, and a reception unit that receives from a user a selection of information to be output from information regarding the internal structure of the output unit and information indicating the orientation of the other output unit.

[0008] From another perspective, the technology disclosed in this specification is an information output method characterized by comprising the steps of: acquiring a response image showing the time response of a change in temperature distribution in an area including a heated portion by irradiating the object with periodic light; and outputting an image of the cross-sectional structure of the object estimated using the acquired response image.

[0009] From another perspective, the technology disclosed in this specification is an information output method comprising the steps of: acquiring training data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of a change in temperature distribution in an area including a heated portion by irradiating the object with periodic light; training a machine learning model using the acquired training data; and inputting the response image of the object into the machine learning model and outputting an estimated image of the internal structure of the object.

[0010] From another perspective, the technology disclosed in this specification is an information output method comprising the steps of: storing a machine learning model trained using training data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of changes in temperature distribution in an area including a location heated by irradiating the object with periodic light; and inputting the response image of an object into the machine learning model and outputting information relating to the internal structure of the object.

[0011] From another perspective, the technology disclosed in this specification is a program that causes a computer to perform the following functions: acquire a response image showing the time response of changes in temperature distribution in an area including a heated portion by irradiating an object with periodic light; and output an image of the cross-sectional structure of the object estimated using the acquired response image.

[0012] From another perspective, the technology disclosed in this specification is a program that causes a computer to execute the following functions: acquire learning data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of changes in temperature distribution in an area including a heated portion by irradiating the object with periodic light; train a machine learning model using the acquired learning data; and input the response image of the object into the machine learning model and output an estimated image of the internal structure of the object.

[0013] From another perspective, the technology disclosed in this specification is a program that causes a computer to perform the following functions: storing a machine learning model trained using training data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of changes in temperature distribution in an area including a portion heated by irradiating the object with periodic light; and inputting the response image of an object into the machine learning model and outputting information related to the internal structure of the object. [Effects of the Invention]

[0014] According to the technology disclosed in this specification, the internal structure of an object can be identified without destroying the object. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a schematic configuration diagram showing an internal structure evaluation device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a functional configuration diagram of a computer. [Figure 3] FIG. 1 illustrates an example of a hardware configuration of a computer. [Figure 4] FIG. 2 is an explanatory diagram showing the principle of measuring in-plane thermal diffusivity. [Figure 5] FIG. 2 is an explanatory diagram showing the principle of measuring thermal diffusivity in the thickness direction. [Figure 6] 1(a) shows the direction of thermal diffusion in the object 1, and FIG. 1(b) shows the relationship between the angle of thermal diffusion, thermal diffusivity, and fiber orientation density. [Figure 7] 10 is a flowchart illustrating the operation of the internal structure evaluation device. [Figure 8] FIG. 10 is a diagram showing an example of a cyclic heating response image measured by a lock-in thermography cyclic heating method. [Figure 9] FIG. 2 is a diagram showing an example of an object used in the present embodiment. [Figure 10] 10A and 10B are a periodic heating response image and a cross-sectional structure image of an object according to the present embodiment. [Figure 11] FIG. 2 is a conceptual diagram of a neural network structure used by the machine learning unit of the present embodiment. [Figure 12] FIG. 2 is an explanatory diagram of learning in a machine learning unit of the present embodiment. [Figure 13] 10 is a diagram illustrating an example of an estimation result by a machine learning unit according to the present embodiment. [Figure 14] 10 is an explanatory diagram of a heating surface and a detection surface of a modified example of the target 1. FIG. [Figure 15] 10 is an explanatory diagram of a heating surface and a detection surface of a modified example of the target 1. FIG. [Figure 16] FIG. 10 is an explanatory diagram of a modified internal structure evaluation device. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, the present embodiment will be described in detail with reference to the accompanying drawings. <Configuration of the internal structure evaluation device 100>

[0017] FIG. 1 is a schematic diagram showing the configuration of an internal structure evaluation device 100 according to this embodiment. First, with reference to FIG. 1, the configuration of an internal structure evaluation device 100 to which this embodiment is applied will be described.

[0018] As shown in Figure 1, the internal structure evaluation device 100 of this embodiment comprises a diode laser 10 that functions as a light source for heating the object 1, a light guiding section 20 that guides the laser light of the diode laser 10 to the object 1, an infrared thermograph (lock-in thermograph) 30 that is arranged opposite the object 1, a computer 50 that receives signals from the infrared thermograph 30, and a periodic signal generator 70 that generates periodic signals and outputs them to the diode laser 10 and the computer 50.

[0019] The light guide section 20 also includes a mirror 21 that reflects the laser light emitted from the diode laser 10, a beam expander 23 that expands the beam diameter of the laser light from the mirror 21, and a holder 24 that holds the object 1. In the internal structure evaluation device 100, the laser light emitted from the diode laser 10 passes through a mirror 21 and a beam expander 23 and is irradiated onto the object 1. In the object 1, the area irradiated with the laser light is periodically heated. That is, a specific point (position) on the surface of the object 1 is periodically spot heated.

[0020] The temperature of the object 1, which has been periodically heated by the laser light from the diode laser 10, is measured from the rear surface of the object 1 by an infrared thermograph 30. The infrared thermograph 30 captures (measures) an infrared image of a predetermined range including a specific point that is periodically spot heated by the diode laser 10. That is, the infrared thermograph 30 measures the temperature response of the rear surface of the object 1 in two dimensions. A periodic signal is input to the infrared thermograph 30 from a periodic signal generator 70. Temperature distribution data, which is data on the temperature measured by the infrared thermograph 30, is then output to a computer 50.

[0021] The computer 50, in conjunction with the infrared thermograph 30, continuously captures and calculates infrared images at a predetermined frame rate, creating averaged image data from the temperature change over time (lock-in method). More specifically, the data obtained by the infrared thermograph 30 is processed by the computer 50 to calculate the direction (angle) from the heating point Hp (see Figure 6(a) below), thermal diffusivity, and orientation. In addition, the process of performing calculations using the image data obtained by the infrared thermography 30 with the computer 50 to calculate the direction (angle) from the heating point Hp of the object 1, thermal diffusivity, and orientation is sometimes referred to as "measurement of the internal structure."

[0022] The computer 50 of this embodiment is also capable of outputting an estimation result of the internal structure of the object 1 using image data obtained by the infrared thermography 30 (for example, a cyclical heating response image Ti described later) and a trained machine learning model (hereinafter referred to as a trained model). This will be explained in detail later. Note that estimating the internal structure of the object 1 based on machine learning by the computer 50 using image data obtained by the infrared thermography 30 is sometimes referred to as "estimating the internal structure."

[0023] In the following description, a direction along the surface of the object 1 in Fig. 1, i.e., the left-right direction in the figure, may be referred to as the x-direction. The up-down direction in Fig. 1 may be referred to as the z-direction. The direction into the paper in Fig. 1 may be referred to as the y-direction.

[0024] <Functional configuration of computer 50> FIG. 2 is a functional block diagram of the computer 50.

[0025] Next, the functional configuration of the computer 50 to which this embodiment is applied will be described. As shown in Fig. 2, a computer 50 to which this embodiment is applied includes a phase delay distribution measurement unit 51 that measures a phase delay distribution based on temperature distribution data and a periodic signal input from the infrared thermography 30 (see Fig. 1), and a thermal diffusivity distribution calculation unit 52 that calculates a thermal diffusivity distribution based on the measured phase delay. Furthermore, the computer 50 includes an orientation calculation unit 53 that calculates the orientation (described below) of the object 1 (see Fig. 1) based on the calculated thermal diffusivity distribution, and a calculation result display unit 54 that displays the calculation results of the thermal diffusivity distribution and the orientation. The computer 50 also includes a machine learning unit 55 that estimates the internal structure of the object 1 based on the time response (phase delay distribution) of the measured temperature distribution change, and a selection receiving unit 56 that receives a selection of the output content of the evaluation result of the object 1.

[0026] <Hardware configuration of computer 50> FIG. 3 is a diagram showing an example of the hardware configuration of the computer 50. As shown in FIG.

[0027] As shown in FIG. 3, computer 50 includes a CPU (Central Processing Unit) 501, which is a computing means, and a main memory 503 and an HDD (Hard Disk Drive) 505, which are storage means. Here, CPU 501 executes various programs such as an OS (Operating System) and application software. Main memory 503 is a storage area that stores various programs and data used for executing the programs. HDD 505 is a storage area that stores input data for the various programs and output data from the various programs. These components included in computer 50 execute the various functions described above in FIG. 2 and elsewhere.

[0028] The computer 50 is equipped with a communication interface (communication I / F) 507 for communicating with external devices such as the infrared thermography 30. The programs executed by the CPU 501 (for example, programs for measuring or estimating the internal structure) can be stored in advance in the main memory 503, or can be stored on a storage medium such as a CD-ROM and provided to the CPU 501, or can be provided to the CPU 501 via a network (not shown).

[0029] <Configuration of the surrounding area of ​​object 1> Next, the configuration of the object 1 and the peripheral configuration of the object 1 in the internal structure evaluation device 100 will be described.

[0030] First, the structure of the object 1 will be described. The object 1 is a flat plate-shaped member. The material of the object 1 is not particularly limited, but it may be made of a composite material such as carbon fiber reinforced plastics (CFRP). As shown in FIG. 1, the object 1 has an upper surface 11, which is the side irradiated with laser light from the diode laser 10, i.e., the upper surface in FIG. 1, and a lower surface 13, which is the side opposite to the side irradiated with laser light from the diode laser 10, i.e., the lower surface in FIG. 1. This lower surface 13 faces the infrared thermography device 30. Additionally, the infrared thermography device 30 measures the heat distribution on the lower surface 13 of the object 1.

[0031] The computer 50 of this embodiment then identifies (specifies) the orientation of the object 1 based on the time response of the change in temperature distribution of the object 1 (see FIG. 1) detected by the infrared thermography 30. To explain further, the computer 50 specifies the orientation of the object 1 based on the delay in the response of the change in temperature distribution of the object 1. In addition, here, the thermal characteristics of the object 1 are assumed to be the orientation of the object 1.

[0032] Here, the object 1 in the illustrated example is a carbon-based composite material, more specifically, carbon fiber reinforced plastics (CFRP). To explain further, the object 1 is made of pitch-based carbon fiber reinforced resin, which is so-called pitch-based carbon fiber (reinforced material) made from pitch impregnated with a resin (matrix) such as epoxy resin. That is, in the object 1, the resin (matrix) such as epoxy resin supports the carbon resin (material).

[0033] Pitch-based carbon fibers have higher thermal conductivity than the resins impregnated into them. Additionally, pitch-based carbon fibers have significantly different thermal conductivity (thermal diffusivity) from resins with relatively low thermal conductivity. Here, pitch-based carbon fiber reinforced resin is merely an example, and any difference in thermal conductivity (thermal diffusivity) between the carbon fiber and the resin is sufficient. For example, polyacrylonitrile (PAN)-based carbon fiber reinforced resins with relatively low thermal diffusivity may also be used. The carbon fiber reinforced resin in the illustrated example is a so-called discontinuous fiber composite material, and the length of the pitch-based carbon fiber, for example, is about 0.1 mm to 10 mm, more specifically, about 1 mm to 5 mm. Here, the discontinuous fiber composite material is merely an example, and a continuous fiber composite material may also be used. For example, the object 1 may be a carbon fiber reinforced resin laminated with continuous fiber prepregs.

[0034] The orientation of the object 1 is an index showing the orientation distribution of the fibers (carbon fibers) contained in the object 1, which is an anisotropic material. In this embodiment, it is shown by the orientation angle (orientation direction) and the dispersion of the orientation angle (variation in the orientation angle). The orientation angle is an angle showing the direction in which the fibers in the object 1 tend to orient. In other words, the orientation angle is the degree of preferential orientation of the fibers.

[0035] <Measurement principle> Next, the principle of the measurement method in this embodiment will be described. 4(a) and 4(b) are explanatory diagrams showing the principle of measuring in-plane thermal diffusivity.

[0036] First, the principle of measuring the in-plane thermal diffusivity of the object 1 will be described with reference to FIGS. 4(a) and 4(b).

[0037] Here, heating light (laser light) of a certain frequency is irradiated onto the surface of the object 1, and measurements are taken from the back side of the object 1 using an infrared thermograph 30. Additionally, the distance dependency of the phase delay is detected here. Now, if the area irradiated by the heating light is considered to be a point heat source, the AC temperature T at a position distant from this point heat source by a distance r isac is expressed by the following equation (1).

[0038]

number

[0039] Here, T0 constant (km) f... Heating frequency (Hz) t time (s) r...distance (m)

[0040] Also, the point heat source and AC temperature T ac The phase difference θ is expressed by the following equation (2).

[0041]

number

[0042] Here, f1... Heating frequency (constant) (Hz) D: Thermal diffusivity (mm 2 / s)

[0043] The in-plane thermal diffusivity D of the object 1 is expressed by the following formula (3).

[0044]

number

[0045] 5(a) and 5(b) are explanatory diagrams showing the principle of measuring thermal diffusivity in the thickness direction. Next, the measurement principle of the thermal diffusivity in the thickness direction of the object 1 will be explained with reference to Figures 5(a) and 5(b). Here, the measurement is performed while changing the frequency of the heating light irradiated on the object 1 with a constant thickness d. In addition, the frequency dependency of the phase delay is detected here.

[0046] The thermal diffusivity D of the object 1 in the thickness direction is expressed by the following formula (4).

[0047]

number

[0048] Here, d: Thickness of the object to be measured (constant) (mm)

[0049] FIG. 6(a) shows the direction of thermal diffusion in the object 1, and FIG. 6(b) shows the relationship between the angle of thermal diffusion, thermal diffusivity, and fiber orientation density. Next, the measurement principle of the orientation distribution of the object 1 will be explained with reference to FIGS. 6(a) and 6(b).

[0050] As shown in Figure 6(a), the thermal diffusion phenomenon from the heated point Hp changes depending on the orientation of the carbon fibers contained in the object 1. That is, heat from the heated point Hp is easily transmitted in the direction along the orientation of the carbon fibers contained in the object 1, but is difficult to transmit in the direction intersecting the orientation of the carbon fibers. This causes the speed of thermal diffusion to differ depending on the angle from the heated point Hp on the object 1. In other words, the thermal diffusivity changes depending on the angle from the heated point Hp on the object 1.

[0051] Here, as shown in Figure 6(b), the thermal diffusivity of the object 1 was measured in multiple directions from the heating point Hp using the internal structure evaluation device 100, and it was confirmed that the thermal diffusivity changes depending on the angle. In this embodiment, the mean and variance of the fiber orientation distribution are calculated based on the obtained thermal diffusivity angular distribution. More specifically, the mean and variance of the fiber orientation distribution are calculated using a fiber orientation distribution density function obtained based on the obtained thermal diffusivity angular distribution. The fiber orientation distribution density function can be obtained, for example, by dividing the thermal diffusivity angular distribution containing multiple peaks into individual peak sections and fitting each peak using the least squares method.

[0052] Here, the fiber orientation distribution density function is expressed by the following equation (5).

[0053]

number

[0054] Here, η offset angle (rad) ξ...parameter that determines the distribution size (dimensionless number) P: First fitting parameter Q: Second fitting parameter

[0055] Also, η≦θ a ≦θ≦θ b ≦180°, and P≧½, Q≧½, ξ≧2, and η≧0. Then, based on P and Q, the mean μ and variance σ of the distribution 2 (Standard deviation σ) is calculated by the following formula (6).

[0056]

number

[0057] Then, based on the average value μ of the distribution, the fiber orientation direction θ0 is determined by the following formula (7): Note that the section including the peak in formula (7) corresponds to the division angle.

[0058]

number

[0059] Also, the variance σ 2 represents the degree of concentration in the orientation direction, and can therefore be rephrased as the degree of fiber orientation.

[0060] <Operation> FIG. 7 is a flowchart illustrating the operation of the internal structure evaluation device 100. Next, the operation of the internal structure evaluation device 100 in this embodiment will be described with reference to FIGS.

[0061] First, the surface of the object 1 is periodically spot heated by the laser light emitted from the diode laser 10 in the internal structure evaluation device 100 (step 701). Then, the phase lag distribution measuring unit 51 measures the phase lag distribution based on the temperature distribution measured by the infrared thermography 30 (step 702). Then, based on the measured phase delay distribution, the thermal diffusivity distribution calculation unit 52 calculates the thermal diffusivity angular distribution (step 703). Then, based on the calculated thermal diffusivity angular distribution, the orientation calculation unit 53 calculates the orientation direction and dispersion (step 704). Then, the calculation result display unit 54 displays the calculation results of the orientation direction and dispersion on a display means (not shown) (step 705).

[0062] In the internal structure evaluation device 100 of the present embodiment described above (see FIG. 1), the upper surface 11 of the object 1 is the surface heated by the diode laser 10, and the lower surface 13 is the surface for detecting the temperature by the infrared thermography 30, but this is not limiting. The internal structure evaluation device 100 may also use the upper surface 11 of the object 1 as the surface heated by the diode laser 10, and the same upper surface 11 as the surface for detecting the temperature by the infrared thermography 30.

[0063] Next, the machine learning unit 55 will be described in detail. FIG. 8 is a diagram showing an example of a cyclic heating response image measured by the lock-in thermography cyclic heating method.

[0064] As described above, the internal structure evaluation device 100 of this embodiment is capable of identifying fiber orientation using thermal diffusivity measurement by the lock-in thermography cyclic heating method. The lock-in thermography cyclic heating method performs lock-in analysis of the thermal response, allowing for non-destructive acquisition of detailed information as amplitude and phase lag. In other words, the internal structure evaluation device 100 can acquire the state of thermal diffusion as the amplitude and phase lag of minute temperature fluctuations.

[0065] When an anisotropic discontinuous fiber CFRP material is measured using the lock-in thermography cyclic heating method with spot heating, a cyclic heating response image, which is a phase-lag image like the one shown in Figure 8, is obtained. This cyclic heating response image does not have a perfect circular distribution, but rather an elliptical distribution that is stretched in the left-right direction due to the influence of the internal carbon fibers. In this way, the cyclic heating response image contains information about the internal structure of the material. Therefore, by quantifying the relationship between the internal structure and the cyclic heating response image, it is possible to estimate the structure.

[0066] The machine learning unit 55 uses a cyclic heating response image that shows the time response of changes in temperature distribution, which can be obtained by the lock-in thermography cyclic heating method. The machine learning unit 55 converts the cyclic heating response image into an internal structure. The machine learning unit 55 also uses deep learning for the conversion into the internal structure. In this embodiment, the training data used for deep learning was prepared by constructing virtual materials and conducting heat transfer simulation analysis. The training data consisted of a large number of pairs of image data of virtual materials with various structures and cyclic heating response image data obtained by cyclic heating of the virtual materials through simulation.

[0067] The network used by the machine learning unit 55 has a structure that combines convolution and deconvolution to convert a cyclic heating response image (input) into an estimated image of the internal structure (output). The machine learning unit 55 constructs a trained model by training the network using the created dataset. The machine learning unit 55 then uses the trained model to enable estimation of the internal structure from the cyclic heating response image.

[0068] In this embodiment, a large number of combinations of "known internal structures" and "periodic heating response images" were prepared as learning data. To create the known internal structures, material development simulation software capable of creating virtual materials was used. On the other hand, software capable of heat transfer simulation analysis was used to create the periodic heating response images of the created known internal structures. In this embodiment, a large number of combinations of data images of "known internal structures" and data images of "periodic heating response images" created and analyzed using this software were created.

[0069] FIG. 9 is a diagram showing an example of the target object 1 used in this embodiment. FIG. 10 shows a periodic heating response image Ti and a cross-sectional structure image Si of the object 1 of this embodiment.

[0070] The object 1 shown in Figure 9 is a virtual material. Unidirectional laminated CFRP is used for the object 1. As shown in Figure 9, a model is used in which bundles of φ200 μm carbon fibers 1c are distributed in one direction within a 2 mm x 2 mm x 1 mm area in the center of a 3 mm x 3 mm x 1 mm resin body of resin 1r. The carbon fibers 1c are randomly distributed based on conditions predetermined by the software.

[0071] In the heat transfer simulation analysis, periodic point heating was performed by heating a single heating point Hp at the center of the upper surface 11 of the object 1. A 2 mm x 2 mm area on the lower surface 13 of the object 1 was used as the measurement surface MA. Here, the size of the heating point Hp in this embodiment is 100 μm in diameter. The periodic heating conditions were a heat flux of 500,000 W / m 2The heating frequency was 0.1 Hz, the measurement frequency was 2 Hz, and the number of cycles was 10.

[0072] Then, as shown in FIG. 10(a), a periodic heating response image Ti is obtained as a result of the heat transfer simulation analysis. This periodic heating response image Ti is a periodic heating response image Ti that shows the phase delay distribution as seen from the bottom surface 13 (xy plane) of the object 1 shown in FIG. 9. In the periodic heating response image Ti, the phase delay is expressed by black and white shading. The phase delay distribution differs depending on the internal structure of the object 1. That is, the image content expressed by black and white shading in the periodic heating response image Ti differs depending on the internal structure of the object 1. In other words, the image content expressed by black and white shading in the periodic heating response image Ti differs depending on the arrangement of the carbon fibers 1c in the cross-sectional structure of the object 1.

[0073] Furthermore, as shown in FIG. 10(b), a cross-sectional structure image Si is obtained that shows the fiber distribution in the cross section of the object 1 that passes through the heating point Hp shown in FIG. 9. This cross-sectional structure image Si is an image of the yz cross section shown in FIG. 9. The cross-sectional structure image Si identifies the positions of the carbon fibers 1c and the resin 1r in the cross section. Note that the cross-sectional structure image Si in this embodiment is coordinate-converted into a monochrome square for easy use in deep learning, which will be described later.

[0074] The machine learning unit 55 of this embodiment uses a combination of the cyclic heating response image Ti and the cross-sectional structure image Si as one piece of data, and uses a data set made up of multiple pieces of data as learning data (teacher data) for deep learning.

[0075] <Machine learning conditions> FIG. 11 is a conceptual diagram of a neural network structure used by the machine learning unit of this embodiment.

[0076] The machine learning unit 55 uses a convolutional neural network (CNN) as an example of a machine learning model. A convolutional neural network is a network structure that can reduce parameters while capturing image features by connecting each layer through convolution processing.

[0077] The machine learning unit 55 of this embodiment uses the structure of U-Net as a reference, but employs a network structure different from U-Net. Here, U-Net is a network structure in which the encoder extracts features from an input image using convolution, and the decoder upsamples the extracted features to output an image of the same size as the input image. In particular, U-Net is primarily used for object detection, and the encoder and decoder are coupled to stabilize the upsampling. For example, U-Net employs a skip structure that sends a feature map (feature values) extracted in a specific convolution layer to the deconvolution layer corresponding to that specific convolution layer.

[0078] In this embodiment, features of the input periodic heating response image Ti are extracted, and then converted into an image that generates an output estimated image representing the internal structure. Therefore, as shown in FIG. 11 , in this embodiment, a network structure was created that removes the connection between the encoder and decoder, based on the structure of U-Net. The machine learning unit 55 of this embodiment uses a neural network including an encoder with five convolutional layers and a decoder with five deconvolutional layers. The encoder has three layers consisting of convolutional layers and pooling layers, and two convolutional layers only. The decoder has three layers consisting of deconvolutional layers and unpooling layers, and two deconvolutional layers only. The neural network of this embodiment does not have a skip connection that sends a feature map (feature) extracted in a specific convolutional layer to the deconvolutional layer corresponding to that specific convolutional layer.

[0079] The internal structure evaluation device 100 of this embodiment realizes the information output method by the following steps. First, the machine learning unit 55 acquires learning data that is a combination of a cross-sectional structure image Si showing the internal structure of the object and a periodic heating response image Ti showing the time response of the change in temperature distribution in an area including a portion heated by irradiating the object with periodic light. Next, the machine learning unit 55 trains a machine learning model using the acquired training data. Furthermore, the machine learning unit 55 inputs the periodic heating response image Ti of an object whose internal structure is unknown to the machine learning model, and outputs a structure estimation image Ei of the internal structure of the object.

[0080] FIG. 12 is an explanatory diagram of learning in the machine learning unit 55 of this embodiment.

[0081] The machine learning unit 55 of this embodiment performs deep learning, training a neural network using a dataset consisting of the above-mentioned pairs. In this embodiment, a total of 122 pairs of datasets were created, with 99 pairs randomly assigned to training data, 20 pairs to validation data, and 3 pairs to test data. In other words, a total of 119 pairs of training data and validation data were used for learning.

[0082] The error function used is the mean square error E, which is highly versatile in regression problems and is shown in equation (8). Adam (Adaptive moment estimation) is used as the optimization method, and learning is performed in the direction that minimizes the error function.

[0083]

number

[0084] where y i is the correct value, and y l^ is the output. First, Model A was trained with hyperparameters set to a maximum of 100 epochs and 16 batches. The training calculation time was 6 minutes 29 seconds on a CPU. As a result, the training error, which is expressed as the magnitude of the error function of the training data, was 0.054, and the validation error, which is expressed as the magnitude of the error function of the validation data, was 0.093. As shown in Figure 12, the errors converged to a downward sloping trend, indicating that training was proceeding normally without overfitting.

[0085] Next, in order to increase the number of epochs while avoiding overfitting, learning is performed using E', which is the error function shown in equation (9) with L2 regularization applied.

[0086]

number

[0087] The regularization parameter λ was set to 0.001. β is the regression coefficient vector. Model B was trained with a maximum number of epochs of 200 and a batch size of 16. The training calculation time was 10 minutes and 41 seconds on a CPU. As a result, the training error, which is expressed as the magnitude of the error function of the training data, was 0.043, and the validation error, which is expressed as the magnitude of the error function of the validation data, was 0.077.

[0088] 13A and 13B show examples of estimation results by the machine learning unit 55 of this embodiment. Note that FIG. 13A shows the results of Model A, and FIG. 13B shows the results of Model B.

[0089] The machine learning unit 55 of this embodiment inputs the cyclic heating response images Ti, which are test data (data 1, data 2, and data 3), into the trained model and estimates the internal structure. 13(a) and 13(b) show, from left to right, the cyclic heating response images Ti as test data, the structure estimation images Ei, which are the estimation results, and the cross-sectional structure images Si, which are the correct data.

[0090] The periodic heating response images Ti shown in FIG. 13 as test data were created by performing a heat transfer simulation analysis on a virtual material having the cross-sectional structure shown in the cross-sectional structure image Si. The structure estimation image Ei, which is the estimation (output) result, is an image that shows the positions of the carbon fibers 1c in the cross section of the object 1. In this way, the machine learning unit 55 is capable of outputting the structure estimation image Ei, which is an image of the cross-sectional structure of the object 1, from the cyclic heating response image Ti of the object 1.

[0091] 13(a) and 13(b), qualitatively, both Model A and Model B estimate the structure with a certain degree of accuracy. In other words, it can be seen that the structure estimation image Ei is an output that reflects the characteristics of the cross-sectional structure image Si, which is the correct structure.

[0092] Next, the accuracy rate obtained by the machine learning unit 55 of this embodiment will be described. The accuracy rate was determined by evaluating the cross-sectional structure image Si, which is an image of a cross section of the object 1, as the correct answer, and the structure estimation image Ei, which is an image output by the trained model.

[0093] In this embodiment, two evaluation methods are used. Specifically, there is an evaluation method called ALL and an evaluation method called Balanced Accuracy (hereinafter referred to as BA). ALL is the proportion of correct answers out of the total, and is also called the accuracy rate. BA is a proportion obtained by averaging the difference in the number of correct answers. In the evaluation, each pixel in the cross-sectional structure image Si of the virtual material was compared with a pixel at a corresponding position in the estimated structure image Ei obtained as an output image.

[0094] Here, for pixels at corresponding positions, the relationship between carbon fiber 1c and other than carbon fiber 1c (resin 1r in this example) in the cross-sectional structure image Si and carbon fiber 1c and other than carbon fiber 1c (resin 1r in this example) in the structure estimation image Ei is as follows: For example, when the cross-sectional structure image Si shows carbon fiber 1c, if it is predicted that the structure estimation image Ei shows carbon fiber 1c, it is considered a true positive (hereinafter referred to as TP). When the cross-sectional structure image Si shows something other than carbon fiber 1c, if it is predicted that the structure estimation image Ei shows carbon fiber 1c, it is considered a false positive (hereinafter referred to as FP). When the cross-sectional structure image Si shows carbon fiber 1c, if it is predicted that the structure estimation image Ei shows something other than carbon fiber 1c, it is considered a false negative (hereinafter referred to as FN). When the cross-sectional structure image Si shows something other than carbon fiber 1c, if it is predicted that the structure estimation image Ei shows something other than carbon fiber 1c, it is considered a true negative (hereinafter referred to as TN).

[0095] ALL is expressed by equation (10).

number

[0096] The ALL values ​​were approximately 65% ​​for Model A (data 1), approximately 66% for Model A (data 2), and approximately 57% for Model A (data 3). The ALL values ​​were approximately 63% for Model B (data 1), approximately 65% ​​for Model B (data 2), and approximately 62% for Model B (data 3). Thus, the ALL values ​​were correct answer rates of approximately 57% to 66%.

[0097] BA is expressed by equation (11).

number

[0098] The BA values ​​were approximately 56% for Model A (data 1), approximately 57% for Model A (data 2), and approximately 50% for Model A (data 3).The BA values ​​were approximately 53% for Model B (data 1), approximately 52% for Model B (data 2), and approximately 55% for Model B (data 3).In this way, the BA values ​​were correct answer rates of approximately 50% to 57%.

[0099] In this embodiment, we focus on the thermal properties specific to the interior of the object 1 (e.g., CFRP, a composite material containing carbon fiber). That is, when the object 1 is heated using a lock-in thermography-based cyclic heating method, the distribution of the heat propagation state (phase delay) from the upper surface 11 to the lower surface 13 of the object 1 can be identified. For example, there is a tendency for the phase delay to be small in the fiber direction of the carbon fiber and large in a direction different from the fiber direction of the carbon fiber. Therefore, the cyclic heating response image Ti, which shows the phase delay distribution obtained by measuring the object 1, reflects the feature amounts of the internal structure of the object 1. Therefore, the machine learning unit 55 of this embodiment extracts the feature amounts through machine learning using a machine learning model. Then, by inputting the cyclic heating response image Ti of the object 1 into the trained machine learning model, a structure estimation image Ei that estimates the internal structure of the object 1 is output.

[0100] In this way, the internal structure evaluation device 100 of this embodiment can estimate the cross-sectional structure of the object 1 from the cyclic heating response image Ti using a trained model. The internal structure evaluation device 100 can then stereoscopically identify the internal structure of the object 1, for example, the three-dimensional orientation of carbon fibers, from the cyclic heating response image Ti shown in two dimensions. In other words, the internal structure evaluation device 100 can estimate the three-dimensional internal structure of the object 1 using the cyclic heating response image Ti.

[0101] Next, the selection receiving unit 56 shown in FIG. 2 will be described. The internal structure evaluation device 100 of this embodiment is capable of performing arithmetic processing by a computer 50 using image data obtained from the object 1 by the infrared thermography 30, and outputting measurement results of the internal structure that calculate the orientation of the object 1. The internal structure evaluation device 100 of this embodiment is also capable of outputting estimation results of the internal structure of the object 1 using a trained model based on the image data obtained by the infrared thermography 30. The selection receiving unit 56 is capable of receiving from the user a selection of whether to output the measurement results of the internal structure or the estimation results of the internal structure.

[0102] Next, variations in learning data when the machine learning unit 55 learns a machine learning model will be described.

[0103] <Modification> 1, the machine learning unit 55 of this embodiment uses, as training data, periodic heating response images Ti obtained by setting the upper surface 11 of the object 1 as the surface heated by the diode laser 10 and the lower surface 13 as the surface for detecting the temperature by the infrared thermography 30. The machine learning unit 55 then trains a machine learning model using, as training data, periodic heating response images Ti obtained for the object 1 under the same heating conditions.

[0104] On the other hand, the training data may be configured to include a plurality of cyclic heating response images Ti obtained under different heating conditions for the same object 1. In this way, by training the machine learning model using training data including a plurality of different cyclic heating response images Ti for the same object 1, it is possible to improve the accuracy of estimating the internal structure of the object 1.

[0105] For example, the machine learning unit 55 may use, as learning data, the periodic heating response image Ti obtained by using the upper surface 11 of the object 1 as the heating surface and the same upper surface 11 as the sensing surface. Then, for a cross-sectional structure image Si showing a cross section of one object 1, learning data is created by combining a periodic heating response image Ti obtained with the upper surface 11 of one object 1 as the heating surface and the lower surface 13 as the sensing surface (see FIG. 1 ) and a periodic heating response image Ti obtained with the upper surface 11 of one object 1 as the heating surface and the upper surface 11 as the sensing surface. Then, the machine learning unit 55 may train a neural network using the created learning data.

[0106] <Modification> FIG. 14 is an explanatory diagram of a heating surface and a detection surface of a modified example of the target 1. In FIG.

[0107] In this embodiment, for example, in the object 1, the heating surface and the detection surface are obtained by a combination of two surfaces that are approximately parallel, such as the upper surface 11 (xy plane) and the lower surface 13 (xy plane), but this is not limited to this form.

[0108] 14, a periodic heat response image Ti may be obtained by providing a heating point Hp on the first side surface 14 (xz plane) of the object 1, with the first side surface 14 serving as the heating surface, and a third side surface 16 (xz plane) of the object 1, which is approximately parallel to the first side surface 14, serving as the detection surface. Alternatively, a periodic heat response image Ti may be obtained by providing a heating point Hp on the second side surface 15 (yz plane) of the object 1, with the second side surface 15 serving as the heating surface, and a fourth side surface 17 (yz plane) of the object 1, which is approximately parallel to the second side surface 15, serving as the detection surface.

[0109] Then, for a cross-sectional structure image Si showing a cross section of the object 1, training data is created by combining a periodic heating response image Ti obtained using the lower surface 13 as the sensing surface, a periodic heating response image Ti obtained using the third side surface 16 as the sensing surface, and a periodic heating response image Ti obtained using the fourth side surface 17 as the sensing surface. In this way, the training data may be composed of multiple periodic heating response images Ti obtained for one object 1 using different heating target surfaces. Then, the machine learning unit 55 may train a neural network using this training data.

[0110] <Modification> FIG. 15 is an explanatory diagram of a heating surface and a detection surface of a modified example of the target 1. In FIG.

[0111] In this embodiment, the upper surface 11 of the object 1 is heated by performing spot heating at one heating point Hp (see FIG. 9, for example) on the upper surface 11 of the object 1, but the present invention is not limited to this example. As shown in Fig. 15, spot heating may be performed at multiple locations on the upper surface 11 of the object 1. In the example shown in Fig. 15, spot heating is performed by irradiating a first heated point Hp1, a second heated point Hp2, a third heated point Hp3, and a fourth heated point Hp4 on the upper surface 11 with light from a diode laser 10. Then, each time spot heating is performed at each location on the upper surface 11 of the object 1, a periodic heating response image Ti is obtained using the lower surface 13 as the detection surface.

[0112] Then, learning data is created by combining a plurality of periodic heat response images Ti obtained by heating at different positions on the upper surface 11 with a cross-sectional structure image Si showing a cross section of the object 1. In this way, the learning data may be configured to include a plurality of periodic heat response images Ti obtained by heating one object 1 at different positions. Then, the machine learning unit 55 may use this learning data to train a neural network.

[0113] <Modification> FIG. 16 is an explanatory diagram of a modified internal structure evaluation device 100.

[0114] As shown in FIG. 16, the basic configuration of the internal structure evaluation device 100 of the modified example is the same as that of the internal structure evaluation device 100 of the above-described embodiment. The internal structure evaluation device 100 of the modified example has a cylindrical lens 25 that converts laser light emitted from a diode laser 10 (see FIG. 1) into sheet light. As shown in FIG. 16, a light irradiation area HA is formed on the upper surface 11 of the object 1. The light irradiation area HA shown in the figure has a substantially elliptical or rectangular shape with its major axis along the x-direction. In addition, the light irradiation area HA has a shape that is elongated in one direction on the upper surface 11 of the object 1. This light irradiation area HA enables line heating (linear heating) on ​​the upper surface 11 of the object 1. The light irradiation area HA can be regarded as a linear heat source.

[0115] Then, line heating is performed on the upper surface 11 of the object 1, and a periodic heating response image Ti is obtained using the lower surface 13 as the detection surface. Furthermore, learning data is created by combining the periodic heating response image Ti obtained by performing line heating with a cross-sectional structure image Si showing a cross section of the object 1. Then, the machine learning unit 55 may use this learning data to train a neural network.

[0116] 16, the object 1 and the cylindrical lens 25 may be moved relative to each other. For example, the cylindrical lens 25 may be fixed, and the object 1 may be moved in the y direction relative to the cylindrical lens 25. This allows line heating to be performed at different locations on the object 1. Then, each time line heating is performed at each location on the upper surface 11 of the object 1, a periodic heating response image Ti can be obtained using the lower surface 13 as the detection surface. Then, learning data is created by combining a plurality of periodic heating response images Ti obtained by different positions of line heating on the upper surface 11 side with a cross-sectional structure image Si showing a cross section of the target object 1. Then, the machine learning unit 55 may train a neural network using this learning data.

[0117] Furthermore, a data set is created by combining a periodic heating response image Ti obtained by performing point heating on one object 1 and a periodic heating response image Ti obtained by performing line heating on one object 1, for a cross-sectional structure image Si showing a cross section of the object 1. In this way, the training data may be configured to include multiple periodic heating response images Ti obtained by varying the heating area of ​​one object 1. Then, the machine learning unit 55 may train a neural network using this training data.

[0118] The internal structure evaluation device 100 may perform surface heating such that the entire upper surface 11 of the object 1 is heated, and the lower surface 13 is used as the detection surface to acquire the periodic heating response image Ti.

[0119] <Modification> The machine learning unit 55 of this embodiment uses, as training data, the periodic heating response images Ti obtained by heating the object 1 at a single heating frequency, but is not limited to this. For example, multiple periodic heating response images Ti obtained by heating a single object 1 at different heating frequencies may be used as training data.

[0120] In particular, when the upper surface 11 of the object 1 is used as the heating surface and the upper surface 11 is used as the detection surface, by obtaining multiple periodic heating response images Ti with different heating frequencies, it becomes possible to selectively obtain information about the internal structure of the object 1 in the thickness direction (z direction).

[0121] Here, when the heating frequency is low, the thermal wavelength becomes longer and the heating range becomes longer from the upper surface 11 to the lower surface 13. In this case, information on the thermal properties of the object 1 in the thickness direction, extending to the lower surface 13, is obtained. On the other hand, when the heating frequency is high, the thermal wavelength becomes shorter and the heating range from the upper surface 11 to the lower surface 13 is shorter. In this case, information on the thermal properties of the object 1 only on the upper surface 11 side in the thickness direction is obtained.

[0122] In this way, by varying the heating frequency for the object 1, it is possible to obtain information on different thermal properties in the thickness direction of the object 1. Then, a neural network can be trained using, as training data, a plurality of periodic heating response images Ti obtained by varying the heating frequency when heating one object 1.

[0123] The diode laser 10 is an example of a heating unit. The infrared thermography 30 is an example of a detection unit. The calculation result display unit 54 is an example of another output unit. The machine learning unit 55 is an example of an acquisition unit, image acquisition unit, learning unit, output unit, and memory unit. The selection receiving unit 56 is an example of a receiving unit. The internal structure evaluation device 100 is an example of an information processing device or information processing system. The cyclic heating response image Ti is an example of a response image. The cross-sectional structure image Si is an example of a structure image. The structure estimation image Ei is an example of an estimation image.

[0124] The internal structure evaluation device 100 of this embodiment uses a trained model of the machine learning unit 55 to estimate the internal structure of the object 1, but is not limited to this. The internal structure evaluation device 100 only needs to be able to estimate the internal structure (cross-sectional structure) of the object 1 based on the cyclic heating response image Ti of the object 1, and can also use an image processing technique using pattern matching, for example.

[0125] 9, for example, in this embodiment, an example of machine learning is described in which a model in which carbon fiber bundles are distributed in one direction is used as the object 1, but the present invention is not limited to this. The machine learning unit 55 of the internal structure evaluation device 100 of this embodiment can also be applied to an object 1 in which the fibers of the carbon fiber bundles are oriented in various directions (multidirectional).

[0126] Furthermore, the object 1 to be evaluated by the internal structure evaluation device 100 of this embodiment is not limited to composite materials. The object 1 may be made of, for example, a single material. In this case, for example, if voids exist in the material, heat conduction is hindered in the voids. Therefore, the presence or absence of voids in the material is reflected in the periodic heating response image Ti as a time response of changes in temperature distribution. The internal structure evaluation device 100 can also measure and estimate the internal structure of an object 1 made of a single material.

[0127] Although various embodiments and modifications have been described above, it is of course possible to combine these embodiments and modifications. Furthermore, the present disclosure is not limited to the above-described embodiments, and can be implemented in various forms without departing from the gist of the present disclosure. [Explanation of symbols]

[0128] 1...object, 1r...resin, 1c...carbon fiber, 10...diode laser, 30...infrared thermography, 50...computer, 51...distribution measurement unit, 52...thermal diffusivity distribution calculation unit, 53...orientation calculation unit, 54...calculation result display unit, 55...machine learning unit, 56...selection reception unit, Si...cross-sectional structure image, Ti...periodic heating response image, Ei...structure estimation image

Claims

1. an acquisition unit that acquires learning data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of a change in temperature distribution in an area including a portion heated by irradiating the object with periodic light; a learning unit that learns a machine learning model using the learning data acquired by the acquisition unit; an output unit that inputs the response image of the object to the machine learning model trained by the learning unit and outputs an estimated image of the internal structure of the object; Equipped with The object has a specific material and a base material that supports the specific material, The estimated image is an image showing the position of the material in a cross section of the object. An information processing system comprising:

2. 2. The information processing system according to claim 1, wherein the learning data includes a plurality of response images of a single object, the response images being generated under different heating conditions.

3. 3. The information processing system according to claim 2, wherein the heating condition is a heating frequency when heating one object.

4. 3. The information processing system according to claim 2, wherein the heating condition is a heating position on one object.

5. 5. The information processing system according to claim 4, wherein the heating condition is a surface of one object to be heated.

6. 5. The information processing system according to claim 4, wherein the heating condition is a heating area for one object.

7. 2. The information processing system according to claim 1, wherein the learning data includes a plurality of response images of a single object, each of which has a different temperature sensing surface.

8. the machine learning model has a neural network structure including an encoder unit having a plurality of convolutional layers and receiving the response image as input, and a decoder unit having a plurality of deconvolutional layers and outputting the estimated image; 2. The information processing system according to claim 1, wherein the features extracted in the convolutional layer are not sent to the deconvolutional layer corresponding to the convolutional layer.

9. a memory unit that stores a machine learning model trained using training data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of a change in temperature distribution in an area including a portion heated by irradiating the object with periodic light; an output unit that inputs the response image of the object to the machine learning model and outputs an estimated image of the internal structure of the object; Equipped with The object has a specific material and a base material that supports the specific material, The estimated image is an image showing the position of the material in a cross section of the object.

1. An information processing device comprising:

10. acquiring learning data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of a change in temperature distribution in an area including a portion heated by irradiating the object with periodic light; training a machine learning model using the acquired training data; inputting the response image of the object into the machine learning model and outputting an estimated image of the internal structure of the object; Equipped with The object has a specific material and a base material that supports the specific material, The estimated image is an image showing the position of the material in a cross section of the object.

10. An information output method comprising:

11. On the computer, A function of acquiring learning data that is a combination of a structural image showing the internal structure of an object and a response image showing the time response of a change in temperature distribution in an area including a portion heated by irradiating the object with periodic light; The function of training a machine learning model using the acquired training data, a function of inputting the response image of the object into the machine learning model and outputting an estimated image of the internal structure of the object, The object has a specific material and a base material that supports the specific material, The estimated image is an image showing the position of the material in a cross section of the object. program.

Images