LEARNING APPARATUS, LEARNING METHOD, AND NON-DESTRUCTIVE INSPECTION SYSTEM

Abstract

To provide a learning device, learning method and nondestructive inspection system which can perform learning for accurately identifying the type of the internal state of a measurement object used in teacher data.SOLUTION: A learning device comprises: a pre-processing unit which performs processing of converting the relative phase difference and relative intensity difference between transmission / reception waves based on the irradiation of an electric wave to a measurement object into color images; and a learning unit which performs learning of an identification model for identifying the type of the internal state by using teacher data in which the first color image and second color image processed by the pre-processing unit are associated with the type of the internal state of the measurement object.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a learning device, a learning method, and a non-destructive inspection system. [Background technology]

[0002] In product inspections of industrial products and food manufacturing processes, there is a demand for understanding the internal state of products that are difficult for users to check visually. For example, even if the surface of a product is in good condition, there is a possibility that there are air bubbles or defects or foreign matter inside. For example, non-destructive inspection of products can be performed using X-ray inspection equipment, but this has issues with cost and safety.

[0003] As a result, there is a growing demand for non-destructive testing systems that use radio waves, which can perform testing more easily and safely.

[0004] As a technology for detecting foreign objects using radio waves, for example, Patent Document 1 discloses a technology in which reflected waves of radio waves transmitted from a transmitter are received by multiple receivers, and if the phase difference between the received waves exceeds a predetermined threshold, it is determined that a foreign object is present in the vicinity of the receiver. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2014-207749 A Summary of the Invention [Problem to be solved by the invention]

[0006] Meanwhile, in product inspection, if it were possible to distinguish the type of internal condition of the product (for example, whether there are air bubbles inside the product or whether metal pieces have been mixed in), in addition to detecting the presence or absence of foreign objects inside the product, it is believed that this would contribute to further improvement of product quality.

[0007] However, in a configuration for determining the presence or absence of a foreign object, such as the technology described in Patent Document 1, only a simple threshold determination is performed, making it difficult to identify the type of internal condition of a product. In addition, for example, when a machine learning algorithm is applied by learning using various teacher data, there is a possibility that a phase jump may occur in detecting the internal condition based on the phase difference between received waves, so there is also room for consideration of a method for determining phase unwrapping.

[0008] As described above, there is room for improvement in terms of accuracy in learning to detect foreign matter inside an object to be measured and to identify the type of foreign matter by learning using teacher data.

[0009] An object of the present disclosure is to provide a learning device, a learning method, and a non-destructive inspection system capable of learning to accurately identify the type of internal condition of a measured object used in teacher data. [Means for solving the problem]

[0010] The learning device according to the present disclosure includes: a pre-processing unit that converts the relative phase difference and the relative intensity difference between a plurality of transmitted and received waves based on the irradiation of radio waves to an object to be measured into a color image; a learning unit that uses teacher data that associates the first color image and the second color image processed by the preprocessing unit with a type of the internal condition of the object to learn an identification model for identifying the type of the internal condition; Equipped with.

[0011] The learning method according to the present disclosure includes: A learning method for a learning device for identifying an internal condition of an object to be measured, comprising the steps of: converting the relative phase difference and the relative intensity difference between a plurality of transmitted and received waves based on irradiation of the radio waves to the object to be measured into a color image including hue, saturation, and brightness; A discrimination model for discriminating the type of the internal condition is trained using training data linking the processed first color image and second color image with the type of the internal condition of the object.

[0012] The non-destructive inspection system according to the present disclosure comprises: a pre-processing unit that converts a relative phase difference and a relative intensity difference between a plurality of transmitted and received waves based on irradiation of radio waves to an object to be measured into a color image including hue, saturation, and brightness; a learning unit that uses teacher data that associates the first color image and the second color image processed by the preprocessing unit with a type of the internal condition of the object to learn an identification model for identifying the type of the internal condition; a discrimination unit that discriminates a type of an internal condition of the object related to the first color image by using the discrimination model; a display unit that displays a recognition result of the recognition unit; Equipped with.

[0013] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0014] According to one aspect of the present disclosure, learning can be performed to accurately identify the type of internal condition of a measured object used in training data.

[0015] Further advantages and benefits of certain aspects of the present disclosure will become apparent from the specification and drawings, in which such advantages and / or benefits are provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of which are provided to obtain one or more identical features. [Brief description of the drawings]

[0016] [Figure 1] 1 is a configuration diagram illustrating an example of a non-destructive inspection system according to an embodiment of the present disclosure. [Diagram 2] 1 is a diagram for explaining a color image in HSV color space in the present embodiment. FIG. [Diagram 3] 11 is a diagram for explaining a conversion process into a color image in the HSV color space in the preprocessing unit. FIG. [Figure 4] 2 is a configuration diagram showing an example of a learning unit and a recognition unit according to the present embodiment. FIG. [Figure 5A] FIG. 13 is a diagram showing feature vectors in an embedding space before learning. [Figure 5B] FIG. 13 shows feature vectors in an embedding space after learning. [Figure 6] 4 is a flowchart showing an example of the operation of learning control in a learning section of the nondestructive inspection system. [Figure 7] 4 is a flowchart showing an example of an inspection control operation in the nondestructive inspection system. [Figure 8A] FIG. 4 is a diagram showing an example of a display on a display unit according to the present embodiment. [Figure 8B] FIG. 4 is a diagram showing an example of a display on a display unit according to the present embodiment. [Figure 9] FIG. 11 is a diagram showing the results of an experiment for confirming the effectiveness of the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] (Embodiment) Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the embodiments described below are merely examples, and the present disclosure is not limited to the following embodiments.

[0018] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.

[0019] It should be noted that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0020] First, a non-destructive inspection system 100 according to an embodiment of the present disclosure will be described with reference to Fig. 1. Fig. 1 is a configuration diagram showing an example of the non-destructive inspection system 100 according to the embodiment of the present disclosure.

[0021] As shown in Fig. 1, the non-destructive inspection system 100 is a system that inspects the internal state of a measured object 101 (such as a substrate, food, or packaged item, the inside of which cannot be seen). For example, when a foreign object 102 (such as a metal piece mixed in a substrate) is present inside the measured object 101, the non-destructive inspection system 100 makes it possible to inspect the internal state of the measured object 101 without destroying the measured object 101. The non-destructive inspection system 100 can also inspect the case where other defects such as air bubbles (voids) are present inside the measured object 101, in addition to the foreign object 102 mixed inside the measured object 101.

[0022] Specifically, the nondestructive inspection system 100 includes a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), and an input / output circuit (not shown). Based on a preset program, the nondestructive inspection system 100 irradiates an object 101 under test with radio waves, receives the reflected waves, and detects and identifies the internal state of the object 101 under test. The nondestructive inspection system 100 then presents (displays) the detection results and identification results of the internal state of the object 101 under test to a user.

[0023] The non-destructive inspection system 100 includes a transmitter / receiver unit 103 , a signal processor unit 104 , a preprocessor unit 105 , a teacher data storage unit 106 , a learning unit 107 , a recognition unit 108 , and a display unit 109 .

[0024] The transmitting / receiving unit 103 has a plurality of transmitting antennas 103A and a plurality of receiving antennas 103B, and is capable of transmitting and receiving radio waves in the millimeter wave band. Specifically, in the transmitting / receiving unit 103, radio waves transmitted (transmitted) from the plurality of transmitting antennas 103A are irradiated onto the object under test 101, and waves reflected from the object under test 101 are received (received) by the plurality of receiving antennas 103B. As the transmitting / receiving unit 103, for example, a multiple-input and multiple-output (MIMO) radar device using a frequency modulated continuous wave (FMCW) system may be used.

[0025] The signal processing unit 104 processes the signal (reflected wave) received by the transmitting / receiving unit 103, and calculates the phase and intensity between a plurality of transmitted and received waves. The signal processing unit 104 can use a method for signal processing that is performed in a general FMCW radar device.

[0026] For example, when an FMCW type device is used, it is possible to estimate the distance to an object (object under test 101) and the relative speed to the object by detecting the time difference between the transmission timing of a transmission signal and the reception timing of a reception signal, and the frequency difference due to the Doppler effect. This makes it possible to calculate different phases and intensities for each combination of multiple transmission antennas 103A and multiple reception antennas 103B.

[0027] The signal processing unit 104 processes the received signal, whose signal waveform is output as digital data, and converts it into a matrix with rows representing virtual arrays (the number of combinations of multiple transmitting antennas and multiple receiving antennas) and columns representing ranges (see, for example, FIG. 3). The range is the distance between the transmitting / receiving unit 103 and the object under test 101. The phase or intensity (or complex number representing the phase and amplitude) calculated for each combination of multiple transmitting antennas and multiple receiving antennas becomes an element of the matrix for each range.

[0028] The pre-processing unit 105 performs pre-processing to convert the information on the phase and intensity between the multiple transmission and reception signals calculated by the signal processing unit 104 into data in a format that is easily identified by the subsequent identification unit 108. Specifically, the pre-processing unit 105 performs processing to convert the relative phase differences and relative intensity differences between the multiple transmission and reception waves based on the irradiation of radio waves to the object under test 101 into a color image. The color image is a color image expressed in a color space (HSV color space) including hue (H), saturation (S), and value brightness (V).

[0029] As shown in FIG. 2, hue H represents colors such as red, green, and blue on a hue wheel with an angle of 0 to 360 degrees. Saturation S represents the vividness of a color as a range of 0 to 100%. For example, the saturation in FIG. 2 indicates that the color becomes more vivid toward the right side in FIG. 2 and becomes more muddy toward the left side. Value V represents the brightness of a color as a range of 0 to 100%. For example, the brightness in FIG. 2 indicates that the color becomes brighter toward the top in FIG. 2 and becomes darker toward the bottom.

[0030] As shown in Fig. 3, the pre-processing unit 105 normalizes the phase and intensity in the matrix from the signal processing unit 104 so that they fall within a predetermined range, calculates the relative phase difference and relative intensity difference in the matrix, and converts them into a predetermined numerical expression. More specifically, the pre-processing unit 105 calculates the relative phase difference and relative intensity difference by subtracting, for example, the component (phase or intensity) of the combination of the transmitting antenna and the receiving antenna corresponding to the first row from the components (phase or intensity) of the other rows for each range (each column). Alternatively, the relative phase difference and relative intensity difference may be calculated by subtracting, for example, the component corresponding to the first column from the components of the other columns for each row. As a result, a relative phase difference matrix and a relative intensity difference matrix are generated with the ranges as columns and the virtual arrays as rows.

[0031] Then, pre-processing unit 105 converts the calculated relative phase difference (for example, in the range of -2π to 2π) into a numerical expression in a predetermined range. Pre-processing unit 105 also converts the calculated relative intensity difference into a numerical expression in a predetermined range. The numerical expression in the predetermined range may be, for example, a real value between 0 and 1.0, or an integer value between 0 and 255.

[0032] The preprocessing unit 105 generates a color image by assigning the relative phase difference matrix to the hue H in the HSV color space, and the relative intensity difference matrix to the saturation S and lightness V in the HSV color space. In this way, a color image can be generated in which the continuity of phase (no phase jump from -π to π occurs) is expressed by the continuity of the hue wheel.

[0033] Furthermore, the pre-processing unit 105 outputs a color image to the display unit 109. In this case, the pre-processing unit 105 may output a color image in the HSV color space, or may convert the color image in the HSV color space into a color image in the RGB color space and output the color image.

[0034] As shown in FIG. 1, the teacher data storage unit 106 associates a color image of the object to be measured 101, whose internal condition is known, with an internal condition label indicating the type of the internal condition, and stores the result as teacher data for discriminative learning of the internal condition.

[0035] Specifically, for example, before learning is performed by the learning unit 107, the non-destructive inspection system 100 measures the object to be measured 101 whose type of internal condition (e.g., normal product (product without defects), presence of foreign matter, presence of air bubbles, etc.) is known, and the color image (second color image) converted by the pre-processing unit 105 is linked to the internal condition label (correct label) and stored in large quantities in the teacher data storage unit 106.

[0036] The color image (second color image) stored in the teacher data storage unit 106 is a measured object used for learning by the learning unit 107, and is therefore a color image obtained by measuring a measured object having a known internal condition and processing it in the pre-processing unit 105. In addition, a plurality of samples of the second color image are stored in the teacher data storage unit 106 for each type of internal condition assumed in advance.

[0037] The learning unit 107 uses the training data to learn a discrimination model for discriminating the type of internal situation. The learning unit 107 will be described in detail later.

[0038] The discrimination unit 108 has the discrimination model described above. Using the discrimination model, the discrimination unit 108 discriminates the type of the internal state of the object 101 related to the color image processed by the pre-processing unit 105. As the discrimination model, a model such as a neural network is used. In the discrimination model, parameters learned by the learning unit 107 are used.

[0039] The classification unit 108 classifies the type of the internal situation and predicts the above-mentioned internal situation label. The classification unit 108 outputs a predicted label which is a prediction result of the internal situation label. In addition, the classification unit 108 also outputs information on an embedding space (feature vector) described later, together with the predicted label. The classification unit 108 will be described in detail later.

[0040] The display unit 109 is a display device capable of displaying to a user the color image processed by the preprocessing unit 105, the predicted labels which are the classification results of the classification unit 108, and the information on the embedding space from the classification unit 108. For example, a user interface such as a touch panel display is used as the display unit 109. The user can determine the internal state of the object under test 101 via the display unit 109 and judge whether the object under test 101 is a pass or fail product.

[0041] Next, a description will be given of the details of learning section 107 and classification section 108. Fig. 4 is a diagram showing an example of the internal configuration of learning section 107 and classification section 108.

[0042] In this embodiment, a convolutional neural network (CNN) is used as an identification model for the identification unit 108. As shown in Fig. 4, the identification unit 108 uses the identification model to embed a feature vector into an embedding space, which is a low-dimensional space. The identification unit 108 includes a feature extraction unit 1081, an embedding space unit 1082, and a label classification unit 1083.

[0043] The feature extraction unit 1081 uses a convolutional neural network to extract features regarding the colors and their arrangement of the color image input from the preprocessing unit 105. As described above, the feature extraction unit 1081 may use a neural network with a structure other than the convolutional neural network that is relatively often used in image recognition.

[0044] The embedding space unit 1082 performs a process of embedding (mapping) a high-dimensional vector indicating the feature extracted by the feature extraction unit 1081 into a low-dimensional embedding space as a low-dimensional vector. The embedding space is preferably a two-dimensional or three-dimensional space that is easy to visualize. The embedding space unit 1082 may also be implemented by a fully connected neural network that converts the output of a convolutional neural network into two-output or three-output.

[0045] The label classification unit 1083 obtains the low-dimensional vector converted by the embedding space unit 1082 and outputs a predicted label. The label classification unit 1083 converts the two-dimensional or three-dimensional low-dimensional vector into a predicted label output according to the type of internal situation to be identified. As the label classification unit 1083, various neural networks and classification algorithms such as the kNN (k-Nearest Neighbor) method and SVM (Support Vector Machine) can be used.

[0046] As described above, the classification unit 108 processes the color image in the feature extraction unit 1081, the embedding space unit 1082, and the label classification unit 1083 in that order, and outputs information on the embedding space including the low-dimensional vector converted in the embedding space unit 1082 and the predicted label classified in the label classification unit 1083.

[0047] The learning unit 107 performs distance learning using the features embedded in the embedding space by the classification unit 108. Specifically, the learning unit 107 performs learning using the low-dimensional vector (hereinafter, feature vector) and predicted label output by the classification unit 108 so that the predicted label matches the teacher data. The learning unit 107 has an error backpropagation unit 1071 and an inter-feature distance learning unit 1072.

[0048] The error backpropagation unit 1071 calculates the error between the predicted label and the correct label in the teacher data storage unit 106, and adjusts the parameters of the discrimination model so as to reduce the error.

[0049] The parameters of the discrimination model are, for example, the weights and bias values ​​of the neurons in the feature extraction unit 1081 and the label classification unit 1083, and the error backpropagation unit 1071 adjusts these parameters in accordance with the error.

[0050] The feature distance learning unit 1072 performs a process of adjusting the distance between feature vectors having the same correct answer label for the first feature vectors output by the discrimination unit 108. Specifically, the feature distance learning unit 1072 adjusts the parameters of the discrimination model so that the feature vectors obtained by converting input images having the same internal situation label in the teacher data are closer to each other, and the feature vectors obtained by converting input images having different internal situation labels are farther away from each other.

[0051] The parameters of the discrimination model are, for example, the weights and bias values ​​of the neurons in the embedding space unit 1082, and the feature distance learning unit 1072 adjusts these parameters.

[0052] Here, the first feature vector is obtained by converting an input image associated with a certain correct answer label. The second feature vector is obtained by converting another input image associated with the same correct answer label as the correct answer label of the first feature vector. Moreover, the third feature vector is obtained by converting another input image associated with a correct answer label different from the correct answer label of the first feature vector. In other words, the feature distance learning unit 1072 adjusts the parameters so that the first feature vector approaches the second feature vector and moves away from the third feature vector.

[0053] For example, as shown in Fig. 5A, in the embedding space 501 before learning by the learning unit 107, each feature vector is arranged (converted) concentrated near one position regardless of the type of internal situation label (because it is still difficult to distinguish). Note that a two-dimensional embedding space is shown as an example. Also, in Figs. 5A and 5B, a normal product is indicated by "○", the presence of a foreign object is indicated by "△", and the presence of an air bubble is indicated by "×".

[0054] 5B, by performing learning by the learning unit 107, each feature vector is distributed and arranged (converted) for each content of the internal situation label in the embedding space 502. Specifically, each feature vector is arranged such that feature vectors having the same type of internal situation label are at different positions from feature vectors having different types of internal situation labels.

[0055] This allows the nondestructive inspection system 100 to easily identify the type of internal situation having similar characteristics, depending on which internal situation label in the embedding space the feature vector is closest to. Also, since the nondestructive inspection system 100 can map the feature vector to a position away from any internal situation label in the embedding space, it is possible for the user to easily recognize the reason why the internal situation is not included in the training data (it is an unknown defect state).

[0056] Next, a description will be given of an example of the operation of the non-destructive inspection system 100. First, a description will be given of an example of the operation of the learning control in the learning unit 107. FIG.

[0057] 6 is premised on the preparation of a plurality of objects 101 to be measured, the internal conditions of which are identified and which are to be used as samples of teacher data, for example. The plurality of objects 101 to be measured are prepared such that the number of objects 101 to be measured having the same type of internal condition is approximately the same for each type of internal condition.

[0058] 6, the nondestructive inspection system 100 irradiates radio waves to the object under test 101 by the transmitting / receiving unit 103 (step S301). After irradiating the radio waves and receiving the reflected waves from the object under test 101 by the transmitting / receiving unit 103, the nondestructive inspection system 100 calculates the relative phase difference and the relative intensity difference between the multiple transmitting antennas 103A and the multiple receiving antennas 103B by the pre-processing unit 105 (step S302).

[0059] After calculating the relative phase difference and the relative intensity difference, the nondestructive inspection system 100 converts the relative phase difference and the relative intensity difference into a color image in the HSV color space in the preprocessing unit 105 (step S303). Then, the nondestructive inspection system 100 associates the color image with the internal situation label and stores it in the teacher data storage unit 106 (step S304).

[0060] Next, the nondestructive inspection system 100 causes the learning unit 107 to learn the discrimination model (step S305). After learning the discrimination model, the nondestructive inspection system 100 causes the learning unit 107 to determine whether or not the discrimination rate is sufficient (step S306).

[0061] The classification rate may be, for example, the ratio of correct answers when all of the multiple objects under test 101 are used as test samples. In addition, the criterion for determining whether the classification rate is sufficient or not may be, for example, that the classification rate is sufficient when it is equal to or greater than a given value (for example, 90%).

[0062] If the discrimination rate is not sufficient (step S306, NO), the process returns to step S301 and the learning flow is repeated. On the other hand, if the discrimination rate is sufficient (step S306, YES), this control ends.

[0063] Next, a description will be given of an example of the operation of inspection control in the nondestructive inspection system 100. Fig. 7 is a flowchart showing an example of the operation of inspection control in the nondestructive inspection system 100. The process in Fig. 7 is based on the premise that the object to be inspected 101 has been prepared.

[0064] 7, the nondestructive inspection system 100 sets parameters of an identification model of the identification unit 108 (step S601). The parameters of the identification model in this case are parameters adjusted by the learning unit 107 through learning.

[0065] The nondestructive inspection system 100 irradiates the object under test 101 with radio waves by the transmitting / receiving unit 103 (step S602). After irradiating the radio waves and receiving the reflected waves from the object under test 101 by the transmitting / receiving unit 103, the nondestructive inspection system 100 calculates the relative phase difference and the relative intensity difference between the multiple transmitting antennas 103A and the multiple receiving antennas 103B by the pre-processing unit 105 (step S603).

[0066] After calculating the relative phase difference and the relative intensity difference, the preprocessing unit 105 of the nondestructive inspection system 100 converts the relative phase difference and the relative intensity difference into a color image in the HSV color space (step S604). After converting into a color image, the identification unit 108 of the nondestructive inspection system 100 converts the color image into a feature vector and identifies a predicted label (step S605). Then, the nondestructive inspection system 100 displays the feature vector and the predicted label on the display unit 109 (step S606).

[0067] Thereafter, the nondestructive inspection system 100 judges whether the inspection has been completed (step S607). If the result of the judgment is that the inspection has not been completed (step S607, NO), the process returns to step S602. On the other hand, if the inspection has been completed (step S607, YES), this control ends.

[0068] Next, a display example of the inspection result by the nondestructive inspection system 100 in this embodiment will be described.

[0069] 8A and 8B, the display unit 109 displays the color image generated by the preprocessing unit 105, the embedding space and feature vector generated by the embedding space unit 1082, and the predicted label that is the classification result. The object to be inspected is indicated by a "■" in the embedding space.

[0070] In FIG. 8A, since the inspection object is located closest to the feature vector "△" indicating "foreign object present", the predicted label states, for example, "foreign object present". In this way, the predicted label and the feature vector of the embedding space are displayed, so that the user can recognize the basis (certainty) of the type of internal situation in the predicted label as visual information. Note that the display of the predicted label may be omitted. Also, when the distance between the measured object "■" and the foreign object presence "△" is within a predetermined range, the measured object "■" and the foreign object presence "△" may be displayed to appeal to the user by flashing or changing the color.

[0071] In FIG. 8B, the inspection target is located at the midpoint of each feature vector. In this case, since the inspection target is located at a position far from any feature vector, the predicted label is written, for example, "unknown defect". By writing it in the predicted label in this way, the user can understand that there is a defect that is not in the training data, and by checking the embedding space, it becomes possible to easily determine the basis. Note that the display of "unknown defect" may be omitted. Also, the display of "○" for normal product, "△" for foreign matter, "×" for air bubble, and "■" for measured object may be displayed to appeal to the user by flashing or changing the color.

[0072] Furthermore, the identification unit 108 can determine whether or not an object to be inspected is an "unknown defect" depending on whether or not the distance between the object to be inspected and each feature vector is equal to or greater than a predetermined threshold value.

[0073] The judgment results of the samples that have been tested may be cumulatively displayed on the display unit 109. This allows the user to check at a glance the degree of variation, such as the proportion of normal products, among multiple samples currently being tested.

[0074] According to the present embodiment configured as described above, the learning unit 107 performs learning using a color image expressed in the HSV color space. That is, since the discrimination model is learned using a color image in which the continuity of phase is expressed by the continuity of a hue wheel, phase differences near ±π, where phase jumps are likely to occur, can be expressed by similar colors. As a result, it is possible to improve the learning effect for identifying the type of internal condition of the object to be measured used in the teacher data, and ultimately to accurately identify the type of internal condition of the object to be measured.

[0075] In addition, since the relative phase difference is assigned to the hue, the phase difference around ±π where the phase jump is likely to occur can be expressed with a similar color. In addition, since the relative intensity difference is assigned to the saturation and brightness, the vividness and brightness of the color can be expressed in detail. As a result, it is possible to easily generate a color image that is easier for the user and the classification model to distinguish, so that the learning effect for distinguishing the type of the internal state of the measured object used in the training data can be further improved, and the classification rate can be improved.

[0076] Furthermore, since the discrimination unit 108 outputs the information (feature vector) of the embedding space in addition to the predicted label via the display unit, it is possible to present the user with the basis for determining which internal situation the feature vector of the object to be measured is similar to, or whether the object has an unknown defect (unknown feature). In other words, it becomes easier for the user to recognize the basis and reliability of discrimination in the nondestructive inspection system 100, making it easier to determine whether the object to be measured is a good or defective item.

[0077] Furthermore, a predetermined experiment was conducted to confirm the effectiveness of the nondestructive inspection system 100 according to this embodiment. The predetermined experiment was an experiment in which a substrate was used as a measurement object, and the discrimination rates for three types of content states, namely, normal product, presence of large bubbles, and presence of small bubbles, and discrimination of whether the product was normal or not were measured.

[0078] As a comparative example, for example, the matrix generated by the signal processing unit was directly converted to an image in the RGB color space without being converted to the HSV color space (the relative phase difference was directly assigned to R (red), and the relative intensity difference was directly assigned to G (green) and B (blue)), and the above classification rate was measured and compared with the classification rate in this embodiment (this example).

[0079] In addition, in this experiment, the inspection items were the presence or absence of a normal product and the type of internal condition for the cases where distance learning was not performed (without distance learning) and where distance learning was performed (with distance learning). Figure 9 shows the experimental results of a given experiment.

[0080] As shown in Fig. 9, it was confirmed that the discrimination rate for all test items in the comparative example was less than 90%. In contrast, it was confirmed that the discrimination rate for all test items in this embodiment was 90% or more. In other words, it was confirmed that the discrimination rate was improved in this embodiment.

[0081] In addition, it can be confirmed that the classification rate is improved when distance learning is performed compared to when distance learning is not performed. In other words, it was confirmed that the classification rate was improved by performing distance learning of the feature vector. From the above, the effectiveness of this embodiment was confirmed.

[0082] In the above embodiment, the relative intensity difference is assigned to saturation and brightness, but the present disclosure is not limited to this, and the relative intensity difference may be assigned to either saturation or brightness. However, from the viewpoint of making the color image easily recognizable, it is preferable to assign the relative intensity difference to both saturation and brightness.

[0083] In addition, in the above embodiment, the transmitting / receiving unit 103 is configured to receive reflected waves from the object to be measured, but the present disclosure is not limited to this, and in a configuration in which the object to be measured is sandwiched between the transmitting unit and the receiving unit, the receiving unit may be configured to receive transmitted waves from the object to be measured.

[0084] In the above-described embodiments, the notation "... part" used for each component may be replaced with other notations such as "... circuit", "... assembly", "... device", "... unit", or "... module".

[0085] Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modified or altered examples within the scope of the claims. It is understood that such modified or altered examples also belong to the technical scope of the present disclosure. In addition, the components in the embodiments may be arbitrarily combined within the scope of the present disclosure.

[0086] The present disclosure can be realized by software, hardware, or software in cooperation with hardware. Each functional block used in the description of the above embodiment may be realized partially or entirely as an LSI (Large Scale Integration) which is an integrated circuit, and each process described in the above embodiment may be controlled partially or entirely by one LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of one chip so as to include some or all of the functional blocks. The LSI may have input and output of data. Depending on the degree of integration, the LSI may be called an IC, a system LSI, a super LSI, or an ultra LSI.

[0087] The method of integration is not limited to LSI, and may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, after LSI manufacturing, a programmable FPGA (Field Programmable Gate Array) or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI may be used. The present disclosure may be realized as digital processing or analog processing.

[0088] Furthermore, if a new integrated circuit technology that can replace LSI appears due to the progress of semiconductor technology or a derivative technology, it is possible to integrate the functional blocks using that technology. The application of biotechnology is also a possibility.

[0089] In addition, the above-mentioned embodiments are merely examples of the embodiment of the present disclosure, and the technical scope of the present disclosure should not be interpreted as being limited by them. In other words, the present disclosure can be embodied in various forms without departing from the gist or main features thereof.

[0090] <Summary of this disclosure> A learning device according to an embodiment of the present disclosure includes: a pre-processing unit that converts the relative phase difference and the relative intensity difference between a plurality of transmitted and received waves based on the irradiation of radio waves to an object to be measured into a color image; a learning unit that uses teacher data that associates the first color image and the second color image processed by the preprocessing unit with a type of the internal condition of the object to learn an identification model for identifying the type of the internal condition; Equipped with.

[0091] A learning method according to another aspect of the present disclosure includes: A learning method for a learning device for identifying an internal condition of an object to be measured, comprising the steps of: converting the relative phase difference and the relative intensity difference between a plurality of transmitted and received waves based on irradiation of the radio waves to the object to be measured into a color image including hue, saturation, and brightness; A discrimination model for discriminating the type of the internal condition is trained using training data linking the processed first color image and second color image with the type of the internal condition of the object.

[0092] A non-destructive inspection system according to yet another aspect of the present disclosure includes: a pre-processing unit that converts a relative phase difference and a relative intensity difference between a plurality of transmitted and received waves based on irradiation of radio waves to an object to be measured into a color image including hue, saturation, and brightness; a learning unit that uses teacher data that associates the first color image and the second color image processed by the preprocessing unit with a type of the internal condition of the object to learn an identification model for identifying the type of the internal condition; a discrimination unit that discriminates a type of an internal condition of the object related to the first color image by using the discrimination model; a display unit that displays a recognition result of the recognition unit; Equipped with. [Industrial Applicability]

[0093] An aspect of the present disclosure is useful for a learning device, a learning method, and a non-destructive inspection system capable of learning to accurately identify the type of internal condition of a measured object used in teacher data. [Explanation of symbols]

[0094] 100 Non-destructive Inspection System 101 Object to be measured 102 Foreign object 103 Transmitter / receiver 104 Signal Processing Section 105 Pretreatment section 106 Teacher Data Storage Unit 107 Learning Department 108 Identification section 109 Display section 1071 Error Backpropagation Unit 1072 Feature Distance Learning Unit 1081 Feature Extraction Unit 1082 Embedded space 1083 Label Classification Department

Claims

1. a pre-processing unit that converts the relative phase difference and the relative intensity difference between a plurality of transmitted and received waves based on the irradiation of radio waves to an object to be measured into a color image; a learning unit that uses the first color image processed by the preprocessing unit and training data linking the second color image with the type of the internal condition of the object to learn an identification model for identifying the type of the internal condition; A learning device comprising:

2. the pre-processing unit assigns the relative phase difference to a hue and the relative intensity difference to at least one of a saturation and a lightness. The learning device according to claim 1 .

3. a discrimination unit that discriminates a type of an internal condition of the object to be measured related to the first color image by using the discrimination model; the learning unit adjusts parameters of the discrimination model in accordance with the discrimination result of the discrimination unit and the teacher data. The learning device according to claim 1 or 2.

4. the identification unit extracts a first feature vector of the first color image using a neural network and predicts a type of the internal condition of the object by embedding the first feature vector in a low-dimensional space; the learning unit adjusts parameters of the discrimination model so that the first feature vector in the predicted internal condition predicted by the discrimination unit substantially coincides with a second feature vector of a second color image in the teacher data that is associated with the same type of internal condition as the internal condition of the object to be measured related to the first color image. The learning device according to claim 3.

5. the learning unit adjusts parameters of the discrimination model such that a distance between the first feature vector and the second feature vector is shorter than a distance between the first feature vector and a third feature vector of a third color image in the teacher data, the third color image being associated with a type of internal condition different from the internal condition of the measured object related to the first color image. The learning device according to claim 4.

6. The identification unit outputs information about the low-dimensional space. The learning device according to claim 5 .

7. A learning method for a learning device for identifying an internal condition of an object to be measured, comprising the steps of: converting the relative phase difference and the relative intensity difference between a plurality of transmitted and received waves based on irradiation of the radio waves to the object to be measured into a color image including hue, saturation, and brightness; learning a discrimination model for discriminating the type of the internal condition of the object using training data linking the processed first color image and the processed second color image with the type of the internal condition of the object; How to learn.

8. a pre-processing unit that converts a relative phase difference and a relative intensity difference between a plurality of transmitted and received waves based on irradiation of radio waves to an object to be measured into a color image including hue, saturation, and brightness; a learning unit that uses the first color image processed by the preprocessing unit and training data linking the second color image with the type of the internal condition of the object to learn an identification model for identifying the type of the internal condition; an identification unit that identifies a type of an internal condition of the object related to the first color image by using the identification model; a display unit that displays a recognition result of the recognition unit; A non-destructive inspection system comprising:

Images