Learning apparatus and learning method

Abstract

The present invention is provided with: a learning image acquisition unit (1101) that acquires an image data set to be examined; an existing feature amount acquisition unit (1102) that acquires feature amounts that are image data sets for which tasks have been executed; a feature quantity extraction unit (1103) that extracts feature quantities of an image data set that is a subject to be inspected; a feature amount comparison unit (1104) that calculates the degree of similarity between the feature amount of the image data set that is the inspection target and the feature amount of the image data set that is the task executed; a model selection unit (1105) that selects, on the basis of the degree of similarity of the feature quantities, one or more trained models corresponding to an image data set as an executed task having a high degree of similarity of the feature quantities with respect to the image data set as the inspection target from among the image data sets as the executed task; a model learning unit (1106) that inputs an image data set to be inspected into the trained model selected by the model selection unit (1105), thereby generating a new trained model that is a distribution of qualified products; a model evaluation unit (1107) that determines whether or not the evaluation of the new trained model is greater than or equal to a threshold value; and a model output unit (1108) that outputs information indicating a new trained model for which the evaluation is determined to be greater than or equal to the threshold value.

Description

Technical Field

[0001] This invention relates to a learning apparatus and a learning method for obtaining a well-trained model. Background Technology

[0002] In production settings, when applying AI-powered automated appearance inspection to a product, the necessary data for achieving the desired functionality requires collecting image datasets of the inspection targets. Therefore, there has been a long-standing pursuit of transfer learning and other methods that utilize limited data from past models for learning.

[0003] On the other hand, when multiple past models exist, to obtain the optimal model for the image dataset being examined, it is necessary to learn and evaluate all past models and select the model with the highest accuracy. Therefore, with a large number of past models, obtaining the optimal model incurs a high learning cost.

[0004] Therefore, a method for transfer learning based on the selection of past models using features from image datasets is proposed.

[0005] As an example of such transfer learning technology, the technology disclosed in Patent Document 1 can be cited.

[0006] This technology combines multiple AI models to improve the accuracy of inspecting both qualified and unqualified products.

[0007] In this technique, firstly, the test data is input into multiple past models, and the past models whose intermediate or final outputs have a correlation less than or equal to a certain level are selected. Then, multiple hybrid model candidates are created using the selected models. Finally, the model with the best accuracy is chosen from the hybrid model candidates.

[0008] As described above, the learning in this technology is carried out in the following manner: the inspection data is input into multiple hybrid models, and the label is correctly determined based on the weighted sum of the outputs of each model.

[0009] Patent Document 1: International Publication No. 2022 / 215559 Summary of the Invention

[0010] As mentioned above, existing transfer learning techniques select multiple model pairs with low correlation between intermediate or final outputs when the inspection data (the image dataset being inspected) is input into past models. In this case, the models are independent of each other, but it may be impossible to select the appropriate model for the inspection data.

[0011] The present invention is proposed to solve the problems mentioned above, and its purpose is to provide a learning device that, compared with the prior art, can obtain a well-trained model for the image dataset that is the object of inspection.

[0012] The learning apparatus of the present invention is characterized by comprising: a learning image acquisition unit that acquires an image dataset as an inspection object; an existing feature acquisition unit that acquires feature quantities of an image dataset as an executed task, the feature quantities being obtained based on the outputs of multiple intermediate layers in a trained model corresponding to the image dataset as an executed task; a feature extraction unit that extracts feature quantities of the image dataset as an inspection object based on the trained model corresponding to the image dataset as an executed task and the image dataset as an inspection object acquired by the learning image acquisition unit, the feature quantities of the image dataset as an inspection object being obtained based on the outputs of multiple intermediate layers in the trained model; and a feature comparison unit that performs feature quantity similarity analysis based on the feature quantities of the image dataset as an inspection object extracted by the feature extraction unit and the feature quantities of the image dataset as an executed task acquired by the existing feature acquisition unit. The system comprises: a calculation unit; a model selection unit, which selects a well-trained model corresponding to an image dataset used for the performed task that has a high similarity in feature quantities to the image dataset used for inspection obtained by the learning image acquisition unit, based on the similarity of feature quantities calculated by the feature quantity comparison unit; a model learning unit, which inputs the image dataset used for inspection into the well-trained model based on the image dataset used for inspection obtained by the learning image acquisition unit and the well-trained model selected by the model selection unit, thereby generating a new well-trained model as a distribution of qualified products; a model evaluation unit, which determines whether the accuracy of the new well-trained model generated by the model learning unit is greater than or equal to a threshold; and a model output unit, which outputs information indicating that the accuracy of the new well-trained model is determined to be greater than or equal to the threshold based on the determination result obtained by the model evaluation unit.

[0013] The effects of the invention

[0014] According to the present invention, due to the configuration shown above, a well-trained model can be obtained appropriately for the image dataset that is the object of inspection, compared to the prior art. Attached Figure Description

[0015] Figure 1 This is a block diagram illustrating a structural example of the learning system involved in Implementation 1.

[0016] Figure 2 This is a block diagram illustrating a structural example of the learning device according to Embodiment 1.

[0017] Figure 3 This is a flowchart illustrating an example of the operation of the learning device according to Embodiment 1.

[0018] Figure 4 This is a diagram showing the overall operation overview of the learning device according to Embodiment 1.

[0019] Figure 5 This is a diagram showing an outline of the operation of the learning device according to Embodiment 1.

[0020] Figure 6 This is a diagram showing an outline of the operation of the learning device according to Embodiment 1.

[0021] Figure 7 This is a diagram showing an outline of the operation of the learning device according to Embodiment 1.

[0022] Figure 8 This is a diagram showing an outline of the operation of the learning device according to Embodiment 1.

[0023] Figure 9 This is a block diagram illustrating a structural example of the learning device according to Embodiment 2.

[0024] Figure 10 This is a flowchart illustrating an example of the operation of the learning device involved in Implementation Method 2.

[0025] Figure 11 This is a diagram showing an outline of the operation of the learning device according to Embodiment 2.

[0026] Figure 12 This is a diagram showing an outline of the operation of the learning device according to Embodiment 2.

[0027] Figure 13 This is a diagram showing an outline of the operation of the learning device according to Embodiment 2.

[0028] In Figure 14, Figure 14A , Figure 14B This is a block diagram illustrating an example of the hardware structure of the learning device according to embodiments 1 and 2. Detailed Implementation

[0029] The embodiments will now be described in detail with reference to the accompanying drawings.

[0030] Implementation Method 1

[0031] Figure 1 This is a diagram illustrating a structural example of the learning system 1 according to implementation method 1.

[0032] like Figure 1As shown, the learning system 1 includes a learning device 11, an operation input device 12, a storage device 13, and a display output device 14.

[0033] The learning device 11 outputs information representing a new trained model based on the image dataset as the object of inspection input via the operation input device 12, the feature quantities of the image dataset as the task performed, the information stored in the storage device 13, and the trained model corresponding to the image dataset as the task performed.

[0034] The structure of the learning device 11 will be described later.

[0035] The operation input device 12 is a device that receives user operations. For example, the operation input device 12 outputs an image dataset, which is the object of inspection, input by the user, to the learning device 11.

[0036] The storage device 13 stores various types of data processed by the learning system 1. For example, the storage device 13 stores feature quantities of an image dataset that has been used for a task and information representing a trained model corresponding to that image dataset.

[0037] Here, the storage device 13 is, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), EEPROM (Electrically EPROM), disk, floppy disk, optical disk, high-density disk, mini disk, or DVD (Digital Versatile Disc).

[0038] The display output device 14 displays the information representing the new trained model output by the learning device 11.

[0039] Next, while referring to Figure 2 The structure of the learning device 11 will be explained.

[0040] like Figure 2 As shown, the learning device 11 includes a learning image acquisition unit 1101, an existing feature acquisition unit 1102, a feature extraction unit 1103, a feature comparison unit 1104, a model selection unit 1105, a model learning unit 1106, a model evaluation unit 1107, and a model output unit 1108.

[0041] The learning image acquisition unit 1101 acquires an image dataset that is the object of inspection. At this time, the learning image acquisition unit 1101 acquires the image dataset that is the object of inspection input via the operation input device 12.

[0042] The existing feature acquisition unit 1102 acquires the feature values ​​of the image dataset as the executed task. At this time, the existing feature acquisition unit 1102 acquires the feature values ​​of the image dataset as the executed task, as shown by the information stored in the storage device 13. In addition, the feature values ​​of the image dataset as the executed task acquired by the existing feature acquisition unit 1102 are feature values ​​based on the outputs of multiple intermediate layers in the trained model corresponding to the image dataset as the executed task.

[0043] The feature extraction unit 1103 extracts features from the image dataset to be inspected based on the trained model corresponding to the image dataset for the performed task and the image dataset to be inspected acquired by the learning image acquisition unit 1101. The features extracted by the feature extraction unit 1103 are features based on the outputs of the multiple intermediate layers in the trained model.

[0044] At this time, for example, the feature extraction unit 1103 may also extract the output of multiple intermediate layers when the image dataset that is the object of inspection or the qualified product image dataset in the image dataset that is the object of inspection is input to the trained model corresponding to the image dataset that has been performed, as a feature quantity.

[0045] Alternatively, for example, the feature extraction unit 1103 may extract a vector group after averaging the output for each channel as a feature. This output is the output of multiple intermediate layers when the image dataset that is the object of inspection or the qualified image dataset in the image dataset that is the object of inspection is input to the trained model corresponding to the image dataset that has been used for the performed task.

[0046] Alternatively, for example, the feature extraction unit 1103 may extract the vector after averaging the overall output as a feature quantity. This output is the output of multiple intermediate layers when the image dataset that is the object of inspection or the qualified image dataset in the image dataset that is the object of inspection is input to the trained model corresponding to the image dataset that has been used for the performed task.

[0047] Furthermore, at this time, the feature extraction unit 1103 obtains the trained model corresponding to the image dataset of the performed task, as shown by the information stored in the storage device 13, and extracts the aforementioned features based on this model.

[0048] The feature comparison unit 1104 calculates the similarity of the features based on the features of the image dataset as the object of inspection extracted by the feature extraction unit 1103 and the features of the image dataset as the task already performed obtained by the existing feature acquisition unit 1102.

[0049] Here, in the learning device 11 according to Embodiment 1, the feature comparison unit 1104 calculates the similarity between the feature quantity of the image dataset as the object of inspection extracted by the feature extraction unit 1103 and the feature quantity of the image dataset as the object of the performed task obtained by the existing feature acquisition unit 1102.

[0050] At this time, for example, the feature comparison unit 1104 may also use the distribution difference to calculate the similarity between the feature quantities of the image dataset as the object of inspection and the feature quantities of the image dataset as the object of the performed task.

[0051] Alternatively, for example, the feature comparison unit 1104 may use common regions distributed among each other to calculate the similarity between the feature quantities of the image dataset as the object of inspection and the feature quantities of the image dataset as the object of the performed task.

[0052] The model selection unit 1105 selects, based on the similarity of the feature quantities calculated by the feature quantity comparison unit 1104, a well-trained model that has a high similarity of feature quantities to the image dataset of the image dataset of the performed task relative to the image dataset of the inspection object obtained by the learning image acquisition unit 1101.

[0053] At this time, for example, the model selection unit 1105 selects one or more well-trained models corresponding to the image dataset from the image dataset that has been performed, in descending order of the similarity of the feature quantities of the image dataset that has been performed to the image dataset that has been examined, which has been obtained by the learning image acquisition unit 1101.

[0054] The model learning unit 1106, based on the image dataset of the inspection object acquired by the learning image acquisition unit 1101 and the trained model selected by the model selection unit 1105, inputs the image dataset of the inspection object into the trained model, thereby generating a new trained model. Furthermore, the new trained model generated by the model learning unit 1106 is a qualified product distribution.

[0055] At this time, for example, the model learning unit 1106 inputs the qualified product image dataset from the image dataset that is the object of inspection into the trained model selected by the model selection unit 1105, and combines all the outputs of the intermediate layers to generate a new trained model.

[0056] The model evaluation unit 1107 determines whether the accuracy of the new trained model generated by the model learning unit 1106 is greater than or equal to a threshold. Furthermore, the threshold can be appropriately set as a value used to evaluate the trained model.

[0057] Based on the determination result obtained by the model evaluation unit 1107, the model output unit 1108 outputs information indicating that the new well-trained model is determined to be evaluated as greater than or equal to the threshold to the outside.

[0058] Next, while referring to Figures 3-8 On the one hand Figure 1 and Figure 2 An example of the operation of the learning device 11 according to Embodiment 1 will be described.

[0059] In the learning device 11 according to Embodiment 1, for example, such as Figure 4 As shown, firstly, feature quantities are extracted from the image dataset that serves as the inspection object (new task) (step ST1). In Figure 4 The example illustrates the following situation: the image dataset to be inspected is image dataset X (bottle neck), and the feature quantities of image dataset X are feature quantities X1 to X3.

[0060] Additionally, the learning device 11 acquires the feature values ​​of the image dataset used for the performed task (step ST2). Figure 4 The example illustrates a scenario where the image dataset used for the performed task consists of three datasets: image dataset A (nut), image dataset B (cable profile), and image dataset C (skin surface). Furthermore, each image dataset used for the performed task is associated with feature values ​​and a trained model. Figure 4 In the example, feature A and the trained model A are associated with image dataset A, feature B and the trained model B are associated with image dataset B, and feature C and the trained model C are associated with image dataset C. Furthermore, information representing the features and trained models corresponding to these image datasets as performed tasks is pre-stored in storage device 13.

[0061] Then, the learning device 11 identifies image datasets from the image datasets for which the task has been performed, whose feature quantities are similar to those of the image datasets for which the inspection is being conducted, and selects the trained model corresponding to the identified image datasets (step ST3). Figure 4 The example illustrates the following situation: feature A is similar to feature X1, feature B is similar to feature X2, and the learning device 11 selects a well-trained model A corresponding to image dataset A and a well-trained model B corresponding to image dataset B.

[0062] Then, the learning device 11 generates a new well-trained model (qualified product distribution) based on the selected well-trained model (step ST4). Figure 4 The example illustrates the following situation: the learning device 11 generates a new trained model X based on the selected trained model A and trained model B.

[0063] Additionally, for example, such as Figure 5 As shown, in the learning apparatus 11 according to Embodiment 1, the premise is that multiple intermediate layers in the intermediate layers of the trained model are used to extract features and generate a new trained model (distribution of qualified products). Figure 5 The example shows the case where the above intermediate layers are three layers (layer a, layer b, and layer c).

[0064] In this case, the learning device 11 according to Embodiment 1 uses a past model (a trained model corresponding to the image dataset of the performed task) to extract features. Furthermore, if based on the premise of generating a qualified product distribution, it is preferable to use multiple intermediate layers useful for feature extraction from the past model.

[0065] Therefore, for example, such as Figure 6 and Figure 7 As shown, in the learning device 11 according to Embodiment 1, past models with high similarity to one or more of the total feature values ​​are selected using feature values ​​based on the outputs of the aforementioned multiple intermediate layers. Figure 6 The example illustrates the case where the learning device 11 compares feature quantity X1 of image dataset X with feature quantity A of image dataset A. In this case, the learning device 11 extracts feature quantity X1 by inputting image dataset X into a trained model A corresponding to image dataset A. Furthermore, in Figure 6 The example illustrates a high similarity between feature X1 and feature A. Additionally, in... Figure 7 The example illustrates the case where the learning device 11 compares feature quantity X3 of image dataset X with feature quantity C of image dataset C. In this case, the learning device 11 extracts feature quantity X3 by inputting image dataset X into a trained model C corresponding to image dataset C. Furthermore, in Figure 7 The example illustrates a case where the similarity between feature X3 and feature C is low.

[0066] Then, for example, such as Figure 8 As shown, the learning device 11 inputs the image dataset used as the inspection object into the selected past model, combines the outputs of all the intermediate layers mentioned above, and thus generates a new trained model. Figure 8The example shows the case where the learning device 11 selects trained model A and trained model B to generate a new trained model X.

[0067] exist Figure 1 and Figure 2 In the example of operation of the learning device 11 according to Embodiment 1 shown, for example, as Figure 3 As shown, firstly, the learning image acquisition unit 1101 acquires an image dataset that is the object of inspection (step ST101). At this time, the learning image acquisition unit 1101 acquires the image dataset that is the object of inspection input via the operation input device 12.

[0068] exist Figures 4-8 In the example, the image acquisition unit 1101 is used to acquire the image dataset X.

[0069] Additionally, the existing feature acquisition unit 1102 acquires the feature values ​​of the image dataset as the executed task (step ST102). At this time, the existing feature acquisition unit 1102 acquires the feature values ​​of the image dataset as the executed task, as shown by the information stored in the storage device 13. Furthermore, the feature values ​​of the image dataset as the executed task acquired by the existing feature acquisition unit 1102 are feature values ​​based on the outputs of multiple intermediate layers in the trained model corresponding to the image dataset as the executed task.

[0070] exist Figures 4-8 In the example, the existing feature acquisition unit 1102 acquires feature A of image dataset A, feature B of image dataset B, and feature C of image dataset C. Furthermore, in... Figure 6 and Figure 7 In the example, the features obtained by the existing feature acquisition unit 1102 are features based on the outputs of layers a, b, and c of the past model.

[0071] Next, the feature extraction unit 1103 extracts features of the image dataset being inspected based on the trained model corresponding to the image dataset of the performed task and the image dataset being inspected acquired by the learning image acquisition unit 1101 (step ST103). The features of the image dataset being inspected extracted by the feature extraction unit 1103 are features based on the outputs of the multiple intermediate layers in the trained model.

[0072] At this time, for example, the feature extraction unit 1103 may also extract the output of multiple intermediate layers when the image dataset that is the object of inspection or the qualified product image dataset in the image dataset that is the object of inspection is input to the trained model corresponding to the image dataset that has been performed, as a feature quantity.

[0073] Alternatively, for example, the feature extraction unit 1103 may extract a vector group after averaging the output for each channel as a feature. This output is the output of multiple intermediate layers when the image dataset that is the object of inspection or the qualified image dataset in the image dataset that is the object of inspection is input to the trained model corresponding to the image dataset that has been used for the performed task.

[0074] Alternatively, for example, the feature extraction unit 1103 may extract the vector after averaging the overall output as a feature quantity. This output is the output of multiple intermediate layers when the image dataset that is the object of inspection or the qualified image dataset in the image dataset that is the object of inspection is input to the trained model corresponding to the image dataset that has been used for the performed task.

[0075] Furthermore, at this time, the feature extraction unit 1103 obtains the trained model corresponding to the image dataset of the performed task, as shown by the information stored in the storage device 13, and extracts the aforementioned features based on this model.

[0076] exist Figures 4-8 In the example, the feature extraction unit 1103 extracts features X1 to X3. Feature X1 is the feature extracted by inputting the image dataset X into the trained model A. Feature X2 is the feature extracted by inputting the image dataset X into the trained model B. Feature X3 is the feature extracted by inputting the image dataset X into the trained model C. Furthermore, in... Figure 6 and Figure 7 In the example, the features extracted by the feature extraction unit 1103 are features based on the outputs of layers a, b, and c of the past model.

[0077] Next, the feature comparison unit 1104 calculates the similarity of the features based on the features of the image dataset as the object of inspection extracted by the feature extraction unit 1103 and the features of the image dataset as the task already performed obtained by the existing feature acquisition unit 1102 (step ST104).

[0078] Here, in the learning device 11 according to Embodiment 1, the feature comparison unit 1104 calculates the similarity between the feature quantity of the image dataset as the object of inspection extracted by the feature extraction unit 1103 and the feature quantity of the image dataset as the object of the performed task obtained by the existing feature acquisition unit 1102.

[0079] At this time, for example, the feature comparison unit 1104 may also use the distribution difference to calculate the similarity between the feature quantities of the image dataset as the object of inspection and the feature quantities of the image dataset as the object of the performed task. At this time, the feature comparison unit 1104 may use, for example, the Frechetinception distance, KL divergence, JS divergence, or Mahalanobis distance as the distribution difference.

[0080] Alternatively, for example, the feature comparison unit 1104 may use common regions distributed among each other to calculate the similarity between the feature quantities of the image dataset as the object of inspection and the feature quantities of the image dataset as the object of the performed task. In this case, the feature comparison unit 1104 may, for example, use histogram intersection as the common regions distributed among each other.

[0081] Next, the model selection unit 1105 selects a trained model (step ST105) that has a high similarity to the image dataset of the performed task relative to the image dataset of the inspection object obtained by the learning image acquisition unit 1101, based on the similarity of the feature quantities calculated by the feature quantity comparison unit 1104.

[0082] At this time, for example, the model selection unit 1105 selects one or more well-trained models corresponding to the image dataset from the image dataset that has been performed, in descending order of the similarity of the feature quantities of the image dataset that has been performed to the image dataset that has been examined, which has been obtained by the learning image acquisition unit 1101.

[0083] exist Figures 4-8 In the example, feature quantity X1 (feature quantity of layers a, b and c as a whole) has high similarity with feature quantity A (feature quantity of layers a, b and c as a whole), and feature quantity X2 (feature quantity of layers a, b and c as a whole) has high similarity with feature quantity B (feature quantity of layers a, b and c as a whole). The feature quantity extraction unit 1103 selected trained model A and trained model B.

[0084] On the other hand, Figures 4-8 In the example, the similarity between feature X3 (the feature of layers a, b and c as a whole) and feature C (the feature of layers a, b and c as a whole) is low, so the feature extraction unit 1103 does not select the trained model C.

[0085] Next, the model learning unit 1106, based on the image dataset of the inspection object acquired by the learning image acquisition unit 1101 and the trained model selected by the model selection unit 1105, inputs the image dataset of the inspection object into the trained model, thereby generating a new trained model (step ST106). Furthermore, the new trained model generated by the model learning unit 1106 is a qualified product distribution.

[0086] At this time, for example, the model learning unit 1106 inputs the qualified product image dataset from the image dataset that is the object of inspection into the trained model selected by the model selection unit 1105, and combines all the outputs of the intermediate layers to generate a new trained model.

[0087] exist Figures 4-8 In the example, the model learning unit 1106 generates a new trained model X based on the trained model A and trained model B. At this time, the model learning unit 1106 inputs the image dataset X into the trained model A and trained model B respectively, and combines all the outputs of the intermediate layers (layer a, layer b, and layer c) of the trained model A and the intermediate layers (layer a, layer b, and layer c) of the trained model B to generate a new trained model X (distribution of qualified products).

[0088] Next, the model evaluation unit 1107 determines whether the accuracy of the new trained model generated by the model learning unit 1106 is greater than or equal to a threshold (step ST107). Furthermore, the threshold can be appropriately set as a value used to evaluate the trained model.

[0089] Next, the model output unit 1108 outputs information indicating that the evaluation of the new trained model is greater than or equal to the threshold to the outside based on the determination result obtained by the model evaluation unit 1107 (step ST108).

[0090] Here, in the prior art, among the selection of multiple past models, it may be impossible to select a model that is appropriately trained for the image dataset being examined.

[0091] In contrast, in the learning apparatus 11 of embodiment 1, a new well-trained model for anomaly detection is generated by comparing the feature quantity (based on the intermediate output of the model) of the image dataset that is the object of inspection with the feature quantity (based on the intermediate output of the model) of the image dataset that has performed the task. Considering this, a well-trained model can be selected by using data similar to the image dataset that is the object of inspection, and high accuracy can be expected.

[0092] As described above, according to this embodiment 1, the learning device 11 includes: a learning image acquisition unit 1101, which acquires an image dataset as an inspection object; an existing feature acquisition unit 1102, which acquires feature quantities of an image dataset as an executed task, the feature quantities being obtained based on the outputs of multiple intermediate layers in a trained model corresponding to the image dataset as an executed task; a feature extraction unit 1103, which extracts feature quantities of the image dataset as an inspection object based on the trained model corresponding to the image dataset as an executed task and the image dataset as an inspection object acquired by the learning image acquisition unit 1101, the feature quantities being obtained based on the outputs of multiple intermediate layers in the trained model; a feature comparison unit 1104, which calculates the similarity of feature quantities based on the feature quantities of the image dataset as an inspection object extracted by the feature extraction unit 1103 and the feature quantities of the image dataset as an executed task acquired by the existing feature acquisition unit 1102; and a model selection unit 110. 5. Based on the similarity of feature quantities calculated by the feature quantity comparison unit 1104, it selects a training model that has a high similarity of feature quantities to the image dataset of the image dataset of the performed task relative to the image dataset of the inspection object obtained by the learning image acquisition unit 1101. The model learning unit 1106 inputs the image dataset of the inspection object obtained by the learning image acquisition unit 1101 and the training model selected by the model selection unit 1105 into the training model, thereby generating a new training model as a distribution of qualified products. The model evaluation unit 1107 determines whether the evaluation of the new training model generated by the model learning unit 1106 is greater than or equal to a threshold. The model output unit 1108 outputs information indicating the new training model whose evaluation is determined to be greater than or equal to the threshold based on the determination result obtained by the model evaluation unit 1107. In particular, in the learning device 11 according to Embodiment 1, the feature comparison unit 1104 calculates the similarity between the feature quantity of the image dataset as the object of inspection extracted by the feature extraction unit 1103 and the feature quantity of the image dataset as the object of the performed task obtained by the existing feature acquisition unit 1102, and selects a well-trained model.

[0093] Therefore, compared with the prior art, the learning device 11 according to Embodiment 1 can obtain a well-trained model that is appropriately trained for the image dataset that is the object of inspection.

[0094] Implementation Method 2

[0095] In the learning apparatus 11 according to Embodiment 1, the similarity between the feature values ​​of the image dataset as a whole, which is the object of inspection, and the feature values ​​of the image dataset as a whole, which has performed a task, is calculated, and a well-trained model is selected. In contrast, in the learning apparatus 11 according to Embodiment 2, the similarity between the feature values ​​of each intermediate layer of the image dataset as the object of inspection and the feature values ​​of each intermediate layer of the image dataset as a task has been performed is calculated, and a well-trained model (intermediate layer) is selected.

[0096] Figure 9 This is a diagram illustrating a structural example of the learning device 11 according to Embodiment 2. Figure 9 In the learning device 11 according to Embodiment 2 shown, relative to Figure 2 In the learning device 11 of Embodiment 1 shown, the feature quantity comparison unit 1104 is changed to the feature quantity comparison unit 1104b, and the model learning unit 1106 is changed to the model learning unit 1106b. Figure 9 Other structural examples in the learning device 11 involved in Embodiment 2 shown are similar to Figure 2 The learning device 11 involved in Embodiment 1 shown has the same structure and is labeled with the same reference numerals; only the different parts are described.

[0097] The feature comparison unit 1104b calculates the similarity of the features based on the features of the image dataset as the object of inspection extracted by the feature extraction unit 1103 and the features of the image dataset as the task already performed obtained by the existing feature acquisition unit 1102.

[0098] Here, in the learning device 11 according to Embodiment 2, the feature comparison unit 1104b calculates the similarity between the feature quantities of each intermediate layer of the image dataset that is the object of inspection extracted by the feature extraction unit 1103 and the feature quantities of the entire intermediate layer of the image dataset that has been executed, which is the image dataset that has been executed, obtained by the existing feature acquisition unit 1102.

[0099] At this time, for example, the feature comparison unit 1104b may also use the distribution difference to calculate the similarity between the feature quantities of each intermediate layer of the image dataset that is the object of inspection and the feature quantities of each intermediate layer of the image dataset that has been performed.

[0100] Alternatively, for example, the feature comparison unit 1104b may use common regions distributed among each other to calculate the similarity between the feature quantities of each intermediate layer of the image dataset that is the object of inspection and the feature quantities of each intermediate layer of the image dataset that has performed the task.

[0101] The model learning unit 1106b, based on the image dataset of the inspection object acquired by the learning image acquisition unit 1101 and the trained model selected by the model selection unit 1105, inputs the image dataset of the inspection object into the trained model, thereby generating a new trained model. Furthermore, the new trained model generated by the model learning unit 1106b is a qualified product distribution.

[0102] At this time, for example, the model learning unit 1106b inputs the qualified product image dataset from the image dataset that is the object of inspection into the trained model selected by the model selection unit 1105, and selectively combines tensors from the outputs of the intermediate layers to generate a new trained model.

[0103] Next, while referring to Figures 10-13 On the one hand Figure 1 and Figure 9 An example of the operation of the learning device 11 according to Embodiment 2 will be described.

[0104] For example, such as Figure 11 and Figure 12 As shown, in the learning device 11 according to Embodiment 2, past models with high similarity to feature quantities greater than or equal to one of the feature quantities of each of the aforementioned intermediate layers are selected using feature quantities based on the outputs of the aforementioned multiple intermediate layers. Figure 11 The example illustrates the case where the learning device 11 compares feature quantity X1 of image dataset X with feature quantity A of image dataset A. In this case, the learning device 11 extracts feature quantity X1 by inputting image dataset X into a trained model A corresponding to image dataset A. Furthermore, in Figure 11 The example illustrates the following situation: regarding layers a and b, the similarity between feature X1 and feature A is high. Additionally, in... Figure 12 The example illustrates the case where the learning device 11 compares feature quantity X2 of image dataset X with feature quantity B of image dataset B. In this case, the learning device 11 extracts feature quantity X2 by inputting image dataset X into a trained model B corresponding to image dataset B. Furthermore, in Figure 12 The example illustrates the following situation: with respect to layer c, the similarity between feature X2 and feature B is high.

[0105] Then, for example, such as Figure 13 As shown, the learning device 11 inputs the image dataset, which is the object of inspection, into the selected past model, and selectively combines tensors from the outputs of the aforementioned intermediate layers, thereby generating a new well-trained model. Figure 13The example illustrates the following situation: the learning device 11 selects trained model A (layers a and b) and trained model B (layer c) to generate a new trained model X.

[0106] exist Figure 1 and Figure 9 In the example of operation of the learning device 11 according to Embodiment 2 shown, for example, as Figure 10 As shown, firstly, the learning image acquisition unit 1101 acquires an image dataset that is the object of inspection (step ST201). At this time, the learning image acquisition unit 1101 acquires the image dataset that is the object of inspection input via the operation input device 12.

[0107] exist Figure 4 , Figure 5 , Figures 11-13 In the example, the image acquisition unit 1101 is used to acquire the image dataset X.

[0108] Additionally, the existing feature acquisition unit 1102 acquires the feature values ​​of the image dataset as the executed task (step ST202). At this time, the existing feature acquisition unit 1102 acquires the feature values ​​of the image dataset as the executed task, as shown by the information stored in the storage device 13. Furthermore, the feature values ​​of the image dataset as the executed task acquired by the existing feature acquisition unit 1102 are feature values ​​based on the outputs of multiple intermediate layers in the trained model corresponding to the image dataset as the executed task.

[0109] exist Figure 4 , Figure 5 , Figures 11-13 In the example, the existing feature acquisition unit 1102 acquires feature A of image dataset A, feature B of image dataset B, and feature C of image dataset C. Furthermore, in... Figure 11 and Figure 12 In the example, the features obtained by the existing feature acquisition unit 1102 are features based on the outputs of layers a, b, and c of the past model.

[0110] Next, the feature extraction unit 1103 extracts features of the image dataset being inspected based on the trained model corresponding to the image dataset of the performed task and the image dataset being inspected acquired by the learning image acquisition unit 1101 (step ST203). The features of the image dataset being inspected extracted by the feature extraction unit 1103 are features based on the outputs of the multiple intermediate layers in the trained model.

[0111] At this time, for example, the feature extraction unit 1103 may also extract the output of multiple intermediate layers when the image dataset that is the object of inspection or the qualified product image dataset in the image dataset that is the object of inspection is input to the trained model corresponding to the image dataset that has been performed, as a feature quantity.

[0112] Alternatively, for example, the feature extraction unit 1103 may extract a vector group after averaging the output for each channel as a feature. This output is the output of multiple intermediate layers when the image dataset that is the object of inspection or the qualified image dataset in the image dataset that is the object of inspection is input to the trained model corresponding to the image dataset that has been used for the performed task.

[0113] Alternatively, for example, the feature extraction unit 1103 may extract the vector after averaging the overall output as a feature quantity. This output is the output of multiple intermediate layers when the image dataset that is the object of inspection or the qualified image dataset in the image dataset that is the object of inspection is input to the trained model corresponding to the image dataset that has been used for the performed task.

[0114] Furthermore, at this time, the feature extraction unit 1103 obtains the trained model corresponding to the image dataset of the performed task, as shown by the information stored in the storage device 13, and extracts the aforementioned features based on this model.

[0115] exist Figure 4 , Figure 5 , Figures 11-13 In the example, the feature extraction unit 1103 extracts features X1 to X3. Feature X1 is the feature extracted by inputting the image dataset X into the trained model A. Feature X2 is the feature extracted by inputting the image dataset X into the trained model B. Feature X3 is the feature extracted by inputting the image dataset X into the trained model C. Furthermore, in... Figure 11 and Figure 12 In the example, the features extracted by the feature extraction unit 1103 are features based on the outputs of layers a, b, and c of the past model.

[0116] Next, the feature comparison unit 1104b calculates the similarity of the features based on the features of the image dataset as the object of inspection extracted by the feature extraction unit 1103 and the features of the image dataset as the task already performed obtained by the existing feature acquisition unit 1102 (step ST204).

[0117] Here, in the learning device 11 according to Embodiment 2, the feature comparison unit 1104b calculates the similarity between the feature values ​​of each intermediate layer of the image dataset as the inspection object extracted by the feature extraction unit 1103 and the feature values ​​of each intermediate layer of the image dataset as the executed task obtained by the existing feature acquisition unit 1102.

[0118] At this time, for example, the feature comparison unit 1104b may also use distribution difference to calculate the similarity between the feature quantities of each intermediate layer of the image dataset that is the object of inspection and the feature quantities of each intermediate layer of the image dataset that has performed the task. At this time, as the distribution difference, the feature comparison unit 1104b may, for example, use Frechet inception distance, KL divergence, JS divergence or Mahalanobis distance.

[0119] Alternatively, for example, the feature comparison unit 1104b may use common regions distributed among them to calculate the similarity between the feature quantities of each intermediate layer of the image dataset being inspected and the feature quantities of each intermediate layer of the image dataset having performed the task. In this case, the feature comparison unit 1104b may, for example, use histogram intersection as the common regions distributed among them.

[0120] Next, the model selection unit 1105 selects a trained model (step ST205) that has a high similarity to the image dataset of the performed task relative to the image dataset of the inspection object obtained by the learning image acquisition unit 1101, based on the similarity of the feature quantities calculated by the feature quantity comparison unit 1104b.

[0121] At this time, for example, the model selection unit 1105 selects one or more well-trained models corresponding to the image dataset from the image dataset that has been performed, in descending order of the similarity of the feature quantities of the image dataset that has been performed to the image dataset that has been examined, which has been obtained by the learning image acquisition unit 1101.

[0122] exist Figure 4 , Figure 5 , Figures 11-13 In the example, the similarity between feature quantity X1 and feature quantity A for layers a and b, and between feature quantity X2 and feature quantity B for layer c, is high. The feature extraction unit 1103 selects the trained model A and the trained model B.

[0123] On the other hand, Figure 4 , Figure 5 , Figures 11-13 In the example, the similarity between feature quantity X3 and feature quantity C of layers a, b and c is low, so the feature extraction unit 1103 does not select the well-trained model C.

[0124] Next, the model learning unit 1106b, based on the image dataset of the inspection object obtained by the learning image acquisition unit 1101 and the trained model selected by the model selection unit 1105, inputs the image dataset of the inspection object into the trained model, thereby generating a new trained model (step ST206). Furthermore, the new trained model generated by the model learning unit 1106b is a qualified product distribution.

[0125] At this time, for example, the model learning unit 1106b inputs the qualified product image dataset from the image dataset that is the object of inspection into the trained model selected by the model selection unit 1105, and selectively combines tensors from the outputs of the intermediate layers to generate a new trained model.

[0126] exist Figure 4 , Figure 5 , Figures 11-13 In the example, the model learning unit 1106b generates a new trained model X based on the trained model A and the trained model B. At this time, the model learning unit 1106 inputs the image dataset X into the trained model A and the trained model B respectively, and combines the outputs of the intermediate layers (layer a and layer b) with high similarity of the trained model A with the outputs of the intermediate layers (layer c) with high similarity of the trained model B, thereby generating a new trained model X (qualified product distribution).

[0127] Next, the model evaluation unit 1107 determines whether the accuracy of the new trained model generated by the model learning unit 1106b is greater than or equal to a threshold (step ST207). Furthermore, the threshold can be appropriately set as a value used to evaluate the trained model.

[0128] Next, the model output unit 1108 outputs information indicating that the evaluation of the new trained model is greater than or equal to the threshold to the outside based on the determination result obtained by the model evaluation unit 1107 (step ST208).

[0129] Here, in the prior art, among the selection of multiple past models, it may be impossible to select a model that is appropriately trained for the image dataset being examined.

[0130] In contrast, in the learning apparatus 11 of embodiment 2, a new well-trained model for anomaly detection is generated by comparing the feature quantity of the image dataset that is the object of inspection (the feature quantity based on the intermediate output of the model) with the feature quantity of the image dataset that has performed the task (the feature quantity based on the intermediate output of the model). Considering this, a well-trained model (intermediate layer) can be selected using data similar to the image dataset that is the object of inspection, and high accuracy can be expected.

[0131] As described above, in the learning apparatus 11 according to Embodiment 2, the feature comparison unit 1104 calculates the similarity between the feature quantities of each intermediate layer of the image dataset being examined, extracted by the feature extraction unit 1103, and the feature quantities of each intermediate layer of the image dataset being processed, obtained by the existing feature acquisition unit 1102, and selects a well-trained model (intermediate layer). Therefore, compared to the prior art, the learning apparatus 11 according to Embodiment 2 can obtain a well-trained model appropriately suited for new data.

[0132] Finally, referring to FIG14, examples of the hardware structure of the learning device 11 according to Embodiments 1 and 2 will be described. Here, the hardware structure example of the learning device 11 according to Embodiment 1 will be described, but the hardware structure example of the learning device 11 according to Embodiment 2 is also the same.

[0133] The functions of the learning image acquisition unit 1101, the existing feature acquisition unit 1102, the feature extraction unit 1103, the feature comparison unit 1104, the model selection unit 1105, the model learning unit 1106, the model evaluation unit 1107, and the model output unit 1108 in the learning device 11 are implemented by the processing circuit 51. The processing circuit 51 can be as follows: Figure 14A The image shows dedicated hardware, which can also be used as follows: Figure 14B The image shows a CPU (Central Processing Unit, also known as a central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor)) 52 executing a program stored in memory 53.

[0134] When the processing circuit 51 is dedicated hardware, it may be a single circuit, a composite circuit, a programmed processor, a parallelized processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The processing circuit 51 can individually implement the functions of the learning image acquisition unit 1101, the existing feature acquisition unit 1102, the feature extraction unit 1103, the feature comparison unit 1104, the model selection unit 1105, the model learning unit 1106, the model evaluation unit 1107, and the model output unit 1108, or it can integrate the functions of each unit and implement them all through the processing circuit 51.

[0135] When the processing circuit 51 is a CPU 52, the functions of the learning image acquisition unit 1101, the existing feature acquisition unit 1102, the feature extraction unit 1103, the feature comparison unit 1104, the model selection unit 1105, the model learning unit 1106, the model evaluation unit 1107, and the model output unit 1108 are implemented by software, firmware, or a combination of software and firmware. The software and firmware are described as programs and stored in the memory 53. The processing circuit 51 implements the functions of each unit by reading and executing the programs stored in the memory 53. That is, the learning device 11 has a memory 53 for storing programs, which, when executed by the processing circuit 51, are evaluated, for example, from the perspective of the results. Figure 3 The steps shown are as follows. Alternatively, these procedures can also be described as a process and method that enables a computer to execute the learning image acquisition unit 1101, the existing feature acquisition unit 1102, the feature extraction unit 1103, the feature comparison unit 1104, the model selection unit 1105, the model learning unit 1106, the model evaluation unit 1107, and the model output unit 1108. Here, the memory 53 can be, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), EEPROM (Electrically EPROM), a disk, floppy disk, optical disk, high-density disk, mini-disk, or DVD (Digital Versatile Disc).

[0136] Furthermore, the functions of the learning image acquisition unit 1101, the existing feature acquisition unit 1102, the feature extraction unit 1103, the feature comparison unit 1104, the model selection unit 1105, the model learning unit 1106, the model evaluation unit 1107, and the model output unit 1108 can be partially implemented by dedicated hardware and partially by software or firmware. For example, the learning image acquisition unit 1101 can be implemented by a processing circuit 51 as dedicated hardware, and the existing feature acquisition unit 1102, the feature extraction unit 1103, the feature comparison unit 1104, the model selection unit 1105, the model learning unit 1106, the model evaluation unit 1107, and the model output unit 1108 can be implemented by the processing circuit 51 reading and executing the program stored in the memory 53.

[0137] As shown above, the processing circuit 51 can implement the above functions through hardware, software, firmware, or a combination thereof.

[0138] Furthermore, it is possible to freely combine the various embodiments, or modify any structural elements of the various embodiments, or omit any structural elements in each embodiment.

[0139] Industrial applicability

[0140] Compared with the prior art, the learning device of the present invention can obtain a well-trained model that is appropriately trained for the image dataset that is the object of inspection, and is suitable for a learning device to obtain a well-trained model.

[0141] Explanation of the label

[0142] 1 Learning system, 11 Learning device, 12 Operation input device, 13 Storage device, 14 Display output device, 51 Processing circuit, 52 CPU, 53 Memory, 1101 Learning image acquisition unit, 1102 Existing feature acquisition unit, 1103 Feature extraction unit, 1104, 1104b Feature comparison unit, 1105 Model selection unit, 1106, 1106b Model learning unit, 1107 Model evaluation unit, 1108 Model output unit.

Claims

1. A learning device, comprising: The learning process utilizes an image acquisition unit to acquire image datasets as objects of inspection. The existing feature acquisition unit acquires the feature quantities of the image dataset that has been used for the performed task. These feature quantities are obtained based on the outputs of multiple intermediate layers in the trained model corresponding to the image dataset that has been used for the performed task. The feature extraction unit extracts features of the image dataset being inspected based on the trained model corresponding to the image dataset for the performed task and the image dataset being inspected obtained by the learning image acquisition unit. The features of the image dataset being inspected are obtained based on the outputs of the plurality of intermediate layers in the trained model. The feature comparison unit calculates the similarity of feature quantities based on the feature quantities of the image dataset as the inspection object extracted by the feature extraction unit and the feature quantities of the image dataset as the executed task obtained by the existing feature acquisition unit. The model selection unit selects, based on the similarity of feature quantities calculated by the feature quantity comparison unit, a trained model that has a high similarity of feature quantities to the image dataset of the image dataset of the performed task relative to the image dataset of the inspection object obtained by the learning image acquisition unit. The model learning unit, based on the image dataset of the inspection object obtained by the learning image acquisition unit and the trained model selected by the model selection unit, inputs the image dataset of the inspection object into the trained model, thereby generating a new trained model as the distribution of qualified products. The model evaluation unit determines whether the accuracy of the new well-trained model generated by the model learning unit is greater than or equal to a threshold. as well as The model output unit outputs information indicating that the new well-trained model has an accuracy greater than or equal to a threshold, based on the judgment result obtained by the model evaluation unit.

2. The learning device according to claim 1, characterized in that, The feature extraction unit extracts the output of multiple intermediate layers of the trained model corresponding to the image dataset that is the object of inspection, obtained by the learning image acquisition unit, or the qualified image dataset from the image dataset that is the object of inspection, as features.

3. The learning device according to claim 1, characterized in that, The feature extraction unit extracts a vector group after averaging the output for each channel as a feature. This output is the output of multiple intermediate layers when the image dataset of the inspection object obtained by the learning image acquisition unit or the qualified product image dataset in the image dataset of the inspection object is input to the trained model corresponding to the image dataset of the performed task.

4. The learning device according to claim 1, characterized in that, The feature extraction unit extracts an overall averaged vector as a feature quantity. This output is the output of multiple intermediate layers when the image dataset of the inspection object obtained by the learning image acquisition unit or the qualified image dataset of the image dataset of the inspection object is input to the trained model corresponding to the image dataset of the performed task.

5. The learning device according to claim 2 or 3, characterized in that, The feature comparison unit uses distribution difference to calculate the similarity between the feature quantities of the image dataset as the inspection object extracted by the feature extraction unit and the feature quantities of the image dataset as the completed task obtained by the existing feature acquisition unit.

6. The learning device according to claim 4, characterized in that, The feature comparison unit uses common regions distributed among themselves to calculate the similarity between the feature quantities of the image dataset as the object of inspection extracted by the feature extraction unit and the feature quantities of the image dataset as the object of the performed task obtained by the existing feature acquisition unit.

7. The learning device according to claim 2 or 3, characterized in that, The feature comparison unit uses distribution difference to calculate the similarity between the feature quantities of each intermediate layer of the image dataset that is the object of inspection extracted by the feature extraction unit and the feature quantities of each intermediate layer of the image dataset that has been tasked and obtained by the existing feature acquisition unit.

8. The learning device according to claim 4, characterized in that, The feature comparison unit uses common regions distributed among themselves to calculate the similarity between the feature quantities of each intermediate layer of the image dataset being inspected, extracted by the feature extraction unit, and the feature quantities of each intermediate layer of the image dataset being performed, obtained by the existing feature acquisition unit.

9. The learning device according to claim 1, characterized in that, The model learning unit inputs the qualified product image dataset from the image dataset of the inspection object obtained by the learning image acquisition unit into the trained model selected by the model selection unit, and combines all the outputs of each intermediate layer to generate a new trained model as the distribution of qualified products.

10. The learning device according to claim 1, characterized in that, The model learning unit inputs the qualified product image dataset from the image dataset of the inspection object obtained by the learning image acquisition unit to the trained model selected by the model selection unit, and selectively combines tensors from the outputs of each intermediate layer, thereby generating a new trained model as the qualified product distribution.

11. A learning method comprising the following steps: Learn to use the image acquisition unit to obtain image datasets as inspection objects; The existing feature acquisition unit acquires the feature quantities of the image dataset that has been used for the performed task. These feature quantities are obtained based on the outputs of multiple intermediate layers in the trained model corresponding to the image dataset that has been used for the performed task. The feature extraction unit extracts features of the image dataset being inspected based on the trained model corresponding to the image dataset for the performed task and the image dataset being inspected obtained by the learning image acquisition unit. The features of the image dataset being inspected are obtained based on the outputs of the plurality of intermediate layers in the trained model. The feature comparison unit calculates the similarity of the feature quantities based on the feature quantities of the image dataset as the inspection object extracted by the feature extraction unit and the feature quantities of the image dataset as the executed task obtained by the existing feature acquisition unit. The model selection unit selects, based on the similarity of the feature quantities calculated by the feature quantity comparison unit, a trained model that has a high similarity of feature quantities to the image dataset of the image dataset of the performed task relative to the image dataset of the inspection object obtained by the learning image acquisition unit. The model learning unit, based on the image dataset of the inspection object obtained by the learning image acquisition unit and the trained model selected by the model selection unit, inputs the image dataset of the inspection object into the trained model, thereby generating a new trained model as the distribution of qualified products. The model evaluation unit determines whether the evaluation of the new well-trained model generated by the model learning unit is greater than or equal to a threshold. as well as Based on the judgment result obtained by the model evaluation unit, the model output unit outputs information indicating that the new well-trained model is judged to have an evaluation greater than or equal to the threshold.

Images