Learning device and learning method

Abstract

This invention provides a learning device and learning method that can easily adjust the size of an AI model to an appropriate size. [Solution] A learning device and a learning method comprising: a learning database unit that stores learning data including multiple images showing a normal state; a feature extraction unit that extracts features from the learning data and obtains the size of each extracted feature; a model size adjustment unit that determines the number of features to be held by the anomaly detection model based on a desired size of the anomaly detection model that detects anomalies using images and the size of each feature obtained by the feature extraction unit; a feature sampling unit that selects the number of features determined by the model size adjustment unit from the features extracted by the feature extraction unit; and an anomaly detection model storage unit that stores an anomaly detection model that holds the features selected by the feature sampling unit.

Description

Technical Field

[0001] The present invention relates to a learning device and a learning method.

Background Art

[0002] Generally, for example, in product image inspection, anomaly detection using an AI (Artificial Intelligence) model may be performed. As an AI model used for such anomaly detection, Non-Patent Document 1 discloses an algorithm called Patch Core, a teacherless anomaly detection method that generates an AI model by machine-learning a plurality of images showing normal states with AI.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the above-mentioned Patch Core uses a learned CNN (Convolutional Neural Network) or the like to extract feature amounts from learning data including a plurality of images showing normal states, and allows an AI model to learn a part of the extracted feature amounts. Therefore, the size of the output AI model changes depending on the number of learned images.

[0005] Furthermore, as the size of the AI ​​model increases, the accuracy of anomaly detection improves, but the processing time increases. Conversely, as the size of the AI ​​model decreases, the accuracy of anomaly detection decreases, but the processing time decreases. Therefore, in order to obtain an AI model of an appropriate size, it is necessary to determine the degree of training while considering processing time, accuracy, and the number of training images, which may increase the time and effort required to generate the AI ​​model.

[0006] The problem that this invention aims to solve is to provide a learning device and a learning method that can easily adjust the size of an AI model to an appropriate size. [Means for solving the problem]

[0007] The learning device according to the present invention comprises: a learning database unit that stores learning data including a plurality of images showing a normal state; a feature extraction unit that extracts features from the learning data and obtains the size of each extracted feature; a model size adjustment unit that determines the number of features to be held by the anomaly detection model based on a desired size of the anomaly detection model that detects anomalies using images and the size of each feature obtained by the feature extraction unit; a feature sampling unit that selects a number of features determined by the model size adjustment unit from the features extracted by the feature extraction unit; and an anomaly detection model storage unit that stores an anomaly detection model that holds the features selected by the feature sampling unit.

[0008] The learning method according to the present invention comprises the steps of: extracting features from training data including multiple images showing a normal state; obtaining the size of each extracted feature; determining the number of features to be held by the anomaly detection model based on the desired size of the anomaly detection model that detects anomalies using images and the size of each obtained feature; selecting the determined number of features from the features extracted from the training data; and storing the anomaly detection model that holds the selected features. [Effects of the Invention]

[0009] According to the present invention, the size of the AI ​​model can be easily adjusted to an appropriate size. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic block diagram of a learning device according to the first embodiment. [Figure 2] This figure shows an example of an image representing a normal state according to the first embodiment, where (a) is a figure showing the original image of the sample, and (b) is a figure showing how the image is divided into grid-like patches. [Figure 3] This is a flowchart illustrating the learning method according to the first embodiment. [Figure 4] This is a flowchart for determining an abnormality according to the first embodiment. [Figure 5] This is a schematic block diagram of a learning device according to the second embodiment. [Figure 6] This is an explanatory diagram showing the case where a data division unit according to the second embodiment is added. [Figure 7] This is a flowchart illustrating the learning method according to the second embodiment. [Modes for carrying out the invention]

[0011] The embodiments relating to this disclosure will be described below with reference to the attached drawings. The embodiments described below are illustrative and should not be interpreted as limiting.

[0012] (First Embodiment) In this embodiment, a learning device 1 is described that includes generating an AI model (hereinafter referred to as the "anomaly detection model") that detects anomalies from training data including multiple images showing a normal state.

[0013] The anomaly detection model stores the feature amounts of learning data including a plurality of images indicating normal states, and its size changes according to the number of the stored feature amounts. The anomaly detection model is used to perform an anomaly determination by comparing the feature amounts stored in the anomaly detection model with the feature amounts of the image to be subjected to the anomaly determination.

[0014] FIG. 1 shows a schematic block diagram of a learning device 1 according to the first embodiment. The learning device 1 includes a learning unit 2 that generates an anomaly detection model, and an inference unit 3 that compares an image captured by an imaging device with the anomaly detection model to perform an anomaly determination and displays the result.

[0015] Each functional block is realized by a processor, a memory, a storage, etc. (not shown) included in the learning device 1 shown in FIG. 1 executing respective system programs and controlling the operations of each part of the learning device 1.

[0016] The processor has, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), or a DSP (Digital Signal Processor), etc., and comprehensively controls the entire learning device 1 and executes various arithmetic processes.

[0017] The memory has, for example, a RAM (Random Access Memory) or a ROM (Read Only Memory), etc., and stores information used for the arithmetic processes executed by the processor. The storage has, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive), etc., and stores various data.

[0018] The learning unit 2 includes a learning database unit 4 that stores learning data including a plurality of images indicating a normal state, and a learning processing unit 5 that performs learning of an anomaly detection model using the learning data of the learning database unit 4 and outputs the result to the inference unit 3.

[0019] The learning database unit 4 may store, for example, images in a normal state among the images captured by the camera unit 21 described later, or may store images in a normal state acquired in advance.

[0020] The learning processing unit 5 performs learning of an anomaly detection model using the learning data stored in the learning database unit 4. Specifically, the learning processing unit 5 includes a feature extraction unit 6, a model size adjustment unit 7, a feature sampling unit 8, and an anomaly detection model storage unit 9.

[0021] The feature extraction unit 6 extracts the feature amount of the learning data from the learning data stored in the learning database unit 4 using a trained CNN or the like, and obtains the size per one extracted feature amount. For example, the feature extraction unit 6 divides the image of the learning data shown in FIG. 2(a) into lattice-like image patches as shown in FIG. 2(b), extracts the feature amount from each of the image patches using a trained CNN or the like, and obtains the size per one feature amount extracted for each image patch.

[0022] Note that the feature extraction unit 6 may extract the feature amount using, for example, VGG (Visual Geometry Group) or ResNet (Residual Network) instead of the trained CNN. Also, the size per one feature amount obtained by the feature extraction unit 6 from the learning data may be the size per one feature vector, which is an aggregate of a plurality of feature amounts arranged side by side.

[0023] The model size adjustment unit 7 determines the number of feature amounts held by the anomaly detection model based on the desired size of the anomaly detection model and the size per one feature amount obtained by the feature extraction unit 6 in order to generate an anomaly detection model of a desired size.

[0024] Specifically, the desired size of the anomaly detection model is determined by, for example, the processing time and accuracy required for anomaly detection. Therefore, the model size adjustment unit 7 calculates the number of features that should be stored in the anomaly detection model of the desired size. In other words, the model size adjustment unit 7 determines the number of features to be held by the anomaly detection model by dividing the desired size of the anomaly detection model by the size of each feature obtained from the training data by the feature extraction unit 6.

[0025] For example, the desired size unit for the anomaly detection model is MB (megabyte). Similarly, for example, the size unit for each feature obtained by the feature extraction unit 6 is KB (kilobyte).

[0026] The feature sampling unit 8 selects a number of features determined by the model size adjustment unit 7 from all the features extracted by the feature extraction unit 6, using an approximation algorithm such as a greedy method. The feature sampling unit 8 may select features using methods other than the greedy method, such as relaxation methods or local search methods.

[0027] The anomaly detection model storage unit 9 stores the features selected by the feature sampling unit 8 as an anomaly detection model.

[0028] The inference unit 3 includes a camera unit 21 that captures images using an imaging device, an image capture database unit 22 that stores the captured images, an anomaly inference unit 23 that applies an anomaly detection model to images input from the image capture database unit 22 to perform anomaly determination inference, and a display unit 24 that displays the inference results.

[0029] The camera unit 21 captures images of the product or other object to be inspected using an imaging device. Examples of imaging devices include area sensor cameras, line sensor cameras, and infrared cameras. The imaging database unit 22 is a database that stores images captured by the imaging device.

[0030] The anomaly inference unit 23 uses the anomaly detection model output from the learning processing unit 5 to perform anomaly detection on images that are subject to anomaly detection input from the image database unit 22. Specifically, the anomaly inference unit 23 applies an anomaly detection model to the image to be judged as an anomaly to obtain an anomaly score value, and determines that there is an anomaly at the anomaly location if the anomaly score value is above a predetermined threshold.

[0031] The display unit 24 is a UI (User Interface) that, when an anomaly is detected by the anomaly inference unit 23, displays the image in which the anomaly was detected, clearly indicating the location of the anomaly, and when no anomaly is detected, displays that no anomaly was detected.

[0032] Next, the learning method according to the first embodiment will be explained with reference to Figure 3. Figure 3 mainly shows the learning method by the learning unit 2. First, the training data is output from the training database unit 4 to the feature extraction unit 6. The output training data is read by the feature extraction unit 6 (S101), features are extracted by the trained CNN, etc., and the size of each extracted feature is obtained (S102).

[0033] Next, the model size adjustment unit 7 determines the number of features to be held by the anomaly detection model based on the desired size of the anomaly detection model and the size of each feature obtained from the training data by the feature extraction unit 6, in order to generate an anomaly detection model of the desired size.

[0034] Specifically, the desired size of the anomaly detection model is determined according to the required processing time and accuracy of anomaly detection, for example. Therefore, the model size adjustment unit 7 calculates the number of features that should be stored in the anomaly detection model of the desired size. In other words, the model size adjustment unit 7 determines the number of features to be held by the anomaly detection model by dividing the desired size of the anomaly detection model by the size of each feature obtained from the training data by the feature extraction unit 6 (S103).

[0035] Then, the feature sampling unit 8 selects a number of features determined by the model size adjustment unit 7 from all the features extracted by the feature extraction unit 6, using an approximation algorithm such as a greedy method (S104). The features selected by the feature sampling unit 8 are stored in the anomaly detection model storage unit 9 as an anomaly detection model (S105), and are output to the inference unit 3 when an anomaly determination is performed (S106).

[0036] In this way, the size of each feature is obtained from the training data, the number of features to be stored in the anomaly detection model is determined based on the size, and the determined number of features are stored as the anomaly detection model. Therefore, the size of the stored anomaly detection model can be easily adjusted to the desired size.

[0037] Next, the flow for determining an anomaly according to the first embodiment will be explained with reference to Figure 4. Figure 4 shows the flow for determining an anomaly performed by the inference unit 3. First, the images to be detected as abnormal are captured by the camera unit 21 (S201) and stored in the image database unit 22 (S202).

[0038] Next, an anomaly detection model is applied to the image to be subjected to anomaly detection to obtain an anomaly score value, and anomaly detection inference is performed (S203). When determining an anomaly score, features may be extracted from the image subject to anomaly detection, and these extracted features may be compared with features of an anomaly detection model that are similar to these features. The distance between these features may then be used to determine the anomaly score.

[0039] Once an abnormality score value is obtained, the abnormality score value is compared with a predetermined threshold, and an abnormality determination is made (S204). Then, if the abnormality score value exceeds a predetermined threshold, the display unit 24 indicates that an abnormality has been detected, clearly shows the location of the abnormality, and displays the result indicating the abnormality (S205). At this time, the display unit 24 may upsample the abnormality score value map, which shows the abnormality score value for each image patch of the image subject to abnormality judgment, and create a visualization image by combining it with the image subject to abnormality judgment, and then display the visualization image.

[0040] On the other hand, if the abnormality score value is below a predetermined threshold, the display unit 24 displays a result indicating that no abnormalities were detected in the image subject to abnormality judgment. In this way, abnormality detection is performed on the image that is subject to abnormality detection. If image acquisition is to be terminated, the processing by the inference unit 3 ends, and if image acquisition is to be continued, the process returns to S201 (S206).

[0041] As described above, the learning device according to this embodiment determines the number of features to be held by the anomaly detection model based on the desired size of the anomaly detection model and the size of each feature, selects the determined number of features from all the features extracted from the training data, and stores the selected features as the anomaly detection model. Therefore, even if the required size of the anomaly detection model changes depending on the processing time and accuracy requirements for anomaly detection, the size of the anomaly detection model can be easily adjusted to an appropriate size.

[0042] (Second embodiment) Figure 5 shows a schematic block diagram of the learning device 1 according to the second embodiment. The learning device 1 according to the second embodiment is the same as the learning device 1 according to the first embodiment (Figure 1), except that a data division unit 10, which will be described later, is added. Therefore, the description of parts that are the same as in the first embodiment will be omitted.

[0043] The data splitting unit 10 divides the training data into multiple split training datasets, each containing an image. Specifically, the data splitting unit 10 divides multiple images contained in the training data into N parts (where N is an integer greater than or equal to 2), generating N split training datasets.

[0044] Figure 6 is a diagram illustrating the flow when the data partitioning unit 10 in Figure 5 is added.

[0045] The training data is output from the training database unit 4 to the data splitting unit 10. The training data output to the data splitting unit 10 is divided into an appropriate number of N parts by the data splitting unit 10 and trained in the following steps. The following describes the case in which divided training datasets 1 to N are generated by the division of the training data into N parts, and training in steps 1 to N is performed.

[0046] First, the divided learning datasets 1 to N are output to the feature extraction unit 6. The feature extraction unit 6 sequentially extracts features from each of the divided learning datasets 1 to N. Then, in step 1, the feature extraction unit 6 extracts features from the divided learning dataset 1 using a pre-trained CNN or the like, and obtains the size of each extracted feature.

[0047] Next, the model size adjustment unit 7 determines the number of features to be held by the anomaly detection model based on the desired size of the anomaly detection model and the size of each feature obtained from the divided learning dataset 1 by the feature extraction unit 6.

[0048] Specifically, the desired size of the anomaly detection model is determined by, for example, the processing time and accuracy required for anomaly detection. Therefore, the model size adjustment unit 7 calculates the number of features that should be stored in the anomaly detection model of the desired size. In other words, the model size adjustment unit 7 determines the number of features to be held by the anomaly detection model by dividing the desired size of the anomaly detection model by the size of each feature obtained from the divided learning dataset 1 by the feature extraction unit 6.

[0049] Then, the feature sampling unit 8 selects a number of features determined by the model size adjustment unit 7 from all the features extracted from the divided learning dataset 1 by the feature extraction unit 6, using an approximation algorithm such as a greedy method. The features selected by the feature sampling unit 8 in step 1 are stored as features of the divided learning dataset 1.

[0050] In Step 2, the same processing as in Step 1 is basically performed on the divided training dataset 2. However, in step 2, when the number of features determined by the model size adjustment unit 7 is selected, the feature sampling unit 8 selects the determined number of features from all the features obtained by adding the already stored features of the divided learning dataset 1 to the features extracted from the divided learning dataset 2.

[0051] Therefore, the feature sampling unit 8 selects a number of features determined by the model size adjustment unit 7 from the features extracted from the divided learning dataset 2 by the feature extraction unit 6 and the features extracted and selected from the divided learning dataset 1. The features selected by the feature sampling unit 8 in step 2 are stored as features of the divided learning dataset 2.

[0052] From step 3 onward, the same processing as in step 2 is performed on the divided learning datasets 3 through N. Once step N is complete, the features of the finally stored divided learning dataset N become the anomaly detection model.

[0053] Next, the learning method of the second embodiment will be described with reference to Figure 7. Figure 7 mainly shows the learning method by the learning unit 2. First, the training data is output from the training database unit 4 to the data splitting unit 10. The output training data is read by the data splitting unit 10 (S301), and the training data is split into N parts (S302). Once the training data has been split and N split training datasets are obtained, the variable i, which specifies the split training dataset to load, is initialized to 1 (S303), and the i-th (in this case, 1st) split training dataset is loaded (S304).

[0054] Next, the i-th segmented learning dataset is output to the feature extraction unit 6. In the feature extraction unit 6, the features of the i-th segmented learning dataset are extracted using the trained CNN, etc., and the size of each extracted feature is obtained (S305). Then, the model size adjustment unit 7 determines the number of features to be held by the anomaly detection model based on the desired size of the anomaly detection model and the size of each feature obtained from the i-th divided learning dataset by the feature extraction unit 6 (S306).

[0055] Specifically, the desired size of the anomaly detection model is determined by factors such as the required processing time and accuracy of anomaly detection. Therefore, the model size adjustment unit 7 calculates the number of features that should be stored in the anomaly detection model of the desired size. In other words, the model size adjustment unit 7 determines the number of features to be held by the anomaly detection model by dividing the desired size of the anomaly detection model by the size of each feature obtained from the segmented learning dataset by the feature extraction unit 6.

[0056] In the case of the i-th (i=1) divided learning dataset, the feature sampling unit 8 selects a number of features determined by the model size adjustment unit 7 from all the features of the i-th (i=1) divided learning dataset extracted by the feature extraction unit 6, using an approximation algorithm such as a greedy method (S307). The anomaly detection model storage unit 9 stores the features of the i-th (i=1) segmented learning dataset selected by the feature sampling unit 8 (S308). Subsequently, 1 is added to the variable i in order to train the i-th (i>1) split learning dataset, and the process returns to S304.

[0057] In the case of the i-th (i>1) split learning dataset, the feature sampling unit 8 adds the features of the (i-1)th split learning dataset (second split learning dataset) that it has already stored to the features of the i-th (i>1) split learning dataset (first split learning dataset) extracted by the feature extraction unit 6 (S309). Then, the feature sampling unit 8 selects a number of features determined by the model size adjustment unit 7 from the features of the i-th (i>1) divided learning dataset and the (i-1)th divided learning dataset using an approximation algorithm such as a greedy method (S310).

[0058] If the variable i (i>1) is not equal to N, the anomaly detection model storage unit 9 stores the i-th (i>1) feature of the divided learning dataset selected by the feature sampling unit 8. Subsequently, 1 is added to the variable i, and the process from S304 to S311 is repeated until the variable i becomes equal to N (S311).

[0059] If the variable i (i>1) is equal to N, the anomaly detection model storage unit 9 stores the features selected by the feature sampling unit 8 as the anomaly detection model (S312). Subsequently, the anomaly detection model stored in the anomaly detection model storage unit 9 is output to the inference unit 3 when an anomaly determination is made (S313). Note that the processing of the inference unit 3 is the same as in the first embodiment, so its explanation will be omitted.

[0060] As described above, the learning device according to this embodiment further includes a data splitting unit that divides the learning data into a plurality of split learning datasets, each containing an image. Therefore, according to this embodiment, the computational resources used to train the anomaly detection model can be reduced compared to training all images included in the training data at once. [Explanation of Symbols]

[0061] 1. Learning device 2. Learning Department 3 Reasoning part 4. Learning Database Department 5. Learning Processing Unit 6. Feature Extraction Unit 7 Model size adjustment section 8. Feature sampling section 9 Anomaly detection model storage unit 10 Data division section 21 Camera Department 22. Photo Database Department 23 Abnormality inference section 24 Display

Claims

1. A learning database unit that stores learning data including multiple images showing a normal state, A feature extraction unit extracts features from the aforementioned training data and obtains the size of each extracted feature, A model size adjustment unit determines the number of features held by the anomaly detection model based on a desired size of the anomaly detection model that detects anomalies using images and the size of each feature obtained by the feature extraction unit. A feature sampling unit selects a number of features determined by the model size adjustment unit from the features extracted by the feature extraction unit, An anomaly detection model storage unit stores an anomaly detection model that holds the features selected by the feature sampling unit, A learning device equipped with this device.

2. The model size adjustment unit determines the number of features held by the anomaly detection model by dividing the desired size of the anomaly detection model by the size of each feature. The learning device according to claim 1.

3. The system further includes a data splitting unit that divides the aforementioned training data into multiple split training datasets, each containing an image. The feature extraction unit sequentially extracts features from each of the multiple divided learning datasets, The feature sampling unit selects a number of features determined by the model size adjustment unit from the features extracted from each of the multiple divided learning datasets by the feature extraction unit. The learning device according to claim 1.

4. The feature sampling unit selects a number of features determined by the model size adjustment unit from the features extracted from the first divided learning dataset by the feature extraction unit and the features extracted from the second divided learning dataset before the first divided learning dataset by the feature extraction unit. The learning device according to claim 3.

5. Features are extracted from training data containing multiple images that show a normal state, and the size of each extracted feature is obtained. Based on the desired size of the anomaly detection model that detects anomalies using images and the size of each acquired feature, the number of features held by the anomaly detection model is determined. The determined number of features are selected from the features extracted from the training data. Store the anomaly detection model that holds the selected features. A learning method that involves a process.

Images