Learning apparatus, method and program

Abstract

To provide a learning apparatus, a method, and a program that can improve a detection accuracy of an instance while teaching in bag units.SOLUTION: A learning apparatus according to an embodiment includes an identification unit, a first calculation unit, a second calculation unit, and an update unit. The identification unit inputs a bag that is a collection of instances into an identifier and outputs a probability row including a probability in which each of the instances belonging to the bag corresponds to a detection target. The first calculation unit calculates a bag likelihood which is a probability in which one or more specific instances corresponding to the detection target is included in the bag. The second calculation unit outputs an expected instance amount related to an amount of the specific instances included in the bag. The update unit updates teaching data which is a numeric value related to the amount of the specific instances included in the bag and parameters of the identifier based on the bag likelihood and the instance amount.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] Embodiments of the present invention relate to a learning device, method, and program.

Background Art

[0002] Machine learning of an identifier for determining normal or abnormal of a product from a bag such as an article image has been performed. In order to obtain high discrimination performance, it is ideal to use, as teaching data, a map in which normal or abnormal is taught for each instance such as a pixel or a partial image of an article image. However, teaching for each instance requires a great deal of cost. When normal or abnormal is taught only for a bag, the learning process is terminated when one instance corresponding to abnormal is found. An identifier obtained by such a learning process cannot detect all of the abnormal portions when the abnormal portion such as a scratch is large.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to provide a learning device, method, and program capable of improving the detection accuracy of instances while teaching at the bag unit.

Means for Solving the Problems

[0005] The learning device according to this embodiment includes an identification unit, a first calculation unit, a second calculation unit, and an update unit. The identification unit inputs a bag, which is a collection of instances, into a classifier and outputs a probability sequence that includes the probability that each instance belonging to the bag corresponds to the detection target. The first calculation unit calculates the bag likelihood, which is the probability that the bag contains one or more specific instances that correspond to the detection target, based on the probability sequence. The second calculation unit outputs the expected instance quantity, which is a numerical value relating to the amount of the specific instances contained in the bag, based on the probability sequence. The update unit calculates a parameter update quantity, which is an amount that changes the parameters of the classifier, based on teaching data, which is a numerical value relating to the amount of the specific instances contained in the bag, the bag likelihood, and the expected instance quantity, and updates the parameters of the classifier based on the parameter update quantity. [Brief explanation of the drawing]

[0006] [Figure 1] This figure shows an example configuration of the learning device according to this embodiment. [Figure 2] This diagram shows an example of an item image (bag) with normal tiles. [Figure 3] This diagram shows an example of an item image (bag) containing tiles with abnormalities. [Figure 4] A diagram illustrating an example of the processing procedure for learning by a learning device. [Figure 5] Figure 4 schematically illustrates the learning process shown in the diagram. [Figure 6] This figure shows an example of the manufacturing process for the product according to this embodiment. [Figure 7] This figure shows an example configuration of the learning dataset generation system according to this embodiment. [Figure 8] This figure shows an overview of the manufacturing process according to this embodiment. [Figure 9] Figure 8 shows an example of an item image of the sheet. [Modes for carrying out the invention]

[0007] The learning device, method, and program according to this embodiment will be described below with reference to the drawings.

[0008] (Learning device) Figure 1 shows an example configuration of the learning device 100 according to this embodiment. As shown in Figure 1, the learning device 100 is a computer having a processing circuit 1, a storage device 2, an input device 3, a communication device 4, and a display device 5. Data communication between the processing circuit 1, the storage device 2, the input device 3, the communication device 4, and the display device 5 is performed via a bus.

[0009] The processing circuit 1 includes a processor such as a CPU (Central Processing Unit) and memory such as RAM (Random Access Memory). The processing circuit 1 includes an acquisition unit 11, an identification unit 12, a first calculation unit 13, a second calculation unit 14, an update unit 15, and a learning control unit 16. The processing circuit 1 realizes each of the above units 11 to 16 by executing a learning program. The learning program is stored in a non-temporary computer-readable recording medium such as a storage device 2. The learning program may be implemented as a single program describing all the functions of the above units 11 to 16, or it may be implemented as multiple modules divided into several functional units. Furthermore, the above units 11 to 16 may be implemented by integrated circuits such as Application Specific Integrated Circuits (ASICs). In this case, they may be implemented in a single integrated circuit, or they may be implemented individually in multiple integrated circuits.

[0010] The acquisition unit 11 acquires training samples from the training dataset. A training sample is a combination of a bag and the corresponding instruction data. A bag is the data that the classifier processes. A bag is a collection of instances. The classifier is a neural network that performs the task of determining whether or not a bag contains a target for detection, such as a normal or abnormal part. An instance is a data element that makes up a bag. An instance is the object of determination for whether or not it is a target for detection. The instruction data is a numerical value relating to the quantity of instances that correspond to the target for detection (hereinafter referred to as "specific instances"). The quantity of specific instances is, for example, the number of specific instances, the proportion that specific instances occupy among the instances that make up the bag, the area occupied by instances within the bag, the length of the outer perimeter of the instances (TV norm), etc.

[0011] The identification unit 12 inputs a bag, which is a collection of instances, into the classifier and outputs a probability sequence containing the probability that each instance belonging to the bag is the target of detection. The classifier is a neural network having a series of network layers for converting the bag into a probability sequence. The network layers may include fully connected layers, convolutional layers, pooling layers, and / or normalization layers.

[0012] The first calculation unit 13 calculates the bag likelihood, which is the probability that one or more specific instances are contained in the bag, based on the probability sequence output by the identification unit 12.

[0013] The second calculation unit 14 outputs the expected instance quantity, which is a numerical value relating to the amount of a specific instance contained in the bag, based on the probability sequence output by the identification unit 12.

[0014] The update unit 15 calculates a parameter update amount, which is the amount by which the classifier parameters are changed, based on teaching data, which is a numerical value relating to the amount of a specific instance contained in the bag, the bag likelihood calculated by the first calculation unit 13, and the expected instance amount calculated by the second calculation unit 14, and updates the classifier parameters based on the parameter update amount.

[0015] The learning control unit 16 controls the learning process for the identifier. The learning control unit 16 determines whether or not the update end condition is satisfied, and until it is determined that the condition is satisfied, it controls the acquisition unit 11, the identification unit 12, the first calculation unit 13, the second calculation unit 14, and the update unit 15 to repeat the learning process. When the learning control unit 16 determines that the update end condition is satisfied, it ends the learning process.

[0016] The storage device 2 is composed of a ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), integrated circuit memory device, etc. The storage device 2 stores a learning program and the like.

[0017] The input device 3 inputs various commands from the user. As the input device 3, a keyboard, a mouse, various switches, a touch pad, a touch panel display, etc. can be used. The output signal from the input device 3 is supplied to the processing circuit 1. Note that the input device 3 may be an input device of a computer connected to the processing circuit 1 via wire or wirelessly.

[0018] The communication device 4 is an interface for performing data communication between the learning device 100 and an external device connected via a network.

[0019] The display device 5 displays various information. As the display device 5, a CRT (Cathode-Ray Tube) display, a liquid crystal display, an organic EL (Electro Luminescence) display, an LED (Light-Emitting Diode) display, a plasma display, or any other arbitrary display known in the art can be appropriately used. Also, the display device 5 may be a projector.

[0020] tHereinafter, an operation example of the learning device 100 according to the present embodiment will be described.

[0021] First, we will explain the relationship between bags and instances, referring to Figures 2 and 3. In the examples in Figures 2 and 3, the bag is an image (hereinafter referred to as the "item image") of an intermediate product (final product) produced during the manufacturing process of the manufactured item, and the instance is a partial image that constitutes the item image. The manufactured item according to this embodiment is applicable to any item, but as an example, let's say it is a tile. The object to be detected is a scratch that has occurred on the tile.

[0022] Figure 2 shows an example of an item image (bag) 20 in which normal tiles have been photographed. As shown in Figure 2, the bag 20 is divided into multiple instances 21. The bag 20 is taught as "normal", and therefore each instance 21 is also "normal".

[0023] Figure 3 shows an example of an item image (bag) 30 in which a tile containing an anomaly has been photographed. As shown in Figure 3, the bag 30 is divided into multiple instances 31. A scratch 32 is visible in the bag 30. Therefore, the bag 30 is taught to be "anomaly". Instance 311, in which the scratch 32 is not visible, is normal, while instances 312 and 313, in which the scratch 32 is visible, are anomaly.

[0024] Here, we will explain the problems with Non-Patent Document 1 (Pathak et al. “FULLY CONVOLUTIONAL MULTI-CLASS MULTIPLE INSTANCE LEARNING,” ICLR, 2015) and Non-Patent Document 2 (Pathak et al. “Constrained Convolutional Neural Networks for Weakly Supervised Segmentation,” Proc. ICCV).

[0025] As described above, in order to obtain high discrimination performance with machine learning, it is ideal to provide an object image with a map that teaches whether each instance is abnormal or normal, but teaching each instance is very costly. As a less costly teaching method, there is a known method called MIL (Multiple Instance Learning), which assigns a binary flag to each image to indicate whether or not the object to be detected is included, as described in Non-Patent Document 1.

[0026] MIL is a framework that can handle problems where direct teaching to instances is not possible, but teaching to bags, such as images, is possible. For example, in the inspection of mass-produced goods, it may not be possible to teach each individual product, but it may be possible to determine the defect rate for a coarse unit, such as a lot of products. Note that the number of instances contained in a bag does not necessarily have to be limited to one. For example, if the minimum number of instances of the target to be detected in the image is known, machine learning can be performed with that constraint, as in Non-Patent Document 2.

[0027] The method described in Non-Patent Document 1 has a problem in use cases such as scratch detection, where the number of instances of the target to be detected is small compared to the total number of instances in the image. In such cases, it only detects deep areas of scratches where there is a clear difference in brightness, and overlooks shallow areas of scratches. This is because, while the classifier's parameters are updated if even one instance is over-detected in an image that has been taught to be scratch-free, the classifier's parameters are not updated in an image that has been taught to be scratch-free as long as one instance is detected, regardless of how large the actual scratch is.

[0028] For example, in the example in Figure 3, instances 312 and 313 contain scratches, but the scratches are deeper in instance 312 and shallower in instance 313. In such a case, the classifier detects instance 312 as abnormal because there is a clear difference in brightness compared to the normal instance 311, but overlooks instance 313, which does not show much difference in brightness. Non-patent literature 2 also states that no improvement can be obtained for this oversight if the minimum number of instances is smaller than the total number of instances in the image. Therefore, this embodiment aims to obtain a classifier that overlooks fewer instances through machine learning, even under conditions where it is only possible to teach a bag with coarser instances than those described above.

[0029] Figure 4 is a diagram showing an example of the processing procedure for the learning process by the learning device 100. Figure 5 is a diagram schematically showing the learning process by the learning device 100. As shown in Figures 4 and 5, the acquisition unit 11 acquires learning samples from the learning dataset (step S1). In step S1, the acquisition unit 11 acquires, as an example, multiple learning samples constituting a minibatch from the learning dataset database 705. As described above, the learning sample includes a bag 51 and teaching data 52. The bag 51 is an image of an item in which a manufactured item to be inspected is captured. The object to be detected is a scratch or other object. The teaching data 52 includes a flag and the amount of correct instances. The flag has a value of "1" if the bag 51 contains a specific instance in which an object such as a scratch is captured, and a value of "0" if the specific instance is not included. The amount of correct instances is a numerical value relating to the amount of a specific instance, and as an example, it is assumed to be the number of specific instances. If the instance is a partial image, the specific instance is a partial image containing a scratch, and if the instance is a pixel, the specific instance is a pixel containing a scratch. A specific instance is also called a faulty instance.

[0030] When step S1 is performed, the identification unit 12 uses the classifier 121 to calculate a probability sequence 53 from the bag 51 acquired in step S1 (step S2). Specifically, the identification unit 12 inputs the bag 51 into the classifier 121. The classifier 121 performs a forward propagation process on the bag 51 to calculate the probability that each instance belongs to a specific instance. The probability sequence 53 is composed of multiple combinations of probabilities corresponding to multiple instances that make up the bag 51.

[0031] When step S2 is performed, the first calculation unit 13 calculates the bag likelihood 54 based on the probability sequence 53 calculated in step S2 (step S3). Specifically, the first calculation unit 13 identifies the maximum value of the multiple probabilities included in the probability sequence 53. The identified maximum value is output as the bag likelihood 54.

[0032] When step S3 is performed, the second calculation unit 14 calculates the expected instance amount 55 based on the probability sequence 53 calculated in step S2 (step S4). Specifically, the second calculation unit 14 calculates the sum of multiple probabilities included in the probability sequence 53. This sum is output as the expected instance amount 55.

[0033] When step S4 is performed, the update unit 15 calculates the parameter update amount 56 based on the bag likelihood 54 calculated in step S3, the expected instance amount 55 calculated in step S4, and the teaching data 52 obtained in step S1 (step S5). Specifically, the update unit 15 first calculates the labeling loss and the instance amount loss.

[0034] The update unit 15 calculates a labeling loss based on the bag likelihood 54 and the teaching data 52. The labeling loss is a binary cross-entropy based on the bag likelihood 54 and the teaching data 52. The labeling loss takes a smaller value the smaller the difference between the correct label and the bag likelihood 54. The correct label takes a predetermined value (e.g., "1") when the number of defective instances is greater than a lower limit (e.g., "0"). The correct label is generated by the update unit 15 based on the number of defective instances. The labeling loss ensures that the identification result of an instance obtained by the classifier 121 is "1" when it is a target for detection and "0" otherwise.

[0035] The update unit 15 calculates instance quantity loss based on the expected instance quantity 55 and the number of bad instances in the training data 52. Instance quantity loss is an index that evaluates the difference between the expected instance quantity 55 and the number of bad instances. Specifically, instance quantity loss is obtained by the mean squared error, which is the average of the squared differences between the expected instance quantity 55 and the number of bad instances across multiple training samples. Labeling loss ensures that the sum of the probability sequence 53 is close to the number of bad instances.

[0036] After calculating the labeling loss and the instance quantity loss, the update unit 15 calculates a parameter update amount 56 to simultaneously reduce the labeling loss and the instance quantity loss. Specifically, the update unit 15 calculates a loss function based on the labeling loss and the instance quantity loss. The loss function is defined by a weighted sum of the labeling loss and the instance quantity loss. The update unit 15 calculates a parameter update amount 56 to minimize the loss evaluated by the loss function.

[0037] When step S5 is performed, the update unit 15 updates the parameters of the classifier 121 based on the parameter update amount 56 calculated in step S5 (step S6). The parameters of the classifier 121 refer to the weight parameters and biases of the transformations between network layers. As an example, the update unit 15 calculates the parameter update amount 56 using stochastic gradient descent. That is, the update unit 15 calculates the gradient of the loss function by backpropagating the classifier 121, and calculates the parameter update amount 56 based on the calculated gradient and the learning rate. Note that the method for calculating the parameter update amount 56 is not limited to stochastic gradient descent, and may be performed by Adam, its evolution, or any other arbitrary method.

[0038] When step S6 is performed, the learning control unit 16 determines whether or not all learning samples have been processed (step S7). The processes from steps S1 to S7 are repeated in mini-batch units, for example. In step S7, it is determined whether or not all learning samples included in all mini-batches have been processed. If it is determined that not all learning samples have been processed (step S7: NO), steps S1 to S7 are executed for a new mini-batch.

[0039] If it is determined in step S7 that all training samples have been processed (step S7: YES), the learning control unit 16 determines whether a predetermined number of processing steps have been performed (step S8). Processing all minibatches sequentially once is called an epoch. The predetermined number of steps refers to the number of epochs at which the learning process is terminated, and can be set to any number. If it is determined that the predetermined number of processing steps have not been performed (step S7: YES), steps S1 to S8 are repeated.

[0040] Then, if it is determined in step S8 that the processing has been performed a predetermined number of times (step S8: YES), the learning control unit 16 terminates the learning process. That is, the learning control unit 16 stores the parameters of the classifier 121 at the time it is determined that the processing has been performed a predetermined number of times as learned parameters in the storage device 2. The classifier 121 to which the learned parameters have been assigned is incorporated into the manufacturing process of the manufactured goods and used to determine whether the manufactured goods are good or bad.

[0041] With the above steps completed, the learning process shown in Figures 4 and 5 is finished.

[0042] (A variation of the learning process) The above learning process can be modified in various ways.

[0043] For example, instance quantity loss does not necessarily have to be the average of the squares of the difference between the correct instance quantity and the expected instance quantity; it could also be the average of the absolute values ​​of that difference.

[0044] The instance quantity loss may be designed to be larger the smaller the expected instance quantity is compared to the lower bound of the ground truth instance quantity minus a predetermined threshold. Specifically, the instance quantity loss L(N,M,T) may be a nonlinear function such as a Marginal Ranking Loss defined using a lower bound N for the ground truth instance quantity, an expected instance quantity M, and a margin T, which differ for each training sample, as shown in equation (1) below.

[0045]

number

[0046] The instance quantity loss L(N,M,T) in equation (1) takes the value "0" if the expected instance quantity M is greater than the value obtained by subtracting the margin (threshold) T from the lower limit N (subtraction value), and takes the value obtained by subtracting the expected instance quantity M from the subtraction value if it is smaller. The instance quantity loss L(N,M,T) in equation (1) works to increase the amount of instances detected when the amount of instances detected by the classifier falls below the taught amount by a margin T or more.

[0047] The learning device 100 is likely to be operated under conditions where the correct instance quantity is not accurately provided. Even in such cases, accuracy can be maintained by using a margin T. If over-detection of instances becomes a problem, the instance quantity loss may be designed to be larger the greater the expected instance quantity is compared to the sum of the lower limit of the correct instance quantity and a predetermined threshold. Specifically, the instance quantity loss L(N,M,T) in equation (2) below may be used.

[0048]

number

[0049] The instance quantity loss L(N,M,T) in equation (2) takes "0" if the value obtained by subtracting the sum of the lower limit N and the margin T from the expected instance quantity M (subtraction value) is less than "0", and takes the subtraction value if it is greater than "0". The instance quantity loss L(N,M,T) in equation (2) works to reduce the number of instances detected when the number of instances detected by the classifier exceeds the taught amount by a margin T or more.

[0050] In addition, the correct instance quantity may be given in a quantized form, in which case the margin T may be set to the step size of the quantization. Furthermore, the instance quantity loss may be calculated by combining the functions described above using methods such as weighted sums. Also, if the frequency distribution of defects is known, the instance quantity loss may be calculated using probability distribution processing such as the Mahalanobis distance.

[0051] (Small summary) As described above, the learning device 100 according to this embodiment includes an identification unit 12, a first calculation unit 13, a second calculation unit 14, and an update unit 15. The identification unit 12 inputs a bag, which is a collection of instances, into the classifier and outputs a probability sequence that includes the probability that each instance belonging to the bag corresponds to the detection target. The first calculation unit 13 calculates the bag likelihood, which is the probability that the bag contains one or more specific instances that correspond to the detection target, based on the probability sequence. The second calculation unit 14 outputs the expected instance quantity, which is a numerical value relating to the amount of specific instances contained in the bag, based on the probability sequence. The update unit 15 calculates a parameter update quantity, which is an amount that changes the parameters of the classifier, based on the teaching data, which is a numerical value relating to the amount of specific instances contained in the bag, the bag likelihood, and the expected instance quantity, and updates the parameters of the classifier based on the parameter update quantity.

[0052] According to the above configuration, the recognition accuracy of instances can be improved by updating the classifier parameters based on the bag likelihood and the expected number of instances. This solves the problem of minimizing the loss, which increases as the number of missed and / or overdetected instances increases, thus reducing missed and / or overdetected instances. Furthermore, since the teaching data in this embodiment is a numerical value relating to the amount of a specific instance contained in the bag, unlike the comparative example which uses teaching data representing good or bad for each instance, there is no need to teach each instance individually. For example, when detecting scratches from an image, the correct number of instances corresponding to the area of ​​the scratch can be used for training, and the parameter update amount can be controlled so that the classifier assigns a high probability to the number of instances corresponding to the area of ​​the scratch. This reduces the chance of the classifier missing scratches without having to teach the exact area of ​​the scratch using a map or the like. Since the "numerical value relating to the amount of a specific instance" used in this embodiment is often managed as yield in the manufacturing process, the effort required to create the teaching data in this embodiment is less than in the comparative example above. In addition, in this embodiment, the loss function used has a positive correlation with the labeling loss, so the recognition accuracy of the bag can be maintained.

[0053] Thus, according to the various embodiments described above, it is possible to provide a learning device, method, and program that can improve the accuracy of instance detection while teaching is performed on a bag-by-bag basis.

[0054] (Training dataset generation system) Next, the learning dataset generation system according to this embodiment will be described. In the following description, the classifier according to this embodiment is used to determine whether an output product at a certain step is good or defective in a manufacturing process consisting of multiple steps.

[0055] Figure 6 shows an example of the manufacturing process for the product according to this embodiment. As shown in Figure 6, component X is processed into product A in process A. Product A is classified as good product A or defective product A by inspection A. Good product A is processed into product B in process B, and further processed into a product in process C. The final target product is manufactured by repeating these processes.

[0056] As shown in Figure 6, in a manufacturing process consisting of multiple steps, automating the inspection of deliverables at each step is useful for improving yield by reducing the amount of defective products carried over to subsequent steps and by identifying the cause of abnormalities in the relevant step. Step A is assumed to output deliverable A, which is a collection of components X, such as a large number of parts or a large sheet of material. In the subsequent inspection A, deliverable X is separated into good products A and defective products A. As a specific example, in inspection A, the appearance of the product is photographed, and if no scratches are visible in the image, it is classified as a good product A; if scratches are visible, it is classified as a defective product A.

[0057] The learning device 100 according to this embodiment is used, for example, to train a classifier for automating inspection A in Figure 6. That is, the classifier classifies the product as either good product A or defective product A based on the item image obtained in inspection A.

[0058] Figure 7 shows an example configuration of the learning dataset generation system 70 according to this embodiment. As shown in Figure 7, the learning dataset generation system 70 includes a measuring instrument 701, a teaching data generator 702, an integration circuit 703, a storage circuit 704, and a learning dataset database 705. The measuring instrument 701, the teaching data generator 702, the integration circuit 703, the storage circuit 704, and the learning dataset database 705 are connected to each other via wired or wireless means so that they can communicate data with one another.

[0059] The measuring instrument 701 measures the output A of process A (hereinafter referred to as output 71) and generates a bag 72, which is measurement data representing the measurement result of output 71. The teaching data generator 702 measures the amount of correct instances contained in output 71, i.e., the amount of instances containing defects (hereinafter referred to as the amount of defective instances), and generates teaching data 74 representing that amount. The integration circuit 703 integrates the bag 72 and the teaching data 74 to generate a training sample 75. The storage circuit 704 stores the training sample 75 in the training data set database 705. Each time output 71 is produced in process A of the manufacturing process shown in Figure 6, the training sample 75 is generated through the cooperation of the measuring instrument 701, the teaching data generator 702, and the integration circuit 703, and is stored in the training data set database 705 by the storage circuit 704. Multiple training samples constitute a training data set. The training data set database 705 is a computer that includes a storage device such as an HDD or SSD that systematically stores multiple training samples. The learning dataset generation system 70 may also include a learning device 100.

[0060] For example, bag 72 is an item image that captures the entirety of a single deliverable A, and instance is a partial image that constitutes the item image, corresponding to a part of deliverable A. A defective instance means a partial image that shows a target for detection, such as a scratch. In this case, measuring instrument 701 includes an optical camera that photographs deliverable A and generates an item image that shows the appearance of deliverable A. Teaching data generator 702 includes an image processing processor that measures the amount of defective instances from the item image obtained by measuring instrument 701. The teaching data generator 702 may perform the process of measuring the amount of defective instances from the item image using a fully automated algorithm, or it may perform the process using a semi-automated algorithm that follows instructions given by a human. Integration circuit 703 includes an information processing processor that associates bag 72 with teaching data 74. The combination of bag 72 and teaching data 74 that are associated with each other constitutes a training sample 75. Storage circuit 704 includes an information processing processor that reads and writes the training sample 75 to a training data set database 705.

[0061] Bag 72 is not limited to images of the entire deliverable A. For example, if deliverable A is a mass-produced part, Bag 72 may be an image showing a series of deliverables A, an instance may be an item image showing each deliverable A from that series, and a defective instance may be an item image showing a defective product from that item image. Furthermore, an instance is not necessarily limited to an image. For example, if deliverable A is a mass-produced part, an instance may be measurement data obtained by measuring the part with some device, and a defective instance may be the measurement data of a defective product from that measurement data. As measurement data, measurement data from various measuring instruments such as current detectors, voltage detectors, vibration detectors, sound collectors, distance measuring instruments, position detectors, and thermal detectors can be used.

[0062] (Example 1) In Example 1, the bag is an image of an item, and the instance is a pixel that makes up the item image. The correct instance quantity is the length of the scratch on the item included in the item image. The expected instance quantity is the sum of multiple probabilities obtained from multiple instances arranged along one direction of the item image.

[0063] Figure 8 is a diagram showing an overview of the manufacturing process according to this embodiment. The product of the manufacturing process shown in Figure 8 is a sheet 80. The surface of the sheet 80 unwound from the unwinding shaft 81 is processed, and the processed sheet 80 is wound up by the winding shaft 82. The camera 83 photographs the surface of the processed sheet 80 for inspection and generates an image corresponding to a bag. Figure 9 is a diagram showing an example of an item image 90 of the sheet 80 shown in Figure 8. As shown in Figure 9, the item image 90 is composed of multiple partial images 91. If the product is defective, the item image 90 will show detection targets such as streaky scratches 92. The learning dataset generation system 70 generates a learning sample from the item image 90.

[0064] If a defect 92 is detected by the teaching data generator 702, the defect is removed during the manufacturing process. Specifically, the defective portion is pulled out from the unwinding shaft 81 and / or the winding shaft 82 and cut. In this case, the correct instance amount is the cut length 93 shown in Figure 9. The cut length 93 is defined, for example, by the amount of defective instances along the winding direction. The teaching data generator 702 processes the image of the item to extract the image region of the defect 92 and counts the number of pixels along the winding direction of the extracted image region. This number of pixels is set as the correct instance amount.

[0065] This section describes how to calculate the expected instance quantity when the correct instance quantity is a cut length of 93. During training, image 90 is processed by a classifier and converted into an anomaly map corresponding to a probability sequence. Each instance in the anomaly map is assigned a probability value corresponding to the part of the sheet that corresponds to that instance being defective. For example, an instance corresponding to a scratch 92 is assigned a high probability value, and an instance without a scratch 92 is assigned a low probability value. In this case, the second calculation unit 14 extracts the image region corresponding to the scratch 92 from the anomaly map, sets a line along the winding direction (hereinafter referred to as the X direction) in the extracted image region, and identifies the maximum value of the probability values ​​of multiple instances along the Y direction (direction orthogonal to the X direction) for each position in the X direction. The second calculation unit 14 can then calculate the expected instance quantity by adding the maximum values ​​of each identified X direction position along the winding direction.

[0066] (Example 2) The bag in Example 2 is an image of an item. The item image is divided into two or more subregions. Each subregion contains two or more instances in the vertical and / or horizontal directions. The expected instance quantity is the sum of probabilities obtained from subregions arranged along one direction of the item image, and the correct instance quantity is the length of the damage to the item contained in the item image.

[0067] Specifically, the second calculation unit 14 in Embodiment 2 divides the anomaly map into multiple sub-regions along the winding direction. Each sub-region has two or more instances in the Y direction and the X direction. The shape of the sub-regions may be changed to suit the actual process. For example, if the output of the process to be automated is in the form of a plate, and in the next process it will be cut into tiles for use, the shape of the sub-regions may be rectangles in the same shape as the tiles. The second calculation unit 14 identifies the maximum probability value of the multiple instances contained in each of the multiple sub-regions. The second calculation unit 14 calculates the expected number of instances by adding the maximum values ​​of each identified sub-region along the winding direction.

[0068] (Example 3) The teaching data generator 702 according to Example 3 generates teaching data 74 using a different algorithm. For example, in the manufacturing process shown in Figure 6, the quantity of component X or the total quantity of product A, the total quantity of good products A, and the quantity of defective products A discarded in inspection A may be recorded for yield management. Even if the total quantity of good products A is unknown, the total quantity of good products A can sometimes be estimated from the quantity of components fed into the subsequent process B or the quantity of products from process B. The teaching data generator 702 may generate teaching data 74, which is the quantity of defective instances, based on this information. In this way, even when inspection A is automated in a manufacturing process consisting of multiple processes as shown in Figure 6, the defect rate of process A can be roughly determined from the quantity of components fed into process B, even without detailed records of inspection A.

[0069] (Small summary) As described above, the learning dataset generation system 70 includes a measuring instrument 701, a teaching data generator 702, an integration circuit 703, a storage circuit 704, and a learning dataset database 705. The measuring instrument 701 measures the product of the manufacturing process and generates a bag, which is measurement data representing the measurement result of the product. The teaching data generator 702 measures the amount of defective parts contained in the product and generates teaching data representing that amount. The integration circuit 703 integrates the bag and the teaching data to generate a learning sample. The storage circuit 704 stores the learning sample in the learning dataset database 705.

[0070] According to the above configuration, training samples to be used in the learning device 100 can be generated and stored. Currently, individually recording which components of the deliverable X are defective and which are good is sometimes not done due to reasons such as increased work costs. For example, even if there is an image of the product's appearance, an instance-level map showing the location of scratches in that image is often not created. In this embodiment, instead of a map representing good / defective, the amount of defective parts contained in the deliverable is used as teaching data, thus reducing the effort and burden of generating a map.

[0071] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of symbols]

[0072] 1... Processing circuit, 2... Memory device, 3... Input device, 4... Communication device, 5... Display device, 11... Acquisition unit, 12... Identification unit, 13... First calculation unit, 14... Second calculation unit, 15... Update unit, 16... Learning control unit, 100... Learning device.

Claims

1. An identification unit inputs a bag, which is a collection of instances, into a classifier and outputs a probability sequence that includes the probability that each instance belonging to the bag corresponds to the detection target. A first calculation unit calculates a bag likelihood, which is the probability that the bag contains one or more specific instances corresponding to the detection target, based on the aforementioned probability sequence. A second calculation unit outputs an expected instance quantity, which is a numerical value relating to the amount of the specific instance contained in the bag, based on the probability sequence. An update unit calculates a parameter update amount, which is an amount that changes the parameters of the classifier, based on teaching data which is a numerical value relating to the amount of the specific instance contained in the bag, the bag likelihood, and the expected instance amount, and updates the parameters of the classifier based on the parameter update amount. A learning device equipped with the following.

2. The bag likelihood is the maximum value of the probabilities included in the probability sequence, The expected instance quantity is the sum of the probabilities included in the probability sequence, The update unit generates a ground truth label that takes a predetermined value when the quantity of the specific instance is greater than the lower limit, and calculates the parameter update amount that simultaneously reduces the labeling loss and the instance quantity loss. The labeling loss takes on a smaller value the smaller the difference between the correct label and the bag likelihood. The instance quantity loss is the difference between the teaching data and the expected instance quantity. The learning device according to claim 1.

3. The instance quantity loss is the average of the absolute values ​​of the difference between the quantity of a particular instance and the expected quantity of instances, or the mean squared error. The update unit calculates the parameter update amount based on a loss function which is a weighted sum of the labeling loss and the instance amount loss. The learning device according to claim 2.

4. The learning device according to claim 2, wherein the instance quantity loss has a larger value the smaller the expected instance quantity is than the value obtained by subtracting a predetermined threshold from the lower limit of the quantity of the specific instance.

5. The learning device according to claim 2, wherein the instance quantity loss has a larger value the more the expected instance quantity is greater than the sum of the lower limit of the quantity of a particular instance and a predetermined threshold.

6. The aforementioned bag is an image of an item, The instance is a pixel that constitutes the image, The quantity of the particular instance is the length of the streaky scratches that have occurred on the article included in the image. The learning device according to claim 2.

7. The learning device according to claim 6, wherein the expected number of instances is the sum of multiple probabilities obtained from a plurality of instances arranged along one direction of the image.

8. The aforementioned image is divided into multiple regions, Each of the aforementioned multiple regions includes two or more instances in the vertical and / or horizontal directions. The expected instance quantity is the sum of probabilities obtained from regions of the plurality of regions that are arranged along one direction of the image. The learning device according to claim 7.

9. The aforementioned bag contains images of items taken during the manufacturing process. The aforementioned identifier determines whether the article is good or bad. The learning device according to claim 1.

10. The aforementioned bag is an image showing a single item, The instance corresponds to a part of the article, The learning device according to claim 9.

11. The aforementioned bag is an image showing a series of items, The instance corresponds to each article in the series of articles, The learning device according to claim 9.

12. A bag, which is a collection of instances, is input to the classifier, and a probability sequence is output that includes the probability that each instance belonging to the bag corresponds to the target for detection. Based on the aforementioned probability sequence, the bag likelihood is calculated, which is the probability that the bag contains one or more specific instances corresponding to the target to be detected. Based on the aforementioned probability sequence, the expected instance quantity, which is a numerical value relating to the amount of the particular instance contained in the bag, is output. Based on teaching data which is a numerical value relating to the amount of the specific instance contained in the bag, the bag likelihood, and the expected instance amount, a parameter update amount is calculated which is the amount by which the parameters of the classifier are changed, and the parameters of the classifier are updated based on the parameter update amount. A computer-implemented learning method that possesses the following characteristics.

13. On the computer, A function that inputs a bag, which is a collection of instances, into a classifier and outputs a probability sequence that includes the probability that each instance belonging to the bag corresponds to the target for detection. A function to calculate the bag likelihood, which is the probability that the bag contains one or more specific instances corresponding to the detection target, based on the aforementioned probability sequence, A function to output the expected instance quantity, which is a numerical value relating to the amount of the specific instance contained in the bag, based on the aforementioned probability sequence, A function that calculates a parameter update amount, which is an amount that changes the parameters of the classifier, based on teaching data which is a numerical value relating to the amount of the specific instance contained in the bag, the bag likelihood, and the expected number of instances, and updates the parameters of the classifier based on the parameter update amount. A learning program that makes this a reality.

Images