Learning assistance device, learning device, learning assistance method, and learning assistance program

Abstract

The learning assistance device has: a derivation unit that derives the feature amount of the teacher data for each teacher data based on a model that has been learned using teacher data in such a way that object data is classified into either a first label or a second label, and the teacher data including first data assigned the first label and second data assigned the second label, and derives the feature amount of the teacher candidate data for each teacher candidate data based on at least one teacher candidate data, each assigned either the first label or the second label, and the model; a calculation unit that calculates at least one of the distance between the teacher candidate data and the first data and the distance between the teacher candidate data and the second data for each teacher candidate data; and a selection unit that selects, based on the distance, data to be added as teacher data from the teacher candidate data.

Description

Technical Field

[0001] The present invention relates to a learning assistance device, a learning device, a learning assistance method, and a learning assistance program. Background Art

[0002] Patent Document 1 discloses a device that uses a model including a neural network and filter coefficients to identify an image. The model inputs a sample image from an input layer of the neural network, performs filtering processing based on the filter coefficients in an intermediate layer, and outputs information (class ID) indicating the classification of the sample image as an identification result in an output layer. Regarding the model, it is necessary to perform pre-learning using a teacher image that is an image to which a correct class ID is assigned. Specifically, the filter coefficients are set so that the neural network that has input the teacher image outputs the correct class ID. Further, the device presents the class ID identified by the model together with the image to the user, and when the class ID is corrected by the user, the model re-learns the image with the corrected class ID.

[0003] Prior Art Documents

[0004] Patent Documents

[0005] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2016-143354 Summary of the Invention

[0006] Problems to be Solved by the Invention

[0007] In addition, images that the model cannot easily identify contribute highly to the determination of the parameters of the neural network and can be teacher data with high learning effects. Therefore, by using images that the model cannot easily identify to re-learn the model, higher learning efficiency can be achieved. However, although the device described in Patent Document 1 re-learns the image with the class ID corrected by the user, in reality, among the images for which the model gives the correct answer, there may be images that are accidentally classified as the correct class with a slight difference. Such images can be said to be images that the model cannot easily identify, but are excluded from the candidates for re-learning. Therefore, the device described in Patent Document 1 may not be able to efficiently make the model learn.

[0008] An object of the present invention is to provide a learning assistance device, a learning device, a learning assistance method, and a learning assistance program that can appropriately assist the learning of a model.

[0009] Means for Solving the Problems

[0010] The learning assistance device of the present invention includes: a teacher data acquisition unit that acquires teacher data, the teacher data including first data assigned with a first label and second data assigned with a second label; a teacher candidate data acquisition unit that acquires at least one teacher candidate data each assigned with either the first label or the second label; a derivation unit that, based on a model that has been learned using the teacher data in such a way as to classify object data into either the first label or the second label and the teacher data, derives, for each teacher data, a feature amount represented by a feature space of a predetermined dimension of the teacher data, and based on the model and the at least one teacher candidate data, derives, for each teacher candidate data, a feature amount represented by the feature space of the teacher candidate data; a calculation unit that, based on the feature amount of the teacher data and the feature amount of the at least one teacher candidate data, calculates, for each teacher candidate data, at least one of the following first distance and second distance, the first distance being the distance in the feature space between the teacher candidate data and the first data, and the second distance being the distance in the feature space between the teacher candidate data and the second data; and a selection unit that selects, based on the distance of each teacher candidate data calculated by the calculation unit, data to be added as teacher data from the at least one teacher candidate data.

[0011] Advantages of the Invention

[0012] According to various aspects and embodiments of the present invention, it is possible to appropriately assist the learning of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram showing an example of the functions of the learning device and the learning assistance device of the embodiment.

[0014] Figure 2 is a diagram showing Figure 1 a block diagram of the hardware structure of the device shown.

[0015] Figure 3 is a schematic diagram of the neural network used in the learning unit.

[0016] Figure 4 is a diagram showing the distribution of the feature amounts calculated by the neural network.

[0017] Figure 5 is an explanatory diagram showing the elements of the acceptable product distance and the non-acceptable product distance.

[0018] Figure 6 is an explanatory diagram showing the elements of the acceptable product distance and the non-acceptable product distance.

[0019] Figure 7 is an explanatory diagram showing the elements of the acceptable product distance and the non-acceptable product distance.

[0020] Figure 8 It is a flowchart of a learning assistance method in a learning device and a learning assistance device.

[0021] Figure 9 It is a flowchart of a learning process.

[0022] Figure 10 The (A) to Figure 10 The (D) of is a diagram showing an example of a screen displayed on a display unit. Detailed implementation mode

[0023] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, in the following description, the same or corresponding elements are denoted by the same reference numerals and repeated descriptions are omitted.

[0024] [Functional structure of learning assistance device]

[0025] Figure 1 It is a block diagram showing an example of the functions of the learning device and the learning assistance device according to the embodiment. Figure 1 The learning device 10 shown is a device for training the model M1. The model M1 has a structure including a neural network and parameters. The neural network has a structure formed by combining a plurality of neurons. As an example, the neural network may also be a hierarchical multi-layer neural network formed by connecting layers obtained by grouping a plurality of neurons. The neural network is defined by the number of neurons and the connection relationship. The connection strength between neurons or between layers is defined using parameters (weight coefficients, etc.). In the neural network, data is input, and based on the operation results of a plurality of neurons and the parameters, the features of the data are output as a solution. The learning device 10 has a learning unit 11 that learns the parameters of the model M1 so as to obtain the target ability. Learning means adjusting the parameters to the optimal values. The details of the neural network will be described later.

[0026] The learning result of the learning device 10 is used in the processing device 12. The processing device 12 has an execution environment in which the model M2 can operate. The model M2 has the same neural network and parameters as the model M1 that is the learning target of the learning device 10. The model M2 is the same model as the model M1, and the model M1 is the main (original) model. In the processing device 12, the object data D1 is input to the model M2, and a result is output from the model M2. The object data D1 is data to be processed for the purpose of the processing device 12, and is, for example, image data, sound data, coordinate graph data, etc. The object data D1 is data before being given the label described later. The purpose of the processing device 12 is identification (classification), determination, etc. The processing device 12 may be physically or logically separated from the learning device 10, or may be integrated with the learning device 10 physically or logically.

[0027] The model M2 of the processing device 12 identifies the content of the object data D1 and outputs a label as the recognition result R1. The label is information for identifying a preset category and is used to classify or discriminate the object data D1. In the case where the object data D1 is image data, the label can be set, for example, as the category of the subject (person, vehicle, animal, etc.), the quality of the subject (qualified product, unqualified product, etc.). The processing device 12 can assign the output label to the object data D1. To assign means to make an association. For example, the relationship between the object data D1 and the label can be recorded in a table or the like, the attribute information of the object data D1 can be changed to include the label, or the label can be embedded in the object data itself.

[0028] Next, a case will be described as an example where the object data D1 with an electronic component as the subject is input and the model M2 of the processing device 12 outputs a label related to the quality of the electronic component. In this case, the learning unit 11 of the learning device 10 causes the model M2 of the processing device 12 to learn the parameters of the neural network of the model M1 in order to accurately discriminate the label of the object data D1.

[0029] The learning unit 11 causes the model M1 to learn based on the teacher data D2. The teacher data D2 is data in the same form as the object data D1 (here, image data) and is pre-assigned with the correct label. For example, an annotator (operator) or the like correctly assigns either a qualified product label (an example of the first label) indicating that the electronic component as the subject meets the appearance quality standard or an unqualified product label (an example of the second label) indicating that the electronic component as the subject does not meet the appearance quality standard to the teacher data D2. Therefore, the teacher data D2 includes qualified product data (an example of the first data) assigned with the qualified product label and unqualified product data (an example of the second data) assigned with the unqualified product label.

[0030] The learning unit 11 causes the neural network of model M1 to learn the characteristics of the qualified product data and the unqualified product data based on the qualified product data and the unqualified product data that are the teacher data D2. For the input teacher data D2, model M1 outputs a score indicating the credibility of belonging to a qualified product (hereinafter referred to as "qualified product score") and a score indicating the credibility of belonging to an unqualified product (hereinafter referred to as "unqualified product score"). In the present embodiment, the qualified product score and the unqualified product score are respectively values in the range of 0.0 to 1.0, and the sum of the qualified product score and the unqualified product score is set to 1.0. The learning unit 11 adjusts the parameters of the neural network of model M1 so that the qualified product score approaches 1.0 and the unqualified product score approaches 0.0 for the qualified product data given the qualified product label. On the other hand, the learning unit 11 adjusts the parameters of the neural network of model M1 so that the qualified product score approaches 0.0 and the unqualified product score approaches 1.0 for the unqualified product data given the unqualified product label. Thus, model M1 acquires the ability to classify the object data D1 into either the qualified product label or the unqualified product label. The parameters learned by the learning unit 11 are output to the processing device 12, and the parameters of model M2 of the processing device 12 are updated. Thus, model M2 of the processing device 12 also acquires the ability to classify the object data D1 into either the qualified product label or the unqualified product label.

[0031] The learning assistance device 20 assists the learning of the learning device 10. The learning assistance device 20 selects additional teacher data D4 for use in re-learning model M1 from the teacher candidate data D3. The teacher candidate data D3 is data of the same form as the teacher data D2 (here, image data), and is pre-labeled by an annotator (operator) or the like.

[0032] The learning assistance device 20 includes a teacher data acquisition unit 21, a teacher candidate data acquisition unit 22, a derivation unit 23, a calculation unit 24, and a selection unit 25.

[0033] The teacher data acquisition unit 21 acquires the teacher data D2 including the qualified product data given the qualified product label and the unqualified product data given the unqualified product label. The teacher data D2 is the data learned by the learning unit 11. The teacher candidate data acquisition unit 22 acquires at least one teacher candidate data D3 given either the qualified product label or the unqualified product label. The teacher candidate data D3 is composed of one or more data. The teacher candidate data D3 may be composed only of the data given the qualified product label, or may be composed only of the data given the unqualified product label. Hereinafter, the teacher candidate data D3 is set to be a plurality of data including both the data given the qualified product label and the data given the unqualified product label.

[0034] The teacher data acquisition unit 21 and the teacher candidate data acquisition unit 22 can acquire the teacher data D2 or the teacher candidate data D3 through communication from a data server (not shown) or the like, or can refer to an external storage medium that can be connected to the learning assistance device 20 or a storage medium included in the learning assistance device 20 to acquire the teacher data D2 or the teacher candidate data D3. The teacher data acquisition unit 21 and the teacher candidate data acquisition unit 22 can also acquire data obtained by a user tagging data obtained through a camera or the like.

[0035] Based on the model M1 learned in the learning unit 11 and the teacher data D2, the derivation unit 23 calculates, for each teacher data D2, a feature amount represented by a feature space of a predetermined dimension. The feature space of the predetermined dimension is a conversion feature space used to facilitate the calculation of feature amounts of a large dimension. Therefore, the dimension of the feature space can be two-dimensional or three-dimensional.

[0036] The feature amount is a vector representing the features of an image and is extracted from the calculation process of the neural network of the model M1 into which the image is input. The derivation unit 23 can also cause the learning device 10 to operate in a manner of extracting feature amounts for each teacher data D2 and obtain the feature amounts from the learning device 10. Alternatively, the derivation unit 23 can prepare a model M3 identical to the model M1 and calculate feature amounts for each teacher data D2 in the learning assistance device 20. The model M3 is a model based on (original) the model M1.

[0037] Based on the model M1 learned in the learning unit 11 and at least one teacher candidate data D3, the derivation unit 23 calculates, for each teacher candidate data D3, a feature amount represented by a feature space having the same dimension as the feature space in which the feature amounts of the teacher data D2 fall. Similar to the teacher data D2, the learning device 10 can be caused to perform extraction of the features of each teacher candidate data D3, or a model M3 identical to the model M1 can be prepared and feature amounts can be calculated for each teacher data D2 in the learning assistance device 20.

[0038] The calculation unit 24 calculates the distance between the teacher data D2 and the teacher candidate data D3 in the feature space. Specifically, the calculation unit 24 calculates the distance between the teacher candidate data D3 and the qualified product data (an example of the first distance), i.e., the qualified product distance, in the feature space for each teacher candidate data D3 based on the feature amounts of the teacher data D2 and the teacher candidate data D3. The calculation unit 24 calculates the distance between the teacher candidate data D3 and the unqualified product data (an example of the second distance), i.e., the unqualified product distance, in the feature space for each teacher candidate data D3 based on the feature amounts of the teacher data D2 and the teacher candidate data D3. The calculation unit 24 may also calculate at least one of the qualified product distance and the unqualified product distance. That is, the calculation unit 24 may calculate only the qualified product distance or only the unqualified product distance. The calculation unit 24 may also calculate an evaluation value using the qualified product distance and the unqualified product distance for each teacher candidate data D3. The detailed description and calculation method of the qualified product distance, the unqualified product distance, and the evaluation value will be described later.

[0039] The selection unit 25 selects the data to be added (the additional teacher data D4) from at least one teacher candidate data D3 as the teacher data D2 based on the distances of each teacher candidate data D3 calculated in the calculation unit 24. As the distance of each teacher candidate data D3, the selection unit 25 may use only the qualified product distance or only the unqualified product distance. In the present embodiment, the selection unit 25 selects the additional teacher data D4 based on both the qualified product distance and the unqualified product distance of each teacher candidate data D3. When the selection unit 25 determines that there is no additional teacher data D4 based on the distance (at least one of the qualified product distance and the unqualified product distance), the selection unit 25 causes the display unit 26 described later to display the determination result. The determination criterion will be described later.

[0040] As a method for the selection unit 25 to select the additional teacher data D4, the following three methods are exemplified. The first method is as follows: the shorter the qualified product distance of the teacher candidate data given an unqualified product label, the higher the probability that the selection unit 25 selects the teacher candidate data from at least one teacher candidate data. The second method is as follows: the shorter the unqualified product distance of the teacher candidate data given a qualified product label, the higher the probability that the selection unit 25 selects the teacher candidate data from at least one teacher candidate data D3. The third method is as follows: the selection unit 25 selects the additional teacher data D4 based on the evaluation value of each teacher candidate data D3. The selection unit 25 can adopt any one of the above three methods or a combination thereof. The detailed situation of each method will be described later.

[0041] The learning assistance device 20 can include a display unit 26, an input unit 27, and a change unit 28.

[0042] The display unit 26 displays the additional teacher data D4 selected by the selection unit 25. The display unit 26 can not only display the image of the additional teacher data D4, but also display the label assigned to the additional teacher data D4, the acceptable product distance, the non-acceptable product distance, the evaluation value, the number of teacher candidate data, etc. The display unit 26 can also display a coordinate graph depicting feature amounts in a space of a specified dimension. The display unit 26 can also display the teacher data D2 and the additional teacher data D4 in a comparable manner. By visualizing the additional teacher data D4 using the display unit 26, it is easy for the user to confirm the deviation in the quality of the additional teacher data D4, as well as to confirm the label, the acceptable product distance, the non-acceptable product distance, the evaluation value, or the number of teacher candidate data.

[0043] When the selection unit 25 determines that there is no additional teacher data D4 based on the distance, the display unit 26 displays, under the control of the selection unit 25, a determination result indicating the case where there is no additional teacher data D4. The selection unit 25 causes the determination result to be displayed on the screen of the display unit 26, whereby it is possible to notify the user that there is no additional teacher data. The user can recognize that there is no additional teacher data D4 for the model M1 to learn, and can easily determine whether to end the learning of parameters such as weight coefficients. The display unit 26 can also notify the user of the determination result in combination with the output of an alarm sound by a speaker (not shown).

[0044] The input unit 27 accepts the input of a user operation. The user operation is an action performed by the user that causes the input unit 27 to operate. As an example, the user operation is a selection operation or an input operation.

[0045] When a user operation for changing the label assigned to the additional teacher data D4 displayed on the display unit 26 is input via the input unit 27, the change unit 28 changes the label assigned to the additional teacher data D4. The change unit 28 causes the display unit 26 to display a screen for the user to confirm whether there is an error in the label previously assigned to the additional teacher data D4. When the user determines that there is an error in the label of the additional teacher data D4, the user changes the label of the additional teacher data D4 from an acceptable product label to a non-acceptable product label, or from a non-acceptable product label to an acceptable product label via the input unit 27 and the change unit 28.

[0046] [Hardware Structure of Learning Assistance Device]

[0047] Figure 2 is a block diagram showing Figure 1 the hardware structure of the device shown. As Figure 2As shown, the learning assistance device 20 is configured as a general computer system including a CPU (Central Processing Unit) 301, a RAM (Random Access Memory) 302, a ROM 303 (Read Only Memory), a graphics controller 304, an auxiliary storage device 305, an external connection interface 306 (hereinafter referred to as "I / F"), a network I / F 307, and a bus 308.

[0048] The CPU 301 is composed of an arithmetic circuit and controls the learning assistance device 20 in a unified manner. The CPU 301 reads out the programs stored in the ROM 303 or the auxiliary storage device 305 into the RAM 302. The CPU 301 executes various processes of the programs read into the RAM 302. The ROM 303 stores system programs and the like used in the control of the learning assistance device 20. The graphics controller 304 generates a screen to be displayed on the display unit 26. The auxiliary storage device 305 has the function of a storage device. The auxiliary storage device 305 stores application programs and the like for executing various processes. As an example, the auxiliary storage device 305 is composed of an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like. The external connection I / F 306 is an interface for connecting various devices to the learning assistance device 20. As an example, the external connection I / F 306 connects the learning assistance device 20, a display, a keyboard, a mouse, and the like. The network I / F 307 communicates with the learning assistance device 20 and the like via the network under the control of the CPU 301. The above-described respective structural parts are connected in a communicable manner via the bus 308.

[0049] The learning assistance device 20 can have hardware other than the above. As an example, the learning assistance device 20 may also have a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), and the like. The learning assistance device 20 does not need to be housed in one housing as hardware and may be separated into several devices.

[0050] Figure 1 The functions of the learning assistance device 20 shown are achieved through Figure 2The hardware implementation shown. The CPU 301 executes programs stored in the RAM 302, ROM 303, or auxiliary storage device 305, and processes data stored in the RAM 302, ROM 303, or auxiliary storage device 305, or data acquired via the external connection I / F 306 or network I / F, thereby implementing the teacher data acquisition unit 21, teacher candidate data acquisition unit 22, derivation unit 23, calculation unit 24, selection unit 25, and change unit 28. The display unit 26 is a display device. The input unit 27 is a mouse, keyboard, touch panel, etc. The function of the change unit 28 can also be further implemented using the graphics controller 304. Figure 1 The processing device 12 and the learning device 10 shown are also composed of Figure 2 a part or all of the hardware shown.

[0051] [Details of the neural network]

[0052] An overview of the neural networks of models M1 to M3 is given. Figure 3 is a schematic diagram of a neural network. As Figure 3 shown, the neural network 400 is a so-called hierarchical neural network, in which multiple artificial neurons (nodes) represented by circles form layers and are connected. The hierarchical neural network has artificial neurons for input, artificial neurons for processing, and artificial neurons for output.

[0053] The data 401 is the object to be processed by the neural network. The data 401 is acquired by the artificial neurons for input in the input layer 402. The artificial neurons for input are arranged in parallel, thereby forming the input layer 402. The data 401 is distributed to the artificial neurons for processing. The signal itself exchanged by the neural network is called a score. The score is a numerical value.

[0054] The artificial neurons for processing are connected to the artificial neurons for input. The artificial neurons for processing are arranged in parallel, thereby forming the intermediate layer 403. The intermediate layer 403 can be multiple layers. In addition, a neural network with three or more layers having the intermediate layer 403 is called a deep neural network.

[0055] The neural network can also be a so-called convolutional neural network. The convolutional neural network is a deep neural network formed by alternately connecting convolutional layers and pooling layers. By sequentially processing using the convolutional layer and the pooling layer, the image of the data 401 is reduced while maintaining features such as edges. When applying the convolutional neural network to image analysis, the image can be classified with high accuracy based on the extracted features.

[0056] The artificial neurons for output output the score to the outside. In Figure 3In the example, the qualified product score and the unqualified product score are output from the artificial neuron for output. That is, in the output layer 404, two artificial neurons are prepared: one for outputting the qualified product score and the other for outputting the unqualified product score. The output layer 404 outputs the qualified product score and the unqualified product score to the outside as the output 405. In the present embodiment, the qualified product score and the unqualified product score are respectively values in the range of 0.0 to 1.0, and the sum of the qualified product score and the unqualified product score is set to 1.0. In the learning process (S510) described later, for the teacher data with the qualified product label, that is, the qualified product data, the neural network 400 is learned in such a way that the qualified product score approaches 1.0 and the unqualified product score approaches 0.0. On the other hand, for the teacher data with the unqualified product label, that is, the unqualified product data, the neural network 400 is learned in such a way that the qualified product score approaches 0.0 and the unqualified product score approaches 1.0.

[0057] [Derivation of Feature Quantities by the Derivation Unit]

[0058] As an example, the derivation unit 23 uses the model M3 including the learned neural network 400 described above to derive the feature quantities represented by the feature space of a predetermined dimension for each teacher data D2. The derivation unit 23 inputs the teacher data D2 obtained by the teacher candidate data acquisition unit 22 as the data 401 to the input layer 402 of the neural network 400. The artificial neurons for processing in the intermediate layer 403 process the input using the learned weight coefficients and propagate the output to other neurons. The derivation unit 23 obtains the operation result of one layer selected from the plurality of intermediate layers 403 as the feature quantity. As an example, the derivation unit 23 projects the operation result of the layer that propagates the score to the output layer 404 (the layer before the output layer 404) among the plurality of intermediate layers 403 into the feature space as the feature quantity. In this way, the derivation unit 23 derives the feature quantity using the learned model M3 and the teacher data D2.

[0059] In addition, the export unit 23 uses the model M3 including the above-mentioned learned neural network 400 to export the feature quantities represented by the feature space of a predetermined dimension for each teacher candidate data D3. The export unit 23 inputs the teacher candidate data D3 acquired by the teacher candidate data acquisition unit 22 as data 401 into the input layer 402 of the neural network 400. The artificial neurons for processing in the intermediate layer 403 process the input using the learned weight coefficients and propagate the output to other neurons. The export unit 23 acquires the operation result of one layer selected from the plurality of intermediate layers 403 as the feature quantity. As an example, the export unit 23 projects the operation result of the layer (the layer before the output layer 404) that propagates scores to the output layer 404 among the plurality of intermediate layers 403 into the feature space as the feature quantity. In this way, the export unit 23 exports the feature quantity using the learned model M3 and the teacher candidate data D3.

[0060] The export unit 23 may also cause the learning device 10 to operate to extract the feature quantity and acquire the feature quantity from the learning device 10. In this case, the learning device 10 uses the model M1 and calculates the feature quantity by the same method as the above method.

[0061] Figure 4 It is a graph showing the distribution of the feature quantities calculated by the neural network. Figure 4 The coordinate diagram shown shows the feature quantities of the teacher data D2 and the teacher candidate data D3 projected into the two-dimensional space. The horizontal axis is the first principal component, and the vertical axis is the second principal component. As Figure 4 shown, the feature quantity 701 of the teacher data D2 labeled as a qualified product, that is, the qualified product data, and the feature quantity 702 of the teacher data D2 labeled as a non-qualified product, that is, the non-qualified product data, respectively form point groups, and there is a boundary surface between the point groups. In Figure 4 the coordinate diagram shown also includes the feature quantity 703 of the teacher candidate data D3 labeled as a qualified product and the feature quantity 704 of the teacher candidate data D3 labeled as a non-qualified product extracted by the export unit 23. The teacher candidate data D3 is plotted regardless of the boundary surface.

[0062] [Calculation of the distance for qualified products and the distance for non-qualified products performed by the calculation unit]

[0063] The calculation unit 24 calculates, for each teacher candidate data D3, the distance between the teacher candidate data D3 and the qualified product data in the feature space, i.e., the qualified product distance, based on the corresponding feature quantity. As an example, as the "distance" used in the expressions of the qualified product distance and the unqualified product distance, the Euclidean distance between the data projected into the feature space can be used. As long as the distance in the feature space can be calculated, it is not limited to the Euclidean distance, and the Mahalanobis distance or the like can also be used. The distance between the teacher data k, which is one data in the teacher data D2, and the teacher candidate data s, which is one data in the teacher candidate data D3, is calculated, for example, using the following Equation 1.

[0064]

Mathematical Equation 1

[0065]

[0066] Here, q (k,i) is the coordinate of the teacher data k in a certain dimension i of the feature space, and p (s,i) is the coordinate of the teacher candidate data s in a certain dimension i of the feature space. d (k,s) is the distance between the teacher data k and the teacher candidate data s, and the vector of q k is the set of coordinate data of the teacher data k in the feature space, and the vector of p k is the set of coordinate data of the teacher candidate data s in the feature space. In addition, k is an integer less than or equal to the number of data in the teacher data (m + n: m and n are integers), i is an integer less than or equal to the number of predetermined dimensions (j) (j is an integer), and s is an integer less than or equal to the number of data in the teacher candidate data (t) (t is an integer).

[0067] When the distance from the teacher candidate data s to the qualified product data OKg, which is one of the qualified product data OK, is set as d (OKg,s) and Equation 1 is used, d is expressed as the following Equation 2. (OKg,s) . In addition, OK in OKg is a symbol indicating a qualified product, and g is an integer less than or equal to the number of data in the qualified product data OK (m).

[0068]

Mathematical Equation 2

[0069]

[0070] q (OKg,i) is the coordinate of the qualified product data Okg in the teacher data D2 in a certain dimension i of the feature space, and the vector of q OKg is the set of coordinate data of the qualified product data Okg in the feature space.

[0071] When the set of distances between the teacher candidate data s and each qualified product data OK is set as d (OK,s)When using the vector of Equation 2, d is expressed as in Equation 3 below. (OK,s) vector.

[0072]

Mathematical Formula 3

[0073]

[0074] The qualified product distance E in the teacher candidate data s (OK,s) For example, it is d (OK,s) the minimum value among the elements of the vector. That is, the qualified product distance E (OK,s) is the minimum value among the elements of the set of distances between the teacher candidate data s and each qualified product data OK, that is, d (OK,s) the minimum value among the elements of the vector. Using Equation 3, the qualified product distance E is expressed as in Equation 4 below. (OK,s) . At this time, the smaller the qualified product distance E (OK,s) , the closer the teacher candidate data s is to any one of the qualified product data OK in the feature space.

[0075]

Mathematical Formula 4

[0076]

[0077] Regarding the qualified product distance E in the teacher candidate data s (OK,s) , for example, a elements can be extracted from the smaller elements among the elements of d (OK,s) the vector, and the qualified product distance E (OK,s) is set to the average value of the a elements. a is a natural number, for example, 3. Using Equation 3, the qualified product distance E in this case is expressed as in Equation 5 below. (OK,s) . At this time, the smaller the qualified product distance E (OK,s) , the closer the teacher candidate data s is to multiple (a) qualified product data OK in the feature space, indicating the group (qualified product cluster) of the teacher candidate data s and the qualified product data OK.

[0078]

Mathematical Formula 5

[0079]

[0080] In addition, the calculation unit 24 calculates the distance between the teacher candidate data D3 and the non - qualified product data in the feature space, that is, the non - qualified product distance, for each teacher candidate data D3 according to the corresponding feature quantity. When the distance from the teacher candidate data s to the non - qualified product data N Gh in the non - qualified product data NG is set as d (NGh,s) , using Equation 1, d is expressed as in Equation 6 below. (NGh,s) . In addition, NG in NGh is a symbol indicating non - qualified products, and h is an integer less than or equal to the number of data (n) of the non - qualified product data NG.

[0081]

Mathematical Formula 6

[0082]

[0083] In addition, q (NGh,i) is the coordinate of the non-conforming data NGh in the teacher data in a certain dimension i of the feature space. The vector of q NGh is the set of coordinate data of the non-conforming data NGh in the feature space. Figure 5 is an explanatory diagram showing elements of the distance of conforming products and the distance of non-conforming products. As Figure 5 shown, for the teacher data D2 and the teacher candidate data s, d (OKk,s) and d (NGk,s) are calculated.

[0084] When the set of distances between the teacher candidate data s and each non-conforming data NG is the vector of d (NG,s) , using Equation 6, the vector of d (NG,s) is expressed as Equation 7 below. Figure 6 is an explanatory diagram showing elements of the distance of conforming products and the distance of non-conforming products. In Figure 6 , the vector of d (OK,s+1) and the vector of d (NG,s+1) for a certain teacher candidate data s+1 are shown.

[0085]

Mathematical Formula 7

[0086]

[0087] The non-conforming product distance E in the teacher candidate data s (NG,s) is, for example, the minimum value among the elements of the vector of d (NG,s) . That is, the non-conforming product distance E (NG,s) is the minimum value among the distances between the teacher candidate data s and each non-conforming data NG. Using Equation 7, the non-conforming product distance E (NG,s) is expressed as Equation 8 below. At this time, the smaller the non-conforming product distance E (NG,s) , the closer the teacher candidate data s is to any one of the non-conforming data NG in the feature space. Figure 7 is an explanatory diagram showing the distance of conforming products and the distance of non-conforming products. In Figure 7 , it shows the case where, in the teacher candidate data s+1, the minimum value of the distance from the conforming product data OK and the minimum value of the distance from the non-conforming product data NG are the conforming product distance E (OK,s+1) and the non-conforming product distance E (NG,s+1) respectively.

[0088]

Mathematical Formula 8

[0089]

[0090] Regarding the defective distance E in the teacher candidate data s (NG,s) , for example, a elements can be extracted from the smaller elements among the elements of the vector of d (NG,s) , and the defective distance E (NG,s) is set to the average value of the a elements. Using Equation 7, the defective distance E in this case is expressed as Equation 9 below (NG,s) . At this time, the smaller the defective distance E (NG,s) , the closer the teacher candidate data s is to multiple (a) defective data NG in the feature space, indicating that the teacher candidate data s is closer to the group (defective cluster) of defective data NG

[0091]

Mathematical Equation 9

[0092]

[0093] In addition, the calculation unit 24 calculates the evaluation value E of the teacher candidate data s using the calculated non-defective distance E (OK,s) and the defective distance E (NG,s) . The evaluation value E s is, for example, the value obtained by dividing the non-defective distance E s by the defective distance E (OK,s) , and is expressed as Equation 10 below (NG,s) .

[0094]

Mathematical Equation 10

[0095]

[0096] For example, the smaller the evaluation value E s is than 1, the smaller the non-defective distance E (OK,s) is than the defective distance E (NG,s) , indicating that the teacher candidate data s is data closer to the non-defective cluster than the defective cluster. Therefore, when the teacher candidate data s is data with a defective label, the smaller the evaluation value E s , the more difficult it is to classify the teacher candidate data s as a non-defective label or a defective label in the models M1, M2, M3 according to the learning results of the current teacher data D2, and the higher the learning effect for the models M1, M2, M3

[0097] On the other hand, for example, the larger the evaluation value E s is than 1, the smaller the defective distance E (NG,s) is than the non-defective distance E (OK,s), it indicates that the teacher candidate data s is data closer to the defective product cluster compared to the qualified product cluster. Therefore, when the teacher candidate data s is data with a qualified product label, the evaluation value E s The larger it is, the more it indicates that the teacher candidate data s is data that is difficult to be classified as a qualified product label or a defective product label in the models M1, M2, and M3 according to the learning results of the current teacher data D2, and it is data with high learning effectiveness for the models M1, M2, and M3.

[0098] In addition, the evaluation value can also be the defective product distance E (NG,s) divided by the qualified product distance E (OK,s) to obtain a value. In this case, the above determination becomes the opposite. That is, the evaluation value E s The larger it is than 1, the shorter the qualified product distance E (OK,s) is compared to the defective product distance E (NG,s) , indicating that the teacher candidate data s is data closer to the qualified product cluster compared to the defective product cluster. And, the evaluation value E s The smaller it is than 1, the shorter the defective product distance E (NG,s) is compared to the qualified product distance E (OK,s) , indicating that the teacher candidate data s is data closer to the defective product cluster compared to the qualified product cluster. In addition, the evaluation value can also be set to the value obtained after performing a specified arithmetic operation on the value obtained by performing the division operation as described above.

[0099] [Method for Selecting Teacher Candidate Data by the Selection Unit]

[0100] The selection unit 25 selects the additional teacher data D4 from the teacher candidate data D3 based on at least one of the qualified product distance E (OK,s) , the defective product distance E (NG,s) and the evaluation value E s . Here, for the learning of the weight coefficients in the neural network 400, the learning effectiveness of the teacher candidate data s that is difficult for the neural network 400 to easily identify is high, and it can shorten the time required for learning. Therefore, the selection unit 25 is required to select the data to be added (the additional teacher data D4) from the teacher candidate data D3 as the teacher data D2 according to the level of learning effectiveness.

[0101] First, the following method will be described: In the selection unit 25, the shorter the qualified product distance E (OK,s) of each teacher candidate data given a defective product label, the higher the probability that the teacher candidate data is selected from at least one teacher candidate data D3. Here, when the qualified product distance E (OK,s) is less than a specified threshold, the shorter the qualified product distance E (OK,s)The shorter the teacher candidate data with a nonconforming product label is, the higher the probability that the teacher candidate data is selected from the teacher candidate data D3. For example, the selection unit 25 selects the conforming product distances E (OK,s) in ascending order from the nearest to the farthest (OK,s) teacher candidate data with a nonconforming product label that is less than a specified threshold until the upper limit number of the predetermined additional teacher data D4 is reached. In Figure 4 , the feature quantity 705 of the teacher candidate data with a nonconforming product label extracted by the derivation unit 23 is projected onto a two-dimensional space. It shows a case where the neural network 400 in the stage of being processed by the teacher data D2 cannot easily identify the teacher candidate data with a nonconforming product label that is close to the conforming product data OK (conforming product cluster) with a conforming product label. In this way, the selection unit 25 can select the additional teacher data D4 with a high learning effect for the neural network 400 by selecting the additional teacher data D4 as described above. In addition, when all the teacher candidate data D3 is only data with a conforming product distance E (OK,s) above a specified threshold, the selection unit 25 determines that there is no additional teacher data D4 and causes the display unit 26 to display the determination result. The selection unit 25 can determine that there is no additional teacher data D4 and cause the display unit 26 to display the determination result when the number of teacher candidate data D3 with a conforming product distance E (OK,s) less than a specified threshold is below a certain threshold.

[0102] In addition, a method is described as follows: In the selection unit 25, the shorter the nonconforming product distance E (NG,s) of each teacher candidate data with a conforming product label is, the higher the probability that the teacher candidate data is selected from at least one teacher candidate data D3. Here, when the nonconforming product distance E (NG,s) is less than a specified threshold, the shorter the nonconforming product distance E (NG,s) of the teacher candidate data with a conforming product label is, the higher the probability that the teacher candidate data is selected from the teacher candidate data D3. For example, the selection unit 25 selects the nonconforming product distances E (NG,s) in ascending order from the nearest to the farthest (NG,s) teacher candidate data with a conforming product label that is less than a specified threshold until the upper limit number of the predetermined additional teacher data D4 is reached. In Figure 4Among them, the feature quantity 706 of the teacher candidate data with a qualified product label extracted by the extraction unit 23 is projected onto a two-dimensional space. It shows a situation where the neural network 400 in the stage of being processed with the teacher data D2 cannot easily identify the teacher candidate data with a qualified product label that is close to the unqualified product data NG (unqualified product cluster) with an unqualified product label. In this way, by selecting the additional teacher data D4 as described above, the selection unit 25 can select the additional teacher data D4 with a high learning effect for the neural network 400. In addition, when all the teacher candidate data D3 are only data with an unqualified product distance E above a specified threshold (NG,s) , the selection unit 25 determines that there is no additional teacher data D4 and causes the display unit 26 to display the determination result. The selection unit 25 may also determine that there is no additional teacher data D4 and cause the display unit 26 to display the determination result when the number of teacher candidate data D3 with an unqualified product distance E (NG,s) less than the specified threshold is below a certain threshold.

[0103] In addition, a method is described as follows: in the selection unit 25, the additional teacher data D4 is selected according to the evaluation value E S of each teacher candidate data. For example, the larger the evaluation value E s of each teacher candidate data s with a qualified product label, the higher the probability that the selection unit 25 selects the teacher candidate data from at least one teacher candidate data D3. For example, the selection unit 25 selects the teacher candidate data with a qualified product label in descending order of the evaluation value E s until the upper limit number of the predetermined additional teacher data D4 is reached. As Figure 7 shown, the teacher candidate data s with a larger evaluation value E s is equivalent to at least any one of the cases where the distance to the qualified product data OK with a qualified product label is longer and the distance to the unqualified product data NG with an unqualified product label is shorter compared to the teacher candidate data s with a smaller evaluation value E s . Therefore, it shows a situation where the neural network 400 in the stage of being processed with the teacher data D2 cannot easily identify the teacher candidate data with a qualified product label. In addition, an evaluation value E s greater than 1 indicates that the teacher candidate data s is data closer to the unqualified product cluster than the qualified product cluster. In this way, the selection unit 25 can select the additional teacher data D4 with a high learning effect for the neural network 400 by, for example, selecting the teacher candidate data with an evaluation value E s greater than 1 and with a qualified product label in descending order of the evaluation value E s as the additional teacher data D4. In addition, when all the teacher candidate data D3 are only data with an evaluation value E less than the specified thresholds In the case of the data, it is determined that there is no additional teacher data D4, and the display unit 26 displays the determination result. The selection unit 25 can also be based on an evaluation value E above a specified threshold s When the number of data of the teacher candidate data D3 is below a certain threshold, it is determined that there is no additional teacher data D4, and the display unit 26 displays the determination result.

[0104] In addition, for example, it can also be that the evaluation value E of each teacher candidate data s with a defective product label s The smaller it is, the higher the probability that the selection unit 25 selects the teacher candidate data from at least one teacher candidate data D3. For example, the selection unit 25 selects teacher candidate data with a defective product label in ascending order of the evaluation value E s until the upper limit number of the predetermined additional teacher data D4 is reached. The teacher candidate data s with a smaller evaluation value E corresponds to s Compared with the teacher candidate data s with a larger evaluation value E, it is at least any one of the cases where the distance to the defective product data NG with a defective product label is longer and the distance to the non-defective product data OK with a non-defective product label is shorter. Therefore, it shows that the neural network 400 in the stage where the teacher data D2 has been processed cannot easily identify the teacher candidate data with a defective product label. In addition, the evaluation value E s Less than 1 indicates that the teacher candidate data s is data closer to the non-defective product cluster than to the defective product cluster. In this way, the selection unit 25 selects teacher candidate data with a defective product label as the additional teacher data D4 in ascending order of the evaluation value E, for example. Thus, it is possible to select the additional teacher data D4 with a high learning effect for the neural network 400. In addition, when all the teacher candidate data D3 are only data with an evaluation value E above a specified threshold s The selection unit 25 determines that there is no additional teacher data D4 and causes the display unit 26 to display the determination result. The selection unit 25 can also be in the case where the number of teacher candidate data D3 with an evaluation value E less than the specified threshold s Is below a certain threshold, it is determined that there is no additional teacher data D4, and the display unit 26 displays the determination result. In addition, the selection unit 25 can also appropriately change the size relationship in combination with the calculation method of the evaluation value E s To select the additional teacher data D4. s s s To select the additional teacher data D4.

[0105] [Operations of the learning device and the learning line-of-sight device]

[0106] Figure 8It is a flowchart of a learning method and a learning assistance method. The learning assistance method performed by the learning assistance device 20 includes an acquisition process (S500, an example of the first step), a derivation process (S520, an example of the second step), a calculation process (S530, an example of the third step), and a selection process (S540, an example of the fourth step). The learning assistance method may also include a display process (S560), an input determination process (S570), a change process (S580), and a notification process (S590). The learning method performed by the learning device 10 has a learning process (S510) (refer to Figure 9 ).

[0107] First, as the acquisition process (S500), the teacher data acquisition unit 21 of the learning assistance device 20 acquires, for example, teacher data D2 having qualified product data OK given a qualified product label and unqualified product data NG given an unqualified product label from a data server. As the acquisition process (S500), the teacher candidate data acquisition unit 22 of the learning assistance device 20 acquires, for example, at least one teacher candidate data D3 given either a qualified product label or an unqualified product label from a data server.

[0108] As the learning process (S510), the learning unit 11 of the learning device 10 learns the teacher data D2 and adjusts the weight coefficients in the neural network 400 of the model M1. Figure 9 It is a flowchart of the learning process. As the arithmetic process (S512), the learning unit 11 causes the neural network 400 of the model M1 to learn the teacher data D2. In this arithmetic process (S512), for the teacher data D2, a qualified product score and an unqualified product score are output from the neural network 400. As the error arithmetic process (S513), the learning unit 11 calculates the error between the label given to the teacher data D2 and the score output for the teacher data D2. As the backpropagation process (S904), the learning unit 11 adjusts the weight coefficients of the intermediate layer 403 of the neural network 400 using the error calculated in the error arithmetic process (S513). As the threshold determination process (S515), the learning unit 11 determines whether the error calculated in the error arithmetic process (S513) is lower than a specified threshold. If it is determined that the error is not lower than the specified threshold (S515: No), the processes of S512 to S515 are repeated again. If it is determined that the error is lower than the specified threshold (S515: Yes), the process proceeds to the completion determination process (S906).

[0109] As a specific example of the arithmetic processing (S512) to the threshold determination processing (S515), a use case where qualified product data OK with a qualified product label "1" is input will be described. When the arithmetic processing (S512) is first performed on the teacher data D2, values such as "0.9" and "0.1" are output from the neural network 400 of the model M1 as the qualified product score and the unqualified product score, respectively. Next, in the error arithmetic processing (S513), the difference "0.1" between the qualified product label "1" and the qualified product score "0.9" is calculated. In addition, in the case of unqualified product data NG with an unqualified product label, the difference from the unqualified product score is calculated. Next, in the error propagation processing (S514), the weight coefficients of the intermediate layer 403 of the neural network 400 of the model M1 are adjusted to make the error calculated in the error arithmetic processing (S513) smaller. In the threshold determination processing (S515), the adjustment of the weight coefficients is repeated until it is determined that the error calculated in the error arithmetic processing (S513) is lower than a specified threshold. Thus, machine learning of the neural network 400 of the model M1 is performed, and the model M1 acquires the ability to classify the target data into either a qualified product label or an unqualified product label.

[0110] Next, in the completion determination processing (S516), it is determined whether the processing for all the teacher data D2 is completed. If it is determined that the processing for all the teacher data D2 is not completed (S516: No), the processing of S511 to S516 is repeated again. If it is determined that the processing for all the teacher data D2 is completed (S516: Yes), Figure 9 the flowchart ends and returns Figure 8 the flowchart.

[0111] As an export processing (S520), the export unit 23 of the learning assistance device 20 exports the feature amounts of the teacher data D2 and the teacher candidate data D3, respectively. The export unit 23 copies the model M1 that has been learned by the learning device 10 to the model M3 of the learning assistance device 20, and uses the model M3 to export the feature amounts of the teacher data D2 and the teacher candidate data D3, respectively. In addition, the export unit 23 may output the teacher candidate data D3 to the learning device 10 to cause the learning device 10 to export the feature amounts of the teacher data D2 and the teacher candidate data D3, respectively. The export unit 23 exports the feature amounts represented by the feature space of a predetermined dimension for each teacher data D2 based on the learned neural network 400 and the teacher data D2. The export unit 23 exports the feature amounts represented by the feature space of a predetermined dimension for each teacher candidate data D3 based on the learned neural network 400 and the teacher candidate data D3.

[0112] As the calculation process (S530), the calculation unit 24 calculates the qualified product distance E for each teacher candidate data D3 according to the feature amounts of the teacher data D2 and at least one teacher candidate data D3 (OK,s) and the unqualified product distance E (NG,s) for at least one of them. The calculation unit 24 calculates the qualified product distance E (OK,s) and the unqualified product distance E (NG,s) for at least one of them (s is an integer from 1 to t). In addition, as the calculation process (S530), the calculation unit 24 calculates the evaluation value E (OK,s) based on the qualified product distance E (NG,s) and the unqualified product distance E s . The calculation unit 24 calculates the evaluation value E s for all the teacher candidate data D3

[0113] As the selection process (S540), the selection unit 25 selects the additional teacher data D4 from the teacher candidate data D3 according to at least one of the qualified product distance E (OK,s) , the unqualified product distance E (NG,s) and the evaluation value E s calculated in the calculation process (S530). The selection unit 25 selects the additional teacher data D4 from the teacher candidate data D3 using a predetermined index among the qualified product distance E (OK,s) , the unqualified product distance E (NG,s) and the evaluation value E s . The selection unit 25 may also, for example, use the weighted values of the qualified product distance E (OK,s) , the unqualified product distance E (NG,s) and the evaluation value E s in combination

[0114] As the end determination process (S550), the selection unit 25 determines whether there is additional teacher data D4 to be added as the teacher data D2 among the remaining teacher candidate data D3. The case where there is no additional teacher data D4 means the case where there is no remaining teacher candidate data D3, or the case where the qualified product distance E (OK,s) , the unqualified product distance E (NG,s) and the evaluation value E s used by the selection unit 25 are equal to or greater than the respective predetermined thresholds or less than the respective thresholds, etc. In the case where it is determined that there is no additional teacher data D4 (S550: no additional teacher data), the process proceeds to the notification process (S590). In the case where it is determined that there is additional teacher data D4 (S550: there is additional teacher data), the process proceeds to the display process (S560).

[0115] When the selection unit 25 determines that there is additional teacher data D4 (S550: there is additional teacher data), as display processing (S560), the display unit 26 displays the additional teacher data D4 selected by the selection unit 25. The user can confirm the additional teacher data D4 displayed on the display unit 26.

[0116] Figure 10 (A) to Figure 10 (D) of FIG. is a diagram showing an example of the screens 610, 620, 630, and 640 displayed on the display unit 26 in the display processing (S560). In Figure 10 (A) to Figure 10 (D), an example where the subject of the additional teacher data D4 is an electronic component is shown. The additional teacher data D41 and D42 are obtained by imaging the data to which the qualified product label is given, and the additional teacher data D43 and D44 are obtained by imaging the data to which the unqualified product label is given.

[0117] Refer to again Figure 8 . As input determination processing (S570), the change unit 28 determines whether a user operation for changing the label given to the additional teacher data D4 displayed on the display unit 26 is input via the input unit 27. When it is determined that a user operation for changing the label given to the additional teacher data D4 displayed on the display unit 26 is input via the input unit 27 (S570: yes), the process proceeds to change processing (S580). When it is determined that a user operation for changing the label given to the additional teacher data D4 displayed on the display unit 26 is not input via the input unit 27 (S570: no), the selection unit 25 adds the additional teacher data D4 to the teacher data D2, and the processes of S500 to S570 are repeated again.

[0118] Figure 10 (A) and Figure 10 (B) of the additional teacher data D41 and D42 are an example of the following data: the outer shape of the subject is consistent with the characteristics of the qualified product data, but the overall tone of the subject is close to the characteristics of the unqualified product data, so the respective unqualified distances are calculated to be short. As an example, when the user determines that the tone of the subject is acceptable, the user presses the input area 611 via the input unit 27, thereby maintaining the qualified product label given to the additional teacher data D41. On the other hand, as an example, when the user determines that the tone of the subject is unacceptable, the user presses the input area 612 via the input unit 27, and thereby the change unit 28 changes the qualified product label given to the additional teacher data D42 to an unqualified product label.

[0119] Figure 10 (C) and Figure 10The additional teacher data D43 and D44 in (D) are examples of the following data: The hue of the main part of the subject is consistent with the characteristics of the non-conforming product data, but the outer shape of the subject is close to the characteristics of the conforming product data. Therefore, the respective distances to the conforming products are calculated to be short. As an example, when the user determines that the main part of the subject includes the defective part 614, the user presses the input area 611 via the input unit 27 to maintain the non-conforming product label assigned to the additional teacher data D43. On the other hand, as an example, when the user determines that the main part of the subject does not include the defective part, the user presses the input area 612 via the input unit 27, and the change unit 28 changes the non-conforming product label assigned to the additional teacher data D44 to a conforming product label. In addition, when the user is unable to determine whether to assign a conforming product label or a non-conforming product label to the additional teacher data D4, the user can also press the input area 613. In this case, the change unit 28 can cancel the addition of the additional teacher data D4 to the teacher data D2.

[0120] As the change process (S580), the change unit 28 changes the label assigned to the additional teacher data D4. The change unit 28 changes the label assigned to the additional teacher data D4 according to the user operation. After the change, the selection unit 25 adds the selected additional teacher data D4 to the teacher data D2. Then, the processes of S500 to S570 are repeated again.

[0121] When the selection unit 25 determines that there is no teacher candidate data D3 that can be selected as the teacher data D2 (S550: no additional teacher data), as the notification process (S590), the selection unit 25 notifies the user via the display unit 26 of the situation where there is no additional teacher data D4. The selection unit 25 controls the screen display of the display unit 26 for a specified time to notify the user of the situation where there is no additional teacher data D4, and after the elapse of the specified time, the Figure 8 flowchart ends.

[0122] [Program]

[0123] The learning assistance program for functioning as the learning assistance device 20 will be described. The learning assistance program has a main module, an acquisition module, an export module, a calculation module, and a selection module. The main module is the part that uniformly controls the device. The functions realized by executing the acquisition module, the export module, the calculation module, and the selection module are the same as the functions of the teacher data acquisition unit 21, the teacher candidate data acquisition unit 22, the export unit 23, the calculation unit 24, and the selection unit 25 of the learning assistance device 20 described above.

[0124] [Summary of the Embodiment]

[0125] In the learning assistance device 20 according to the present embodiment, the teacher data acquisition unit 21 and the teacher candidate data acquisition unit 22 acquire teacher data D2 and teacher candidate data D3. The derivation unit 23 derives feature amounts for each teacher data D2 and each teacher candidate data D3 according to the model M3 that has been learned using the teacher data D2. The calculation unit 24 calculates the acceptable product distance E (OK,s) and the non-acceptable product distance E (NG,s) for at least one of them. The selection unit 25 selects additional teacher data D4 from the teacher candidate data D3 according to the distance (the acceptable product distance E (OK,s) and the non-acceptable product distance E (NG,s) for at least one of them) calculated by the calculation unit 24. As an example of the learning of the weight coefficients in the neural network 400 which is the model M1, M2, M3, the learning effect of the teacher candidate data D3 that cannot be easily recognized by the neural network 400 is high, and the learning time required can be shortened. Therefore, it is required that the selection unit 25 selects the data to be added as the teacher data D2 from the teacher candidate data D3 according to the level of the learning effect. The teacher candidate data D3 with a high learning effect is the teacher candidate data with a non-acceptable product label that is close to the acceptable product data OK in the feature space, or the teacher candidate data with an acceptable product label that is close to the non-acceptable product data NG in the feature space. By using at least one of the acceptable product distance E (OK,s) and the non-acceptable product distance E (NG,s) calculated by the calculation unit 24 as an index, the selection unit 25 can improve the efficiency of the process of selecting the data to be added as the teacher data D2 from the teacher candidate data D3 according to the level of the learning effect. Thus, the learning assistance device 20 can appropriately assist the learning of the model M1. In addition, the learning assistance method and the learning assistance program also achieve the same effects as described above.

[0126] The learning device 10 can efficiently learn the model M1 (the weight coefficients in the neural network 400) using the teacher data D2 with a high learning effect selected by the selection unit 25.

[0127] The shorter the acceptable product distance E (OK,s) of the teacher candidate data with a non-acceptable product label, the higher the probability that the selection unit 25 selects the teacher candidate data from at least one teacher candidate data D3. In this case, the selection unit 25 can obtain the teacher candidate data with a high learning effect that is close to the acceptable product data OK in the feature space and has a non-acceptable product label as the teacher data D2.

[0128] The non-acceptable product distance E (NG,s)The shorter it is, the more the selection unit 25 increases the probability that the teacher candidate data is selected from at least one teacher candidate data. In this case, the selection unit 25 can obtain the teacher candidate data D3 with a high learning effect that is assigned a qualified product label and is close to the non-conforming product data NG in the feature space as the teacher data D2.

[0129] The selection unit 25 uses the qualified product distance E for each teacher candidate data D3 (OK,s) and the non-conforming product distance E (NG,s) to calculate the evaluation value E s , and selects the additional teacher data D4 from at least one teacher candidate data D3. By using both the qualified product distance E (OK,s) and the non-conforming product distance E (NG,s) , the selection unit 25 can improve the efficiency of the process of selecting the teacher candidate data D3 with a high learning effect for the neural network 400 as the teacher data D2.

[0130] The learning device 10 and the learning assistance device 20 further include a display unit 26 that displays the teacher candidate data D3 selected by the selection unit 25. Thus, the user can identify the teacher candidate data D3 with a high learning effect.

[0131] In addition, the learning assistance device 20 further includes: an input unit 27 that accepts the input of a user operation; and a change unit 28 that changes the label assigned to the teacher candidate data D3 displayed by the display unit 26 when a user operation for changing the label assigned to the teacher candidate data D3 is input to the input unit 27. Thus, the user can correct the qualified product label or non-conforming product label previously assigned to the teacher candidate data D3 while confirming the display unit 26.

[0132] In addition, when the selection unit 25 determines based on the distance that there is no data (additional teacher data D4) to be added as the teacher data D2 among at least one teacher candidate data D3, the selection unit 25 causes the display unit 26 to display the determination result. In this case, the user can identify that there is no additional teacher data D4 for the neural network 400 to learn, and can easily determine whether to end the learning of the weight coefficients.

[0133] The embodiments of the present invention have been described above. However, the present invention is not limited to the above embodiments. In the above embodiments, a structure in which the learning device 10 and the learning assistance device 20 are physically or logically separated is described. However, the learning device 10 and the learning assistance device 20 may also be combined and integrated physically or logically. That is, the learning device 10 may also be a structure that includes the learning assistance device 20.

[0134] Each structural element of the learning assistance device 20 may also be configured as an aggregate in which devices corresponding to the respective functions of the structural elements are connected via a communication network.

[0135] In the case where the learning assistance device 20 does not have the display unit 26, the learning assistance method may also not perform the display process (S560). In the case where the learning assistance device 20 does not have the input unit 27 and the change unit 28, the learning assistance method may also not perform the input determination process (S570).

[0136] Reference numeral description

[0137] 10: Learning device; 11: Learning unit; 20: Learning assistance device; 21: Teacher data acquisition unit; 22: Teacher candidate data acquisition unit; 23: Derivation unit; 24: Calculation unit; 25: Selection unit; 26: Display unit; 27: Input unit; 28: Change unit; 400: Neural network.

Claims

1. A learning assistance device, comprising: a teacher data acquisition unit that acquires teacher data, the teacher data including first data assigned with a first label and second data assigned with a second label; a teacher candidate data acquisition unit that acquires at least one teacher candidate data each assigned with any one of the first label and the second label; a derivation unit that, based on a model obtained by learning using the teacher data in a manner of classifying object data into any one of the first label and the second label and the teacher data, derives a feature quantity represented by a feature space of a predetermined dimension for each of the teacher data, and based on the model and the at least one teacher candidate data, derives a feature quantity represented by the feature space for each of the teacher candidate data; a calculation unit that, based on the feature quantity of the teacher data and the feature quantity of the at least one teacher candidate data, calculates at least one of a first distance and a second distance for each of the teacher candidate data, the first distance being a distance between the teacher candidate data and the first data in the feature space, and the second distance being a distance between the teacher candidate data and the second data in the feature space; and a selection unit that selects, as data to be appended as the teacher data, from the at least one teacher candidate data according to the distance of each of the teacher candidate data calculated by the calculation unit.

2. The learning assistance device according to claim 1, wherein the shorter the first distance of the teacher candidate data assigned with the second label is, the higher the probability that the selection unit selects the teacher candidate data from the at least one teacher candidate data is.

3. The learning assistance device according to claim 1, wherein the shorter the second distance of the teacher candidate data assigned with the first label is, the higher the probability that the selection unit selects the teacher candidate data from the at least one teacher candidate data is.

4. The learning assistance device according to any one of claims 1 to 3, wherein the calculation unit calculates an evaluation value for each of the teacher candidate data using the first distance and the second distance, and the selection unit selects, as data to be appended as the teacher data, from the at least one teacher candidate data according to the evaluation value of each of the teacher candidate data.

5. The learning assistance device according to any one of claims 1 to 3, wherein the learning assistance device further includes a display unit that displays the data selected by the selection unit.

6. The learning assistance device according to claim 4, wherein the learning assistance device further includes a display unit that displays the data selected by the selection unit.

7. The learning assistance device according to claim 5, wherein the learning assistance device further includes: an input unit that accepts input of a user operation; and A change unit that changes the label assigned to the data displayed by the display unit when a user operation for changing the label assigned to the data is input to the input unit.

8. The learning assistance device according to claim 5, wherein when it is determined based on the first distance and the second distance that there is no data to be added as the teacher data among the at least one teacher candidate data, the selection unit causes the display unit to display the determination result.

9. A learning device, comprising: a teacher data acquisition unit that acquires teacher data including first data assigned a first label and second data assigned a second label; a teacher candidate data acquisition unit that acquires at least one teacher candidate data each assigned either the first label or the second label; a derivation unit that derives, for each teacher data, a feature amount represented by a feature space of a predetermined dimension of the teacher data based on a model learned using the teacher data to classify object data into either the first label or the second label and the teacher data, and derives, for each teacher candidate data, a feature amount represented by the feature space based on the model and the at least one teacher candidate data; a calculation unit that calculates, for each teacher candidate data, at least one of a first distance and a second distance based on the feature amount of the teacher data and the feature amount of the at least one teacher candidate data, where the first distance is the distance in the feature space between the teacher candidate data and the first data, and the second distance is the distance in the feature space between the teacher candidate data and the second data; a selection unit that selects, based on the distances of each teacher candidate data calculated by the calculation unit, data to be added as the teacher data from the at least one teacher candidate data; and a learning unit that causes the model to learn using the data selected by the selection unit.

10. A learning assistance method, comprising the following steps: a first step of acquiring teacher data including first data assigned a first label and second data assigned a second label, and at least one teacher candidate data each assigned either the first label or the second label; a second step of deriving, for each teacher data, a feature amount represented by a feature space of a predetermined dimension of the teacher data based on a model learned using the teacher data to classify object data into either the first label or the second label and the teacher data, and deriving, for each teacher candidate data, a feature amount represented by the feature space based on the model and the at least one teacher candidate data; In the third step, based on the feature quantities of the teacher data and the feature quantities of the at least one teacher candidate data, at least one of a first distance and a second distance is calculated for each of the teacher candidate data. The first distance is the distance between the teacher candidate data and the first data in the feature space, and the second distance is the distance between the teacher candidate data and the second data in the feature space; And In the fourth step, based on the distances of each of the teacher candidate data calculated in the third step, data to be added as the teacher data is selected from the at least one teacher candidate data.

11. A program product comprising a learning assistance program for causing a computer to function as the learning assistance device according to any one of claims 1 to 8.

Images