Learning apparatus and learning system

Abstract

A learning device according to the present disclosure includes: a learning image acquisition unit (21) that acquires a learning image; a feature amount extraction unit (22) that calculates a shared feature space feature amount of the learning image; an existing feature amount acquisition unit (23) that acquires a pre-stored learned model and an existing feature amount corresponding to the learned model; a feature amount comparison unit (24) that calculates, as an index, the distance or similarity between the shared feature space feature amount and the existing feature amount in the shared feature space; a model selection unit (25) that selects, as a base model, one learned model suitable for the learning purpose from among the learned models on the basis of the index; a model learning unit (26) that performs re-learning of the base model; a model evaluation unit (27) that evaluates the estimation performance of the re-learned base model; and a learned model output unit (28) that outputs the re-learned base model.

Description

Technical Field

[0001] The disclosed technology relates to a learning device and a learning system. Background Art

[0002] In the technical field of machine learning, it is well known that methods of improving development efficiency by making good use of existing learning models rather than starting from scratch.

[0003] For example, Patent Document 1 describes an information processing device that uses the following method: a model that has been learned using data corresponding to the attributes of the identification object and has good recognition performance for the identification object is selected from a plurality of models. According to Patent Document 1, for example, in the learning of a model for identifying cracks, the attribute of the data is exemplified as "the type of infrastructure structure such as a bridge or a tunnel".

[0004] The attributes of data that Patent Document 1 assumes are defined by humans as users and are based on human concepts such as “bridge”, “tunnel”, and “dam”.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2021-81793 Summary of the invention

[0008] Problems to be solved by the invention

[0009] Patent document 1 focuses on the attributes of data, but the attributes of data alone cannot quantitatively indicate whether a learning model is suitable for the purpose of learning. For example, when the attributes of data are the same as "bridge", "tunnel" and "dam", but there are learning models A and B learned using learning data sets with different images themselves, it is unknown which learning model is better to use. In the prior art, in this case, it is necessary to implement recognition performance tests of both learning model A and learning model B. In this way, development efficiency is not much improved.

[0010] The purpose of the disclosed technology is to provide a learning device and a learning system that introduce an indicator that quantitatively indicates whether an existing learning model is suitable for the learning purpose, thereby improving development efficiency compared to the past.

[0011] Means for solving problems

[0012] The learning device of the disclosed technology includes: a learning image acquisition unit, which acquires a learning image; a feature extraction unit, which calculates a shared feature space feature of the learning image; an existing feature acquisition unit, which acquires a pre-stored learned model and the existing feature corresponding to it; a feature comparison unit, which calculates the distance or similarity between the shared feature space feature and the existing feature in the shared feature space as an indicator; a model selection unit, which selects one of the learned models suitable for the learning purpose as a basic model based on the indicator; a model learning unit, which relearns the basic model; a model evaluation unit, which evaluates the inference performance of the relearned basic model; and a learned model output unit, which outputs the relearned basic model.

[0013] Effects of the Invention

[0014] Since the learning device of the disclosed technology has the above-mentioned structure, it is possible to quantitatively indicate whether the existing learning model is suitable for the learning purpose, and the development efficiency is better than before. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is an explanatory diagram showing the configuration of a learning system according to the first embodiment.

[0016] Figure 2 This is a block diagram showing the functional structure of the learning device according to the first embodiment.

[0017] Figure 3 This is an explanatory diagram showing the hardware configuration of the learning device according to the first embodiment.

[0018] Figure 4 This is a flowchart showing the processing procedure of the learning device according to the first embodiment. DETAILED DESCRIPTION

[0019] It can be said that the problem to be solved by the learning system 1 of the disclosed technology is a kind of optimization problem, just like the general machine learning problem. Regarding optimization problems, the no free lunch theorem is well known. In simple terms, the no free lunch theorem ideally states that a model with high generality and high performance at the same time is theoretically impossible. A certain model (MA) performs better than other models (MB) because the model (MA) is more specialized or specialized for the specific problem to be solved.

[0020] It can be said that the feature of the disclosed technology is that, by design, a model that is learned generally in a wide range and a model that is learned specifically in a specific range are used separately.

[0021] Implementation Method 1

[0022] Figure 11 is an explanatory diagram showing the structure of the learning system 1 according to the first embodiment. Figure 1 As shown, the learning system 1 of the first embodiment includes a learning device 2 , an operation input device 3 , a storage device 4 , and a display output device 5 .

[0023] like Figure 1 As shown, the learning device 2, the operation input device 3, the storage device 4, and the display output device 5 can be connected to each other via an information network such as a LAN (Local Area Network).

[0024] 《Learning device 2 constituting learning system 1》

[0025] The learning device 2 constituting the learning system 1 is a device having a mathematical model (also referred to as a “learning model”) capable of performing supervised learning based on a learning data set composed of learning images and the like.

[0026] The details of the learning device 2 will become clear from the following description.

[0027] <<Operation input device 3 constituting learning system 1>>

[0028] The operation input device 3 constituting the learning system 1 is a user interface that receives user operations (hereinafter referred to as "user operations") and outputs corresponding operation signals. Specifically, the operation input device 3 is a keyboard, a pointing device, a touch panel, or a touch sensor.

[0029] An operation signal output from the operation input device 3 is sent to the learning device 2 .

[0030] <Storage device 4 constituting learning system 1>

[0031] The storage device 4 constituting the learning system 1 is a device for storing information necessary for the processing performed by the learning device 2. Figure 1 As shown, the storage device 4 is provided outside the learning device 2. The memory 7 described later is located inside the learning device 2, and the two are distinguished in this specification.

[0032] like Figure 1 As shown, the storage device 4 can be accessed from the learning device 2 . Therefore, the learning device 2 can read out information stored in the storage device 4 , and can also store information in the storage device 4 .

[0033] <Display output device 5 constituting learning system 1>

[0034] The display output device 5 constituting the learning system 1 is a user interface that provides visual information to the user. Specifically, the display output device 5 is a display. The visual information to be provided to the user is transmitted from the learning device 2.

[0035] Figure 2 1 is a block diagram showing the functional structure of the learning device 2 according to the first embodiment. Figure 2 As shown, the learning device 2 includes a learning image acquisition unit 21, a feature extraction unit 22, an existing feature acquisition unit 23, a feature comparison unit 24, a model selection unit 25, a model learning unit 26, a model evaluation unit 27 and a learned model output unit 28.

[0036] <Learning Image Acquisition Unit 21 in Learning Device 2>

[0037] The learning image acquisition unit 21 in the learning device 2 is a component that acquires a learning data set (for example, a learning image).

[0038] Generally, there are a plurality of learning images. For example, it is assumed that the learning images are stored in the storage device 4. The learning image acquisition unit 21 acquires the learning images stored in the storage device 4.

[0039] The learning image acquired by the learning image acquisition unit 21 is sent to the feature amount extraction unit 22 .

[0040] <Feature Quantity Extraction Unit 22 in Learning Device 2>

[0041] The feature extraction unit 22 in the learning device 2 is a component that extracts feature values ​​from the transmitted learning image. Here, feature value (Feature) is a term generally used in the technical field of machine learning. In simple terms, it is numerical data that expresses the characteristics of an object or phenomenon. More strictly speaking, feature value (Feature) is a characteristic or attribute of data that can be used by a model for prediction.

[0042] The feature extraction unit 22 functions as an AI device that has performed pre-learning in image recognition, that is, has previously learned through learning how to convert the target data into a representation that makes it easy to solve the problem. The representation method obtained through this representation learning is the feature.

[0043] In order to distinguish from the feature quantity (Feature) used as a common noun, the feature quantity extracted by the feature quantity extraction unit 22 is referred to as a "shared feature space feature quantity" in this specification. The shared feature space feature quantity can also be considered as a vector defined in the "shared feature space". Generally speaking, when certain image data is input into a learning device, the feature quantity (Feature) obtained as an intermediate product may change due to the learning process (progress). In contrast, the shared feature space feature quantity defined in the disclosed technology does not change as a representation of certain image data.

[0044] If we use a metaphor, the shared feature space is like the world map. Figure 1 Although a world map is suitable for publishing information over a wide range and looking down, it is not suitable for investigating detailed information. The details of how to use the shared feature space in the disclosed technology will become clear through the following description.

[0045] The feature extraction unit 22 has a mathematical model for extracting the feature of the shared feature space. A well-known mathematical model is an artificial neural network (Artificial Neural Network, sometimes also referred to as "neural network" for short).

[0046] In the learning device 2 of the disclosed technology, the mathematical model of the feature extraction unit 22 is preferably ResNet learned using ImageNet, which is a general image data set. The following documents can be used as references for ResNet.

[0047] "Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun", "Deep ResidualLearning for Image Recognition", "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016pp.770-778".

[0048] Generally speaking, features include local features in the image to global features. Local features should also be considered to include features about edges and features about partial shapes. Global features should also be considered to include, for example, characteristics specific to the category to which the supervisory label belongs.

[0049] The fully learned learning model (called the "learned model") extracts appropriate features to achieve the learning purpose. In this way, the learned model extracts the features required to achieve the learning purpose, and therefore may not be consistent with the features of the image that people think of, such as "metal-like texture", "triangle-shaped as a whole", "there is a dog", etc. This is because "metal", "texture", "triangle", "dog", etc. are concepts created by people, and mathematical models do not have these concepts. Artificial intelligence with a learned model simply extracts appropriate features (Feature) to achieve the learning purpose through machine learning.

[0050] The feature extraction unit 22 may also use a method called BOW (Bag Of Visual Words) to extract the shared feature space feature. BOW is a method of compressing image dimensions into a one-dimensional vector (also called a "feature vector"). BOW is sometimes also called Bag of Words or Bag of Features. The following documents can be used as references for BOW.

[0051] "Csurk, Gabriella, et al.", "Visual categorization with bags of keypoints.", Workshop on statistical learning in computer vision, ECCV.Vol.1.No.1-22.2004.

[0052] Thus, the feature extraction unit 22 can use a known learning model. Recently, applications that can be searched by simply taking a picture or holding up a camera have become popular, and the feature extraction unit 22 can adopt the technology used in such applications. In addition, the feature extraction unit 22 can also use a known feature extraction method.

[0053] However, as mentioned above, since the shared feature space is like the world map Figure 1 Therefore, in order to make it universal, it is preferred to use a learning model that has been pre-learned using a general image with many classification levels and a wide range. In addition, it is desired that the shared feature space feature quantity does not change as a representation of a certain image data. For this reason, the learning model for extracting the shared feature space feature quantity by the feature quantity extraction unit 22 is not learned, but the parameters are fixedly used.

[0054] The shared feature space feature extracted by the feature extraction unit 22 is sent to the feature comparison unit 24 together with the original learning image.

[0055] <Existing Feature Quantity Acquisition Unit 23 in Learning Device 2>

[0056] The existing feature quantity acquisition unit 23 in the learning device 2 is a component that acquires the “existing feature quantity” stored in advance in the storage device 4 .

[0057] The learning system 1 of the disclosed technology has several types of learned models in advance in the storage device 4. These learned models are pre-stored in the storage device 4 as candidates for learning models selected according to the learning purpose. In contrast to the learned models that are widely learned according to the general purpose of the feature extraction unit 22, the learned models stored in the storage device 4 are learned models that are specialized and learned according to a specific purpose. The learned models stored in the storage device 4 literally mean "already learned models", but are further learned (referred to as "relearning") in the model learning unit 26 described later. The details of the relearning will become clear through the description described later.

[0058] The learning system 1 of the disclosed technology adds an index to the learned model stored in the storage device 4 , the basis of which is the existing feature quantity.

[0059] For example, it is assumed that the learned model (MP) stored in the storage device 4 is a model learned specifically for image recognition of plant flowers. When image data is input, the learned model (MP) can also generate a feature quantity (Feature) as an intermediate product. If expressed metaphorically, the feature space in the learned model (MP) is a detailed map related only to a specific region. The feature quantity (Feature) generated by the learned model (MP) is a quantity that is defined only in the learned model (MP) and has meaning only in the feature space, and cannot be used as an indicator for comparison with other models.

[0060] For example, suppose the learned model (MP) is learned as Another learned model (MV) is a model learned specifically for vertebrates and is learned as The meanings and dimensions of the elements of the feature vector obtained as an intermediate product are generally not uniform.

[0061] The disclosed technology focuses on what kind of learning data set is used to learn the learned model (MP). Moreover, the disclosed technology calculates the shared feature space feature values ​​for each image used in the learning of the learned model (MP). The calculation of the shared feature space feature values ​​here is pre-calculated using the same means as the feature value extraction unit 22. The representative points of the shared feature space feature values ​​obtained here are stored in the storage device 4 as existing feature values. By calculating the shared feature space feature values ​​in this way, the learned model (MP) can be represented as the shared feature space feature values ​​of the image used for learning drawn in the shared feature space. If expressed metaphorically, the learned model (MP) can be represented as a plot distributed in the area corresponding to "plant flowers" in the shared feature space as a world map.

[0062] For example, the disclosed technology uses a representative point such as the centroid of a shared feature space feature quantity drawn in a shared feature space as an existing feature quantity that determines an index of a learned model (MP). If expressed metaphorically, the existing feature quantity of the learned model (MP) can be said to be the center of the "plant flower" area in the shared feature space as a world map.

[0063] The learned models stored in the storage device 4 are considered to be specialized models of insects, birds, animals, great men, national flags, fine arts, and other subjects that can be seen in illustrations in addition to plants and flowers. Furthermore, the learned models stored in the storage device 4 are also considered to be specialized models of unmanned cash registers, specialized models of character recognition such as OCR (Optical Character Recognition), specialized models of biometric authentication, specialized models of three-dimensional shape recognition, specialized models of robot control, specialized models of autonomous driving, specialized models of prevention, and specialized models of various other fields.

[0064] The candidates for the learning model and the existing feature quantities acquired by the existing feature quantity acquisition unit 23 are sent to the feature quantity comparison unit 24 .

[0065] <Feature Quantity Comparison Unit 24 in Learning Device 2>

[0066] The feature quantity comparison unit 24 in the learning device 2 is a component for comparing shared feature space feature quantities in the shared feature space. More specifically, the feature quantity comparison unit 24 performs a process of calculating distances between two or more shared feature space feature quantities (vectors) drawn in the shared feature space.

[0067] In other words, the fact that the distance between two shared feature space feature quantities in the shared feature space is close means that the characteristics or attributes of the original two images are close.

[0068] The starting point of the distance in the shared feature space calculated by the feature comparison unit 24 is the shared feature space feature extracted by the feature extraction unit 22. The end point of the distance calculated by the feature comparison unit 24 is a plurality of existing feature values ​​corresponding to the plurality of learned models acquired by the existing feature acquisition unit 23.

[0069] As described above, there may be a plurality of learning images input to the learning image acquisition unit 21, and therefore, the shared feature space feature quantities associated with the learning images plotted in the shared feature space form a distribution. Therefore, the feature quantity comparison unit 24 may use the Mahalanobis distance often used in statistics as the distance in the shared feature space.

[0070] The essence of the processing performed by the feature value comparison unit 24 is to search for a learned model learned using image data with similar characteristics or attributes from the input learning image. From this point of view, the value calculated by the feature value comparison unit 24 is not limited to "distance" such as Euclidean distance or Mahalanobis distance. The feature value comparison unit 24 can also calculate a value using a similarity concept such as cosine similarity.

[0071] The values ​​(distance, similarity, etc.) calculated by the feature value comparison unit 24 for each learned model are sent to the model selection unit 25 as an index for selecting a learned model suitable for the purpose of learning.

[0072] 《Model selection unit 25 in learning device 2》

[0073] The model selection unit 25 in the learning device 2 is a structural element that directly selects a learned model determined to be most suitable for the learning purpose based on the index sent from the feature value comparison unit 24, or proposes candidates of the learned model to be selected to the user.

[0074] When the user's judgment is not necessary, specifically, the model selection unit 25 directly selects a learned model with the shortest distance or the highest similarity in the shared feature space based on the index sent from the feature quantity comparison unit 24 .

[0075] When the learned model to be used is determined based on the user's judgment, the model selection unit 25 lists the learned models in order of distance from near to far in the shared feature space or in order of similarity from high to low, and displays them on the display output device 5. The user operates the input device 3 to answer the question of which learned model to use from the list displayed on the display output device 5.

[0076] The learned model directly or indirectly selected in the model selection unit 25 is sent to the model learning unit 26 as a base model.

[0077] 《Model learning unit 26 in learning device 2》

[0078] The model learning unit 26 in the learning device 2 is a component that relearns the base model selected by the model selection unit 25. The learning data set used by the model learning unit 26 to relearn the base model may be the learning image acquired by the learning image acquisition unit 21.

[0079] As described above, the learning images used to calculate the shared feature space feature quantity in the feature quantity extraction unit 22 may be one or more learning images acquired via the learning image acquisition unit 21. On the other hand, the model learning unit 26 excludes the learning images retained for evaluation from among the learning images acquired via the learning image acquisition unit 21, and in principle uses all the remaining learning images for re-learning.

[0080] The relearned basic model and the learning images retained for evaluation are sent to the model evaluation unit 27 .

[0081] 《Model evaluation unit 27 in learning device 2》

[0082] The model evaluation unit 27 in the learning device 2 is a component that evaluates the estimation performance of the relearned basic model using the learning images reserved for evaluation.

[0083] The evaluation result performed by the model evaluation unit 27 may be displayed on the display output device 5 to inform the user. In addition, the learning system 1 of the disclosed technology may be configured to display the learning performance evaluation on the display output device 5 together with the estimation performance evaluation.

[0084] The user determines whether to complete the re-learning or to add a learning data set and continue the re-learning based on the evaluation result displayed on the display output device 5 .

[0085] When it is determined that the re-learning is completed, the re-learned basic model is sent to the learned model output unit 28 .

[0086] <<Learned model output unit 28 in learning device 2>>

[0087] The learned model output unit 28 in the learning device 2 is a component that stores the relearned basic model in the storage device 4 .

[0088] Figure 3 This is an explanatory diagram showing the hardware configuration of the learning device 2 according to the first embodiment.

[0089] In the learning device 2 of the first embodiment, the functions of the learning image acquisition unit 21, the feature extraction unit 22, the existing feature acquisition unit 23, the feature comparison unit 24, the model selection unit 25, the model learning unit 26, the model evaluation unit 27 and the learned model output unit 28 are implemented by a processing circuit. The processing circuit may be a processor 6 (see Figure 3 A), or a processing circuit 8 as dedicated hardware (see Figure 3 B).

[0090] Figure 3A is a hardware configuration diagram when each function of the learning device 2 is executed by software. Figure 3 In the case of the hardware structure shown in A, the various functions of the learning device 2 are implemented by software, firmware, or a combination of software and firmware. Software and firmware are described as programs and stored in the memory 7. The processor 6 (sometimes also referred to as a CPU, a central processing unit, a processing unit, a microprocessor, a microcomputer, or a DSP) implements the functions of each part by reading out and executing the program stored in the memory 7. That is, the learning device 2 has a memory 7, which is used to store programs that, when executed by the processor 6, result in the execution of the various functions of the learning device 2. In addition, it can also be said that these programs cause the processor 6 to execute the process and method of the learning device 2. Here, the memory 7 can be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, an EPROM, etc. In addition, the memory 7 can also be a structure having a disk such as a magnetic disk, a floppy disk, an optical disk, a high-density disk, a mini disk, a DVD, etc. Furthermore, the memory 7 can also be in the form of an HDD or an SSD.

[0091] Figure 3 B is a hardware configuration diagram in which each function of the learning device 2 is executed by dedicated hardware. Figure 3 In the case of the hardware structure shown in B, the processing circuit 8 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. Each function of the learning device 2 can be implemented by a separate processing circuit 8 or by a single processing circuit 8.

[0092] In addition, a part of each function of the learning device 2 may be implemented by software or firmware, and the rest may be implemented by dedicated hardware. In this way, the learning device 2 can implement each function by hardware, software, firmware, or a combination thereof.

[0093] Figure 4 : is a flowchart showing the processing steps of the learning device 2 according to the first embodiment. Figure 4 As shown, the processing steps of the learning device 2 of embodiment 1 include: a processing step of obtaining a learning image (ST201), a processing step of extracting a feature value (ST202), a processing step of obtaining an existing feature value (ST203), a processing step of calculating a similarity (ST204), a processing step of selecting a model (ST205), a processing step of relearning a model (ST206), a processing step of evaluating a model (ST207), and a processing step of outputting learned model information (ST208).

[0094] The learning image acquisition process step ( ST201 ) is a process step performed by the learning image acquisition unit 21 .

[0095] The feature extraction process step ( ST202 ) is a process step performed by the feature extraction unit 22 .

[0096] The process step of acquiring the existing feature amount ( ST203 ) is a process step performed by the existing feature amount acquisition unit 23 .

[0097] The similarity calculation process step ( ST204 ) is a process step performed by the feature quantity comparing unit 24 .

[0098] The model selection process step ( ST205 ) is a process step performed by the model selection unit 25 .

[0099] The model re-learning process ( ST206 ) is a process performed by the model learning unit 26 .

[0100] The model evaluation process step ( ST207 ) is a process step performed by the model evaluation unit 27 .

[0101] The processing step of outputting the learned model information ( ST208 ) is a processing step performed by the learned model output unit 28 .

[0102] One of the excellent effects of the learning system 1 of the disclosed technology is that it proposes distinguishing between the use of a model learned generally in a wide range and a model learned specifically in a specific range.

[0103] One of the excellent effects of the learning system 1 of the disclosed technology is that, by defining a shared feature space, an index (distance, similarity) that quantitatively indicates whether an existing learning model is suitable for the learning purpose is proposed.

[0104] One of the excellent effects of the learning system 1 of the disclosed technology is that unnecessary recognition performance tests can be reduced, thereby improving development efficiency compared to the past.

[0105] Industrial Applicability

[0106] The learning system 1 and the learning device 2 of the disclosed technology can be widely applied to the technical field of image recognition and have industrial applicability.

[0107] Description of symbols

[0108] 1: Learning system; 2: Learning device; 3: Operation input device; 4: Storage device; 5: Display output device; 6: Processor; 7: Memory; 8: Processing circuit; 21: Learning image acquisition unit; 22: Feature extraction unit; 23: Existing feature acquisition unit; 24: Feature comparison unit; 25: Model selection unit; 26: Model learning unit; 27: Model evaluation unit; 28: Learned model output unit.

Claims

1. A learning device, wherein: The learning device includes: a learning image acquisition unit that acquires a learning image; a feature extraction unit that calculates a shared feature space feature of the learning image; an existing feature quantity acquisition unit that acquires a pre-stored learned model and an existing feature quantity corresponding thereto; a feature quantity comparison unit that calculates, in a shared feature space, a distance or similarity between the shared feature space feature quantity and the existing feature quantity as an index; A model selection unit that selects one of the learned models that is suitable for the learning purpose as a basic model based on the indicator; A model learning unit, which relearns the basic model; a model evaluation unit that evaluates the inference performance of the relearned basic model; and A learned model output unit outputs the relearned basic model.

2. A learning system comprising a learning device, an operation input device, a storage device and a display output device connected to each other, wherein: The learning device comprises: a learning image acquisition unit that acquires a learning image; a feature extraction unit that calculates a shared feature space feature of the learning image; an existing feature quantity acquisition unit that acquires a pre-stored learned model and an existing feature quantity corresponding thereto; a feature quantity comparison unit that calculates, in a shared feature space, a distance or similarity between the shared feature space feature quantity and the existing feature quantity as an index; A model selection unit that selects one of the learned models that is suitable for the learning purpose as a basic model based on the indicator; A model learning unit, which relearns the basic model; a model evaluation unit that evaluates the inference performance of the relearned basic model; and A learned model output unit outputs the relearned basic model.

Images