Computer system and triplet generation method

Abstract

To generate a triplet of multi-label data.SOLUTION: A computing system is configured to: acquire multiple pieces of input data; select one piece of input data as query input data; execute retrieval processing to retrieve similar input data which is input data similar to the query input data; analyze, based on a result of the retrieval processing, appearance frequency of each of multiple labels; calculate, based on a result of the analysis, for each of the labels, an index for evaluating a degree of relationship in characteristics between the label and the input data; calculate, based on the indices, label similarity representing a degree of similarity between multiple labels added to the query input data and multiple labels added to the similar input data; and select, based on a difference in label similarity of each of the multiple pieces of similar input data, similar input data which serves as a negative instance and a positive instance, to generate a triplet.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a technique for generating training data for use in a triplet network. [Background technology]

[0002] One of the training methods for deep neural networks is the triplet network, which trains a model (e.g., a neural network) that outputs feature vectors in which the distance between data with the same label is small and the distance between data with different labels is large.

[0003] FIG. 8 is a diagram showing the concept of learning in a conventional triplet network. In a triplet network, learning is performed using a triplet 800 consisting of reference data (anchor) 801, data (positive case) 802 to which the same label as that of the anchor 801 has been assigned, and data (negative case) 803 to which a label different from that of the anchor 801 has been assigned. Each piece of data 801, 802, and 803 that make up the triplet 800 is input to a neural network 810 with the same weights. The neural network 810 outputs a feature vector 820 corresponding to each piece of data 801, 802, and 803. For one triplet 800, the feature vector 820 is used to calculate the distance D between the anchor 801 and the positive case 802. + and the distance D between the anchor 801 and the negative example 803 - The distance D between the multiple triplets 800 is calculated. + and distance D - The neural network 810 is trained using the sum of the differences between the above as a loss function, which is the objective function of the training.

[0004] For example, Non-Patent Document 1 describes a technique for learning feature quantities for performing personal authentication using face images using a triplet network.

[0005] There are countless combinations of data that make up triplets for a learning set. Therefore, it is important to construct triplets that are suitable for efficient learning. In response to this, Non-Patent Document 2 describes a method for generating triplets that realize efficient learning using a similar vector search process. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Schroff, Florian, Dmitry Kalenichenko, James Philbin. "Facenet: A unified embedding for face recognition and clustering", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2015) [Non-patent document 2] Yasumitsu Ikeura, Koichi Okamoto, Ryohei Kashima, and Atsushi Hiroike: Multimodal Deep Learning Platform for IoT Data, Hitachi Review, 102-3, 119-123 (2020) Summary of the Invention [Problem to be solved by the invention]

[0007] Conventional techniques assume that one label is assigned to one piece of data. Therefore, in the case of data with multiple labels (multi-label data), the relationship between positive and negative examples cannot be uniquely defined.

[0008] In addition, labels assigned to data may not adequately represent the characteristics of the data. For example, consider a case where multiple users assign search keywords to data such as images. If each user assigns keywords at their own discretion, the keywords may not necessarily represent the characteristics of the data.

[0009] An object of the present invention is to provide a technique for generating triplets to be used in training a triplet network for a model that handles multi-label data. [Means for solving the problem]

[0010] A representative example of the invention disclosed in the present application is as follows: That is, a computer system having at least one computer retains information of a model for calculating a feature vector of input data assigned with a plurality of labels, acquires a plurality of the input data, selects one of the input data as query input data, executes a search process to search for similar input data that is input data similar to the query input data based on the feature vector, calculates, for each of the plurality of labels based on a result of the search process, an index for evaluating a degree of relationship between the label and characteristics of the input data, calculates, based on the index, label similarities indicating a degree of similarity between the plurality of labels assigned to the query input data and the plurality of labels assigned to the similar input data, selects the similar input data to be a negative example and the similar input data to be a positive example based on a difference in the label similarities between each of the plurality of similar input data, and generates a triplet consisting of the query input data and the selected two similar input data. [Effects of the Invention]

[0011] According to the present invention, triplets to be used for training a triplet network can be generated for a model that handles multi-label data. Problems, configurations, and effects other than those described above will become clear from the following description of the embodiments. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 2 is a diagram illustrating an example of a functional configuration of a learning device according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a learning device according to a first embodiment. [Figure 3] 10 is a flowchart illustrating an example of a process for generating learning data according to the first embodiment. [Figure 4] 4 is a flowchart illustrating an example of a learning process executed by the learning device according to the first embodiment. [Figure 5] 10 is a flowchart illustrating an example of triplet generation processing executed by the learning device according to the first embodiment. [Figure 6] FIG. 4 is a diagram illustrating details of a triplet generation process executed by the learning device of the first embodiment. [Figure 7] 11 is a flowchart illustrating an example of triplet generation processing executed by the learning device according to the third embodiment. [Figure 8] FIG. 1 is a diagram illustrating the concept of learning a conventional triplet network. DETAILED DESCRIPTION OF THE INVENTION

[0013] Examples of the present invention will be described below with reference to the drawings. However, the present invention should not be construed as being limited to the description of the following examples. Those skilled in the art will readily understand that the specific configuration can be modified without departing from the spirit or scope of the present invention. In the configurations of the invention described below, identical or similar components or functions are designated by the same reference numerals, and redundant description will be omitted. The designations "first," "second," "third," and the like in this specification are used to identify components and do not necessarily limit the number or order. The position, size, shape, and range of each component shown in the drawings may not represent the actual position, size, shape, and range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not limited to the position, size, shape, and range, etc., disclosed in the drawings, etc. [Example]

[0014] In the first embodiment, a system is assumed that handles image data to which a plurality of words are assigned as labels. Fig. 1 is a diagram illustrating an example of a functional configuration of a learning device according to the first embodiment. Fig. 2 is a diagram illustrating an example of a hardware configuration of the learning device according to the first embodiment.

[0015] The learning device 100 uses a triplet network to learn a model that outputs feature vectors used for classifying multi-label data, etc. As shown in FIG. 2, the learning device 100 has a processor 201, a main memory device 202, a secondary memory device 203, and a network interface 204. The hardware elements are connected to each other via a bus. The learning device 100 may have input devices such as a keyboard, a mouse, and a touch panel, and may also have output devices such as a display and a printer.

[0016] The processor 201 executes a program stored in the main memory device 202. The processor 201 executes processing in accordance with the program, thereby operating as a functional unit (module) that realizes a specific function. In the following description, when a processing is described using a functional unit as the subject, it indicates that the processor 201 is executing a program that realizes the functional unit.

[0017] The main memory device 202 is a storage device such as a DRAM (Dynamic Random Access Memory) and stores programs executed by the processor 201 and data used by the programs. The main memory device 202 is also used as a work area. The secondary memory device 203 is a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) and permanently stores data.

[0018] The programs and data stored in the main memory device 202 may be stored in the secondary memory device 203. In this case, the processor 201 reads the programs and data from the secondary memory device 203 and loads them into the main memory device 202.

[0019] The learning device 100 includes an inference unit 101, a similar vector search unit 102, a triplet generation unit 103, a model update unit 104, and a model storage unit 105. The learning device 100 also stores a dataset 110 and model information 120.

[0020] The dataset 110 is composed of a plurality of training data 111. The training data 111 includes images and a plurality of labels.

[0021] The model information 120 is information for managing a model that outputs a feature vector of an image. The model is, for example, a deep neural network. The model information 120 stores information about the structure and weights of the network. The weights of the deep neural network at the start of learning may be set randomly, but are assumed to be set to arbitrary values ​​in advance to improve learning efficiency.

[0022] The dataset 110 and the model information 120 may be set in advance in the learning device 100, obtained from an external source, or input by the user.

[0023] The inference unit 101 inputs images included in the training data 111 into a model defined by the model information 120, and outputs a feature vector.

[0024] The similar vector search unit 102 searches for similar feature vectors in the feature space.

[0025] The triplet generation unit 103 generates triplets by selecting a set of images (learning data 111) that are anchors, positive examples, and negative examples based on the search results.

[0026] The model update unit 104 executes a model learning process using triplets to update the model. Note that the present invention may be applied to any learning method using triplets.

[0027] The model storage unit 105 reflects the learning result in the model information 120. The model storage unit 105 may output the learning result to the outside as necessary.

[0028] In the first embodiment, the learning data 111 including a plurality of labels is assumed to be prepared in advance, but may be generated using document data including images and text.

[0029] The functions of the learning device 100 may be realized using a computer system consisting of multiple computers.

[0030] FIG. 3 is a flowchart showing an example of the process of generating the training data 111 in the first embodiment. Here, the description is given assuming that the training device 100 receives multiple document data and executes the following process. Here, the document data includes text and one or more images. The text is composed of multiple paragraphs, and the images are assigned identification numbers. Note that the following process is executed for one document data.

[0031] The learning device 100 selects an image included in the document data (step S101).

[0032] Learning device 100 analyzes the document for each paragraph and extracts the paragraph containing the identification number of the selected image as related text (step S102). Learning device 100 stores the image associated with the related text in a work area. For example, in the case of document data in HTML format, text containing a tag indicating a link to the image is extracted as related text.

[0033] The learning device 100 determines whether or not processing has been completed for all images included in the document data (step S103).

[0034] If processing has not been completed for all images included in the document data, the learning device 100 returns to step S101.

[0035] When processing has been completed for all images included in the document data, the learning device 100 selects an image included in the document data (step S104) and extracts words by performing morphological analysis on the related text associated with the image (step S105).

[0036] The learning device 100 generates images labeled with the extracted words as learning data 111 (step S106).

[0037] The learning device 100 determines whether or not processing has been completed for all images included in the document data (step S107).

[0038] If processing has not been completed for all images included in the document data, the learning device 100 returns to step S104.

[0039] When processing has been completed for all images included in the document data, the learning device 100 ends the learning data generation process.

[0040] The above process makes it possible to generate training data 111 containing multiple labels (words). However, the number of different labels assigned to images is enormous, making it difficult to handle as a simple label estimation problem. On the other hand, in a triplet network that handles multi-label training data, there is no constraint on the number of different labels, so it can be handled formally. However, the labels assigned to images by the above process do not necessarily represent the characteristics of the image.

[0041] In the first embodiment, label similarity is introduced, taking into consideration the association between image characteristics and labels, and the training data 111 that constitutes triplets is selected.

[0042] FIG. 4 is a flowchart illustrating an example of the learning process executed by the learning device 100 according to the first embodiment.

[0043] The inference unit 101 uses the model information 120 to execute an inference process for outputting a feature vector of the training data 111 (step S201).

[0044] The similar vector search unit 102 executes a clustering process of the feature vectors (step S202). In the clustering process, clusters of the feature vectors are generated in the feature space. This process is a preprocessing for speeding up the search for similar feature vectors. The clustering process does not necessarily have to be executed.

[0045] The triplet generation unit 103 executes triplet generation processing in cooperation with the similar vector search unit 102 (step S203). The triplet generation processing will be described in detail later.

[0046] The triplet generation unit 103 selects a triplet to be used for learning from the triplets generated in the triplet generation process (step S204). The criteria for selecting triplets will be described later. The triplet generation unit 103 outputs a triplet array with the selected triplet as an element to the model update unit 104. The array stores the triplets in an ordered manner.

[0047] The model update unit 104 updates the model using the selected triplet (step S205).

[0048] The learning device 100 determines whether or not a learning termination condition is met (step S206). The termination condition will be described later.

[0049] If the learning termination condition is not met, the learning device 100 returns to step S201 and executes the same process. If the learning termination condition is met, the learning device 100 ends the learning process.

[0050] The processing from step S201 to step S205 is defined as one learning cycle (one epoch).

[0051] Fig. 5 is a flowchart illustrating an example of triplet generation processing executed by the learning device 100 of the first embodiment. Fig. 6 is a diagram illustrating details of the triplet generation processing executed by the learning device 100 of the first embodiment.

[0052] The similar vector search unit 102 selects one query image from the images (learning data 111) (step S301). The query image may be selected randomly, or a selection method may be specified in advance. The similar vector search unit 102 assigns identification information for processing to the selected query image. Here, the identifier of the query image is represented by the letter i.

[0053] The similar vector search unit 102 searches for images similar to the query image using the feature vectors of the images (step S302). For example, the similar vector search unit 102 calculates the distance between images in the feature space using the feature vectors, and outputs images whose distance is smaller than a threshold as similar images. Note that the search process can be sped up by setting a cluster that includes the feature vector of the query image as the search range.

[0054] The similar vector search unit 102 selects R similar images (step S303). For example, the R similar images are selected in ascending order of the distance between the images in the feature space. Note that R can be set arbitrarily.

[0055] The similar vector search unit 102 determines whether to end the search (step S304). For example, if the number of searches is greater than a predetermined threshold, the similar vector search unit 102 ends the search. Note that although the search may be performed on all images, due to requirements such as limitations on the amount of calculation, the search is performed on a predetermined number of images.

[0056] The triplet generation unit 103 selects one label from the label set (step S305). Specifically, the triplet generation unit 103 generates a set of independent labels based on the labels assigned to each image, and selects one label from the set. Here, the number of labels (number of different labels) included in the label set is defined as K, and the identifier of the label included in the set is represented by the letter k.

[0057] The triplet generation unit 103 calculates the label information amount of the selected label (step S306). The label information amount is an index for evaluating the degree of association between the label and the characteristics of the image. The method for calculating the label information amount will be described below.

[0058] The triplet generation unit 103 uses formula (1) to calculate the log likelihood that an image with the selected label k will be found in a search for similar images to the query image i. Note that formula (1) assumes that the appearance frequency of images with the selected label k follows a binomial distribution.

[0059]

number

[0060] Here, r_ik represents the number of similar images with label k in the search for similar images to query image i, θ_ik represents the parameter that defines the binomial distribution, and C represents the number of combinations.

[0061] If we assume that the frequency of appearance of images assigned label k is the same as when an image is randomly selected from all images, then θ_ik will be a value that does not depend on the query image i, as shown in equation (2). Hereinafter, the value of equation (2) will be referred to as p_k. Note that N represents the number of images (learning data 111), and n_k represents the number of images assigned label k.

[0062]

number

[0063] Assuming that the frequency of appearance of images assigned the selected label k follows a distribution specific to the query image i, θ_ik is the maximum likelihood estimator that maximizes equation (1), as shown in equation (3). Hereinafter, the value of equation (3) will be referred to as q_k.

[0064]

number

[0065] The difference in log-likelihood under the two assumptions is a value that statistically represents the degree to which the frequency of appearance of similar images labeled k differs from random sampling in a search for similar images to query image i. Therefore, as shown in equation (4), f_k, which is the average difference in log-likelihood for all query images, is an index that represents the degree to which the frequency of appearance of images labeled k is affected in the search for similar images. Note that equation (4) is a formula when a search is performed for all images.

[0066]

number

[0067] In the first embodiment, f_k is treated as the label information amount of label k. In a full search, the label information amount f_k is 0 when q_k=p_k, and is a value greater than 0 in other cases.

[0068] The value f obtained by averaging the label information amounts f_k of all labels, as shown in equation (5), is an index that indicates the degree to which the search for similar images affects the frequency distribution of the labels.

[0069]

number

[0070] Next, the triplet generating unit 103 determines whether or not the processing has been completed for all labels (step S307).

[0071] If the processing has not been completed for all labels, the triplet generating unit 103 returns to step S305.

[0072] When the processing has been completed for all labels, the triplet generating unit 103 selects one query image from the images selected as query images (step S308).

[0073] The triplet generating unit 103 calculates the label similarity between the selected query image and the similar image (step S309).

[0074] Label similarity represents the similarity between images based on labels. The label similarity s(x, y) between a pair of images x and y is defined by Equation (6). Here, X represents the set of labels assigned to image x, and Y represents the set of labels assigned to image y.

[0075]

number

[0076] When set X and set Y match, the label similarity s(x, y) takes the maximum value of "1", and when set X and set Y do not match at all (when the intersection of set X and set Y is an empty set), the label similarity s(x, y) takes the minimum value of "0". The label similarity takes a larger value when labels with a larger label information content match.

[0077] The triplet generating unit 103 generates triplets based on the label similarity of each similar image (step S310).

[0078] In Example 1, we consider the difference (gap) in label similarity between similar images. The gap g(q, x, y) in label similarity is defined by Equation (7), where q represents the query image.

[0079]

number

[0080] Figure 6 shows the relationship between the label similarity between a query image and similar images. The horizontal axis represents the similarity ranking (similarity ranking) of feature vectors, and the vertical axis represents label similarity. By combining pairs of similar images with a large gap in label similarity as positive and negative examples, triplets that enable efficient learning can be generated.

[0081] Specifically, the triplet generation unit 103 selects one similar image and calculates the label similarity gap between the selected similar image and a similar image with a lower similarity rank. The triplet generation unit 103 extracts a combination of similar images that maximizes the gap in label similarity. From the extracted set of similar images, the triplet generation unit 103 selects a similar image with a lower similarity rank as a positive example, and a similar image with a higher similarity rank as a negative example. Furthermore, the triplet generation unit 103 generates a triplet including the query image, the positive example, and the negative example. At this time, the triplet generation unit 103 associates the label similarity gap with the triplet.

[0082] Note that one or more triplets are generated for one query image.

[0083] The triplet generating unit 103 determines whether or not the processing has been completed for all images selected as query images (step S311).

[0084] If the processing has not been completed for all images selected as query images, the triplet generating unit 103 returns to step S308.

[0085] When the processing is completed for all images selected as query images, the triplet generation unit 103 ends the triplet generation processing.

[0086] Here, the triplet selection method in step S204 will be described. The triplet generation unit 103 selects a predetermined number of triplets in descending order of the gap in label similarity.

[0087] Here, a method for updating the model will be described. The model update unit 104 sequentially reads triplets for a mini-batch from the triplet array and executes model learning processing.

[0088] Possible termination conditions for this implementation include when the number of epochs is greater than a threshold, or when there is no increase in the average amount of label information. If the average amount of label information does not increase even after epoch processing, the learning device 100 determines that the fluctuation in the effect of similar image search on the label frequency distribution has converged. In this case, the learning device 100 may adopt the model updated in processing the epoch in which the average amount of label information is maximized.

[0089] As described above, according to the first embodiment, by introducing label similarity into multi-label data, triplets useful for learning a model that outputs a feature vector of an image can be generated.

[0090] In the first embodiment, an image is used as an example of multi-label data, but the present invention can also be applied to data other than images.

[0091] It should be noted that triplet generation and model learning may be implemented as separate devices. [Example]

[0092] In the second embodiment, the generation process of the training data 111 and the method of calculating the label information amount are different. Hereinafter, the second embodiment will be described, focusing on the differences from the first embodiment.

[0093] The hardware configuration and functional configuration of the learning device 100 of the second embodiment are the same as those of the first embodiment.

[0094] In the second embodiment, there is a difference in the process of generating the training data 111. Specifically, in step S106, the training device 100 generates the training data 111 by assigning words and the frequency of appearance of the words to images.

[0095] Here, the number of labels assigned to image j is defined as t_j. The total number of labels assigned to each image is defined as T. Also, the frequency of appearance of label k is defined as m_k. In this case, the relationship shown in equation (8) holds.

[0096]

number

[0097] The learning process of the second embodiment is the same as that of the first embodiment. The triplet generation process of the second embodiment is the same as that of the first embodiment. However, in the second embodiment, the label information amount is calculated based on the frequency of appearance of the label, rather than the number of images to which the label is assigned.

[0098] Since the number of labels assigned to images varies, the total number of labels assigned to similar images found by similar image search varies for each query image. Assuming a binomial distribution, the log-likelihood is given by equation (9). Here, S_i represents the total number of labels assigned to similar images of query image i. Also, s_ik represents the frequency with which label k appears in similar images of query image i.

[0099]

number

[0100] If we assume that the frequency of appearance of images with label k is the same as when images are randomly selected, the parameter θ_ik is given by equation (10), which we will call p_k.

[0101]

number

[0102] Assuming that the frequency of occurrence of images with the selected labels follows a distribution specific to the query image i, the maximum estimator is given by Equation (11), which we denote as q_k.

[0103]

number

[0104] In the second embodiment, the label information amount f_k is defined by equation (12).

[0105]

number

[0106] As in the first embodiment, the average f of the label information amount f_k defined by equation (13) is an index that indicates the degree to which the search for similar images affects the frequency distribution of the labels.

[0107]

number

[0108] In the second embodiment, the label similarity is defined by equation (14). Here, c_k(x) represents the frequency with which label k appears in image x. When c_k(x) is 0 or 1, equation (14) is equal to equation (6).

[0109]

number

[0110] The second embodiment also provides the same effects as the first embodiment. [Example]

[0111] In the first embodiment, triplets are generated using search results of similar images for each of a plurality of query images. However, this method has a problem in that a large memory capacity is required to hold the search results. In the third embodiment, a processing method for calculating label features with a small memory area will be described.

[0112] The hardware configuration and functional configuration of the learning device 100 of the third embodiment are the same as those of the first embodiment. The generation process and learning process of the learning data 111 of the third embodiment are the same as those of the first embodiment.

[0113] The triplet generation process of the third embodiment is different from that of the first embodiment. Fig. 7 is a flowchart showing an example of the triplet generation process executed by the learning device 100 of the third embodiment.

[0114] The triplet generation unit 103 initializes a set of triplets to an empty set (step S401), and also initializes the label information amount of each label (step S402). Here, an initial value other than 0 is set for the label information amount of each label.

[0115] The similar vector search unit 102 selects one query image from the images (step S403). The process of step S403 is the same as the process of step S301.

[0116] The similar vector search unit 102 searches for images similar to the query image using the feature vector calculated from the image (step S404). The process of step S404 is the same as the process of step S302.

[0117] The similar vector search unit 102 selects R similar images (step S405). The process of step S405 is the same as the process of step S303.

[0118] The triplet generation unit 103 acquires the label information amount of each label calculated in the processing of the previous epoch (step S406). In the processing of the first epoch, the initial value of the label information amount is acquired.

[0119] The triplet generation unit 103 calculates the label similarity between the query image and the similar image (step S407). The method of calculating the label similarity is the same as in Example 1 or Example 2. However, Example 3 differs in that it uses the label information amount of each label calculated in the processing of the previous epoch.

[0120] The triplet generating unit 103 generates triplets based on the label similarity of each similar image (step S408). The process of step S408 is the same as the process of step S310.

[0121] The triplet generation unit 103 updates the label information amount of each label (step S409). Specifically, the triplet generation unit 103 adds a term related to the search execution (term related to q_k) to the label information amount of each label based on the search result of step S404.

[0122] The triplet generation unit 103 determines whether to end the search (step S410). The process of step S410 is the same as the process of step S304.

[0123] If the search is not to be ended, the triplet generation unit 103 returns to step S403.

[0124] When the search is completed, the triplet generation unit 103 calculates the average of the label information amounts (step S411), and then ends the triplet generation process.

[0125] In the third embodiment, it is only necessary to store the label information amount of each label and the search results of one query image, so that the storage capacity required for processing can be reduced and the calculation speed can be improved.

[0126] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments are provided to explain the present invention in detail, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.

[0127] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, solid-state drives (SSDs), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, and ROMs.

[0128] Furthermore, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Python, and Java (registered trademark).

[0129] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or storage medium.

[0130] In the above-described embodiment, the control lines and information lines are shown as those considered necessary for the explanation, and not all control lines and information lines are necessarily shown in the product. All components may be interconnected. [Explanation of symbols]

[0131] 100 Learning Device 101 Reasoning part 102 Similar Vector Search Unit 103 triplet generator 104 Model Update Section 105 Model Storage Unit 110 datasets 111 Training Data 120 Model Information 201 processor 202 Main storage 203 Secondary storage device 204 Network Interface

Claims

1. A computer system having at least one computer, It stores information about a model that calculates feature vectors for input data with multiple labels. acquiring a plurality of said input data; selecting one of the input data as query input data, and performing a search process to search for similar input data, which is the input data similar to the query input data, based on the feature vector; analyzing the frequency of appearance of each of the plurality of labels based on the results of the search process; calculating, for each of the plurality of labels based on the results of the analysis, an index that evaluates the degree of relationship between the label and the characteristics of the input data; calculating a label similarity indicating a degree of similarity between the plurality of labels assigned to the query input data and the plurality of labels assigned to the similar input data based on the index; selecting the similar input data to be negative examples and the similar input data to be positive examples based on a difference in the label similarity between each of the plurality of similar input data; A computer system that generates a triplet consisting of the query input data and two selected similar input data.

2. 2. The computer system of claim 1, selecting a predetermined number of the triplets in descending order of the difference in label similarity of the similar input data constituting the plurality of triplets; A computer system that outputs the selected triplet.

3. 2. The computer system of claim 1, performing a training process to update a model using the triplets; a computer system that repeatedly executes the triplet generation and the learning process until the average value of the indices of the plurality of labels no longer increases.

4. A method for generating triplets executed by a computer system, comprising: The computer system at least one computer; It stores information about a model that calculates feature vectors for input data with multiple labels. The method for generating triplets includes the steps of: a first step in which the at least one computer acquires a plurality of pieces of input data; a second step in which the at least one computer selects one of the input data as query input data, and executes a search process to search for similar input data, which is the input data similar to the query input data, based on the feature vector; a third step in which the at least one computer analyzes the frequency of appearance of each of the plurality of labels based on the result of the search process, and calculates, for each of the plurality of labels, an index for evaluating the degree of relationship between the label and the characteristics of the input data based on the result of the analysis; a fourth step in which the at least one computer calculates, based on the index, a label similarity indicating a degree of similarity between the plurality of labels assigned to the query input data and the plurality of labels assigned to the similar input data; a fifth step in which the at least one computer selects the similar input data to be a negative example and the similar input data to be a positive example based on a difference in the label similarity between each of the plurality of similar input data; a sixth step in which the at least one computer generates a triplet consisting of the query input data and the selected two similar input data.

5. 5. A method for generating triplets according to claim 4, comprising: a step of selecting a predetermined number of the triplets in descending order of the difference in label similarity between the similar input data constituting the plurality of triplets by the at least one computer; and a step of outputting the selected triplet by the at least one computer.

6. 5. A method for generating triplets according to claim 4, comprising: a seventh step in which the at least one computer executes a learning process to update a model using the triplets; A method for generating triplets, characterized in that the second to seventh steps are repeatedly executed until the average value of the indices of the plurality of labels no longer increases.

Images