Learning model updating device and learning model updating method

Abstract

To enable triplet network learning for making multi-label learning data as an object.SOLUTION: A learning model update device 210 for accepting a learning model for calculating an image feature amount vector of image data attached with a plurality of labels to update the learning model, includes: a label similarity calculation unit 213a for calculating a label similarity between a query being one image data selected from a plurality of image data accepted as learning data 220 to the learning model and a plurality of image data different from the image data selected as the query; a selection processing unit 213b for selecting a group of image data in which a label similarity gap satisfies a prescribed condition among the plurality of image data used in calculating the label similarity as a group of positive cases and a group of negative cases; and a model update processing unit 214 for updating the learning model on the basis of the selected groups of positive cases and negative cases.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a learning model updating device and a learning model updating method. [Background technology]

[0002] The increasing capacity of various storage devices, improvements in computer computing performance, and faster transfer speeds over networks have made it possible to store large amounts of electronic history information in various business operations. In terms of information content, it is now possible to store not only traditional bibliographic information, but also various media information such as videos and images.

[0003] Against this background, there is an increasing need for information processing technology to utilize this electronic history information. For example, in the examination process for patents, trademarks, and designs, a variety of information such as the documents applied for, related images, and bibliographic information attached to the examination process, is accumulated as work history information. If this history information could be utilized to improve the efficiency of the examination process and the work related to applications, it would greatly contribute to the development of the industrial sector.

[0004] Business information is often stored with an arbitrary number of keywords for images. There is also a demand for a system that automatically assigns keywords to images acquired in business.

[0005] Training data in which multiple labels are assigned to the data is called multi-label training data. In training deep neural networks that target multi-label training data, a method that uses Cross-Entropy Loss is commonly used. With this method, a network is constructed that outputs values ​​equal to the "number of different" labels. However, when assigning keywords, it is often difficult to predict in advance the number of different keywords that will be assigned. Also, when the number of different keywords to be assigned is extremely large, problems arise in the configuration of the network model and in the execution of processing.

[0006] On the other hand, triplet network learning, which is one of the learning methods for deep neural networks, trains the neural network to output feature vectors in which the distance between data with the same label is small and the distance between data with different levels is large.

[0007] The triplet concept is illustrated in Figure 1. In triplet network learning, data with the same level as the data of interest (anchor) 111 is treated as positive examples 112, and data with a different level is treated as negative examples 113, and a triplet 110 consisting of the anchor, positive examples, and negative examples is created.

[0008] Each data constituting the triplet is input to a neural network model 120 having the same weight, and the neural network constituting the neural network model 120 outputs a feature vector 130 corresponding to each data.

[0009] 1 illustrates a CNN (Convolutional Neural Network) as the neural network model 120, but there is no intention to limit the neural network model 120 to a CNN, and any known neural network model can be applied. Throughout this specification, any known network model can be suitably applied.

[0010] Next, the distance D between the anchor 111 and the positive example 112 in the feature space + 140, and the distance D between the anchor 111 and the negative example 113 in the feature space - 150 is calculated. The loss function, which is the learning objective function, is defined as the sum of the differences between these two distances calculated for each triplet. In Non-Patent Document 1, this triplet network learning is applied to learning features for personal authentication using face images.

[0011] When a learning set is given, there are an infinite number of triplets. Non-Patent Document 2 discusses a method for selecting triplets that realize efficient learning by applying a similar vector search process. [Prior art documents] [Non-patent literature]

[0012] [Non-Patent Document 1] Schroff, Florian, Dmitry Kalenichenko, James Philbin. "Facenet: A unified embedding for face recognition and clustering", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2015) [Non-Patent Document 2] Yasumitsu Ikeura, Koichi Okamoto, Ryohei Kashima, Atsushi Hiroike: Multimodal Deep Learning Platform for IoT Data, Hitachi Review, 102-3, 119-123 (2020) Summary of the Invention [Problem to be solved by the invention]

[0013] The triplet network learning described above is an effective method for obtaining feature vectors, but it can only be applied when a single label is assigned to each data. On the other hand, in the case of multi-label learning data, the relationship between positive and negative examples between the anchor data and other data cannot be uniquely defined, so triplets cannot be constructed.

[0014] The present invention has been made in consideration of the above problems, and an object of the present invention is to provide a learning model updating device and a learning model updating method that enable triplet network learning targeting multi-label learning data. [Means for solving the problem]

[0015] In order to solve the above problem, a learning model updating device according to one aspect of the present invention is a learning model updating device that accepts a learning model that calculates an image feature vector of image data to which multiple labels have been assigned, and updates this learning model, and is characterized in that it includes: a query, which is one image data selected from multiple image data accepted as learning data for the learning model, a label similarity calculation unit that calculates label similarity between the image data selected as the query and multiple image data different from the image data selected as the query, a selection processing unit that selects, from the multiple image data used when calculating the label similarity, a pair of image data whose label similarity gap satisfies a predetermined condition as a pair of positive and negative cases, and a model updating processing unit that updates the learning model based on the selected pair of positive and negative cases. Effect of the Invention

[0016] According to the present invention, it is possible to realize a learning model updating device and a learning model updating method that enable triplet network learning for multi-label learning data. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a schematic diagram of general triplet network learning. [Diagram 2] FIG. 1 is a configuration diagram of a learning model updating device according to a first embodiment. [Diagram 3] 4 is a flow of processing of the learning model updating device according to the first embodiment. [Figure 4] 4 is a flow of a triplet set generation process of the learning model updating device according to the first embodiment. [Diagram 5] FIG. 4 is an explanatory diagram of a triplet generation process for each search result in the learning model updating device according to the first embodiment. [Figure 6] 11 is a flow chart showing a triplet generation process for each search result in the learning model updating device according to the first embodiment. [Figure 7]FIG. 11 is a configuration diagram of a learning model updating device according to a second embodiment. [Figure 8] 11 is a flow of processing of a learning model updating device according to a second embodiment. [Figure 9] 13 is a flow of a triplet set generation process in the learning model updating device according to the second embodiment. [Figure 10] 13 is a flow of a triplet generation process for each anchor in the learning model updating device according to the second embodiment. [Figure 11] FIG. 11 is a configuration diagram of a search device using a network model updated according to the first or second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the embodiments described below do not limit the invention according to the claims, and all of the elements and combinations thereof described in the embodiments are not necessarily essential to the solution of the invention.

[0019] In the drawings explaining the embodiments, the same reference numerals are used to designate parts having the same functions, and repeated explanations thereof will be omitted.

[0020] In the following description, an expression such as "xxx data" may be used as an example of information, but the information may have any data structure. In other words, to show that the information does not depend on the data structure, "xxx data" may be referred to as "xxx table." Furthermore, "xxx data" may simply be referred to as "xxx." In the following description, the structure of each piece of information is an example, and the information may be divided and stored, or combined and stored.

[0021] In the following explanation, the process may be described with a "program" as the subject, but since a program is executed by a processor (e.g., a CPU (Central Processing Unit)) to perform a defined process using storage resources (e.g., a memory) and / or a communication interface device (e.g., a port) as appropriate, the subject of the process may also be the program. Processes explained with a program as the subject may also be processes performed by a processor or a computer having that processor.

[0022] The learning model updating device of this embodiment has, as an example, the following configuration.

[0023] In multi-label learning data, the label similarity between data is defined based on the label assigned to each piece of data. When a certain piece of data is considered as an anchor, the set of data with the largest difference (gap) in label similarity with the anchor, or the set of data sets with a difference (gap) in label similarity equal to or greater than a certain value is extracted. Among each set of data, those with high label similarity with the anchor are considered as positive example candidates, and those with low label similarity are considered as negative example candidates. From the extracted positive and negative example candidate pairs, the necessary number of pairs are selected in order of smallest distance to the anchor in the feature vector space to form a set of triplets.

[0024] By executing the above process using each data in the training data as an anchor, a universal set of triplets to be learned is constructed. A loss function is defined for the universal set of triplets constructed, and learning for extracting feature vectors is performed by minimizing the loss function. EXAMPLES

[0025] This embodiment realizes efficient learning by combining the technique of this embodiment with a similar vector search technique.

[0026] FIG. 2 is a configuration diagram of this embodiment.

[0027] The learning model updating device 210 of this embodiment accepts a set of image data to which multi-label information is added as learning data 220. In normal operation, when learning a deep neural network, it is rare for the network weights to start from a completely random state. The efficiency of learning is improved by providing a network model 230 that has been trained in advance on another learning task as the initial state of learning.

[0028] Inside the learning model update device 210, the inference processing unit 211 extracts feature vectors of the learning data using the state of the network model at that point in the learning process. The similar vector search processing unit 212 performs a similar vector search on the set of feature vectors extracted by the inference processing unit 211.

[0029] The triplet set generation processing unit 213 configures a set of triplets to be used for learning based on the result of the similar vector search processing by the similar vector search processing unit 212. More specifically, the label similarity calculation unit 213a of the triplet set generation processing unit 213 calculates the label similarity between a query, which is one image data selected from multiple image data that are the learning data 220, and multiple image data different from the image data selected as the query, and the selection processing unit 213b selects, from the multiple image data used in calculating the label similarity, a pair of image data whose label similarity gap satisfies a predetermined condition as a pair of positive and negative cases.

[0030] The model update processing unit 214 performs a process of updating the network model using the set of learning triplets generated by the triplet set generation processing unit 213. The state of the updated network model 240 is stored in an external storage medium by the model storage processing unit 215 as necessary.

[0031] FIG. 3 shows a process flow of the learning model updating device 210.

[0032] In the learning model update device 210, first, learning data and an initial state of the network model are acquired (301). In the subsequent inference process (302), a feature vector of the learning data is extracted.

[0033] The next clustering process (303) is a pre-processing for quickly executing a similar vector search performed in the triplet set generation process (304). Similar vector search using clustering is discussed in the above-mentioned Non-Patent Document 2, etc. The subsequent triplet set generation process (304) will be described later.

[0034] From the set of triplets obtained in the triplet set generation process (304), a learning triplet to be actually used for learning is selected (305) based on a certain criterion. The content of this criterion will be described later. A learning triplet is an ordered array whose elements are triplets. The network model is updated (306) using the learning triplet obtained in this way. This updated network model will be used in the next inference process (302).

[0035] In this embodiment, the process from the inference process (302) to the update of the network model (306) is considered as one cycle of learning. In the process of determining whether or not to end learning (307), learning is usually ended when the number of cycles reaches a preset maximum number of cycles, but learning may also be ended based on other evaluation criteria.

[0036] FIG. 4 shows the flow of the triplet set generation process (304).

[0037] First, the elements of the triplet set are cleared (401). Next, one piece of data in the training data is selected as a query (402), data having a feature vector similar to that of the query is searched for (403), and search results are obtained (404). Using the obtained search results, a triplet generation process (405) is performed for each of the next search results. The details of this process will be described later.

[0038] Next, the triplet set for each obtained search result is added to the triplet set of this process (406). The processes from 402 to 406 are usually performed using all learning data as a query, but an upper limit may be set on the number of pieces of data to be used as a query due to requirements such as restrictions on the amount of calculation. In this case, a method such as using a certain number of pieces of data randomly selected from the learning data as the query is used.

[0039] Next, the triplet generation process (405) for each search result will be described in detail.

[0040] First, the label similarity when multiple labels are assigned is defined as follows.

number

[0041] Next, for a data pair (x, y) consisting of two different data in the search results, the difference (gap) g(q, x, y) in label similarity with the query q is defined as follows:

number

[0042] FIG. 5 shows a schematic diagram of the transition of the label similarity in the search results.

[0043] The horizontal axis of Figure 5 is the ranking in the search results, and the vertical axis is the label similarity of each search result. If there is a high correlation between the similarity in the feature vector space and the label similarity, it is expected that the label similarity will monotonically decrease as the search ranking increases. However, in reality, a discrepancy between the two occurs, and data with a low search ranking may show a high label similarity, or data with a high search ranking may show a low label similarity.

[0044] Therefore, we use the concept of label similarity gap in formula (2). When focusing on a certain search ranking data, we calculate the label similarity gap between that data and data with a lower ranking. According to the variable names in formula (2), the lower ranking data corresponds to x, and the focused data corresponds to y.

[0045] In this embodiment, for the data pairs in the search results, the pair with the largest gap in label similarity is extracted, and the data with the lower ranking among the data constituting each pair is set as a candidate positive case for forming a triplet, and the data with the higher ranking is set as a candidate negative case.

[0046] When a search result is given, there are usually multiple pairs with the largest gap value. The gap and its underlying label similarity are defined as real numbers between 0 and 1, but as is clear from formula (1), the label similarity is defined as a combination of not-so-large integer values, such as the number of labels assigned to each data. As a result, the gap value is not distributed as a continuous value, but as a number of discrete values.

[0047] Therefore, the pair that gives the maximum value is not unique, but usually occurs in many cases. Also, if there are multiple pairs that share the same negative example candidate among the pairs with the largest gap, the pair with the higher ranked positive example candidate, i.e., the pair with the smaller gap in search ranking, is selected.

[0048] From the set of positive and negative example candidate pairs extracted using the above method, a pre-specified number of pairs are selected in order of the negative example candidate's search ranking, and a triplet set is constructed by combining each pair with the query. If the number of positive and negative example candidate pairs is less than the specified number, a triplet set is constructed using all of the positive and negative example candidate pairs. Note that if the maximum gap value is smaller than a pre-specified threshold, no triplets are constructed. In other words, in this case, the set of triplets generated is an empty set.

[0049] FIG. 6 shows the flow of the process of generating a triplet set for each search result (405) described above.

[0050] First, the label similarity between the query and the data in the search results is calculated (601). Next, the pair of search results with the largest gap in label similarity is extracted as a pair of positive and negative example candidates (602). Next, from the extracted pairs of positive and negative example candidates, N negative example candidates with the highest search rankings are selected (603). Finally, N triplets are constructed by combining the query and the selected pairs of positive and negative example candidates (604).

[0051] For each generated triplet, we define the following value (loss function for each triplet):

number

[0052] In the selection of learning triplets (305) in Fig. 3, the triplet set obtained by the triplet set generation process (304) is sorted in ascending order of the value of formula (3). If the number of elements in the triplet set is greater than the specified number, the lower triplets, i.e., the triplets with relatively large values ​​of formula (3), are discarded.

[0053] In updating the network model (306) in FIG. 3, optimization is performed so as to reduce the loss function of the following equation.

number

[0054] In this example, we do not optimize all training triplets at once. As with most deep neural network training, we divide the entire array of training triplets into small arrays called mini-batches, and update the network sequentially in mini-batches.

[0055] Therefore, even if the loss function for each triplet in formula (3) satisfies the non-negative condition in the initial state of a certain period, the difference in squared distance between positive and negative examples may become a negative value due to fluctuations in the state of the network during the update process. The value b in formula (4) is intended to prevent learning from failing due to instability in the numerical calculations when updating the network based on an extremely negative value. From this perspective, the value of b in formula (4) is set in advance as an appropriate scalar value.

[0056] Moreover, the network update process (306) is performed in the order sorted in process (305). Therefore, learning using triplets with smaller values ​​of formula (3) is prioritized. Learning triplets with smaller loss functions per triplet first means that triplets that are expected to require smaller corrections in the feature space, that is, triplets that are expected to be easier to learn, are prioritized. This is a measure to realize a more stable learning process.

[0057] In the triplet set generation process for each search result described in FIG. 6, the pair of positive and negative example candidates with the largest gap in label similarity is extracted, but all pairs with a gap value equal to or greater than a certain threshold may be extracted. In addition, from the extracted pairs of positive and negative example candidates, pairs with high rankings of negative example candidates are selected with priority, but pairs with small loss function for each triplet in Equation (3) may be selected with priority, or may be selected randomly. In addition, in the selection of learning triplets (305) in FIG. 3, pairs with large gaps in label similarity may be selected with priority, or may be selected randomly.

[0058] In addition, other definitions may be adopted in the method of defining the label similarity in formula (1). For example, the following formula is another definition method using set operations, similar to formula (1).

number

[0059] Furthermore, as a representation format for multi-labeled data, there is also a method of expressing the data in vector format, assuming a vector whose dimension is the total number of existing labels, and representing data in vector format by assigning 1 to the data when it is assigned and 0 to the data when it is not assigned. In this case, the label similarity can be defined from the inner product between vectors or the distance between vectors.

[0060] As described above in detail, according to the present embodiment, it is possible to realize the learning model updating device 210 and the learning model updating method that enable triplet network learning for multi-label learning data. EXAMPLES

[0061] In this embodiment, a learning process that does not involve a similar vector search will be described.

[0062] FIG. 7 is a configuration diagram of this embodiment.

[0063] Many of the processing units constituting this embodiment provide the same functions as the corresponding processing units in the first embodiment shown in FIG.

[0064] The learning model update device 710 receives a set of image data to which multi-label information has been added as learning data 720. If necessary, the learning model update device 710 also receives a network model 730 that has been trained in advance on another learning task as the initial state of learning.

[0065] Inside the learning model updating device 710, a triplet set generation processing unit 711 generates a set of triplets to be used for learning, and a model updating processing unit 712 performs a network model update process using the generated set of triplets for learning. The state of the updated network model 740 is saved in an external storage medium by a model saving processing unit 713 as necessary.

[0066] FIG. 8 shows a process flow of the learning model updating device 710.

[0067] In the learning model update device 710, first, the learning data and the initial state of the network model are acquired (801). Next, the label similarity of equation (1) is calculated between all the learning data, and the value is stored (802).

[0068] Next, a triplet set generation process (803) in this embodiment is performed. The details of this process will be described later. From the set of triplets acquired in the triplet set generation process (803), a learning triplet to be actually used for learning is selected (804) based on a certain criterion. The details of this criterion will be described later. A learning triplet is an ordered array with triplets as elements. The network model is updated (805) using the learning triplet acquired in this way.

[0069] In this embodiment, one learning cycle is defined as the process from triplet set generation process (803) to network model update (805). In the learning end determination process (806), learning is usually terminated when the number of cycles reaches a preset maximum number of cycles, but learning may also be terminated based on other evaluation criteria.

[0070] FIG. 9 shows the flow of the triplet set generation process (803).

[0071] First, the elements of the triplet set are cleared (901). Next, one piece of data in the training data is selected as an anchor (902). Based on the data selected as the anchor, a triplet generation process (903) is performed for each anchor. The details of this process will be described later.

[0072] Next, the triplet set for each acquired anchor is added to the triplet set of this process (904). The processes from (902) to (904) are usually performed by selecting all the learning data as anchors, but an upper limit may be set on the number of data items to be used as anchors due to requirements such as restrictions on the amount of calculation. In this case, a certain number of data items randomly selected from the learning data are used as anchors.

[0073] FIG. 10 shows the flow of the triplet generation process (903) for each anchor.

[0074] First, a set S of pairs of positive and negative example candidates, which is data temporarily held during this process, is emptied (1001). Next, a data pair (x, y) is randomly selected from the learning data excluding the anchor a (1002). Next, a gap g(a, x, y) in the label similarity of equation (2) is calculated from the label similarity between a and x, and between a and y (1003). Here, the one corresponding to the query q in equation (2) is the anchor a in this embodiment. If the gap g(a, x, y) is equal to or greater than a predefined threshold, (x, y) is added to the set S (1005).

[0075] The process from (1002) to (1005) is repeatedly executed until a certain termination condition is met (1006). In this embodiment, this repeated process is terminated when the number of elements in set S reaches a predefined number, or when the number of times the process has been executed reaches a predefined maximum number.

[0076] Next, the set S of pairs of positive and negative example candidates extracted in this way is sorted in order of the largest gap in label similarity, and the top pairs are selected (1007). There is an upper limit to the number of pairs that can be selected, and pairs exceeding this limit, i.e. pairs with relatively small gaps in label similarity, are discarded. A set of triplets is constructed from the pairs of positive and negative example candidates selected in this way (1008).

[0077] Unlike the first embodiment, the triplet generated in the triplet set generation process (803) of this embodiment does not have any constraints on the feature vector space regarding the selection of positive and negative cases. Therefore, the loss function for each triplet in formula (3) may be a negative value. In the selection of learning triplets (804), the set of generated triplets is sorted in ascending order of the absolute value of the loss function for each triplet in formula (3), and a certain number of top triplets are selected as triplets to be used for final learning. In the subsequent network model update (805), the model is updated using each triplet according to the sorted order.

[0078] Therefore, this embodiment can also provide the same effects as the above-mentioned first embodiment. EXAMPLES

[0079] In this embodiment, a search device 1100 that searches for similar image data using network models 240, 740 updated by the learning model updating devices 210, 710 of the first and second embodiments will be described.

[0080] FIG. 11 is a configuration diagram of a search device 1100 using the network model 240, 740 updated according to the first or second embodiment.

[0081] When image data that is a query (search target) is input to an image input unit 1101 of a search device 1100, a feature calculation unit 1102 calculates the feature of the input image data. Next, a search unit 1103 inputs the calculated feature to a network model 240, 740, and obtains image data that is similar (determined to be similar) to the query, which is the search result. Then, a search result display unit 1104 displays the image data that is the search result obtained by the search unit 1103 using a display device such as a display (not shown).

[0082] In addition, the above-mentioned embodiments are described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Also, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.

[0083] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium storing it constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, etc.

[0084] Furthermore, the program code for realizing the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Java (registered trademark), and Python.

[0085] Furthermore, all or part of the program code of the software that realizes the functions of each embodiment may be stored in advance in the storage of the machine learning system, or, if necessary, may be stored in the storage from a non-transitory storage device of another device connected to the network, or from a non-transitory storage medium via an external I / F (not shown) provided by the machine learning system.

[0086] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed over a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read out and execute the program code stored in the storage means or storage medium.

[0087] In the above embodiment, the control lines and information lines are shown as they are considered necessary for the explanation. Not all control and information lines on the product are shown. They may be connected. (Appendix 1) A learning model update device that accepts a learning model that calculates an image feature vector of image data to which a plurality of labels are assigned, and updates the learning model, A label similarity calculation unit that calculates label similarity between a query, which is one of the image data selected from the plurality of image data accepted as learning data for the learning model, and a plurality of image data different from the image data selected as the query; a selection processing unit that selects, from among the plurality of image data used in calculating the label similarity, a pair of image data in which a gap in the label similarity satisfies a predetermined condition as a pair of a positive case and a negative case; a model update processing unit that updates the learning model based on the selected set of positive examples and negative examples; A learning model updating device comprising: (Appendix 2) The learning model updating device according to (Appendix 1), wherein the predetermined condition is that the gap is maximum. (Appendix 3) The learning model updating device described in (Appendix 1), wherein the specified condition is that the gap is greater than or equal to a predetermined threshold. (Appendix 4) a similar vector search processing unit that performs a similar vector search for each of the plurality of positive examples and the plurality of negative examples for the query based on the learning model, and calculates a ranking of similar vector search results for the query; The selection processing unit selects a set of the positive case and the negative case based on the predetermined condition and the ranking. The learning model updating device according to (Supplementary Note 2) or (Supplementary Note 3), (Appendix 5) The selection processing unit extracts a plurality of pairs of the positive case and the negative case in which the gap in the label similarity satisfies a predetermined condition as candidates, identifies the negative case that is ranked up to a predetermined rank in descending order of the ranking, and selects a pair of the positive case and the negative case including the identified negative case from the extracted plurality of candidate pairs. The learning model updating device according to (Appendix 4) is characterized in that: (Appendix 6) The selection processing unit extracts a plurality of pairs of the positive example and the negative example in which the gap in the label similarity satisfies a predetermined condition as candidates, and selects the pair of the positive example and the negative example having the smallest triplet loss function from the extracted plurality of candidate pairs. The learning model updating device according to (Appendix 4) is characterized in that: (Appendix 7) The learning model updating device described in (Appendix 6) is characterized in that the triplet loss function is calculated based on a squared sum error of a distance between the feature vector of the query and the feature vector of the positive example, and a distance between the feature vector of the query and the feature vector of the negative example. (Appendix 8) The learning model updating device described in (Appendix 1) is characterized in that the label similarity is calculated based on the number of elements in the union and the number of elements in the intersection of the set of labels assigned to the query and the set of labels assigned to multiple pieces of image data other than the image data selected as the query. (Appendix 9) A learning model updating method using a learning model updating device that accepts a learning model that calculates an image feature vector of image data to which a plurality of labels are assigned, and updates the learning model, comprising: Calculate label similarity between a query, which is one of the image data selected from the plurality of image data accepted as learning data for the learning model, and a plurality of image data different from the image data selected as the query; selecting a pair of image data, the pair of which has a gap in the label similarity that satisfies a predetermined condition, as a pair of a positive case and a negative case, from among the plurality of image data used in calculating the label similarity; A learning model updating method using a learning model updating device, characterized in updating the learning model based on the selected set of positive examples and negative examples. [Explanation of symbols]

[0088] 110...triplet 111...anchor 112...positive example 113...negative example 120...deep neural network 130...feature vector 140...distance between positive example and anchor 150...distance between negative example and anchor 210, 710...learning model update device 211...inference unit 212...similar vector search processing unit 213, 711...triplet set generation processing unit 213a...label similarity calculation unit 213b...selection processing unit 214, 712...model update processing unit 215, 713...model storage processing unit 220...learning data 230, 240, 730, 740...network model

Claims

1. A learning model update device that accepts a learning model that calculates an image feature vector of image data to which a plurality of labels are assigned, and updates the learning model, a label similarity calculation unit that calculates label similarity between a query, which is one of the image data selected from the plurality of image data accepted as learning data for the learning model, and a plurality of image data different from the image data selected as the query; a selection processing unit that selects, from among the plurality of image data used in calculating the label similarity, a pair of image data in which a gap in the label similarity satisfies a predetermined condition as a pair of a positive case and a negative case; a model update processing unit that updates the learning model based on the selected set of positive examples and negative examples; A learning model updating device comprising:

2. 2. The learning model updating device according to claim 1, wherein the predetermined condition is that the gap is a maximum.

3. 2. The learning model updating device according to claim 1, wherein the predetermined condition is that the gap is equal to or greater than a predetermined threshold value.

4. a similar vector search processing unit that searches for the image data having the image feature vector similar to the image feature vector of the query, obtains a similar vector search result for the query, and calculates a ranking in the similar vector search result for the query; The selection processing unit selects a set of the positive case and the negative case based on the predetermined condition and the ranking.

4. The learning model updating device according to claim 2 or 3.

5. The selection processing unit extracts a plurality of pairs of the positive case and the negative case in which the gap in the label similarity satisfies a predetermined condition as candidates, identifies a pre-specified number of the negative cases in descending order of the rank, and selects a pair of the positive case and the negative case including the identified negative case from the extracted plurality of candidate pairs.

5. The learning model updating device according to claim 4.

6. The selection processing unit extracts a plurality of pairs of the positive example and the negative example, the gap of which satisfies a predetermined condition, as candidates, and selects, from among the extracted plurality of candidate pairs, a candidate pair having a small triplet loss function as the pair of the positive example and the negative example, with a priority.

5. The learning model updating device according to claim 4.

7. 7. The learning model updating device according to claim 6, wherein the triplet loss function is calculated based on a squared sum error of a distance between the feature vector of the query and the feature vector of the positive example, and a distance between the feature vector of the query and the feature vector of the negative example.

8. The label similarity is calculated based on the number of elements in a union and a product of a set of the labels assigned to the query and a set of the labels assigned to a plurality of pieces of image data other than the image data selected as the query.

2. The learning model updating device according to claim 1 .

9. A learning model updating method using a learning model updating device that accepts a learning model that calculates an image feature vector of image data to which a plurality of labels are assigned, and updates the learning model, comprising: Calculating label similarity between a query, which is one image data selected from the plurality of image data accepted as learning data for the learning model, and a plurality of image data different from the image data selected as the query; selecting a pair of image data, the pair of which has a gap in the label similarity that satisfies a predetermined condition, as a pair of a positive case and a negative case, from among the plurality of image data used in calculating the label similarity; The learning model is updated based on the selected set of positive examples and negative examples. A learning model updating method using a learning model updating device.

Images