Information processing apparatus and control method therefor

Abstract

To improve model learning accuracy.SOLUTION: An information processing apparatus comprises: model management means that manages, for each of a plurality of learning models, first information for identifying a dataset used to train the learning model, and second information for identifying an initial model used to train the learning model; data management means that manages third information concerning a plurality of datasets each identified by the first information of each of the plurality of learning models; reception means that receives an instruction of incremental learning using, as an initial model, a predetermined learning model included in the plurality of learning models; and determination means that determines a dataset to be used for the incremental learning from the plurality of datasets. The determination means determines, as a dataset to be used for the incremental learning, a dataset used to train a learning model whose evaluation accuracy is equal to or higher than a predetermined accuracy, the evaluation accuracy being obtained when inputting a predetermined dataset to each of at least one learning model used as the initial model of the predetermined learning model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a management technique for a learning model and learning data. [Background technology]

[0002] In recent years, the accuracy of image recognition technologies such as image classification, object detection, and object tracking has improved dramatically with the advent of deep neural networks (DNNs). Generally, training a DNN model requires a large amount of data, but it is difficult for individuals to collect such a large amount of data. Therefore, a technique called fine-tuning is sometimes used, in which a model trained with a large amount of data is used as the initial model and only a portion of the layers of the DNN model are trained. Furthermore, fine-tuning is known to tend to forget information acquired during the initial model's tasks. Therefore, situations can arise in which fine-tuning is repeated using training data unrelated to the initial model.

[0003] Patent Document 1 discloses a method for selecting data useful for learning by calculating the similarity between labels, and classifying high similarity as a positive example and low similarity as a negative example. Non-Patent Document 1 also discloses a method for calculating the similarity of training data based on a portion of the output of each layer of a DNN model, and correcting the training data to obtain plausible labels based on the similarity. These methods can improve learning accuracy even when the training data contains incorrect true values. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-179162 [Non-patent literature]

[0005] [Non-Patent Document 1] Jiangfan Han, Ping Luo, Xiaogang Wang, "Deep Self-Learning From Noisy Labels", arXiv:1908.02160, 2019 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the method of Patent Document 1 has a problem that the learning accuracy is likely to decrease when the training data contains data that has been arbitrarily annotated by a user. Also, Non-Patent Document 1 has a problem that, because the training data is corrected using an existing trained model, when a group of arbitrarily annotated data exists in the training data, it is not possible to effectively correct or select the training data.

[0007] The present invention has been made in view of such problems, and aims to provide a technique that can improve the learning accuracy of a model. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, an information processing device according to the present invention has the following configuration. That is, the information processing device that manages a plurality of learning models includes: A model management means for managing, for each of the plurality of learning models, first information that identifies a dataset used in learning the learning model and second information that identifies an initial model used in learning the learning model; a data management means for managing third information regarding a plurality of data sets identified by the first information for each of the plurality of learning models; a receiving means for receiving an instruction for additional learning using a given learning model included in the plurality of learning models as an initial model; a determination means for determining a dataset to be used for the additional learning from the plurality of datasets based on the first information, the second information, and the third information; Equipped with The determination means determines, as the dataset to be used for the additional learning, the dataset used to train a learning model whose evaluation accuracy when a given dataset is input to each of one or more learning models used as an initial model of the given learning model is equal to or greater than a predetermined accuracy.

[0009] Alternatively, an information processing device that manages a plurality of learning models may A model management means for managing, for each of the plurality of learning models, first information that identifies a dataset used in learning the learning model and second information that identifies an initial model used in learning the learning model; a data management means for managing third information regarding a plurality of data sets identified by the first information for each of the plurality of learning models; a receiving means for receiving an instruction for additional learning using a given learning model included in the plurality of learning models as an initial model; a determination means for determining a dataset to be used for the additional learning from the plurality of datasets based on the first information, the second information, and the third information; Equipped with The determination means determines, from among one or more datasets used in training each of the one or more learning models used as initial models for the given learning model, a dataset whose similarity to the first dataset used in training the given learning model is equal to or greater than a predetermined value as the dataset to be used for the additional training. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a technique that can improve the learning accuracy of a model. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 2 is a block diagram showing a functional configuration of the information processing device. [Figure 2] FIG. 2 is a diagram showing a processing flow in an information processing device. [Figure 3]10 is a flowchart of a learning process in the information processing device. [Figure 4] FIG. 10 is a diagram illustrating an example of information assigned to a model. [Figure 5] FIG. 10 is a diagram illustrating an example of information added to data. [Figure 6] FIG. 10 is a diagram illustrating an example of traceability information. [Figure 7] FIG. 10 is a diagram illustrating an example of learning data. [Figure 8] FIG. 10 is a diagram illustrating the difference in true values ​​between different objects. [Figure 9] 10 is a flowchart of a learning process in an information processing device (second embodiment). [Figure 10] FIG. 10 is a diagram illustrating an example of learning data (modification). [Figure 11] FIG. 10 is a diagram illustrating an example of information assigned to a model (modification). DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0013] (First embodiment) As a first embodiment of an information processing device according to the present invention, an information processing device that manages the histories of a learning model and learning data as traceability information will be described below as an example.

[0014] <Summary> When additional training is performed on a training model (pre-trained model) using training data, the information on the task targeted by the initial model (e.g., human torso) tends to be lost and the model tends to shift to a new task (e.g., whole body). This shift is called catastrophic forgetting.

[0015] Therefore, the information processing device of this embodiment manages traceability information that indicates the process by which models and data were created in the past (i.e., the past history of models and data).The information processing device then selects training data to be used for additional learning based on the evaluation accuracy when a given data set is input to each model obtained by tracing the traceability information.This makes it possible to select training data with similar learning targets and true values ​​during additional learning, improving the learning accuracy of the model.

[0016] <Device configuration> FIG. 1 is a block diagram showing the functional configuration of an information processing device 11. The information processing device 11 manages "models" and "data" used for model learning and evaluation. In addition, the information processing device 11 learns models, edits data, and registers model information in response to requests from users. Here, it is assumed that the models and data are available for use by third parties via a learning model publishing platform. In other words, the user is a user of the learning model publishing platform.

[0017] The information processing device 11 is communicably connected to the user terminal 12 and the user terminal 13 via a communication network such as a local area network (LAN) or the Internet. The method of connection is not particularly limited. For example, each may be connected separately via a wired connection or via wireless communication. Although two user terminals are shown in FIG. 1, this is not limiting. Each user terminal is assumed to be an information terminal such as a personal computer (PC), a mobile phone, or a tablet terminal device.

[0018] The information processing device 11 includes a control unit 10 and a storage unit 20. The control unit 10 can be realized, for example, by a central processing unit (CPU) executing various programs stored in the storage unit 20. The storage unit 20 is a large-capacity storage device such as random access memory (RAM), read-only memory (ROM), or a hard disk drive (HDD) or solid-state drive (SSD).

[0019] The control unit 10 includes a model management unit 101, a data management unit 102, a traceability information management unit 103, a related model / data management reception unit 104, a learning / evaluation unit 105, and a display control unit 107. As described above, each of these functional units can be realized by, for example, a CPU executing various programs. However, some or all of these functional units may also be realized by hardware such as an application specific integrated circuit (ASIC).

[0020] The model management unit 101 manages models registered in the information processing device 11. In this embodiment, the model is assumed to be an "object detection model" intended to perform object detection, which is a form of image recognition, but is not limited to this. It may also be a voice recognition model, a natural language processing model, or a generative artificial intelligence (AI) model. The model management unit 101 assigns model information to the model and manages it.

[0021] 4 is a diagram showing examples of model information 401, 402 assigned to two models. The model information is attribute information of the model, and includes information on the model ID, initial model ID, learning data ID, tag, task, evaluation data ID, and evaluation result. In other words, it includes identification information (ID) that identifies the initial model and data used to generate the model.

[0022] The model ID is a model-specific identification information (ID) for identifying each model. The initial model ID is a model ID that specifies the model that became the initial parameter when each model was trained.

[0023] 4, in model information 401, the model ID is "m0001" and the initial model ID is "m0000". In addition, in model information 402, the model ID is "m0002" and the initial model ID is "m0001". This indicates that the model with the model ID "m0002" is a model obtained as a result of learning using the model with the model ID "m0001" as the initial model.

[0024] The training data ID is a unique ID for the training data that identifies the dataset used to train each model. The tag contains information about the object that each model targets, as well as information necessary for users to search for models. The task contains the type of processing that each model targets (object detection, image generation, etc.). The evaluation data ID is a data ID that specifies the dataset used when evaluating each model. The evaluation results contain numerical values ​​obtained as a result of the evaluation.

[0025] 4, model information 401 and 402 are both model information for an object detection model. For example, if the model is another type of model (such as a voice recognition model), the items included in the model information may be different from those in the example of FIG.

[0026] The data management unit 102 manages data registered in the information processing device 11. The data in this embodiment is image data for object detection. The data management unit 102 accepts registration of a data set from a user, and manages the data set by assigning data information to the data set.

[0027] 5 is a diagram showing an example of data information 501, 502 assigned to data (image data set). The data information is attribute information of the data, and includes information on a data ID, an initial data ID, a tag, and a task.

[0028] The data ID is a data-specific ID used to identify each piece of data. The initial data ID is the data ID of the initial dataset from which the data was created. The tag contains information about the object targeted by each piece of data and information necessary for users to search for data. The task contains the type of processing targeted by each piece of data (object detection, image generation, etc.).

[0029] For example, the initial data ID in data information 502 is "d0001," which indicates that the data corresponding to data information 502 was edited (created) using the data corresponding to data information 501 as the initial data. Editing is performed by changing annotations, processing images, etc.

[0030] 5, data information 501 and 502 are both data information of data used in a model for an object detection model. For example, if the data is used in another type of model (such as a voice recognition model), the items included in the data information may be different from those in the example of FIG.

[0031] The traceability information management unit 103 manages traceability information of models and data. Here, the traceability information is information indicating the process by which each model and each data was created in the past (i.e., the past history).

[0032] FIG. 6 is a diagram showing an example of traceability information. In FIG. 6, solid arrows indicate derivation relationships between models and between data. That is, the relationships between multiple learning models and multiple data sets published on the learning model publishing platform are integrated and managed. Also, in FIG. 6, dotted arrows indicate the data used to train each model. Furthermore, evaluation data ("e0000" to "e0002") is associated with each model-data pair. As mentioned above, evaluation data is the data set used when evaluating each model.

[0033] For example, model 602 ("m0002") indicates that it is a derived model created by additional learning using data ("d0002") with model 601 ("m0001") as the initial model. Also, data ("d0002") indicates that it is derived data created (edited or added) based on data ("d0001").

[0034] In this way, traceability information shows the records of how each model and data has been edited in the past. Therefore, by tracing the traceability information, it is possible to identify one or more models that were used to create each model. It is also possible to identify one or more data that were used to create each data.

[0035] The related model data management unit 104 acquires related models, evaluation data, and training data based on a model acquisition instruction from a user. Details of the related model data management unit 104 will be described later with reference to FIG.

[0036] The learning and evaluation unit 105 learns and evaluates models based on available models and data. When a new model is learned, the traceability information management unit 103 updates the traceability information based on the model information and data information associated with the model and data.

[0037] The display control unit 107 controls the display of the results of the information processing device 11 in response to requests from the user terminals 12 and 13 .

[0038] The storage unit 20 stores the various programs executed by the CPU described above, as well as the models, model information, data, data information, evaluation data, traceability information, and the like described above.

[0039] <Device Operation> This embodiment assumes that the history of trained models (initial model, training data, and their modification history) is registered as traceability information (FIG. 6) in a training model publishing platform. A method for achieving higher training accuracy when additional training is performed on the trained model described above is described. Specifically, one or more models linked by traceability information are evaluated using predetermined evaluation data, and training data linked to a model with a high evaluation value is selected for additional training (relearning).

[0040] Fig. 2 is a diagram showing the flow of processing in an information processing device in response to a request from a user terminal. Processes 1 to 7 in Fig. 2 exemplify processes of adding new data to an existing dataset and performing additional learning on a model. On the other hand, processes 8 to 15 in Fig. 2 exemplify processes of editing at least a part of an existing dataset and performing additional learning on a model.

[0041] In the following description, it is assumed that models 600 to 602 have been registered in advance through processes 1 to 7 or processes 8 to 15 in FIG. 2. Here, it is assumed that user 0 has registered model 600 ("m0000"), user 1 has registered model 601 ("m0001"), and user 2 has registered model 602 ("m0002"). As each user registers a model, models 600 to 602 are stored in storage unit 12. In addition, the above-mentioned model information (FIG. 4), data information (FIG. 5), and traceability information (FIG. 6) are also registered (however, model 607 and related information have not yet been registered).

[0042] It is also assumed here that user 2 wishes to detect "the entire body of a person." However, the target object can be anything, and there are no limitations on the target object. The purpose of human object detection is, for example, to "focus the camera on the entire body of a person." It is also assumed that user 0 wishes to detect "a dog," and user 1 wishes to detect "a person's torso."

[0043] FIG. 3 is a flowchart for additional learning (relearning) of a model in the information processing device 11. Here, an example will be described in which user 2 performs additional learning on model 602 and registers model 607 obtained by the additional learning. Specifically, a method for improving the learning accuracy of model 602 (i.e., more appropriately detecting a "full body of a person") will be described. The following processing is started when the information processing device 11 receives an instruction for additional learning of model 602 ("m0002") as a given model from user 2 via the user terminal.

[0044] In S301, the related model data management unit 104 instructs the model management unit 101 to acquire model 602 ("m0002"). Furthermore, it references the traceability information 606 assigned to model 602. As a result, model 601 ("m0001") linked as the initial model is acquired as a related model. Similarly, model 600 ("m0000") linked as the initial model in model 601 is acquired as a related model. In other words, one or more related models (model 601, model 600) linked to model 602 are acquired (model information acquisition) based on the traceability information 606. Furthermore, the related model data management unit 104 acquires evaluation data ("e0002") corresponding to model 602 from the data management unit 102.

[0045] 7 is a diagram showing an example of a dataset used for learning or evaluation in object detection. The dataset used for learning or evaluation includes an image 701 and a true value 704. The image 701 is an image showing a subject that a user wants to recognize. In this example, the dataset includes a person 702, which is a subject that user 2 wants to recognize, as well as a dog 703, which is a subject that does not need to be recognized.

[0046] Furthermore, in order to learn object detection, not only images but also subject information as ground truth (GT) is required. The subject information is, for example, bounding box information (BB information) corresponding to information on the position and size of the object to be detected in the image. BB 705 is a BB corresponding to person 702. That is, the dataset includes one or more pairs of an image and a ground truth corresponding to the image.

[0047] In S302, the learning and evaluation unit 105 acquires an evaluation value indicating the evaluation accuracy of the related models (models 600 to 602) using the evaluation data ("e0002") acquired in S301. For example, as the evaluation value, a value indicating how close each model is to the true value assigned to the evaluation data ("e0002") is derived (calculated). As an example, the IoU (Intersection over Union), which indicates how close the true BB is to the BB estimated by the model, can be derived as the evaluation value. In this case, it is advisable to calculate the average IoU for all BBs assigned to the evaluation data. Other evaluation values ​​that can be used include precision, recall, ROC (Receiver Operating Characteristic curve), and AOU (Area under an ROC curve).

[0048] In S303, the learning and evaluation unit 105 determines whether or not any of the training data is useful for improving the accuracy of model 602 ("m0002") based on the evaluation values ​​obtained from each related model. For example, if the evaluation value for a certain related model is equal to or greater than a predetermined accuracy, the training data used to train the related model is determined to be useful. The reason why data with a higher evaluation value is determined to be useful is that the higher the evaluation value, the more likely it is that the training data used for the corresponding related model has the same true value as the target currently being learned.

[0049] FIG. 8 is a diagram illustrating differences in true values ​​for different objects. Image 801 exemplarily shows true values ​​assigned by user 1. On the other hand, image 802 exemplarily shows true values ​​assigned by user 0. That is, as described above, user 1 wants to detect a "person's torso," and therefore sets BB corresponding to "person's torso" as true value 803 for image 801. On the other hand, user 0 wants to detect a "dog," and therefore sets BB corresponding to "dog" as true value 804 for image 802. However, user 2 wants to detect a "whole person" (corresponding to BB705).

[0050] In this way, even for the same learning data and the same object (a person in this case), different true values ​​may be set depending on the user (users 1 and 2 in this case). Furthermore, it is possible that a true value may be given to an object (a dog in this case) that user 2 did not expect at all, as in true value 804. On the other hand, the evaluation data ("e0002") prepared by user 2 is often assigned a "full-body person" like BB705, just like the learning data ("d0002").

[0051] Therefore, the evaluation values ​​for model 600 ("m0000") and model 601 ("m0001") trained with BB803, which has a different range of true values, and BB804, which has a different target for the true values, tend to be low. However, BB803 and BB705, which assign true values ​​to the same target (a person in this case) but have different ranges, are in an inclusive relationship. Therefore, in evaluation values ​​such as IoU, the evaluation value for a model ("m0001") trained with true values ​​in an inclusive relationship is somewhat high. However, the evaluation value for a model ("m0000") trained with true values ​​that are not in an inclusive relationship is low. In other words, a model with a higher evaluation value for the evaluation data ("e0002") is more likely to have learned true values ​​for the same target (a person in this case). Increasing the amount of training data with similarly defined true values ​​for the same target also increases the variation in the training data. In addition, increasing the variation can suppress overfitting.

[0052] Alternatively, initial models may be traced back from model 602 based on the traceability information 606, and models with evaluation values ​​of a predetermined accuracy or higher may be selected as related models. Then, the training data used to train the related models whose evaluation values ​​are equal to or higher than the predetermined accuracy is determined to be useful and used for training in S304, which will be described later. This makes it possible to select training data for the same subject and with similar definitions.

[0053] Furthermore, by examining the fluctuations in the evaluation values ​​for each model by tracing the traceability information, if a tendency for an almost linear transition is observed, it may be determined that the learning order, like curriculum learning, has had an effect of improving accuracy. By performing additional learning while maintaining the learning order, it is possible to increase the variation in the learning data and improve the learning accuracy while maintaining the accuracy improvement effect of the initial learning of model 602.

[0054] In S304, the learning and evaluation unit 105 acquires (acquires data information) one or more pieces of learning data (data sets) that were determined to be useful in S303, and performs learning. For example, if learning data ("d0001") is determined to be useful in S303, this learning data is added to learning data ("d0002"), and learning is performed. Note that the model to be subjected to additional learning may be model 602 or model 601.

[0055] As a learning method for object detection, for example, a method using a neural network exists. The learning method for object detection using a neural network is described in detail in Literature A. In the evaluation performed by the learning and evaluation unit 105 in S303, the data used for learning or any data may be used. (Reference A) Tian et al., "FCOS: Fully Convolutional One-Stage Object Detection", arXiv:1904.01355, 2019

[0056] In S305, the learning and evaluation unit 105 registers and stores in the model management unit 101 the model 607 (“m0002′”) obtained by the additional learning.

[0057] In S306, the traceability information management unit 103 updates the traceability information. For example, when additional learning is performed on the model 602 using learning data (“d0002” and “d0001”) to create a model 607, the traceability information management unit 103 updates the traceability information as shown in FIG. 6.

[0058] As described above, according to the first embodiment, when determining the training data to be used for additional training (relearning) of a model, the data to be used for additional training is determined using traceability information. The traceability information indicates a record of past edits of each model and data. In particular, based on the evaluation value obtained from a related model of the model to be trained, a related model trained using training data having a true value with a close definition is determined. Additional training (relearning) of a model using the training data used to train the determined related model enables training with an increased variety of training data, thereby improving learning accuracy.

[0059] In the above description, a set of an image and a true value is given as an example of the learning data, but it does not necessarily have to be a set of an image and a true value. For example, it may be a set of a voice or text and a true value, or it may be only an image, a voice, or a text. It may also be a combination of an image and a text. Furthermore, it may be applied not only to image recognition such as the object detection described above, but also to natural language processing such as prediction and recognition of time-series data and text classification. Note that instead of centrally managing the traceability information in the information processing device 11, it may be configured to manage it in a distributed manner using blockchain technology or the like.

[0060] (Second embodiment) In the second embodiment, a form will be described in which learning data linked by traceability information is evaluated using intermediate features. That is, it differs from the first embodiment in that the evaluation method in the learning and evaluation unit 105 is different. The functional configuration (FIG. 1) is the same as in the first embodiment, so a description thereof will be omitted. In addition, a case will be described in which the model information (FIG. 4), data information (FIG. 5), and traceability information (FIG. 6) are also the same as in the first embodiment.

[0061] <Device Operation> 9 is a flowchart of the learning process in the information processing device according to the second embodiment. As in the first embodiment, a situation will be described as an example in which user 2 performs additional learning on model 602 and registers model 607 obtained by the additional learning. The following process starts when information processing device 11 receives an instruction for additional learning on model 602 ("m0002") from user 2 via the user terminal.

[0062] In S901, similarly to the first embodiment, the related model data management unit 104 acquires the model 602 and one or more related models (model 601, model 600) linked to the model 602 based on the traceability information 606. In addition, the related model data management unit 104 acquires, from the data management unit 102, the learning data ("d0001" and "d0000") used in learning the related models (model 601, model 600). In S902, the learning and evaluation unit 105 calculates features related to the training data acquired in S901. Specifically, part or all of the training data acquired in S901 ("d0001" and "d0000") is input to the model 602 ("m0002") that is the target of additional training, and the features extracted by the model at that time are used as the features. In the case of a DNN-based model similar to the object detection described in the first embodiment, the features extracted here are part of the output of each layer obtained during inference ("intermediate features").

[0063] The learning and evaluation unit 105 uses the extracted intermediate features to calculate the similarity between the data ("d0002") and the other data (here, "d0000" and "d0001"). Any method for calculating the similarity can be used as long as it can quantitatively measure the relationship between the intermediate features in the feature space. Here, we assume that COS similarity is used as an example of similarity. Non-Patent Document 1 provides details on a method for comparing intermediate features of DNNs using COS similarity.

[0064] In addition, if each training data set contains a large amount of data, m samples of data from each training data set may be used to calculate the similarity between the training data sets. As a sampling method, it is preferable to use a method that is less likely to bias the population (such as random sampling).

[0065] In S903, the learning and evaluation unit 105 selects the training data whose similarity calculated in S902 is equal to or greater than a predetermined value as related training data. Alternatively, the similarity between training data can be calculated based on the average or variance of each COS similarity and used as the selection criterion. The training data selected in this way is set as related data.

[0066] Similarity comparison will be explained in detail below. Here, as an example, a method for comparing the similarity between data ("d0001") and data ("d0002") will be explained. First, m samples are extracted from data ("d0001"), and m samples are also extracted from data ("d0002"). At this time, intermediate features are calculated for each of the m samples using model 602 ("m0002"). This results in m intermediate features for each of data ("d0001") and data ("d0002"). The COS similarity is calculated for all combinations (m x m combinations) of these intermediate features, and the sum of the COS similarities for all combinations is taken as the similarity between the training data.

[0067] Note that as the number of samples (m) increases, the possibility of outliers being mixed in the samples also increases. Therefore, it is possible to calculate the COS similarity for all combinations (m × m combinations) and select a smaller number of plausible representative values ​​from among them. Specific methods are described in detail in Non-Patent Document 1.

[0068] Thereafter, in steps S904 to S906, additional learning of the model 602 is performed in the same manner as in the first embodiment, the obtained model is registered / stored, and the traceability information is updated.

[0069] As described above, according to the second embodiment, when determining the training data to be used for additional training (relearning) of a model, the data to be used for additional training is determined using traceability information. In particular, the similarity between the training data used in training the model to be trained and the training data used in training each related model is calculated. By additionally training (relearning) the model using training data with higher similarity, it becomes possible to train with a greater variety of training data, thereby improving the training accuracy.

[0070] (Variation) In the above embodiment, the learning of an object detection model has been described as an example, but the learning of other types of models can be similarly applied. Below, as a modified example, a form applied to an image generation model used in an image generation task will be described. Image generation AI is an image generation technology as exemplified by Document B. (Reference B) Rombach et al., "High-Resolution Image Synthesis with Latent Diffusion Models", CVPR 2022, arXiv:2112.10752, 2021

[0071] The functional configuration and operation of the information processing device are the same as those of the first embodiment (FIGS. 1 to 3), and therefore will not be described here. However, the difference is that character strings (text) are used instead of BBs as true value data used for model learning.

[0072] FIG. 10 is a diagram showing an example of training data in a modified example. The training data includes an image 1001 and a true value 1002. The image 1001 is an image showing a subject (a person in this case) that the user wants to recognize. Furthermore, in order to train a model for an image generation task, not only the image but also subject information as a true value (GT) is required. The subject information is, for example, text that represents the subject in the image. In this case, the true value 1002 is specified as the text corresponding to the subject shown in the image 1001, "person."

[0073] Figure 11 is a diagram showing an example of information attached to data in a modified example. Here, it is assumed that user 1 creates a model ("m0001") using data ("d0001") as shown in Figure 10 for an initial model ("m0000") and publishes it on a learning model publishing platform. In this case, user 2, who uses the learning model publishing platform, can create a model ("m0002") for the published model ("m0001") using data ("d0002") that he or she created.

[0074] In this modified example, the true value is not BB but a character string that describes the subject. In this case, the evaluation value can be a value calculated using an error function, or a value obtained by converting each evaluation image into a predetermined intermediate feature (embedded representation) and using the average or variance of these. As an example, we will explain a method of evaluating a model for evaluation data using FID (Fréchet Inception Distance).

[0075] First, two types of embedded representations are calculated: one is obtained by inputting the evaluation data (FIG. 10) into a trained model (here, "m0001"), and the other is obtained by inputting the image output by the model ("m0002") into the trained model. Then, the mean and variance of these embedded representations in the embedded representation space are calculated, and the similarity between the two types of embedded representations is calculated. In the case of a task that requires generating a more plausible image, a lower FID can be said to indicate a higher evaluation value. Then, as in the first embodiment, the training data used for training a model with an evaluation value higher than a predetermined value is determined to be useful, and is used for training the model ("m0002").

[0076] Furthermore, similar to the second embodiment, the model ("m0002") can be used to calculate the similarity for some or all of the training data linked by the traceability information. That is, m samples are extracted for each related model. From the obtained samples, an embedded representation is calculated using the model ("m0002"), and the similarity between the training data is calculated using an index such as the FID described above. It can be said that the higher the similarity of the training data, the closer its characteristics are to the properties of the already trained model ("m0002"). Therefore, if the similarity is less than a predetermined value, the data is determined to be related training data. Details of the training of the model ("m0002") (S304) are described in the aforementioned document B.

[0077] The disclosure of this specification includes the following information processing device, control method, and program. (Item 1) An information processing device that manages a plurality of learning models, A model management means for managing, for each of the plurality of learning models, first information that identifies a dataset used in learning the learning model and second information that identifies an initial model used in learning the learning model; a data management means for managing third information regarding a plurality of data sets identified by the first information for each of the plurality of learning models; a receiving means for receiving an instruction for additional learning using a given learning model included in the plurality of learning models as an initial model; a determination means for determining a dataset to be used for the additional learning from the plurality of datasets based on the first information, the second information, and the third information; Equipped with The determination means determines, as the dataset to be used in the additional learning, a dataset used for learning a learning model whose evaluation accuracy when a given dataset is input to each of one or more learning models used as an initial model of the given learning model is equal to or greater than a predetermined accuracy. 1. An information processing device comprising: (Item 2) The model management means further manages fourth information that identifies a dataset used in evaluation of the learning model; The given dataset is a dataset used in evaluating the given learning model. 2. The information processing device according to item 1, (Item 3) The given data set includes one or more pairs of images and corresponding truth values. 3. The information processing device according to item 1 or 2. (Item 4) the plurality of learning models are object detection models; The true value is bounding box information of the detection target object included in the image. 4. The information processing device according to item 3, (Item 5) the plurality of learning models are image generation models, The true value is a character string indicating a subject included in the image. 4. The information processing device according to item 3, (Item 6) An information processing device that manages a plurality of learning models, A model management means for managing, for each of the plurality of learning models, first information that identifies a dataset used in learning the learning model and second information that identifies an initial model used in learning the learning model; a data management means for managing third information regarding a plurality of data sets identified by the first information for each of the plurality of learning models; a receiving means for receiving an instruction for additional learning using a given learning model included in the plurality of learning models as an initial model; a determination means for determining a dataset to be used for the additional learning from the plurality of datasets based on the first information, the second information, and the third information; Equipped with The determination means determines, as a dataset to be used in the additional learning, a dataset having a similarity to a first dataset used in learning of the given learning model that is equal to or greater than a predetermined value, from among one or more datasets used in learning of each of the one or more learning models used as an initial model of the given learning model. 1. An information processing device comprising: (Item 7) The determining means derives the similarity by comparing a feature obtained by inputting the one or more data sets into the given learning model with a feature obtained by inputting the first data set into the given learning model. 7. The information processing device according to claim 6, (Item 8) A control method for an information processing device that manages a plurality of learning models, comprising: a model information acquisition step of acquiring, for each of the plurality of learning models, first information that identifies a dataset used in learning the learning model and second information that identifies an initial model used in learning the learning model; a data information acquisition step of acquiring third information regarding a plurality of data sets identified by the first information for each of the plurality of learning models; a receiving step of receiving an instruction for additional learning using a given learning model included in the plurality of learning models as an initial model; a determination step of determining a dataset to be used for the additional learning from the plurality of datasets based on the first information, the second information, and the third information; Including, In the determination step, a dataset used for training a learning model whose evaluation accuracy when a given dataset is input to each of one or more learning models used as an initial model of the given learning model is equal to or greater than a predetermined accuracy is determined as the dataset to be used in the additional learning. A control method comprising: (Item 9) A control method for an information processing device that manages a plurality of learning models, comprising: a model information acquisition step of acquiring, for each of the plurality of learning models, first information that identifies a dataset used in learning the learning model and second information that identifies an initial model used in learning the learning model; a data information acquisition step of acquiring third information regarding a plurality of data sets identified by the first information for each of the plurality of learning models; a receiving step of receiving an instruction for additional learning using a given learning model included in the plurality of learning models as an initial model; a determination step of determining a dataset to be used for the additional learning from the plurality of datasets based on the first information, the second information, and the third information; Including, In the determination step, a dataset having a similarity to a first dataset used in training of the given learning model that is equal to or greater than a predetermined value is determined as a dataset to be used in the additional training from among one or more datasets used in training of each of the one or more learning models used as initial models of the given learning model. A control method comprising: (Item 10) 10. A program for causing a computer to execute the control method according to item 8 or 9.

[0078] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0079] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0080] 10 control unit; 11 information processing device; 20 storage unit; 12, 13 user terminal

Claims

1. An information processing device that manages a plurality of learning models, a model management means for managing, for each of the plurality of learning models, first information that identifies a dataset used in learning the learning model and second information that identifies an initial model used in learning the learning model; a data management means for managing third information regarding a plurality of data sets identified by the first information for each of the plurality of learning models; a receiving means for receiving an instruction for additional learning using a given learning model included in the plurality of learning models as an initial model; a determination means for determining a data set to be used for the additional learning from the plurality of data sets based on the first information, the second information, and the third information; Equipped with The determination means determines, as the dataset to be used in the additional learning, a dataset used for learning a learning model whose evaluation accuracy when a given dataset is input to each of one or more learning models used as an initial model of the given learning model is equal to or greater than a predetermined accuracy.

1. An information processing device comprising:

2. The model management means further manages fourth information that identifies a dataset used in evaluation of the learning model; The given dataset is a dataset used in evaluating the given learning model.

2. The information processing apparatus according to claim 1, wherein:

3. The given data set includes one or more pairs of images and corresponding truth values.

2. The information processing apparatus according to claim 1, wherein:

4. the plurality of learning models are object detection models; The true value is bounding box information of the detection target object included in the image.

4. The information processing apparatus according to claim 3,

5. the plurality of learning models are image generation models, The true value is a character string indicating a subject included in the image.

4. The information processing apparatus according to claim 3,

6. An information processing device that manages a plurality of learning models, a model management means for managing, for each of the plurality of learning models, first information that identifies a dataset used in learning the learning model and second information that identifies an initial model used in learning the learning model; a data management means for managing third information regarding a plurality of data sets identified by the first information for each of the plurality of learning models; a receiving means for receiving an instruction for additional learning using a given learning model included in the plurality of learning models as an initial model; a determination means for determining a data set to be used for the additional learning from the plurality of data sets based on the first information, the second information, and the third information; Equipped with The determining means determines, as a dataset to be used in the additional learning, a dataset having a similarity to a first dataset used in learning of the given learning model that is equal to or greater than a predetermined value, from among one or more datasets used in learning of each of the one or more learning models that were used as initial models of the given learning model.

1. An information processing device comprising:

7. The determining means derives the similarity by comparing a feature obtained by inputting the one or more data sets into the given learning model with a feature obtained by inputting the first data set into the given learning model.

7. The information processing apparatus according to claim 6,

8. A control method for an information processing device that manages a plurality of learning models, comprising: a model information acquisition step of acquiring, for each of the plurality of learning models, first information that identifies a dataset used in learning the learning model and second information that identifies an initial model used in learning the learning model; a data information acquisition step of acquiring third information regarding a plurality of data sets identified by the first information for each of the plurality of learning models; a receiving step of receiving an instruction for additional learning using a given learning model included in the plurality of learning models as an initial model; a determination step of determining a dataset to be used for the additional learning from the plurality of datasets based on the first information, the second information, and the third information; Including, In the determination step, a dataset used for training a learning model whose evaluation accuracy when a given dataset is input to each of one or more learning models used as an initial model of the given learning model is equal to or greater than a predetermined accuracy is determined as the dataset to be used in the additional learning. A control method comprising:

9. A control method for an information processing device that manages a plurality of learning models, comprising: a model information acquisition step of acquiring, for each of the plurality of learning models, first information that identifies a dataset used in learning the learning model and second information that identifies an initial model used in learning the learning model; a data information acquisition step of acquiring third information regarding a plurality of data sets identified by the first information for each of the plurality of learning models; a receiving step of receiving an instruction for additional learning using a given learning model included in the plurality of learning models as an initial model; a determination step of determining a dataset to be used for the additional learning from the plurality of datasets based on the first information, the second information, and the third information; Including, In the determination step, a dataset having a similarity to a first dataset used in training of the given learning model that is equal to or greater than a predetermined value is determined as a dataset to be used in the additional training from among one or more datasets used in training of each of the one or more learning models used as initial models of the given learning model. A control method comprising:

10. A program for causing a computer to execute the control method according to claim 8 or 9.

Images