Information processing device and its control method

Abstract

To control disclosure of a learning model.SOLUTION: An information processing apparatus for managing a plurality of learning models disclosed in a model disclosure platform comprises: management means which, with respect to each of the plurality of learning models, manages first information about a learning history of the learning model and second information about a base model used for generation of the learning model; and determination means which determines a disclosure range of each of the plurality of learning models in the model disclosure platform on the basis of both of the first information and the second information.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to a technology for managing learning models and learning data.

Background Art

[0002] In recent years, artificial intelligence (AI) technology using machine learning has achieved remarkable development and been applied to various products and services including image recognition and automatic translation. Recently, an AI called generative AI, which generates content such as text and images based on user input, has also emerged and attracted much attention.

[0003] Many of these AI technologies are implemented by learning models (trained models) created by companies, research institutions, and the like. However, it is expected that it will become common practice in the future for general users to perform machine learning according to their own purposes and create learning models. In line with this, it is also expected that the performance of machine learning by general users and the publication of learning models, etc., will spread via services provided over the Internet. Such services are referred to herein as "learning model publishing platforms".

[0004] On a learning model publishing platform, users can download and use learning models published by other users. At that time, it is also conceivable that paying a part of the download fee as a reward to the model creator will activate model creation by users on the platform. In model creation, it is conceivable that a user uses a learning model published by another user as an initial model, performs additional training according to the user's own purpose, and publishes the obtained learning model. However, published learning models and data used for training need to comply with ethics and must not infringe on copyrights or portrait rights.

[0005] Patent Document 1 discloses a technology that performs AI ethics judgment from learning conditions at the start of machine learning, and executes learning only when it is determined that AI ethics are complied with, thereby enabling creation of a learning model that complies with AI ethics. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2023-64636 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, the AI ​​ethics guidelines that learning models must adhere to may be revised. In the technology described in Patent Document 1, even if the learning model adheres to the guidelines at the time of training, there is no guarantee that a learning model created after the guidelines are revised will comply with the revised AI ethics guidelines.

[0008] Therefore, problems may arise even after a learning model created according to the technology described in Patent Document 1 is published on a learning model publishing platform. Specifically, derivative models created by further training using the published learning model as the initial model may not satisfy the AI ​​ethics after the revision of the rules. Furthermore, there is a risk that derivative models with insufficient additional training (derivative models that are substantially indistinguishable from the initial model) may be published on the learning model publishing platform. Thus, the challenge is to manage and control not only the publication of learning models but also the publication of derivative models.

[0009] This invention has been made in view of these problems and aims to provide a technology for appropriately controlling the publication of learning models. [Means for solving the problem]

[0010] To solve the above-mentioned problems, the information processing device according to the present invention has the following configuration. That is, the information processing device that manages multiple learning models published on the model publishing platform is For each of the aforementioned multiple learning models, a management means for managing first information relating to the learning history of the learning model and second information relating to the base model used to create the learning model, A determination means for determining the scope of publication of each of the plurality of learning models in the model publication platform based on both the first and second pieces of information, Equipped with 、 The determination means determines, based on the first information, disclosure criteria information that shows the difference between the learning history of the learning model of interest and the learning history of the base model identified by the second information used to create the learning model of interest, and determines the scope of disclosure for each of the multiple learning models based on the disclosure criteria information and the second information corresponding to each of the multiple learning models. . [Effects of the Invention]

[0011] According to the present invention, it is possible to provide a technology for appropriately controlling the publication of learning models. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing the functional configuration of an information processing device. [Figure 2] This figure shows an example of the information that can be added to a model. [Figure 3] This figure shows an example of training data. [Figure 4] This figure shows examples of information that can be attached to data. [Figure 5] This figure shows an example of traceability information. [Figure 6] This figure shows an example of publicly available criteria information. [Figure 7] This is a flowchart of the processing in an information processing device (first embodiment). [Figure 8] This is a flowchart of the processing in the information processing device (second embodiment). [Figure 9] This is a flowchart of the processing in the information processing device (third embodiment). [Figure 10] This is a flowchart of the processing in an information processing device (modified version). [Figure 11] This figure shows an example of a traceability information management screen. [Modes for carrying out the invention]

[0013] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and overlapping descriptions are omitted.

[0014] (First Embodiment) As a first embodiment of the information processing apparatus according to the present invention, an information processing apparatus that controls the disclosure range of a learning model will be described below as an example. In particular, an example will be described in which, when learning data includes an image that is ethically or legally defective (in terms of copyright or portrait rights), the disclosure range of the learning model is narrowed (limited disclosure) or disclosure is prohibited (non-disclosure).

[0015] <Overview> The information processing apparatus of the present embodiment manages information (learning data ID) related to data used for training a learning model (trained model) in association with the learning model. Further, when the learning model is a derived model, information (initial model ID) related to the base model (initial model) used is managed in association with the learning model. Then, the information processing apparatus manages traceability information (past history of learning models) based on the respective initial model IDs and learning data IDs of the plurality of learning models, and controls the disclosure range of each learning model based on the traceability information.

[0016] <Terminology> "Ethically inappropriate images" include, for example, images that suggest violence or discrimination, or images with sexual depictions. "Copyright and portrait rights inappropriate images" are images that infringe on copyright or portrait rights. For example, images whose reproduction or widespread public use, such as machine learning, is not permitted by the rights holder. Examples include photographs of affiliated entertainers published by talent agencies or images of animated characters. In the following explanation, "ethically inappropriate images" and "copyright and portrait rights inappropriate images" may be collectively referred to as "NG images."

[0017] <Device configuration> Figure 1 is a block diagram showing the functional configuration of the information processing device 11. The information processing device 11 manages the "model" and the "data" used for training and evaluating the model. The information processing device 11 also performs model training and data editing in response to requests from users (users of the learning model publishing platform), and edits model information (publication decision) in response to requests from administrators (administrators of the learning model publishing platform).

[0018] The information processing device 11 is connected to the administrator terminal 13 and user terminal 14 via a communication network such as a local area network (LAN) or the internet. The method of connection is not particularly limited. For example, they may be connected separately by wired connections or by wireless communication. Also, although Figure 1 shows one administrator terminal and one user terminal, it is not limited to this. The administrator terminal 13 and user terminal 14 are envisioned to be information terminals such as personal computers (PCs), mobile phones, and tablet devices.

[0019] The information processing device 11 includes a control unit 10 and a storage unit 12. The control unit 10 can be implemented, for example, by a central processing unit (CPU) executing various programs stored in the storage unit 12. The storage unit 12 is a large-capacity storage device such as random access memory (RAM), read-only memory (ROM), or a hard disk drive (HDD) or solid-state drive (SSD).

[0020] The control unit 10 includes a model management unit 101, a data management unit 102, a traceability information management unit 103, a public access standard information output unit 104, a public access determination unit 105, a registration unit 106, a learning and evaluation unit 107, a display control unit 108, and a communication control unit 109. As described above, each of these functional units can be implemented, for example, by a CPU executing various programs. However, some or all of them may be implemented by hardware such as application-specific integrated circuits (ASICs).

[0021] The model management unit 101 manages the models registered in the information processing device 11. In this embodiment, the model is assumed to be an "object detection model" intended for object detection from images, but it is not limited to this. It may also be a speech recognition model, a natural language processing model, or a generative artificial intelligence (AI) model.

[0022] Figure 2 shows an example of model information 201 and 202 assigned to two models. Model information is attribute information of the model and includes information such as model ID, user ID, initial model ID, training data ID, tags, task, training parameters, evaluation data ID, evaluation results, and publication scope. In other words, it includes the training history, training data ID, and training parameters.

[0023] The Model ID is a model-specific identifier (ID) used to identify each model. The User ID is a user-specific ID used to identify the user who trained each model. The Initial Model ID specifies the model that served as the initial parameter when training each model.

[0024] In Figure 2, in model information 201, the model ID is "m0001" and the initial model ID is "m0000". Also, in model information 202, the model ID is "m0002" and the initial model ID is "m0001". Therefore, the model with model ID "m0002" is a model obtained by training with the model with model ID "m0001" as the initial model.

[0025] The training data ID is a unique ID for the training data used to train each model. Tags contain information about the objects targeted by each model, as well as information necessary for users to search for models. The task describes the type of processing each model targets (e.g., object detection, image generation). Training parameters are the hyperparameters used during training each model (e.g., learning rate, number of training iterations). The evaluation data ID is a data ID that specifies the dataset used to evaluate each model. Evaluation results describe the numerical results obtained from the evaluation. The access scope indicates the access scope (the scope of access permitted) for each model. For example, it may be "all" (public to everyone), "private," or one or more user IDs. If user IDs are listed, only users with the listed IDs can access the model, and other users cannot.

[0026] As shown in Figure 2, model information 201 and 202 are both model information for an object detection model. For example, if the model is of a different type (such as a speech recognition model), the items included in the model information may differ from the example in Figure 2.

[0027] The data management unit 102 manages the data registered in the information processing device 11. The data in this embodiment is image data for object detection.

[0028] Figure 3 shows an example of training data. The training data includes image 301 and true values ​​304. Image 301 is an image showing the subjects that the user wants to recognize (in this case, a person 302 and a dog 303). In addition to images, subject information as true values ​​is also required to train object detection. Subject information is, for example, bounding box (BB) information corresponding to the position and size of the subjects in the image. BB305 is the BB corresponding to person 302, and BB306 is the BB corresponding to dog 303.

[0029] Figure 4 shows examples of data information 401 and 402 assigned to data (image dataset). Data information is attribute information of the data and includes data ID, user ID, tags, task, and image ID.

[0030] The Data ID is a unique identifier for each piece of data. The User ID is a unique identifier for the user who created each piece of data. Tags contain information about the object targeted by each piece of data, as well as information necessary for users to search for data. Tasks describe the type of processing each piece of data is targeted by (e.g., object detection, image generation). Image IDs are unique identifiers for one or more images contained in the data. Each piece of data contains one or more image IDs.

[0031] As shown in Figure 4, data information 401 and 402 are both data information for data used in the object detection model. For example, if the data is used in other types of models (such as speech recognition models), the items included in the data information may differ from the example in Figure 4.

[0032] The data management unit 102 accepts data registrations from users and registers new datasets.

[0033] The traceability information management unit 103 manages the traceability information of the models. Here, traceability information refers to information that shows the history of how each model was created in the past (i.e., the past history of the model), and is information that is generated based on model information, for example.

[0034] Figure 5 shows an example of traceability information. In Figure 5, the arrows indicate the derivation relationships of each model. For example, Model 502 (Model ID: "m0002") is a derived model created by further training using Model 501 (Model ID: "m0001") as the initial model. Similarly, Model 503 is a derived model of Model 502, and Model 504 is a derived model of Model 501. Note that in Figure 5, the initial model ID shown for Model 501 is "-", which indicates that the initial model is an untrained model.

[0035] Thus, since traceability information shows the records of how each model has been edited in the past, it is possible to identify one or more models used to create the model in question by tracing the traceability information. For example, from the traceability information shown in Figure 5, it can be said that models 502 to 504 are all derived models of model 501. Furthermore, the training data IDs of each model in Figure 5 can be used to identify the data used in training each model, and the relationship between the models and the data can be shown as indicated by the dotted lines in Figure 5.

[0036] The public access criteria information output unit 104 outputs public access criteria information, which is used as one of the criteria for determining whether a model should be made public. As will be described in detail later, the public access criteria information is output for each model based on the model information obtained from the model management unit 101 and the data information obtained from the data management unit 102.

[0037] Figure 6 shows an example of disclosure criteria information. For example, disclosure criteria information indicates whether each model is using a specific image (an image with an image ID specified by the administrator terminal 13) as training data. For example, suppose the image with the image ID shown in bold in Figure 5 ("imgAAAA1002") is an "NG image," and the administrator has designated this image as unusable via the administrator terminal 13. From Figure 5, the data IDs of the data containing this image are "d0002" and "d0004," and the models that use this data are Model 502 and Model 504. Therefore, as shown in Figure 6 (center column), the disclosure criteria information is "FALSE" for Model 501 and Model 503, and "TRUE" for Model 502 and Model 504.

[0038] The disclosure determination unit 105 makes a disclosure determination for the model based on the disclosure criteria information (first determination result) output by the disclosure criteria information output unit 104 and the determination result based on the traceability information managed by the traceability information management unit 103 (second determination result). Here, as shown in Figure 6 (right column), the determination result based on the traceability information is whether or not each model is a derived model of a model trained using "NG images". For example, referring to Figure 5, since model 503 is a derived model of model 502, the determination result is "TRUE".

[0039] The disclosure determination unit 105 then makes a disclosure determination for the model based on the determination results from the disclosure criteria information and traceability information. Here, models 502 and 504, whose disclosure criteria information is "TRUE," are determined to be "private," while model 503, whose disclosure criteria information is "FALSE" but whose traceability information is "TRUE," is determined to be "limited disclosure (disclosure to one or more specific user IDs)." Models that do not fall into any of these categories are determined to be "fully public."

[0040] The registration unit 106 updates the value of the public access range in the model information of each model registered in the information processing device 11 based on the determination result (public, private, limited access) of the public access determination unit 105.

[0041] The learning and evaluation unit 107 performs model training and evaluation based on the models and data registered in the information processing device 11.

[0042] The display control unit 108 controls the display of results from the information processing device 11 in response to requests from the administrator terminal 13 and the user terminal 14. For example, it also generates the management screen (described in the fourth embodiment) that is displayed on the administrator terminal 13. The communication control unit 109 controls the sending and receiving of information between the administrator terminal 13 and the user terminal.

[0043] The memory unit 12 stores various programs executed by the CPU as described above, as well as models, model information, data, data information, traceability information, and the like.

[0044] <Device Operation> Figure 7 is a flowchart of the processing in the information processing device 11. Figure 7(a) is a flowchart for learning and registering a model, and Figure 7(b) is a flowchart for determining / updating the scope of public access to the model.

[0045] First, referring to Figure 7(a), we will explain the process when a user operates the user terminal 14, registers data in the information processing device 11, and trains a model. Here, we will explain the situation in which new data (dataset) is registered and one of the models already registered with that data is further trained.

[0046] In S701, when the registration unit 106 receives a data addition request from the user terminal 14, it registers the data in the storage unit 12. The registered data is registered with data information such as that shown in data information 401. When registering data, information such as the scope of publication of the data, tags, and tasks can be added. Here, the data ID of the data registered by the user is "d0001".

[0047] In S702, when the learning and evaluation unit 107 receives a learning execution request from the user terminal 14, it performs learning using the data registered by the user. For example, it receives a model specification from the user (model ID is "m0000") and performs learning on that model using data (data ID is "d0001"). It is assumed that the model with model ID "m0000" is an untrained model and has been pre-registered in the storage unit 12.

[0048] Regarding object detection learning methods, for example, there are methods using neural networks (NN). For details on object detection learning methods using NNs, please refer to reference A. The learning and evaluation unit 107 may evaluate the trained model using the data used for training or arbitrary data.

[0049] (Reference A) Tian et al., "FCOS: Fully Convolutional One-Stage Object Detection", arXiv:1904.01355, 2019 In S703, the registration unit 106 registers the model learned by the user in the storage unit 12. When registering a model, model information such as that shown in model information 201 is assigned. The user ID is the ID of the user operating the user terminal 14.

[0050] In S704, the traceability information management unit 103 updates the traceability information. Specifically, as shown in Figure 5, it manages the model 501 (model ID "m0001") by linking it to the data (data ID "d0001").

[0051] As a result, the data prepared by the user (data ID "d0001") and the model 501 trained using that data (model ID "m0001") are registered in the information processing device 11.

[0052] Next, referring to Figure 7(b), we will explain the process for determining / updating the scope of model publication. Here, we will explain the situation when an administrator newly specifies (registers) an NG image.

[0053] In S711, when the registration unit 106 receives a request to register an NG image from the administrator terminal 13, it registers the NG image. For example, it may directly receive the image ID of the NG image, or it may display multiple images on the display unit (not shown) of the administrator terminal 13 and receive the selection of an NG image from the administrator. Here, "imgAAAA1002" shown in Figure 5 is received as the image ID of the NG image and registered.

[0054] In S712, the data management unit 102 obtains the data ID of the data containing the image ID of the NG image. In the situation shown in Figure 5, "d0002" and "d0004" are obtained.

[0055] In S713, the model management unit 101 obtains model IDs that have the data ID acquired in S712 as their training data ID. In the situation shown in Figure 5, "m0002" and "m0004" are obtained.

[0056] In S714, the traceability information management unit 103 acquires traceability information. Here, the traceability information shown in Figure 5 is acquired.

[0057] In S715, the public standard information output unit 104 outputs public standard information for each model ID obtained in S713. Here, "TRUE" is output for models 502 and 504 corresponding to the model IDs obtained in S713, and "FALSE" is output for the other models (Figure 6 (center column)).

[0058] In S716, the disclosure determination unit 105 makes a disclosure determination for the models stored in the storage unit 12 based on the disclosure criteria information output in S715 and the traceability information acquired in S714. First, based on the traceability information acquired in S714, it determines whether each model is a derived model of a model trained using "NG images" (derived model information). Here, "TRUE" is output for model 503, which is the currently focused learning model, and "FALSE" is output for the other models (Figure 6 (right column)). Then, a disclosure determination is made based on both pieces of information for each model (disclosure criteria information and derived model information).

[0059] In S717, the registration unit 106 updates the value of the disclosure range in the model information of each model based on the result of the disclosure determination in S716.

[0060] As explained above, according to the first embodiment, it is possible to effectively determine the scope of disclosure of the model that used the NG images for training and its derived models.

[0061] (Extreme Variation 1-1) In the aforementioned disclosure determination unit 105, models whose disclosure criteria information is "FALSE" and whose derived model information is "TRUE" were determined to be for limited disclosure. However, the model performance verification of the model may be performed before determining that it is for limited disclosure, and this may be the final disclosure determination result.

[0062] Here, model performance verification is the verification of whether each model has the performance specified by the administrator. For example, if the model is a personal authentication model, verification is performed to determine whether it has the performance to identify the person in the image specified on the administrator terminal 13 (for example, the person included in the NG image mentioned above).

[0063] The disclosure determination unit 105 then sets models whose model performance verification result is "TRUE" (= has the ability to identify a person) as "private" and models whose result is "FALSE" as "fully public" or "limited public".

[0064] (Second Embodiment) In the second embodiment, other information processing devices that control the scope of access to a learning model are described below as examples. In particular, an example of setting the access scope to "limited access" or "private access" based on the generated images obtained from the execution results of the model will be described.

[0065] <Device configuration> The functional configuration of the information processing device according to the second embodiment is the same as that of the first embodiment (Figure 1), so a detailed explanation will be omitted. Below, we will mainly describe the functional parts that differ in operation from those of the first embodiment. In the following description, the model will be an "image generation model" that generates illustrations and photographs based on input text information, but it is not limited to this.

[0066] The public access criteria information output by the public access criteria information output unit 104 according to the second embodiment is the similarity between the correct image and the generated image for a given keyword (for example, the name of an animation character that is not permitted to be used due to copyright restrictions). Here, the generated image is the image generated when the model executes the keyword entered from the administrator terminal 13. Note that the keyword may be a string of characters from a watermark image embedded in the image (such as a logo or copyright notice).

[0067] The disclosure determination unit 105 performs a disclosure determination on the models registered in the storage unit 12 based on the disclosure criteria information output by the disclosure criteria information output unit 104 and the traceability information management unit 103.

[0068] As mentioned above, the criteria information for publication is the similarity between the correct image and the generated image. Therefore, the correct image indicated by the keyword entered from the administrator terminal 13 is prepared and registered in advance. For example, if the entered keyword is "Character A", the correct image of Character A is prepared and registered in advance.

[0069] The similarity calculation unit included in the public access determination unit 105 calculates the similarity between the correct image and the generated image, and if the similarity exceeds a predetermined threshold, the similarity determination is set to "TRUE". When the similarity determination is "TRUE", it indicates that there is a high possibility that the training data contains images that are not permitted to be used under copyright law (although they are not identical to the registered NG image). Therefore, the public access determination unit 105 determines that the model for which the similarity determination is "TRUE" and its derivative models are "private" or "limited access". Models that do not fall into either of these categories are determined to be "fully public".

[0070] In this example, the similarity between the correct image for the input keyword and the generated image was used as the determination criterion, but this is not the only option. Alternatively, a classification unit could be provided in the public determination unit 105, and the determination criterion could be whether the classification result obtained by processing the generated image with a classifier matches the keyword.

[0071] <Device Operation> Of the processing steps in the information processing device according to the second embodiment, the process of learning and registering a model is the same as in the first embodiment (Figure 7(a)), so a description will be omitted.

[0072] Figure 8 is a flowchart for determining / updating the scope of model disclosure in the second embodiment.

[0073] In S801, when the public access information output unit 104 receives a keyword registration request from the administrator terminal 13, it registers the keyword.

[0074] In S802, when the public determination unit 105 receives a registration request for a correct image corresponding to a keyword registered in S801 from the administrator terminal 13, it registers the correct image in association with the keyword.

[0075] In S803, the traceability information management unit 103 acquires traceability information. Here, the traceability information shown in Figure 5 is acquired. In the second embodiment, the system may be configured to acquire only the derivative information between models (arrows in Figure 5).

[0076] In S804, the public reference information output unit 104 inputs the keywords registered in S801 into the public model stored in the storage unit 12 and generates an image. Then, it outputs the similarity between the ground truth image and the generated image for the keyword as public reference information.

[0077] In S805, the disclosure determination unit 105 makes a disclosure determination for the model stored in the storage unit 12 based on the disclosure criteria information output in S804 and the traceability information acquired in S803. The detailed operation of the disclosure determination in the disclosure determination unit 105 is as described above.

[0078] In S806, the registration unit 106 updates the value of the disclosure range in the model information of each model based on the result of the disclosure determination in S805.

[0079] As described above, according to the second embodiment, keywords related to the NG image are input to the model, and the similarity between the generated image obtained and the correct image for that keyword is calculated. Based on the similarity, the scope of publication for the model and its derivative models is determined. In particular, if the similarity exceeds a predetermined threshold, it is determined that there is a high possibility that images similar to the NG image were used in the model's training, and the model and its derivative models are set to "private" or "limited publication." Therefore, it is possible to effectively determine the scope of publication for models and their derivative models that use not only the registered NG image but also images similar to that NG image.

[0080] (Third embodiment) In the third embodiment, another information processing device that controls the scope of disclosure of the learning model will be described below as an example.

[0081] <Overview> One challenge with publishing learning models on a platform is that derivative models with virtually identical performance to highly-rated models may be registered on the platform. In such cases, the rewards that should rightfully be paid to the user who created the highly-rated model end up being paid to the third party who registered the derivative model. Therefore, it is necessary to ensure a sense of fairness and satisfaction for the user who created the highly-rated model.

[0082] Therefore, when a derived model (also called a child model) is newly registered in the information processing device of this embodiment, the information processing device checks the degree of similarity between the derived model and the initial model (also called a parent model). Then, if the degree of similarity between the derived model and the initial model is higher than a predetermined threshold (i.e., substantially identical to the initial model), the information processing device sets the derived model to "limited access" or "private access".

[0083] <Device configuration> The functional configuration of the information processing device according to the third embodiment is the same as that of the first embodiment (Figure 1), so a detailed explanation will be omitted. Below, we will mainly describe the functional parts that differ in operation from those of the first embodiment. In the following description, the model will be the "object detection model" as in the first embodiment, but it is not limited to this.

[0084] In the third embodiment, the model management unit 101 acquires model information for the model (child model) and its parent model registered by the user on the user terminal 14 from the storage unit 12, based on the traceability information managed by the traceability information management unit 103. The acquired model information is passed to the public standard information output unit 104.

[0085] The public access criteria information output unit 104 obtains the learning history (learning data ID, learning parameters) from the model information of both the parent model and the child model. It then determines whether the learning data and learning parameters are the same between the two models and outputs the "determination result" and the "number of learning iterations of the child model" as public access criteria information.

[0086] The identity of the training data between the two models is determined by the identity / similarity of the training data used by each model during training. For example, this can be determined by whether the training data IDs are the same. Alternatively, the identity / similarity of multiple image IDs included in the training data of each model can be determined. For example, if multiple image IDs included in the training data of the child model and multiple image IDs included in the training data of the parent model overlap by a predetermined percentage or more (e.g., 99% or more) (if the difference is less than the predetermined percentage), the data of both models may be considered substantially identical.

[0087] Furthermore, the identity of the learning parameters between the two models is determined by comparing the learning parameters, excluding the number of training iterations. For example, if only the learning parameters that do not affect model performance (such as the storage location of log data during the training process) differ, the learning parameters of both models can be considered identical.

[0088] The disclosure determination unit 105 makes a disclosure determination for the child model based on the disclosure criteria information output by the disclosure criteria information output unit 104. For example, if the determination result of the identity of the learning data and learning parameters of the parent model and the child model is "TRUE" and the number of learning iterations of the child model is less than a predetermined number, the child model is set to "non-disclosure". In other words, in this case, the child model has hardly progressed in learning from the parent model, and the two models can be considered to be almost identical.

[0089] <Device Operation> Figure 9 is a flowchart of the processing in the information processing device 11 according to the third embodiment. Figure 9(a) is a flowchart for learning and registering a model, and Figure 9(b) is a flowchart for determining / updating the scope of public access to the model.

[0090] First, referring to Figure 9(a), we will explain the process when a user operates the user terminal 14, registers data in the information processing device 11, and learns the model. Since steps S901 to S904 are the same as steps S701 to S704 in the first embodiment (Figure 7(a)), we will omit their explanation.

[0091] In S905, the publication determination unit 105 executes the publication determination process for the model shown in Figure 9(b). That is, the publication determination is performed at the time of model registration. This prevents inappropriate models (child models that are substantially identical to the parent model) from being published.

[0092] In S911, the model management unit 101 acquires traceability information from the traceability information management unit 103.

[0093] In S912, the model management unit 101 obtains model information for the model (child model) registered in S903 and its initial model (parent model) from the acquired traceability information. The acquired model information is passed to the public standard information output unit 104.

[0094] In S913, the public standard information output unit 104 obtains the training data ID and training parameters from the model information of both the parent model and the child model. It then outputs the "determination result" of the identity of the training data of both models and the "number of training sessions for the child model" as public standard information.

[0095] In S914, the publication determination unit 105 makes a publication determination for the child model based on the publication criteria information. The detailed operation of the publication determination in the publication determination unit 105 is as described above.

[0096] In S915, the registration unit 106 updates the value of the public access range in the child model's model information based on the result of the public access determination in S914.

[0097] In the above explanation, the public access determination of a registered model is performed at the time of new model registration, but this is not the only way. For example, at any time, the administrator terminal 13 can select any model and perform the public access determination process shown in Figure 9(b). In this case, if the public access determination result for the selected model is "private" or "limited access," derived models of that model may also be set to "private" or "limited access" based on traceability information.

[0098] As described above, according to the third embodiment, when a child model is newly registered in the information processing device, the degree of similarity between the child model and its parent model is checked, and the scope of public access for the child model is determined based on the degree of similarity. In particular, if the degree of similarity exceeds a predetermined threshold, the child model is set to "limited access" or "private access". This makes it possible to prevent child models that are substantially identical to the parent model from being made public.

[0099] (Variation 3-1) In the above explanation, the public access criteria information output unit 104 output the "determination result" of the identity of the training data of both models and the "number of training sessions for the child model" as public access criteria information, but other information may also be output as public access criteria information.

[0100] For example, object detection processing is performed on the evaluation image indicated by the evaluation data ID in the child model's model information using both models. Then, the Intersection over Union (IoU) between the detection result (BB) from the child model and the detection result (BB) from the parent model is calculated, and the IoU value may be output as publicly available reference information. IoU is an index that represents the degree of overlap between the two detection results (BB), and IoU is a value between 0 and 1, with a larger IoU indicating a greater degree of overlap between the BBs. If there are multiple evaluation images, the average IoU for each evaluation image may be output as publicly available reference information.

[0101] In this case, the public determination unit 105 evaluates the similarity between the two models as high if the IoU value (or average value) exceeds a predetermined value (for example, 0.99). In other words, if the detection performance for detection targets in the same image is about the same for both models, the two models can be considered to be approximately identical.

[0102] (Variation 3-2) Furthermore, the public standard information output unit 104 may calculate the weight difference between the parent model and the child model's neural networks (NN) (for each layer and kernel) and output the sum of the absolute values ​​of these differences as public standard information.

[0103] In this case, the disclosure determination unit 105 evaluates the similarity between the two models as high if the calculated sum is less than a predetermined value. That is, because the difference in the weights of the neural networks of the two models is small, the two models can be considered to be almost identical.

[0104] (Modified example 3-3) Furthermore, the learning and evaluation unit 107 may be configured to store time-series data of the loss during learning in the storage unit 12 during model learning. Here, the loss is the difference between the predicted value by the neural network and the true value of the learning data.

[0105] The public access information output unit 104 then calculates the public access information from the following formula (1) based on the learning rate of the model information's learning parameters and the time-series data of the loss. Public access criteria information = lr × Σ|∂Loss / ∂w| ···(1)

[0106] In equation (1), lr is the learning rate, Loss is the loss at a given time, w is the weight of the model's neural network (NN), ∂ is the partial derivative, || is the absolute value, and Σ represents the time cumulative value. In other words, equation (1) is the value obtained by multiplying the time cumulative absolute value of the loss gradient by the learning rate. A large value means that the NN weights change significantly, while a small value means that the weights change only slightly. In other words, the publicly available reference information corresponds to the learning progress, indicating how far the learning has progressed.

[0107] The disclosure determination unit 105 uses the traceability information managed by the traceability information management unit 103 to determine whether a model registered in S903 has a parent model, and makes a disclosure determination for the model based on the disclosure criteria information described above. If the calculated disclosure criteria information is less than a predetermined value, it can be determined that the weights of the model's neural network have hardly changed from the parent model in the child model, and the similarity between the two models is high.

[0108] Figure 10 is a flowchart for determining / updating the scope of model publication in Modification 3-3. The process of training and registering the model is the same as in the third embodiment (Figure 9(a)), so the explanation is omitted.

[0109] In S1011, the model management unit 101 obtains model information for newly registered models from the storage unit 12 via the user terminal 14. The model information is then passed to the public standard information output unit 104.

[0110] In S1012, the learning and evaluation unit 107 retrieves time-series data of the loss during training for the model registered in S903 from the storage unit 12. The time-series data of the loss is passed to the public standard information output unit 104.

[0111] In S1013, the public access information output unit 104 calculates the public access information using formula (1).

[0112] In S1014, the disclosure determination unit 105 acquires the traceability information managed by the traceability information management unit 103.

[0113] In S1015, the publication determination unit 105 checks whether a parent model exists for the model registered in S903. If a parent model exists, it makes a publication determination for the model registered in S903 based on the publication criteria information calculated in S1013.

[0114] In S1016, the registration unit 106 updates the value of the public access range in the model information of the model registered in S903, based on the result of the public access determination in S1015.

[0115] (Fourth embodiment) In the fourth embodiment, other information processing devices that control the scope of disclosure of the learning model will be described below as examples. In particular, a graphical user interface (GUI) that manages the traceability information provided by the display control unit 108 to the administrator terminal 13 and the user terminal 14 will be described. Note that the administrator terminal 13 and the user terminal 14 are assumed to be PCs, but are not limited to them. Also, the colors and line types on the GUI are examples only and are not limited to them.

[0116] <Traceability Information Management Screen> Figure 11 shows an example of the traceability information management screen 1100. As described above, the management screen 1100 is provided to the administrator terminal 13 and / or user terminal 14 by the display control unit 108. The management screen 1100 includes a traceability information screen 1110 and a detailed information screen 1120. For example, the detailed information screen 1120 displays detailed information of the GUI component selected by the user on the traceability information screen 1110.

[0117] The traceability information screen 1110 displays models 1101 to 1104, represented by rectangles, and the derivation relationships between the models are indicated by arrows according to the traceability information (Figure 5). A cursor 1105 is also displayed to indicate the position relative to the management screen 1100. The cursor 1105 can be moved by the user, for example, by operating a mouse (not shown). The traceability information screen 1110 also has buttons 1106 to 1108 for instructing the execution of various processes.

[0118] Models 1101 through 1104 display their respective model IDs and access permissions. Here, the access permissions are displayed as text within the rectangle representing each model, but it is also possible to configure the system to display the access permissions as the color of the rectangle. For example, a model with an access permission of "Public (all)" could be displayed as a blue rectangle, a model with an access permission of "(Limited access to specific users)" as a light blue rectangle, and model 1104 with an access permission of "Private" as a gray rectangle.

[0119] The arrows connecting the models indicate the derivation relationships between the trained models. The model connected to the base of the arrow is the initial model, and the model connected to the tip of the arrow is a derived model generated based on the initial model. In other words, in Figure 11, model 1102 is a derived model of model 1101, model 1103 is a derived model of model 1102, and model 1104 is a derived model of model 1101. Model 1101 does not have a corresponding initial model and indicates that it is an untrained model (for example, a model in which the NN weights are set to random values).

[0120] Figure 11 shows that Model 1101 is selected by cursor 1105. To clearly indicate that it is selected, the rectangle representing Model 1101 is displayed with a thick border. Also in Figure 11, the model information for Model 1101 is displayed on the detailed information screen 1120.

[0121] When a model is selected, if the user clicks the mouse button on the learning button 1106, the user is redirected to the learning management screen (not shown), and the learning process using the selected model 1101 as the initial model can begin. Also, when a model is selected, if the user clicks the evaluation button 1107, the user is redirected to the evaluation management screen (not shown), and the evaluation process using the evaluation data specified in the evaluation data ID can be executed for the selected model 1101. Furthermore, when a model is selected, if the user clicks the publication decision button 1108, the user is redirected to the publication decision management screen (not shown), and the publication decision process for the selected model 1101 and its derived models can be executed.

[0122] As described above, according to the fourth embodiment, a GUI can be provided (to the administrator terminal 13 and / or user terminal 14) that allows users to easily understand the publication status of each model and the derivation relationships between each model. Furthermore, through the GUI, users can intuitively check detailed information of each model and perform various processes (learning, evaluation, publication decision) on each model.

[0123] The disclosures herein include the following information processing devices, control methods, and programs. (Item 1) An information processing device for managing multiple learning models published on a model publishing platform, For each of the aforementioned multiple learning models, a management means for managing first information relating to the learning history of the learning model and second information relating to the base model used to create the learning model, A determination means for determining the scope of publication of each of the plurality of learning models in the model publication platform based on both the first and second pieces of information, An information processing device characterized by comprising: (Item 2) The first piece of information mentioned above includes information about the dataset used to train the learning model, The determination means determines, based on the first information, disclosure criteria information indicating whether specific data is used in the learning of each of the plurality of learning models, and determines the scope of disclosure for each of the plurality of learning models based on the disclosure criteria information corresponding to each of the plurality of learning models and the second information. The information processing device described in item 1, characterized by the features described herein. (Item 3) The system further includes a means for receiving the specification of the aforementioned specific data from the administrator of the model publishing platform. The information processing device described in item 2, characterized by the features described herein. (Item 4) The aforementioned specific data is data whose use is restricted ethically and / or legally. The determination means determines, as the disclosure criteria information, to be limited disclosure that narrows the scope of disclosure or non-disclosure that prohibits disclosure, when the specific data is used, and determines, as the disclosure criteria information, to be full disclosure without any restrictions on the scope of disclosure, when the specific data is not used. An information processing device according to item 2 or 3, characterized by the features described herein. (Item 5) The determination means determines that the learning model is either publicly accessible or private if the base model identified by the second information used to create the learning model has been determined to be either publicly accessible or private. The information processing device described in item 4, characterized by the features described herein. (Item 6) The system further includes verification means for verifying whether the learning model of interest possesses the model performance specified by the administrator of the aforementioned model publishing platform. The decision means determines whether to restrict access to the learning model of interest having the aforementioned model performance, thereby limiting its public access, or to keep it private, thereby prohibiting its public access. The information processing device according to item 2, characterized in that it is a processing device. (Item 7) The determination means determines, based on the first information, disclosure criteria information that shows the difference between the learning history of the learning model of interest and the learning history of the base model identified by the second information used to create the learning model of interest, and determines the scope of disclosure for each of the multiple learning models based on the disclosure criteria information and the second information corresponding to each of the multiple learning models. The information processing device described in item 1, characterized by the features described herein. (Item 8) The first piece of information described above includes the dataset and training parameters used in training the learning model, and the number of training iterations. The determination means determines, if the difference between the dataset and learning parameters used to train the focus learning model and the dataset and learning parameters used to train the base model is less than a predetermined percentage and the number of training iterations of the focus learning model is less than a predetermined number of iterations, to restrict access to the focus learning model (limited access) or prohibit access (private access). The information processing device described in item 7, characterized by the features described herein. (Item 9) If the base model used to create the learning model has been determined to be either publicly accessible or private, the determination means determines that the learning model is either publicly accessible or private. The information processing device described in item 8, characterized by the features described herein. (Item 10) The first piece of information includes the detection results obtained by a learning model on a given dataset. The determination means determines the Intersection over Union (IoU) between the detection result of the learning model of interest and the detection result of the base model identified by the second information used to create the learning model of interest as the public access criteria information for the learning model of interest, and determines the scope of public access for each of the learning models The information processing device described in item 1, characterized by the features described herein. (Item 11) The first piece of information mentioned above includes the weights of the neural networks (NNs) that make up each learning model. The determination means determines the sum of the absolute values ​​of the differences between the weights of the neural network constituting the learning model of interest and the weights of the neural network constituting the base model identified by the second information used to create the learning model of interest as the public access criteria information for the learning model of interest, and determines the scope of public access for each of the multiple learning models based on the public access criteria information and the second information corresponding to each of the multiple learning models. The information processing device described in item 1, characterized by the features described herein. (Item 12) The first piece of information includes time-series data of the learning rate and loss of the learning parameters during training for each learning model. The determination means determines the learning progress calculated based on the learning rate of the learning model of interest and the time-series data as the public access criteria information for the learning model of interest, and determines the scope of public access for each of the multiple learning models based on the public access criteria information corresponding to each of the multiple learning models and the second information. The information processing device described in item 1, characterized by the features described herein. (Item 13) An information processing device for managing multiple learning models published on a model publishing platform, The aforementioned multiple learning models are object detection models for detecting objects from images, The aforementioned information processing device is A registration means for registering keywords specified by the administrator of the aforementioned model publishing platform and the correct images corresponding to those keywords, For each of the aforementioned multiple learning models, a management means manages first information regarding the similarity between the generated image generated by inputting the keyword into the learning model and the ground truth image, and second information regarding the base model used to create the learning model. A determination means for determining the scope of publication of each of the plurality of learning models in the model publication platform based on both the first and second pieces of information, An information processing device characterized by comprising: (Item 14) The model publishing platform further comprises a screen generation means for generating an administration screen that is displayed on an administrator terminal operated by the administrator of the aforementioned model publishing platform, The management screen is configured to display the derivation relationships between the multiple learning models and the publication status of each of the multiple learning models as a graphical user interface (GUI) based on the second information. An information processing device according to any one of items 1 to 13, characterized by the above. (Item 15) A method for controlling an information processing device that manages multiple learning models published on a model publishing platform, For each of the aforementioned multiple learning models, the process involves obtaining a first piece of information regarding the learning history of the learning model and a second piece of information regarding the base model used to create the learning model. A decision step of determining the scope of publication for each of the multiple learning models in the model publication platform based on both the first and second pieces of information, A control method characterized by including (Item 16) A program to cause a computer to execute the control method described in item 15.

[0124] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0125] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of symbols]

[0126] 10 Control unit; 11 Information processing unit; 12 Storage unit; 13 Administrator terminal; 14 User terminal

Claims

1. An information processing device for managing multiple learning models published on a model publishing platform, For each of the aforementioned multiple learning models, a management means for managing first information relating to the learning history of the learning model and second information relating to the base model used to create the learning model, A determination means for determining the scope of publication of each of the plurality of learning models in the model publication platform based on both the first and second pieces of information, Equipped with, The determination means determines, based on the first information, disclosure criteria information that shows the difference between the learning history of the learning model of interest and the learning history of the base model identified by the second information used to create the learning model of interest, and determines the scope of disclosure for each of the multiple learning models based on the disclosure criteria information and the second information corresponding to each of the multiple learning models. An information processing device characterized by the following:

2. The first piece of information described above includes the dataset and training parameters used in training the learning model, and the number of training iterations. The determination means determines, if the difference between the dataset and learning parameters used to train the focus learning model and the dataset and learning parameters used to train the base model is less than a predetermined percentage and the number of training iterations of the focus learning model is less than a predetermined number of iterations, to restrict access to the focus learning model (limited access) or prohibit access (private access). The information processing apparatus according to feature 1.

3. If the base model used to create the learning model has been determined to be either publicly accessible or private, the determination means determines that the learning model is either publicly accessible or private. The information processing apparatus according to feature 2.

4. An information processing device for managing multiple learning models published on a model publishing platform, For each of the aforementioned multiple learning models, a management means for managing first information relating to the learning history of the learning model and second information relating to the base model used to create the learning model, A determination means for determining the scope of publication of each of the plurality of learning models in the model publication platform based on both the first and second pieces of information, Equipped with, The first piece of information includes the detection results obtained by a learning model on a given dataset. The determination means determines the Intersection over Union (IoU) between the detection result of the learning model of interest and the detection result of the base model identified by the second information used to create the learning model of interest as the public access criteria information for the learning model of interest, and determines the scope of public access for each of the learning models An information processing device characterized by the following:

5. An information processing device for managing multiple learning models published on a model publishing platform, For each of the aforementioned multiple learning models, a management means for managing first information relating to the learning history of the learning model and second information relating to the base model used to create the learning model, A determination means for determining the scope of publication of each of the plurality of learning models in the model publication platform based on both the first and second pieces of information, Equipped with, The first piece of information mentioned above includes the weights of the neural networks (NN) that constitute each learning model. The determination means determines the sum of the absolute values ​​of the differences between the weights of the NN constituting the learning model of interest and the weights of the NN constituting the base model identified by the second information used to create the learning model of interest as the disclosure criteria information for the learning model of interest, and determines the disclosure scope for each of the multiple learning models based on the disclosure criteria information and the second information corresponding to each of the multiple learning models. An information processing device characterized by the following:

6. An information processing device for managing multiple learning models published on a model publishing platform, For each of the aforementioned multiple learning models, a management means for managing first information relating to the learning history of the learning model and second information relating to the base model used to create the learning model, A determination means for determining the scope of publication of each of the plurality of learning models in the model publication platform based on both the first and second pieces of information, Equipped with, The first piece of information includes time-series data of the learning rate and loss of the learning parameters during training for each learning model. The determination means determines the learning progress calculated based on the learning rate of the learning model of interest and the time series data as the public access criteria information for the learning model of interest, and determines the scope of public access for each of the multiple learning models based on the public access criteria information corresponding to each of the multiple learning models and the second information. An information processing device characterized by the following:

7. An information processing device for managing multiple learning models published on a model publishing platform, The aforementioned learning models are image generation models that generate images for keywords, The aforementioned information processing device is A registration means for registering keywords specified by the administrator of the aforementioned model publishing platform and the correct images corresponding to those keywords, For each of the aforementioned multiple learning models, a management means manages first information regarding the similarity between the generated image generated by inputting the keyword into the learning model and the ground truth image, and second information regarding the base model used to create the learning model. A determination means for determining the scope of publication of each of the plurality of learning models in the model publication platform based on both the first and second pieces of information, An information processing device characterized by comprising:

8. The model publishing platform further comprises a screen generation means for generating an administration screen that is displayed on an administrator terminal operated by the administrator of the aforementioned model publishing platform, The management screen is configured to display, as a graphical user interface (GUI), the derivation relationships between the multiple learning models and the publication status of each of the multiple learning models, based on the second information. The information processing apparatus according to any one of claims 1 to 7.

9. A method for controlling an information processing device that manages multiple learning models published on a model publishing platform, For each of the aforementioned multiple learning models, the process involves obtaining a first piece of information regarding the learning history of the learning model and a second piece of information regarding the base model used to create the learning model. A decision step of determining the scope of publication for each of the multiple learning models in the model publication platform based on both the first and second pieces of information, Includes, In the aforementioned determination step, based on the first information, disclosure criteria information is determined that shows the difference between the learning history of the learning model of interest and the learning history of the base model identified by the second information used to create the learning model of interest, and the scope of disclosure for each of the multiple learning models is determined based on the disclosure criteria information and the second information corresponding to each of the multiple learning models. A control method characterized by the following:

10. A control method for an information processing device that manages multiple learning models published on a model publishing platform, For each of the aforementioned multiple learning models, the process involves obtaining a first piece of information regarding the learning history of the learning model and a second piece of information regarding the base model used to create the learning model. A decision step of determining the scope of publication for each of the multiple learning models in the model publication platform based on both the first and second pieces of information, Includes, The first piece of information includes the detection results obtained by a learning model on a given dataset. In the aforementioned determination step, the Intersection over Union (IoU) between the detection result from the learning model of interest and the detection result from the base model identified by the second information used to create the learning model of interest is determined as the public access criteria information for the learning model of interest, and the scope of public access for each of the learning models is determined based on the public access criteria information corresponding to each of the learning models and the second information. A control method characterized by the following:

11. A control method for an information processing device that manages multiple learning models published on a model publishing platform, For each of the aforementioned multiple learning models, the process involves obtaining a first piece of information regarding the learning history of the learning model and a second piece of information regarding the base model used to create the learning model. A decision step of determining the scope of publication for each of the multiple learning models in the model publication platform based on both the first and second pieces of information, Includes, The first piece of information mentioned above includes the weights of the neural networks (NN) that constitute each learning model. In the aforementioned determination step, the sum of the absolute values ​​of the differences between the weights of the NN constituting the learning model of interest and the weights of the NN constituting the base model identified by the second information used to create the learning model of interest is determined as the disclosure criteria information for the learning model of interest, and the disclosure scope for each of the multiple learning models is determined based on the disclosure criteria information corresponding to each of the multiple learning models and the second information. A control method characterized by the following:

12. A control method for an information processing device that manages multiple learning models published on a model publishing platform, For each of the aforementioned multiple learning models, the process involves obtaining a first piece of information regarding the learning history of the learning model and a second piece of information regarding the base model used to create the learning model. A decision step of determining the scope of publication for each of the multiple learning models in the model publication platform based on both the first and second pieces of information, Includes, The first piece of information includes time-series data of the learning rate and loss of the learning parameters during training for each learning model. In the aforementioned determination step, the learning progress calculated based on the learning rate of the learning model of interest and the time-series data is determined as the public access criteria information for the learning model of interest, and the scope of public access for each of the multiple learning models is determined based on the public access criteria information corresponding to each of the multiple learning models and the second information. A control method characterized by the following:

13. A control method for an information processing device that manages multiple learning models published on a model publishing platform, The aforementioned learning models are image generation models that generate images for keywords, The control method described above is A registration step involves registering a keyword specified by the administrator of the aforementioned model publishing platform and associating it with a correct image corresponding to that keyword. For each of the aforementioned multiple learning models, an acquisition step is made to acquire first information regarding the similarity between the generated image generated by inputting the keyword into the learning model and the ground truth image, and second information regarding the base model used to create the learning model. A decision step of determining the scope of publication for each of the multiple learning models in the model publication platform based on both the first and second pieces of information, A control method characterized by including

14. A program for causing a computer to execute the control method described in any one of claims 9 to 13.

Images