Information processing device, information processing system, and information processing method

Abstract

To improve the recognition accuracy of the recognition model. [Solution] The information processing device 10 comprises a generation unit 10B and a learning unit 10C. The learning unit 10C inputs a first category that classifies the data 24 and a second category that classifies the data 24 in more detail than the first category into a generation model (generation model 20), and generates a first data 24A belonging to the first category and a second data 24B belonging to the second category as output from the generation model. The learning unit 10C uses the data 24, including the first data 24A and the second data 24B, to learn a recognition model 26 that outputs recognition results from the data 24.

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus, an information processing system, and an information processing method.

Background Art

[0002] A method of learning a recognition model using image data has been disclosed. For example, a method of unsupervised learning of a recognition model using pseudo samples generated using a pre-trained generation model has been disclosed (see, for example, Patent Document 1). Further, a method of supervised learning of a recognition model for classification using an image generated by inputting text of a category name into a generation model has been disclosed (see, for example, Non-Patent Document 1).

[0003] However, in the prior art, since the recognition model is learned using only data that can be used for supervised learning or only data that can be used for unsupervised learning, the learning lacks either faithfulness or diversity. For this reason, in the prior art, it may not be possible to improve the recognition accuracy of the recognition model.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The present invention has been made in view of the above, and aims to provide an information processing device, an information processing system, and an information processing method that can improve the recognition accuracy of a recognition model. [Means for solving the problem]

[0007] The information processing device of this embodiment comprises a generation unit and a learning unit. The generation unit inputs a first category, which classifies the data, and a second category, which classifies the data in more detail than the first category, into a generation model, and generates first data belonging to the first category and second data belonging to the second category as output from the generation model. The learning unit uses the data, including the first data and the second data, to learn a recognition model that outputs recognition results from the data. [Brief explanation of the drawing]

[0008] [Figure 1] A block diagram showing the configuration of an information processing system. [Figure 2A] A schematic diagram of the data. [Figure 2B] A schematic diagram of the data. [Figure 3] A flowchart illustrating the flow of information processing performed by an information processing device. [Figure 4] A block diagram showing the configuration of an information processing system. [Figure 5] A schematic diagram of the display screen. [Figure 6] A flowchart illustrating the flow of information processing performed by an information processing device. [Figure 7] Hardware configuration diagram. [Modes for carrying out the invention]

[0009] The information processing apparatus, information processing system, and information processing method of this embodiment will be described in detail below with reference to the attached drawings.

[0010] (First Embodiment) Figure 1 is a block diagram showing an example of the configuration of the information processing system 1 of this embodiment.

[0011] Information processing system 1 is a system for learning the recognition model 26. Details of the recognition model 26 will be described later. Information processing system 1 includes an information processing device 10. The information processing device 10 is an information processing device for learning the recognition model 26. In this embodiment, the information processing device 10 learns the generation model 20 and the recognition model 26. Details of the generation model 20 and the recognition model 26 will be described later.

[0012] The information processing device 10 includes a generative model learning unit 10A, a generation unit 10B, and a learning unit 10C.

[0013] The generative model learning unit 10A, the generation unit 10B, and the learning unit 10C are implemented by, for example, one or more processors. For example, each of the above units may be implemented by having a processor such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) execute a program, i.e., by software. Each of the above units may be implemented by a dedicated IC or other processor, i.e., by hardware. Each of the above units may be implemented by using both software and hardware. When multiple processors are used, each processor may implement one of the above units, or two or more of the above units.

[0014] The generation model 20 and recognition model 26, which will be described in detail later, may be stored in a memory unit provided in the information processing device 10. Alternatively, at least one of the generation model 20 and recognition model 26 may be stored in a memory unit provided outside the information processing device 10. Furthermore, the memory unit and at least one of the multiple functional units included in the information processing device 10 may be mounted on an external information processing device that is communicated with the information processing device 10 via a network or the like.

[0015] The generative model learning unit 10A learns the generative model 20 using the training data 22.

[0016] The learning data 22 includes at least one of supervised learning data 22A and unsupervised learning data 22B.

[0017] The supervised learning data 22A is composed of pairs of the third data 24C and the third category to which the third data 24C belongs. That is, the supervised learning data 22A is data with the third category to which the third data 24C belongs assigned to the third data 24C.

[0018] The third data 24C is an example of the data 24.

[0019] The data 24 is data to be recognized by the recognition model 26. The data 24 is, for example, image data, but is not limited to image data. In the present embodiment, a form in which the data 24 is image data will be described as an example.

[0020] The third category is an example of a category.

[0021] A category is data that directly or indirectly represents correct data to be output from the recognition model 26 when the data 24 is input to the recognition model 26. In the present embodiment, a category is information representing a label or class of each group obtained by classifying a plurality of data 24 into a plurality of groups according to a predetermined classification rule. For example, assume that the data 24 is image data in which a car is reflected. In this case, the category is, for example, "car", "car model", "color of the car", etc., but is not limited thereto.

[0022] In the supervised learning data 22A, information representing the category to which the third data 24C included in the supervised learning data 22A belongs is pre-assigned as the third category to the third data 24C.

[0023] The unsupervised learning data 22B is data composed of the third data 24C. That is, the unsupervised learning data 22B is data that does not include the third category corresponding to the third data 24C.

[0024] The generative model 20 is a machine learning model that takes categories as input and outputs data 24 belonging to the input categories. The generative model 20 uses image generation models such as stable diffusion (Reference 1) and generative adversarial networks (GAN) (Reference 2), which can be used to output data 24 by adding conditions using arbitrary text.

[0025] ·Reference 1: R. Rombach, et al., “High-Resolution Image Synthesis with Latent Diffusion Models”, Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2022. • Reference 2: Ian Goodfellow, et al. "Generative adversarial nets," Advances in neural information processing systems 27, 2014.

[0026] The generative model learning unit 10A learns the generative model 20 by using the first category and the second category as categories, as well as using multiple training data 22.

[0027] Category 1 is an example of a category. Compared to Category 2, Category 1 is a category that classifies Data 24 into broader categories.

[0028] The second category is an example of a category. Compared to the first category, the second category is a more detailed classification of data 24.

[0029] Therefore, when classifying multiple data points 24, the number of data points 24 belonging to the first category will be greater than or equal to the number of data points 24 belonging to each of the one or more second categories. Furthermore, the first category will include data points 24 belonging to multiple different second categories, resulting in a diverse range of data points 24. On the other hand, the second category is a category that classifies the data points 24 in more detail than the first category. Therefore, the second category will include one or more data points 24 that are highly faithful to that second category.

[0030] The generative model learning unit 10A trains the generative model 20 so that, upon input of the first category, it outputs data 24 belonging to the first category. The generative model learning unit 10A also trains the generative model 20 so that, upon input of the second category, it outputs data 24 belonging to the second category.

[0031] The generative model learning unit 10A identifies the category to which all of the multiple third data 24C included in the multiple training data 22 belong as the first category.

[0032] For example, consider the case where training data 22 includes supervised training data 22A. In this case, the generative model learning unit 10A identifies a higher-level category that collectively refers to multiple third categories contained in multiple supervised training data 22A as the first category.

[0033] For example, let's assume that all of the multiple third data points 24C are data points 24 in which a car is visible. Let's also assume that the third category included in the supervised learning data 22A is information representing the type of car. In this case, the generative model learning unit 10A identifies "car" as the first category, for example, as a higher-level category that encompasses multiple types of car models.

[0034] The generative model learning unit 10A then inputs text containing the first category into the generative model 20 and learns the generative model 20 using a known method with multiple training data 22 so that the generative model 20 generates and outputs first data 24A belonging to the first category.

[0035] The text input to the generative model 20 could be, for example, "a photo of a {category}." Alternatively, the text input to the generative model 20 could also be, for example, "a sks {category}.", which adds an arbitrary string that is not commonly used as an identifier, or "a [v] {category}.", which adds a learnable prompt [v].

[0036] The first data 24A is an example of data 24. The first data 24A is data 24 belonging to the first category. When the first category is "car", the generative model learning unit 10A trains the generative model 20 so that, upon input of the first category, it outputs multiple first data 24A with various diversity in which "cars" are depicted.

[0037] Furthermore, the generative model learning unit 10A uses each of the multiple third categories contained in the multiple supervised learning data 22A as the second category. If the learning data 22 does not contain the supervised learning data 22A, the generative model learning unit 10A may identify a lower-level concept than the first category, that is, a category that classifies the multiple data 24 in more detail, as the second category. In this case, the generative model learning unit 10A may also identify a pre-stored category as the second category. The user may pre-input a desired category that can be used in the recognition result output from the recognition model 26 as the second category, and the information processing device 10 may pre-store the second category that has been received as input in the storage unit.

[0038] For example, let's assume that all of the multiple third data points 24C are data points 24 in which a car is visible. Let's also assume that the generative model learning unit 10A identifies "car" as the first category. Let's also assume that the third category included in the supervised learning data 22A is information representing the type of car. In this case, the generative model learning unit 10A identifies each of the multiple types of car models as the second category.

[0039] The generative model learning unit 10A then inputs text containing the second category into the generative model 20 and learns the generative model 20 using a known method with multiple training data 22 so that the generative model 20 generates and outputs second data 24B belonging to the second category.

[0040] Second data 24B is an example of data 24. Second data 24B is data 24 belonging to the second category. When the second category is "vehicle type", the generative model learning unit 10A learns the generative model 20 so that, upon input of the second category, it outputs multiple second data 24B that are more faithful to the input specific "vehicle type" and include images of that "vehicle type".

[0041] For training the generative model 20 by the generative model learning unit 10A, methods such as fine-tuning, which updates all model parameters, adding a task-specific layer to the model and updating only a portion of the model parameters, and tuning only the prompt portion of the input text without updating the model parameters can be used.

[0042] The generation unit 10B generates data 24 for each category. The generation unit 10B inputs the categories into the generation model 20 and retrieves the data 24 output from the generation model 20 to generate data 24 belonging to the input categories. Category input can be performed by inputting text containing categories into the generation model 20, similar to when the generation model learning unit 10A trains the generation model 20.

[0043] In detail, the generation unit 10B inputs the first category to the generation model 20 and generates the first data 24A belonging to the first category as output from the generation model 20. The generation unit 10B also inputs the second category to the generation model 20 and generates the second data 24B belonging to the second category as output from the generation model 20.

[0044] Furthermore, when inputting text containing categories into the generation model 20, the generation unit 10B may set a fidelity parameter (CFG Scale: Classifier-free guidance Scale) (Reference 3) and generate data 24.

[0045] ·Reference 3:3. J. Ho, et al., "Classifier-free diffusion guidance."ArXiv preprint arXiv:2207.12598, 2022.

[0046] The fidelity parameter is a numerical value that specifies how faithful the generated data 24 is to the input text containing the category. The higher the fidelity parameter value, the more faithful the generated model 20 is to the input text containing the category.

[0047] The fidelity and diversity of data 24 are in a trade-off relationship. The higher the fidelity, the more data 24 with the same structure are generated, and the lower the fidelity, the more diverse data 24 are generated. In other words, the higher the fidelity parameter set in the generative model 20, the more data 24 with the same structure are generated by the generative model 20. Conversely, the lower the fidelity parameter set in the generative model 20, the more data 24 with different structures, i.e., more diversity, are generated by the generative model 20.

[0048] For example, the generation unit 10B sets a first fidelity parameter in the generation model 20 and inputs a first category to the generation model 20, generating first data 24A as output from the generation model 20. Also, for example, it sets a second fidelity parameter in the generation model 20 and inputs the second category to the generation model 20, generating second data 24B as output from the generation model 20.

[0049] The first fidelity parameter is an example of a fidelity parameter and is set in the generative model 20 when inputting data into the generative model 20 of the first category. The second fidelity parameter is an example of a fidelity parameter and is set in the generative model 20 when inputting data into the generative model 20 of the second category.

[0050] The generation unit 10B can be configured to set arbitrary values ​​as the first fidelity parameter and the second fidelity parameter. Preferably, the generation unit 10B sets the second fidelity parameter, which represents a higher fidelity than the first fidelity parameter, to the generated model 20.

[0051] In other words, it is preferable for the generation unit 10B to generate the first data 24A by setting a first fidelity parameter that represents a fidelity lower than the second fidelity parameter in the generation model 20 and inputting the first category into the generation model 20. In this case, the generation unit 10B can lower the fidelity for the input category (first category) compared with the second data 24B, and cause the generation model 20 to generate a more diverse range of first data 24A with greater diversity.

[0052] Furthermore, it is preferable that the generation unit 10B generates the second data 24B by setting a second fidelity parameter that represents a higher fidelity than the first fidelity parameter in the generation model 20 and inputting the second category into the generation model 20. In this case, the generation unit 10B can increase the fidelity to the input category (second category) compared with the first data 24A, thereby suppressing the generation of data 24 as the second data 24B, which has higher fidelity and is less likely to belong to the second category.

[0053] The generation unit 10B may, instead of the generation model 20, use a machine learning model trained on an information processing device other than the information processing device 10 to generate the first data 24A belonging to the first category and the second data 24B belonging to the second category.

[0054] For example, if the training data 22 is not data from a special domain, the generation unit 10B may use a generative model pre-trained with a large amount of general image data or other data 24 to generate the first data 24A and the second data 24B.

[0055] Figures 2A and 2B are schematic diagrams of an example of data 24.

[0056] Figure 2A is a schematic diagram of an example of first data 24A generated by the generation unit 10B. Figure 2A is a schematic diagram of an example of multiple first data 24A output from the generation model 20 when the generation unit 10B uses "car" as the first category.

[0057] Figure 2B is a schematic diagram of an example of the second data 24B generated by the generation unit 10B. Figure 2B is a schematic diagram of multiple examples of the second data 24B output from the generation model 20 when the generation unit 10B uses "Vehicle type "Toshiba S4 Sedan 2012"" as the second category.

[0058] Returning to Figure 1, we continue the explanation.

[0059] The learning unit 10C trains the recognition model 26 using data 24, which includes the first data 24A and the second data 24B.

[0060] The recognition model 26 is a machine learning model that takes data 24 as input and outputs the recognition result of the input data 24.

[0061] The recognition result should be the recognition result of data 24. The recognition result may be, for example, the recognition result of an object contained in data 24, the category to which the object contained in data 24 belongs, etc. In this embodiment, one example will be described in which the recognition result output from the recognition model 26 is the category to which data 24 belongs. A category may be referred to as a class.

[0062] The recognition model 26 is a known neural network. In this embodiment, one example described is a form in which the recognition model 26 is a machine learning model that executes a classification algorithm such as a classification task. Note that the algorithm executed by the recognition model 26 is not limited to a classification task, but may further include one or more tasks such as object detection and segmentation.

[0063] For example, recognition model 26 may use a CNN (Convolutional Neural Network) such as VGG (Reference 4) or ResNet (Reference 5) as its backbone. Another example of recognition model 26 is a method that directly performs object class classification and region regression for each pixel of the feature map to estimate the class identification of an object region. Examples of this method include single-stage detectors such as SSD (Single Shot Multibox Detector) (Reference 6) and FCOS (Fully Convolutional One-Stage Object Detection) (Reference 7).

[0064] Furthermore, the recognition model 26 may also use a two-stage detector that performs class classification and object region regression after extracting candidate object regions. Examples of two-stage detectors include Faster R-CNN (Reference 8).

[0065] ·Reference 4: Simonyan, Karen, and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv:1409.1556 (2014). ·Reference 5: He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. ·Reference 6: Liu Wei, et al. "SSD: Single shot multibox detector." European conference on computer vision. Springer, Cham, 2016. ·Reference 7: Zhi Tian, ​​et al. "Fcos: Fully convolutional one-stage object detection." Proceedings of the IEEE / CVF international conference on computer vision. 2019. ·Reference 8: Ren, Shaoqing, et al. "Faster r-cnn: Towards real-time object detection with region proposal networks." Advances in neural information processing systems. 2015.

[0066] The learning unit 10C learns the recognition model 26 through self-supervised learning using the first data 24A. The learning unit 10C also learns the recognition model 26 through supervised learning using the second data 24B and the second category used to generate the second data 24B by the generation unit 10B.

[0067] Furthermore, the learning unit 10C may use the third data 24C included in the training data 22 as additional data 24, in addition to the first data 24A and the second data 24B, to train the recognition model 26. That is, the learning unit 10C trains the recognition model 26 using data 24 which includes the third data 24C, the first data 24A, and the second data 24B.

[0068] The learning unit 10C trains the recognition model 26 using data 24, which includes the third data 24C, the first data 24A, and the second data 24B. This makes it possible to generate an expanded, large amount of data 24 and use it to train the recognition model 26, even when training the recognition model 26 for data 24 in a special domain where it is difficult to prepare a large amount of data 24 in advance.

[0069] The learning unit 10C should train the recognition model 26 through self-supervised learning using unsupervised learning data 22B, and then train the recognition model 26 through supervised learning using supervised learning data 22A.

[0070] When learning with the second data 24B and the second category, and when learning with supervised learning data 22A, the learning unit 10C should perform supervised learning of the recognition model 26 using methods such as cross-entropy loss, similar to known supervised learning. Furthermore, when learning with the first data 24A, and when learning with unsupervised learning data 22B, the learning unit 10C should perform self-supervised learning to train the recognition model 26.

[0071] Next, an example of the information processing flow executed by the information processing device 10 of this embodiment will be described.

[0072] Figure 3 is a flowchart showing an example of the information processing flow performed by the information processing device 10.

[0073] The generative model learning unit 10A learns the generative model 20 using the training data 22 (step S100).

[0074] The generation unit 10B generates data 24 for each category using the generative model 20 learned in step S100 (step S102). The generation unit 10B inputs the first category into the generative model 20 and generates first data 24A belonging to the first category as output from the generative model 20. The generation unit 10B also inputs the second category into the generative model 20 and generates second data 24B belonging to the second category as output from the generative model 20. Furthermore, as described above, the generation unit 10B may set a fidelity parameter when inputting text containing categories into the generative model 20 and generate data 24.

[0075] Next, the learning unit 10C trains the recognition model 26 using the data 24, which includes the first data 24A and the second data 24B, generated in step S102 (step S104).

[0076] Then, this routine ends.

[0077] As described above, the information processing device 10 of this embodiment comprises a generation unit 10B and a learning unit 10C. The learning unit 10C inputs a first category that classifies the data 24 and a second category that classifies the data 24 in more detail than the first category into a generation model (generation model 20), and generates a first data 24A belonging to the first category and a second data 24B belonging to the second category as output from the generation model. The learning unit 10C uses the data 24, including the first data 24A and the second data 24B, to learn a recognition model 26 that outputs recognition results from the data 24.

[0078] In conventional techniques, recognition models were trained using either data suitable for supervised learning or data suitable for unsupervised learning. This resulted in learning that lacked either fidelity or diversity. Consequently, conventional techniques sometimes failed to improve the recognition accuracy of recognition models.

[0079] On the other hand, in the information processing device 10 of this embodiment, the recognition model 26 is trained using first data 24A belonging to the first category, and second data 24B belonging to the second category, which is a more detailed classification of data 24 compared to the first category.

[0080] As described above, Category 1 is a category that classifies the data 24 in a broader, more general way compared to Category 2. Category 2 is a category that classifies the data 24 in more detail compared to Category 1.

[0081] Therefore, when multiple data points 24 are classified, the number of data points 24 belonging to the first category will be greater than or equal to the number of data points 24 belonging to each of the one or more second categories. Furthermore, the data points 24 belonging to the first category include data points 24 belonging to multiple different second categories. Thus, the first category will contain a variety of data points 24 with diversity. On the other hand, the second category is a category that classifies the data points 24 in more detail than the first category, so the second category will contain a variety of data points 24 that are highly faithful to that second category.

[0082] In this embodiment, the information processing device 10 learns the recognition model 26 using both the multiple first data 24A and the multiple second data 24B.

[0083] Therefore, the information processing device 10 of this embodiment can learn a recognition model 26 that satisfies both fidelity and diversity.

[0084] Therefore, the information processing device 10 of this embodiment can improve the recognition accuracy of the recognition model 26.

[0085] (Second embodiment) In this embodiment, we describe a method for training the generative model 20 based on derived fidelity parameters.

[0086] In this embodiment, the same reference numerals are used for the same parts as in the above embodiment, and detailed descriptions are omitted.

[0087] Figure 4 is a block diagram showing an example of the configuration of the information processing system 1B of this embodiment.

[0088] The information processing system 1B comprises an information processing device 11 and a UI (user interface) unit 12. The information processing device 11 and the UI unit 12 are connected via a network or the like, enabling communication by wired or wireless means.

[0089] Information processing system 1B is the same as information processing system 1 of the above embodiment, except that it includes an information processing device 11 instead of an information processing device 10, and also includes a UI unit 12.

[0090] The UI unit 12 includes a display unit 14 and an input unit 16. The display unit 14 displays various types of information. The display unit 14 is, for example, a display, a projection device, etc. The input unit 16 accepts user input. The input unit 16 is, for example, a pointing device such as a mouse and touchpad, a keyboard, etc. The display unit 14 and the input unit 16 may be integrated into a touch panel. The information processing device 11 may also be configured to include the UI unit 12. Furthermore, the UI unit 12 may be provided on an external information processing device that is wirelessly or wiredly connected to the information processing device 11 for communication.

[0091] The information processing device 11 includes a generation model learning unit 10A, a generation unit 11B, a learning unit 10C, a display control unit 11D, an input receiving unit 11E, a calculation unit 11F, and a derivation unit 11G.

[0092] The generative model learning unit 10A, generation unit 11B, learning unit 10C, display control unit 11D, input receiving unit 11E, calculation unit 11F, and derivation unit 11G are implemented by, for example, one or more processors. For example, each of the above units may be implemented by having a processor such as a CPU or GPU execute a program, i.e., by software. Each of the above units may be implemented by a processor such as a dedicated IC, i.e., by hardware. Each of the above units may be implemented by using both software and hardware. When multiple processors are used, each processor may implement one of the above units, or it may implement two or more of the above units.

[0093] The generation model 20 and the recognition model 26 may be stored in a memory unit provided in the information processing device 11. Alternatively, at least one of the generation model 20 and the recognition model 26 may be stored in a memory unit provided outside the information processing device 11. Furthermore, the memory unit and at least one of the multiple functional units included in the information processing device 11 may be mounted on an external information processing device that is communicably connected to the information processing device 11 via a network or the like.

[0094] The generation model learning unit 10A and the learning unit 10C are the same as in the above embodiment. The information processing device 11 is the same as the information processing device 10 of the above embodiment, except that it includes a generation unit 11B instead of a generation unit 10B, and further includes a display control unit 11D, an input receiving unit 11E, a calculation unit 11F, and a derivation unit 11G.

[0095] The generation unit 11B generates data 24 for each category, similar to the generation unit 10B. Specifically, the generation unit 11B inputs the first category into the generation model 20 and outputs the first data 24A belonging to the first category from the generation model 20. The generation unit 11B also inputs the second category into the generation model 20 and outputs the second data 24B belonging to the second category from the generation model 20.

[0096] In this embodiment, the generation unit 11B obtains the fidelity parameters derived for each category by the derivation unit 11G, which will be described in detail later, from the derivation unit 11G.

[0097] Then, the generation unit 11B sets the first fidelity parameter, which is the fidelity parameter derived for the first category by the derivation unit 11G, into the generation model 20, and inputs the first category into the generation model 20, generating the first data 24A as the output from the generation model 20. The generation unit 11B can input text containing the first category into the generation model 20, just as with the generation unit 10B.

[0098] Furthermore, the generation unit 11B sets the second fidelity parameter, which is the fidelity parameter derived for the second category by the derivation unit 11G, into the generation model 20, and inputs the second category into the generation model 20, generating the second data 24B as the output from the generation model 20. The generation unit 11B can be used in the same way as the generation unit 10B, by inputting text containing the second category into the generation model 20.

[0099] The display control unit 11D displays the first data 24A and the second data 24B generated by the generation unit 11B on the display unit 14.

[0100] Figure 5 is a schematic diagram of an example of a display screen 30 displayed on the display unit 14.

[0101] The display screen 30 shows the data 24 generated by the generation unit 11B, categorized by type.

[0102] For example, the display control unit 11D displays on the display screen 30 the prompt 31A input to the generation model 20, which is text containing "car", an example of a first category; the fidelity score 32A, which is the value of the first fidelity parameter set in the generation model 20 when the first category was input to the generation model 20; and a plurality of first data 24A belonging to the first category.

[0103] Furthermore, the display control unit 11D displays on the display screen 30 the prompt 31B input to the generation model 20, which is text including "Vehicle type" "Toshiba S4 Sedan 2012", an example of the second category; the fidelity score 32B, which is the value of the second fidelity parameter set in the generation model 20 when the second category was input to the generation model 20; and a plurality of second data 24B belonging to the second category.

[0104] The user can view the list of generated data 24 by looking at the multiple first data 24A and multiple second data 24B belonging to the first and second categories, respectively, displayed on the display screen 30. In other words, the display control unit 11D can provide the user with a list of the multiple data 24 generated for each category by the generation unit 11B in a viewable format.

[0105] The display screen 30 includes a regeneration button 34 for receiving a regeneration instruction and a learning start button 35 for receiving a learning start instruction. When the user selects the regeneration button 34 on the display screen 30 based on an instruction from the user to the input unit 16, the input receiving unit 11E accepts a regeneration instruction. When the user selects the learning start button 35 on the display screen 30 based on an instruction from the user to the input unit 16, the input receiving unit 11E accepts a learning start instruction.

[0106] Furthermore, the user may change at least one of the fidelity score 32A, which is the value of the displayed first fidelity parameter, and the fidelity score 32B, which is the value of the displayed second fidelity parameter, by operating the input unit 16. For example, if multiple data 24 data 24 that are biased towards the same or similar composition are generated as data 24 belonging to each category, the user will input a lower fidelity parameter value. When the input unit 16 is operated by the user and the fidelity parameters for each category are received, the input receiving unit 11E receives the fidelity parameters entered for the first category and the second category, respectively.

[0107] Furthermore, the user can select the data 24 to be used for training. Specifically, the display screen 30 includes checkboxes near each of the multiple data 24 displayed to accept the selection of whether or not to use the data 24. The user can select the data 24 to be used for training the recognition model 26 by operating the input unit 16 and checking the checkboxes corresponding to the desired data 24 to be used for training the recognition model 26. The input receiving unit 11E should accept the identification information of the data 24 corresponding to the checked checkboxes as the identification information of the data 24 to be used for training the recognition model 26, based on the user's instructions to operate the input unit 16.

[0108] Returning to Figure 4, we continue the explanation.

[0109] When the input receiving unit 11E receives a regeneration instruction from the input unit 16, the derivation unit 11G derives different fidelity parameters for each of the first and second categories compared to when the previous data 24 was created. The derivation unit 11G then outputs the first fidelity parameter, which is the fidelity parameter derived for the first category, and the second fidelity parameter, which is the fidelity parameter derived for the second category, to the generation unit 11B.

[0110] Furthermore, consider the case where the input receiving unit 11E receives input values ​​for the first fidelity parameter for the first category and the second fidelity parameter for the second category from the input unit 16. Then, consider the scenario where the input receiving unit 11E receives a regeneration instruction from the input unit 16. In this case, the derivation unit 11G derives the first fidelity parameter and the second fidelity parameter that were received as input for each of the first and second categories as the fidelity parameters to be set next time. The derivation unit 11G then outputs the derived first fidelity parameter and the derived second fidelity parameter to the generation unit 11B.

[0111] In detail, the user changes at least one of the fidelity score 32A, which is the value of the displayed first fidelity parameter, and the fidelity score 32B, which is the value of the displayed second fidelity parameter, by operating the input unit 16. For example, if multiple data 24 data belonging to each category are generated and are biased towards the same or similar composition, the user inputs a lower fidelity parameter value. Upon receiving the user's input of fidelity parameters for each category, the input receiving unit 11E receives the input fidelity parameters for the first category and the second category, respectively. The derivation unit 11G then outputs the input first fidelity parameter and the second fidelity parameter to the generation unit 11B.

[0112] The generation unit 11B sets the first fidelity parameter, which is the fidelity parameter derived for the first category by the derivation unit 11G, into the generation model 20. Then, the generation unit 11B inputs the first category into the generation model 20 and generates the first data 24A as the output from the generation model 20. The generation unit 11B can input the first category into the generation model 20 by inputting text containing the first category into the generation model 20, similar to the generation unit 10B.

[0113] Furthermore, the generation unit 11B sets the second fidelity parameter, which is the fidelity parameter derived for the second category by the derivation unit 11G, into the generation model 20. Then, the generation unit 11B inputs the second category into the generation model 20 and generates the second data 24B as the output from the generation model 20. In the same way as the generation unit 10B, the generation unit 11B can input the second category into the generation model 20 by inputting text containing the second category into the generation model 20.

[0114] On the other hand, when the input receiving unit 11E receives a learning start instruction from the input unit 16, the learning unit 10C learns the recognition model 26 using the data 24, which includes the first data 24A and the second data 24B, in the same manner as in the above embodiment.

[0115] Furthermore, consider the case where the input receiving unit 11E receives identification information for data 24 corresponding to a checked checkbox in response to an operation instruction from the user to the input unit 16, and uses this identification information for data 24 to be used for training the recognition model 26. In this case, the learning unit 10C can train the recognition model 26 using the data 24 identified by the received identification information from among the multiple data 24 displayed on the display unit 14.

[0116] The calculation unit 11F calculates the first and second accuracy using the evaluation data 28.

[0117] Evaluation data 28 is data used to evaluate the recognition model 26. Evaluation data 28 includes pairs of reference data 24D and the correct recognition results of reference data 24D. Reference data 24D is an example of data 24. The correct recognition result is the correct recognition result for reference data 24D. As described in the above embodiment, in this embodiment, the form in which the recognition result of the recognition model 26 is a category is described as an example. For this reason, the correct recognition result is described assuming a form in which it is the correct category to which the reference data 24D belongs.

[0118] The calculation unit 11F calculates a first precision for each of the multiple evaluation data 28, which represents the degree of agreement between the recognition result output from the recognition model 26 and the correct recognition result corresponding to the reference data 24D, by inputting the reference data 24D included in the evaluation data 28 into the recognition model 26.

[0119] Furthermore, the calculation unit 11F calculates the second accuracy of the comparison target to the first accuracy.

[0120] The calculation unit 11F inputs the reference data 24D into a comparison model trained using only one or more first data 24A, only one or more second data 24B, or only one or more training data 22. The calculation unit 11F calculates the degree of agreement between the comparison recognition result output by this comparison model and the correct recognition result corresponding to the reference data 24D as the second precision. The comparison model is a machine learning model that includes the same algorithm as the recognition model 26, except that the data used for training is different.

[0121] Furthermore, the calculation unit 11F may calculate the degree of agreement between the comparative recognition result output by inputting the reference data 24D to the learning recognition model 26, which is a recognition model 26 whose learning progress is at an earlier stage than the current one, and the correct recognition result corresponding to the reference data 24D, as the second precision.

[0122] The calculation unit 11F calculates a pair of first accuracy and second accuracy for each of the multiple evaluation data 28. The calculation unit 11F then outputs the calculated pair of first accuracy and second accuracy, along with the correct recognition result included in the evaluation data 28 used to calculate the first accuracy included in the pair, to the derivation unit 11G for each correct recognition result.

[0123] The derivation unit 11G performs the following processing for each group of first precision and second precision pairs received from the calculation unit 11F, and the correct recognition results included in the evaluation data 28 used to calculate the first precision included in the pair.

[0124] In detail, the derivation unit 11G performs at least one of the following: deriving a fidelity parameter with a smaller value than the one previously derived, as a fidelity parameter for a category (first category or second category) corresponding to the correct recognition result, where the first accuracy is equal to or greater than the second accuracy; and deriving a number of generated data 24 that is greater than the number calculated previously.

[0125] A category corresponding to the correct answer recognition result means a category (first category or second category) that matches the correct answer recognition result in at least part, includes the correct answer recognition result, or is most similar to the correct answer recognition result.

[0126] Furthermore, the derivation unit 11G derives a fidelity parameter with a higher value than the one obtained in the previous derivation, as a fidelity parameter for a category (first category or second category) corresponding to the correct recognition result where the first accuracy is less than the second accuracy, and derives a number of generated values ​​that is lower than the one obtained in the previous derivation.

[0127] In other words, for categories where the first precision is equal to or greater than the second precision, the derivation unit 11G performs adjustment processing such as lowering the value of the fidelity parameter or increasing the number of data 24 generated so that multiple data 24 with greater diversity are generated.

[0128] Furthermore, for categories where the first accuracy is less than the second accuracy, the derivation unit 11G performs adjustment processing such as increasing the value of the fidelity parameter or decreasing the number of data 24 generated so that multiple data 24 with higher fidelity are generated for that category.

[0129] Therefore, in this embodiment, the derivation unit 11G can derive at least one of the fidelity parameters and the number of generated values ​​that can improve the recognition accuracy of the recognition model 26, for each category.

[0130] Furthermore, for the derivation of the fidelity parameters by the derivation unit 11G, a general hyperparameter search method such as grid search or Bayesian optimization-based search may be used to search for the fidelity parameters.

[0131] The generation unit 11B sets the first fidelity parameter, which is the fidelity parameter derived for the first category by the derivation unit 11G, into the generation model 20, and also inputs the first category into the generation model 20. Similar to the generation unit 10B, the generation unit 11B can input the first category into the generation model 20 by inputting text containing the first category into the generation model 20. Furthermore, if the number of data 24 generated for the first category has been derived by the derivation unit 11G, the generation unit 11B sets the number of first data 24A generated into the generation model 20. Then, as output from the generation model 20, the generation unit 11B generates the first data 24A with the number of generation derived for the first category by the derivation unit 11G.

[0132] Furthermore, the generation unit 11B sets the second fidelity parameter, which is the fidelity parameter derived for the second category by the derivation unit 11G, into the generation model 20, and also inputs the second category into the generation model 20. Similar to the generation unit 10B, the generation unit 11B can input the second category into the generation model 20 by inputting text containing the second category into the generation model 20. Also, if the number of data 24 generated for the second category is derived by the derivation unit 11G, the generation unit 11B sets the number of second data 24B generated into the generation model 20. Then, as output from the generation model 20, the generation unit 11B generates the second data 24B with the number of generation derived for the second category by the derivation unit 11G.

[0133] Therefore, the information processing device 11 of this embodiment can generate first data 24A of the first category and second data 24B of the second category using a generation model 20 in which fidelity parameters are set according to the accuracy of the recognition model 26 or the results of verification of the user-generated data 24. Furthermore, the information processing device 11 can generate data 24 used for training the recognition model 26, which can further improve the recognition accuracy of the recognition model 26.

[0134] Next, an example of the information processing flow executed by the information processing device 11 of this embodiment will be described.

[0135] Figure 6 is a flowchart showing an example of the information processing flow performed by the information processing device 11.

[0136] The generative model learning unit 10A learns the generative model 20 using the training data 22 (step S200).

[0137] The generation unit 11B sets the fidelity parameters and the number of data 24 to be generated for each category in the generative model 20 learned in step S200 (step S202). For example, in step S202, the generation unit 11B sets the fidelity parameters and the number of data to be generated, which are set in advance for each category, as initial values.

[0138] The generation unit 11B inputs text containing the category into the generative model 20 learned in step S200, each time it sets the fidelity parameter and the number of items to be generated for each category. The generation unit 11B then generates category-specific data 24 by acquiring the data 24 output from the generative model 20 (step S204).

[0139] In detail, the generation unit 11B inputs the first category into a generation model 20, which has a first fidelity parameter and a number of items to be generated for the first category, and generates first data 24A of the number of items to be generated belonging to the first category as output from the generation model 20. The generation unit 11B also inputs the second category into a generation model 20, which has a second fidelity parameter and a number of items to be generated for the second category, and generates second data 24B of the number of items to be generated belonging to the second category as output from the generation model 20.

[0140] The display control unit 11D displays the first data 24A and the second data 24B generated in step S204 on the display unit 14 for each category (step S206). As a result of the processing in step S206, the display unit 14 displays, for example, the display screen 30 shown in Figure 5.

[0141] Next, the input receiving unit 11E determines whether or not it has received a regeneration instruction (step S208). If the input receiving unit 11E determines that it has received a regeneration instruction (step S208: Yes), the derivation unit 11G derives, for example, different fidelity parameters for each of the first and second categories than those used when the previous data 24 was created (step S210). Also, if the input receiving unit 11E receives input values ​​for the first fidelity parameter for the first category and the second fidelity parameter for the second category from the input unit 16, the derivation unit 11G derives the input first fidelity parameter and second fidelity parameter for each of the first and second categories.

[0142] The generation unit 11B sets the fidelity parameters derived in step S210 and the number of data 24 to be generated for each category in the generation model 20 learned in step S200 (step S212). Then it proceeds to step S204.

[0143] If the input receiving unit 11E determines in step S208 that it has received a learning start instruction (step S208: No), the process proceeds to step S214.

[0144] In step S214, the learning unit 10C trains the recognition model 26 using the data 24, which includes the first data 24A and the second data 24B, generated in step S204 (step S214).

[0145] The calculation unit 11F uses the evaluation data 28 to calculate the first and second accuracy of the evaluation data 28 for each category (step S216). For each of the multiple evaluation data 28, the calculation unit 11F calculates the first accuracy, which represents the degree of agreement between the recognition result output from the recognition model 26 by inputting the reference data 24D contained in the evaluation data 28 into the recognition model 26 and the correct recognition result corresponding to the reference data 24D. The calculation unit 11F also calculates the second accuracy of the comparison target for the first accuracy for each category.

[0146] Next, the information processing device 11 determines whether or not the learning completion conditions for the recognition model 26 are met (step S218). For example, the information processing device 11 makes the determination in step S218 by determining whether or not the number of learning iterations in step S214 has exceeded a predetermined threshold. If the determination in step S218 is positive (step S218: Yes), this routine is terminated. If the determination in step S218 is negative (step S218: No), the process proceeds to step S220.

[0147] In step S220, the derivation unit 11G derives the fidelity parameter and the number of data 24 generated for each category based on the pairs of first and second precision calculated for each category in step S214 (step S220).

[0148] The derivation unit 11G receives multiple groups from the calculation unit 11F, each group consisting of a pair of first precision and second precision, and the correct recognition results included in the evaluation data 28 used to calculate the first precision included in the pair. The derivation unit 11G then performs the following processing for each group.

[0149] In detail, the derivation unit 11G performs at least one of the following: deriving a fidelity parameter with a smaller value than the one previously derived, as a fidelity parameter for a category (first category or second category) corresponding to the correct recognition result, where the first accuracy is equal to or greater than the second accuracy; and deriving a number of generated data 24 that is greater than the number calculated previously.

[0150] Furthermore, the derivation unit 11G derives a fidelity parameter with a higher value than the one obtained in the previous derivation, as a fidelity parameter for a category (first category or second category) corresponding to the correct recognition result where the first accuracy is less than the second accuracy, and derives a number of generated values ​​that is lower than the one obtained in the previous derivation.

[0151] The generation unit 11B sets the fidelity parameters derived in step S220 and the number of data 24 to be generated for each category in the generative model 20 learned in step S200 (step S222). Then, it proceeds to step S204.

[0152] As described above, in the information processing device 11 of this embodiment, the derivation unit 11G derives fidelity parameters for each category, including the first category and the second category. The generation unit 11B sets the first fidelity parameter, which is the fidelity parameter derived for the first category by the derivation unit 11G, into the generation model 20, and also inputs the first category into the generation model 20. The generation unit 11B then generates the first data 24A as output from the generation model 20. The generation unit 11B also sets the second fidelity parameter, which is the fidelity parameter derived for the second category by the derivation unit 11G, into the generation model 20, and also inputs the second category into the generation model 20. The generation unit 11B then generates the second data 24B as output from the generation model 20.

[0153] Therefore, in the information processing device 11 of this embodiment, the derivation unit 11G derives at least one of the fidelity parameter and the number of data 24 generated, depending on the accuracy of the recognition model 26 or the result of verification of the user-generated data 24, thereby enabling the generation of data 24 used for training the recognition model 26, which can further improve the recognition accuracy of the recognition model 26.

[0154] Therefore, in addition to the effects of the above embodiment, the information processing device 11 of this embodiment can further improve the recognition accuracy of the recognition model.

[0155] Next, an example of the hardware configuration of the information processing device 10 and information processing device 11 of the above embodiment will be described.

[0156] Figure 7 is a hardware configuration diagram of an example of the information processing device 10 and information processing device 11 of the above embodiment.

[0157] The information processing devices 10 and 11 in the above embodiment have a CPU (Central Processing Unit) 81, ROM (Read Only Memory) 82, RAM (Random Access Memory) 83, and communication I / F 84, etc., interconnected by a bus 85, and have a hardware configuration that utilizes a normal computer.

[0158] The CPU 81 is an arithmetic unit that controls the information processing device 10 and the information processing device 11 in the above embodiment. The ROM 82 stores programs and the like that realize various processes performed by the CPU 81. Although a CPU is used in this explanation, a GPU (Graphics Processing Unit) may also be used as the arithmetic unit that controls the information processing device 10 and the information processing device 11. The RAM 83 stores data necessary for various processes performed by the CPU 81. The communication I / F 84 is an interface that connects to the display unit 60 and the like for sending and receiving data.

[0159] In the information processing devices 10 and 11 of the above embodiment, the CPU 81 reads a program from the ROM 82 onto the RAM 83 and executes it, thereby realizing each of the above functions on the computer.

[0160] The programs for executing the above-described processes performed by the information processing device 10 and information processing device 11 in the above-described embodiment may be stored in an HDD (hard disk drive). Alternatively, the programs for executing the above-described processes performed by the information processing device 10 and information processing device 11 in the above-described embodiment may be pre-installed and provided in a ROM 82.

[0161] Furthermore, the programs for executing the above-described processes performed by the information processing devices 10 and 11 of the above-described embodiment may be stored in an installable or executable file format on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or flexible disk (FD), and provided as a computer program product. Alternatively, the programs for executing the above-described processes performed by the information processing devices 10 and 11 of the above-described embodiment may be stored on a computer connected to a network such as the Internet and provided by allowing download via the network. Alternatively, the programs for executing the above-described processes performed by the information processing devices 10 and 11 of the above-described embodiment may be provided or distributed via a network such as the Internet.

[0162] Although embodiments of the present invention have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0163] 1. 1B Information Processing System 10, 11 Information Processing Equipment 10A Generative Model Learning Unit 10B, 11B generation section 10C Learning Department 11D Display Control Unit 11F Calculation section 11G Derivation part

Claims

1. A generation unit inputs a first category that classifies the data and a second category that classifies the data in more detail than the first category into a generation model, and generates a first data belonging to the first category and a second data belonging to the second category as output from the generation model. A learning unit that learns a recognition model that outputs a recognition result from the data, including the first data and the second data, An information processing device equipped with the following features.

2. The aforementioned learning unit, The recognition model is trained by self-supervised learning using the first data, The recognition model is trained by supervised learning using the second data and the second category used to generate the second data by the generation unit. The information processing apparatus according to claim 1.

3. A generative model learning unit that learns the generative model using training data including the third data, The information processing apparatus according to claim 1, comprising:

4. The aforementioned learning unit, The recognition model is trained using the data, including the third data, the first data, and the second data. The information processing apparatus according to claim 3.

5. The generative model learning unit, The generative model is trained using the category to which all of the multiple third data included in the multiple training data belong as the first category. The information processing apparatus according to claim 3.

6. The aforementioned training data is Includes supervised learning data including the third data and the third category to which the third data belongs, and unsupervised learning data consisting of the third data, The generative model learning unit, The generative model is trained by supervised learning using the third category included in the supervised learning data as the second category. The information processing apparatus according to claim 3.

7. The generation unit is The first fidelity parameter is set in the generative model, the first category is input to the generative model, and the first data is generated as the output from the generative model. The second fidelity parameter is set in the generative model, the second category is input to the generative model, and the second data is generated as the output from the generative model. The information processing apparatus according to claim 1.

8. The generation unit is A second fidelity parameter, representing a higher fidelity than the first fidelity parameter, is set in the generative model. The information processing apparatus according to claim 7.

9. The system includes a derivation unit that derives fidelity parameters for each category, including the first category and the second category, The generation unit is The first fidelity parameter, which is the fidelity parameter derived for the first category in the derivation unit, is set in the generation model, the first category is input to the generation model, and the first data is generated as the output from the generation model. The second fidelity parameter, which is the fidelity parameter derived for the second category in the derivation unit, is set in the generation model, the second category is input to the generation model, and the second data is generated as the output from the generation model. The information processing apparatus according to claim 7.

10. The evaluation data includes pairs of reference-generated data and the correct recognition results of the reference-generated data, and includes a first precision that represents the degree of agreement between the recognition result output from the recognition model when the reference-generated data is input to the recognition model and the correct recognition result, A calculation unit that calculates the second accuracy of the comparison target, Equipped with, The aforementioned derivation section is, The first accuracy is greater than or equal to the second accuracy, and at least one of the following is performed: deriving a fidelity parameter for the category corresponding to the correct recognition result that has a smaller value than the one previously derived, and deriving a number of generated data that is greater than the number calculated previously, If the first accuracy is less than the second accuracy, derive a fidelity parameter for the category corresponding to the correct recognition result that has a higher value than the one derived previously, and derive a number of generated items that is fewer than the one derived previously. The generation unit is The first fidelity parameter, which is the fidelity parameter derived for the first category in the derivation unit, is set in the generation model, the first category is input to the generation model, and the first data of the number of generations derived for the first category in the derivation unit is generated as the output from the generation model. The second fidelity parameter, which is the fidelity parameter derived for the second category in the derivation unit, is set in the generation model, the second category is input to the generation model, and the second data of the number of generations derived for the second category in the derivation unit is generated as the output from the generation model. The information processing apparatus according to claim 9.

11. The calculation unit described above, The degree of agreement between the comparison recognition result output by inputting the aforementioned reference-generated data into a comparison model trained using only the first data, only the second data, or only the training data, and the aforementioned correct recognition result, or The degree of agreement between the comparative recognition result output by inputting the reference generation data into the learning recognition model, which is the recognition model whose learning progress is earlier than the current state, and the correct recognition result, The second precision described above is expressed as, The information processing apparatus according to claim 10.

12. A display control unit that displays the first data and the second data on a display device, The information processing apparatus according to claim 1, comprising:

13. A generation unit inputs a first category that classifies the data and a second category that classifies the data in more detail than the first category into a generation model, and generates a first data belonging to the first category and a second data belonging to the second category as output from the generation model. A learning unit that learns a recognition model that outputs a recognition result from the data, including the first data and the second data, An information processing system equipped with the following features.

14. An information processing method performed by an information processing device, A generation step in which a first category classifying the data and a second category classifying the data in more detail than the first category are input into a generative model, and a first data belonging to the first category and a second data belonging to the second category are generated as output from the generative model, A learning step of learning a recognition model that outputs a recognition result from the data, using the data including the first data and the second data, Information processing methods including

Images