Information processing method, program and information processing system

Abstract

To obtain training data for allowing a learned model to provide outputs with results thereof conforming to a target format, as well as, having diversity.SOLUTION: An information processing method according to an embodiment is executed by an information processing device, and includes: a target information acquisition step of acquiring target information being information relating to a target in processing results; an input information generation step of generating input information to be input to a learned model so as to allow the learned model to output processing results matching the target information; an evaluation step of evaluating the input information and the processing results having been output from the learned model by the inputting the input information; and a training data creation step of selecting the processing results having been evaluated to conform to a prescribed evaluation criterion in the evaluation step and generating a training data set for a machine learning model from the selected processing results.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing method, a program, and an information processing system. [Background technology]

[0002] Patent Document 1 discloses an information processing method for generating training data used for learning (training) a machine learning model. This information processing method modifies an input image according to a predetermined rule to generate a modified image. That is, image parts are selected from multiple image databases, and a composite image (input image for learning) is generated by combining them, and a modified image for learning is generated from a specific part image, thereby generating a large amount of training data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-102041 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, when it is desired to standardize the output of a trained model (hereinafter simply referred to as a "trained model"), which is a machine learning model trained with training data, to a specific target format, this can be achieved by training (including re-training) or fine-tuning the trained model, but training data containing information of the specific format is required. In this case, the inclusion of information of the specific format may cause bias in the content of the training data, resulting in a loss of diversity in the output results of the trained model, and there is room for improvement in this regard.

[0005] Taking into consideration the above-mentioned facts, the present invention aims to obtain training data that will enable output results from a learned model to be output in a targeted manner while having diversity. [Means for solving the problem]

[0006] According to one embodiment, the information processing method is executed by an information processing device, and includes a goal information acquisition step of acquiring goal information, which is information regarding a goal in a processing result; an input information generation step of generating input information to be input to a trained model so that the output from the trained model becomes the processing result that matches the target information; an evaluation step of evaluating the input information and the processing result output from the trained model by inputting the input information; and a training data creation step of selecting the processing result that has been evaluated in a predetermined manner in the evaluation step, and creating a training dataset for a machine learning model from the selected processing result.

[0007] According to a program of one embodiment, an information processing device is caused to execute an information processing method including: a goal information acquisition step of acquiring goal information, which is information regarding a goal in a processing result; an input information generation step of generating input information to be input to a trained model so that the output from the trained model becomes the processing result that matches the target information; an evaluation step of evaluating the input information and the processing result output from the trained model by inputting the input information; and a training data creation step of selecting the processing result that has been evaluated in a predetermined manner in the evaluation step, and creating a training dataset for a machine learning model from the selected processing result.

[0008] According to one embodiment, the information processing system is executed by an information processing device and includes a goal information acquisition step of acquiring goal information, which is information regarding a goal in a processing result; an input information generation step of generating input information to be input to a trained model so that the output from the trained model becomes the processing result that matches the target information; an evaluation step of evaluating the input information and the processing result output from the trained model by inputting the input information; and a training data creation step of selecting the processing result that has been evaluated in a predetermined manner in the evaluation step and creating a training dataset for a machine learning model from the selected processing result. Effect of the Invention

[0009] According to one embodiment, training data can be obtained that enables output results from a learned model to be output in a targeted manner while having diversity. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a server according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a processing flow of an information processing system according to an embodiment. [Diagram 5] 5 is a flowchart illustrating an example of processing executed by the information processing system according to the embodiment. [Figure 6] FIG. 11 is a diagram showing an example of a process flow in a target information acquisition step according to the embodiment. [Figure 7] 11 is an example of a screen display when executed by the information processing system according to the embodiment. [Figure 8] 11 is an example of a screen display when executed by the information processing system according to the embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a process flow in a target information acquisition step according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An embodiment of an information processing system according to the present invention will be described below with reference to Figures 1 to 8. In each drawing, the same or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0012] (System Overview) First, an overview of the information processing system 10 according to the present embodiment will be described. The information processing system 10 according to the present embodiment is a system that automatically executes a series of processes, including acquiring target information for a specific goal, generating input information for a trained model based on the target information, evaluating a processing result that is an output from the trained model, and creating training data for a machine learning model other than the trained model described above based on the evaluation result. In the present embodiment, as an example, the information processing system 10 is an information processing system that automatically creates a training dataset for training a machine learning model so that the machine learning model can generate an image according to the style information when it acquires information on the style of an image (hereinafter simply referred to as "style information") as target information. The created training dataset is used for learning and fine tuning of the machine learning model. Note that the training dataset is intended for a machine learning model other than the trained model that outputs the processing result, but is not limited to this, and may be created for the trained model.

[0013] (System Configuration) Fig. 1 is a diagram showing an example of a configuration of an information processing system 10 according to the present embodiment. As shown in Fig. 1, the information processing system 10 according to the present embodiment includes a user terminal 12 and a server 14, which are communicably connected to each other via a network N. The network N is, for example, a wired LAN (Local Area Network), a wireless LAN, the Internet, a public line network, a mobile data communication network, or a combination of these.

[0014] The user terminal 12 is an example of an information processing device that is operated by a user U to input and display various information. The user terminal 12 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, a wearable device, or a combination of these.

[0015] The server 14 is an example of an information processing device that acquires information input from the user terminal 12, processes the information, and outputs the results. The server 14 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, or a combination of these. The specific configuration and operation of the server 14 will be described later.

[0016] (Hardware configuration - Server) 2 is a block diagram showing a hardware configuration of the server 14. The server 14 includes a processor 20, a memory 22, a storage 24, and a communication I / F 26, which are communicatively connected to each other via a bus B.

[0017] The processor 20 controls each component of the server 14 and realizes the functions of the server 14 by expanding various programs stored in the storage 24 into the memory 22 and executing them. The programs executed by the processor 20 include, but are not limited to, an OS (Operating System) and a program 32 described below. A part of the state visualization method according to this embodiment is realized by the processor 20 executing these programs. The processor 20 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), or a combination of these.

[0018] The memory 22 is, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), or a combination thereof. The ROM is, for example, a Programmable ROM (PROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), or a combination thereof. The RAM is, for example, a Dynamic RAM (DRAM), a Static RAM (SRAM), a Magnetoresistive RAM (MRAM), or a combination thereof.

[0019] The storage 24 stores the OS, various programs described below, and various data. The storage 24 is, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), a storage class memory (SCM), or a combination of these.

[0020] The communication I / F 26 is an interface for connecting the server 14 to external devices including the user terminal 12 and the image capturing device 16 via the network N and for controlling communication. The communication I / F 26 is, for example, an adapter compliant with Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication (e.g., Fibre Channel), but is not limited thereto.

[0021] The hardware configuration of the user terminal 12 is substantially the same as the hardware configuration of the server 14 described above, and therefore a detailed description thereof will be omitted.

[0022] (Functional configuration - Server) Next, the functional configuration of the server 14 will be described. Fig. 3 is a diagram showing an example of the functional configuration of the server 14. When executing various programs, the server 14 realizes various functions using the above-mentioned hardware resources. The server 14 has a storage unit 30, a processing unit 40, and a communication unit 60 as the functional configuration realized by the server 14. Each functional configuration is realized by the processor 20 reading and executing a program 32 stored in the memory 22 or the storage 24.

[0023] The communication unit 60 is realized by the communication I / F 26. The communication unit 60 transmits and receives information to and from the user terminal 12 via the network N. The communication unit 60 receives information input from the user terminal 12. In addition, the communication unit 60 transmits information to the user terminal 12, and receives a request by a user U from the user terminal 12.

[0024] The storage unit 30 is realized by the memory 18 and the storage 24. The storage unit 30 stores a program 32, a trained model 34, and a result DB 36.

[0025] The trained model 34 is composed of a plurality of trained machine learning models. Specifically, the model includes a language model, an image model, an image caption model, and an evaluation model as a trained model for evaluation. As an example, the language model is a large-scale language model that is trained with a large amount of text data from articles, books, websites, etc. on the Internet, and when text data called a prompt is input, the language model generates and outputs text data corresponding to the prompt. The language model in the trained model 34 of this embodiment generates a short sentence corresponding to the prompt using a corpus based on the acquired target information. In addition, the language model generates text data that is supplementary information and supplementary correction information based on the acquired processing result (details will be described later).

[0026] As an example, the image generation model is a model that has been trained with a large amount of image data, and when a prompt is input, the image generation model generates and outputs image data corresponding to the prompt. The image generation model in the trained model 34 of this embodiment generates and outputs an image based on target information (details will be described later).

[0027] As an example, an image caption model is a model that has learned the semantic associations between image data and text data, and when image data is input, it generates and outputs text (caption) data corresponding to the image data.

[0028] As an example, the evaluation model is a type of reinforcement learning, and is a model that has learned human evaluation criteria for the quality of image data generated by an image generation model from text data. As a specific example, when image data that looks like a real photo is generated for text data that instructs the output of an illustration-like image, the evaluation model determines that the quality is not as intended and outputs a low evaluation for the image data. In this embodiment, each model constituting the trained model 34 described above is configured to be stored in the storage unit 30, but this is not limited thereto, and at least one model may be stored externally and connected to the information processing system 10 so as to be communicable.

[0029] The result DB 36 is a database that stores information related to the information acquired so far by the information processing system 10 and the results of processing executed. The result DB 36 stores, for example, the target information, input information, evaluation results, and training data sets acquired so far, linked to each other.

[0030] The processing unit 40 is realized by the processor 20 reading and executing the program 32 from the memory 22 (see FIG. 2) and cooperating with other hardware configurations. The processing unit 40 includes a request information acquisition unit 42, a topic generation unit 44, a goal information acquisition unit 46, an input information generation unit 48, an evaluation unit 50, a training data creation unit 52, and an output unit 54.

[0031] The request information acquisition unit 42 acquires request information, which is information related to a request regarding the processing result (corresponding to A of B1 in FIG. 4). This request information corresponds to a prompt input by the user to the user terminal 12, and is text data expressing at least a part of the request regarding the processing result that the user wants to obtain through the information processing system 10. As an example, when a user needs a training data set to obtain a trained model that outputs images with a style that gives a soft impression, the user inputs "Please give me an example of a warm and fuzzy illustration" as the request information in the text data PT (see FIG. 7), and the request information acquisition unit 42 acquires the text data PT as the request information and stores it in the result DB 36.

[0032] As shown in FIG. 6, the topic generation unit 44 generates at least one of a plurality of topics TP related to the requested information and information (images IG) related to the topic based on the acquired requested information (corresponding to B and C of B1 in FIG. 4). That is, the topic generation unit 44 generates text data that is information related to the requested information by inputting a prompt to the language model to output information related to the requested information. In addition, the topic generation unit 44 is capable of generating images as information related to the topic by inputting a prompt to the image generation model to output an image corresponding to the requested information or the text data related to the requested information described above. As an example, in response to the requested information of "Please give an example of a warm and cozy illustration," the topic generation unit 44 outputs text data TP such as "Girls relaxing at a cafe" and "Mermaid living in a fantastical underwater world" as topics related to "warm and cozy illustration" (see FIG. 7). In addition, the topic generation unit 44 generates generated images IG (see FIG. 8) representing "Girls relaxing at a cafe" and "Mermaid living in a fantastical underwater world" as images related to these topics.

[0033] The target information acquisition unit 46 selectably outputs at least one of the topic generated by the topic generation unit 44 and information related to the topic, and acquires the selected information as target information (corresponding to D of B1 in FIG. 4). That is, as shown in FIG. 7, the target information acquisition unit 46 selectably outputs at least one of the topic and the generated image IG (see FIG. 8) with the topic as a prompt as information related to the topic to the user terminal 12, acquires information on the result of the selection made by the user at the user terminal 12, acquires at least one of the selected generated image IG and the topic corresponding to the generated image IG as target information, and stores it in the result DB 36. As an example, when only the topic "Girls relaxing at a cafe" is selected from the text data of the topics "Girls relaxing at a cafe" and "Mermaid living in a fantastic underwater world", the text data "Girls relaxing at a cafe" is set as the target information. Furthermore, when a user U selects an image of a particular style from among a plurality of generated images IG with different styles related to the topic "Girls Relaxing in a Cafe" using the selection input unit SB in FIG. 8, at least one of the selected image itself and information on the style (style information) is set as the target information. Note that "art style" here refers to the style of an image, and includes various styles such as realism, impressionism, pop art, illustration, manga, gekiga, and shadow art, as examples. In addition, the selected topic and the generated image IG as information related to the topic are configured to be acquired as the target information, but this is not limiting, and only the topic or only the generated image IG may be acquired as the target information.

[0034] The input information generating unit 48 creates a corpus (corresponding to A of B2 in FIG. 4, details will be described later) and generates a prompt, which is input information for obtaining a processing result related to the acquired target information (corresponding to B of B2 in FIG. 4). That is, the input information generating unit 48 generates a prompt such that the output (in other words, the processing result) from the image generation model in the trained model 34 is related to the target information. The input information generating unit 48 may generate a prompt based on request information, or may generate a prompt based on target information. In addition, the input information generating unit 48 generates a prompt using a corpus. That is, the input information generating unit 48 creates a corpus, which is a data set of language collected and organized for a specific purpose, and randomly samples at least one of nouns, adjectives, and verbs from the corpus to generate a short sentence that becomes a prompt. In this corpus, as an example, it is limited to simple words, specifically, it is set to the A1 or A2 level in the CEFR (Common European Framework of Reference for Languages). The corpus is not limited to a newly created one, and an existing language database such as WordNet may be used.

[0035] The input information generating unit 48 also clusters the generated prompts based on a predetermined criterion (corresponding to C of B2 in FIG. 4). As an example, the input information generating unit 48 converts each of the generated prompts into a vector of numerical values, that is, maps them into a high-dimensional vector space, and clusters prompts whose relative positions are equal to or smaller than a predetermined criterion (in other words, prompts that have similar meanings because they are close in position) into the same class.

[0036] The evaluation unit 50 receives the target information and the input information, outputs the processing result from the trained model 34, and evaluates the processing result. That is, the evaluation unit 50 acquires the target information and the input information (corresponding to A of B3 in FIG. 4), and inputs the acquired information to the image generation model in the trained model 34, thereby generating image data according to the target information by the image generation model (corresponding to B of B3 in FIG. 4). This image data is data of an image whose style and content match based on the information on the style obtained from the target information and the input information representing the content of the image. The evaluation unit 50 generates and outputs image data for each class clustered by the input information generation unit 48. In addition, the image data generated by the evaluation unit 50 does not necessarily match the target information and the input information completely, but at least a part of the image is matched or similar to at least one of the target information and the input information.

[0037] In addition, the evaluation unit 50 evaluates the generated image data with respect to the target information and the input information (corresponding to C of B3 in FIG. 4). That is, the evaluation unit 50 evaluates the generated image data using the evaluation model of the trained model 34.

[0038] The training data creation unit 52 creates a training data set for an external machine learning model (not shown) from the image data that has been given a predetermined evaluation by the evaluation unit 50. Specifically, the evaluation unit 50 acquires the image data with the highest evaluation score (as an example, a quantified score such that the more an image conforms to the target information or input information, the higher the evaluation) as a predetermined evaluation from among the multiple image data generated for each clustered class, and the input information (prompt) corresponding to the image data (corresponding to A of B4 in FIG. 4), and creates a training data set by processing the acquired information.

[0039] That is, the training data creation unit 52 generates a caption and supplementary correction information as supplementary information (corresponding to B and C of B4 in FIG. 4), and creates a training data set for the above-mentioned image data including the corresponding input information, supplementary information, and supplementary correction information. That is, the training data creation unit 52 creates a training data set including text data (caption) generated from an image caption model as supplementary information using image data that has been given a predetermined evaluation by the evaluation unit 50 as input. As an example, if the image data is "an illustration of a family relaxing and chatting in a living room," the training data creation unit 52 generates captions such as "a family sitting on a sofa," "a cartoon-style family sitting around a table," and "an illustration of a family sitting in a living room" as supplementary information from the image data. The training data creation unit 52 combines the input information and the supplementary information into one sentence using a language model or the like, and creates training data including the sentence that is the result of the combination. In addition, if multiple image data are generated based on at least one of the same input information and target information, the training data creation unit 52 generates supplementary information including common predetermined information. As an example, when multiple image data are generated based on the same input information, the training data creation unit 52 generates common supplementary information such as “style A” for each image data using a language model or the like and links them.

[0040] Furthermore, when the generated supplemental information includes information that is estimated to be highly related to the target information, the training data creation unit 52 creates a training data set including supplemental correction information generated by deleting the related information. As an example, when the target information is a "warm illustration", the supplemental information for the image generated based on the input information is a caption such as "cartoon-style family sitting around a table" or "illustration of a family sitting in the living room". In this case, since "illustration" in the target information and "cartoon-style" and "illustration" in the supplemental information are highly related information, the training data creation unit 52 generates supplemental correction information of "family sitting around a table" and "family sitting in the living room" from which the highly related information has been deleted, and creates a training data set including this. In other words, supplemental correction information from which related information that is easily linked to the style information acquired as the target information has been deleted is generated, and a training data set including this is created. Note that the degree of this relevance may be determined by using a trained model that determines the relevance based on whether or not it is equal to or exceeds a predetermined standard, or may be determined in advance based on a rule base based on whether or not the relevance between information is equal to or exceeds a predetermined standard, or may be something else.

[0041] Furthermore, the training data creation unit 52 uses a language model, an evaluation model, etc. to determine how much at least a part of the generated image data matches the input information, and creates a training data set including supplemental correction information with unmet information indicating that the matching is not reached if a predetermined matching level is not reached. As an example, for input information of "a family relaxing and chatting in a living room", it determines how much the part of the generated image data corresponding to the faces of the people who make up the family matches the input information of "family", in other words, whether the faces of the people who make up the family in the image data can be recognized as people's faces. Then, if the determination result does not satisfy the predetermined matching level of "natural as a person's face", it creates a training data set including supplemental correction information with unmet information of "bad face" added, since the face does not meet the level required by the input information. In other words, the unmet information is information that indicates at least one of the parts and contents of the processing result that are poorly done with respect to the input information.

[0042] When the output unit 54 receives a request to download the training data set from the user terminal 12 after the training data creation unit 52 has completed creating the training data set, the output unit 54 transmits the training data set to the user terminal 12 (corresponding to D of B4 in Figure 4).

[0043] (Processing performed by information processing system) Next, the operation of the information processing system 10 will be described. Fig. 5 is a flow chart showing an example of the flow of processing by the information processing system 10. The processor 20 reads out the program 32 stored in the storage 24, expands it in the memory 22, and executes it, thereby performing processing. Although not shown, when the processor 20 receives operation information for terminating the operation of the information processing system 10, or information on the end of an operation from the user terminal 12 during the determination processing being executed (these will be simply referred to as "termination operation"), the processor 20 ends the processing based on the program 32 being processed.

[0044] The processor 20 determines whether or not request information has been input from the user terminal 12 (step S100). If request information has not been input (step S100: NO), the processor 20 repeats the process of step S100. On the other hand, if request information has been input (step S100: YES), the processor 20 acquires the request information (step S102), generates a topic from the request information (step S104, see FIG. 7), and generates an image, which is information related to the topic (step S106).

[0045] The processor 20 transmits the generated topics and images related to the topics to the user terminal 12 in a selectable manner (see FIG. 8), and determines whether or not they have been selected by the user terminal 12 (step S108). If they have not been selected (step S108: NO), the processor 20 shifts the process to step S106. On the other hand, if they have been selected (step S108: YES), the processor 20 acquires at least one of the selected topic and the image related to the topic as target information (step S110).

[0046] The processor 20 generates input information from the corpus (step S112) and clusters the input information (step S114). The processor 20 then generates an image based on the input information for each class and the target information (step S116), and evaluates the generated image (step S118).

[0047] The processor 20 determines whether or not a predetermined evaluation has been made for the image (step S120). If the predetermined evaluation has not been made (step S120: NO), the processor 20 proceeds to the process of step S116. On the other hand, if the predetermined evaluation has been made (step S120: YES), the processor 20 generates supplemental information corresponding to the image (step S124).

[0048] The processor 20 determines whether the supplemental information includes information that is estimated to be highly related to the target information (step S126). If there is no information that is estimated to be highly related (step S126: NO), the processor 20 proceeds to step S130, which will be described later. On the other hand, if there is information that is estimated to be highly related (step S126: YES), the processor 20 generates supplemental correction information by deleting the information that is estimated to be highly related to the supplemental information (step S128).

[0049] The processor 20 determines whether or not the degree of match between at least a portion of the image data and the input information is equal to or greater than a predetermined degree (step S130). If the degree of match is equal to or greater than the predetermined degree (step S130: YES), the processor 20 proceeds to step S134, which will be described later. On the other hand, if the degree of match is not equal to or greater than the predetermined degree (step S130: NO), the processor 20 adds unachieved information to the supplemental information (step S132).

[0050] The processor 20 determines whether the processing results are multiple image data with common input information (step S134). If the processing results do not have common input information (step S134: NO), the processor 20 proceeds to step S138, which will be described later. On the other hand, if the processing results have common input information (step S134: YES), the processor 20 generates common correction information and adds it to the previously generated correction information (step S136).

[0051] The processor 20 creates a training data set for the image data, including the corresponding input information, supplementary information, and supplementary correction information (step S138), and outputs the data when an output request is received (step S140), and then ends the process.

[0052] (Effects of this embodiment) According to the information processing system 10 of this embodiment, the following steps are performed: a goal information acquisition step of acquiring goal information, which is information about a goal in a processing result; an input information generation step of generating input information to be input to the trained model 34 so that the output from the trained model becomes a processing result related to the goal information; an evaluation step of evaluating the processing result output from the trained model 34 by inputting the goal information and the input information; and a training data creation step of selecting the processing result that has been evaluated in the evaluation step and creating a training data set for a machine learning model from the selected processing result. That is, when the goal information in the processing result is acquired, input information (prompt) that becomes a processing result related to the goal information is generated, the processing result obtained by the prompt is evaluated, and the processing result to be used in the training data set is selected from the evaluation result, so that a processing result according to the goal information can be obtained. In addition, instead of generating a processing result directly from the goal information, a prompt is generated so that the processing result becomes related to the goal information, and the processing result generated by the prompt is used, so that a variety of processing results that can be interpreted from the prompt can be obtained. That is, training data can be obtained so that the output result from the trained model is output in a targeted style while having diversity. Also, from another perspective, by inputting target information, user U can obtain training data based on a series of information generated from the target information even if user U does not have training data at hand.

[0053] In addition, the training data creation step generates supplementary information by text data for the processing result, and generates a training data set including the input information and the supplementary information, so that the actually generated image data can have additional information in addition to the input information by the supplementary information. In other words, by adding the supplementary information, it is possible to create a training data set that reduces ambiguity that occurs between image data and text data. As a result, by training with the training data set, a machine learning model that outputs a desired result with a high probability can be obtained.

[0054] Furthermore, in the training data creation step, the training data creation unit 52 combines the input information and the supplementary information into one sentence using a language model or the like, and creates training data including the sentence resulting from the combination, making it possible to create a training data set that reduces ambiguity that occurs between the image data and the input information and supplementary information. As a result, by training with the training data set, it is possible to obtain a machine learning model that outputs a desired result with a higher probability.

[0055] Furthermore, the training data creation step generates supplementary information about the processing results, and generates a training data set including supplementary correction information excluding information in the supplementary information that is estimated to be highly related to the target information, so that it is possible to suppress learning of the target information in association with the actually generated image data. In other words, when the target information is input, it is possible to suppress biased output of only specific image data. This makes it possible to obtain a training data set for obtaining a machine learning model with diversity in the output results.

[0056] In addition, the training data creation step generates supplementary information about the processing result, and generates a training data set including supplementary correction information with non-achievement information added when at least a part of the processing result does not reach a predetermined degree of agreement with the target information, so that the machine learning model can be trained including the non-achievement information. This makes it possible to obtain a training data set for obtaining a machine learning model that suppresses the output of undesired results.

[0057] Furthermore, the training data creation step generates supplemental information including common predetermined information for multiple processing results with the same input information, and generates a training dataset including the supplemental information. Therefore, even if multiple processing results with different contents are generated for one input information, they can be determined to be related processing results, so that the output results of the machine learning model trained on the training dataset can be made more diverse.

[0058] Furthermore, in the target information acquisition step, the request information, which is information regarding a request for the processing result, is acquired, and at least one of a plurality of topics related to the request information and information related to the topic is generated and output selectably, and the selected topic is acquired as the target information. Therefore, the content of the target information can be made related to the request information and diverse. As a result, by training with the training data set, it is possible to achieve both that the output result from the model is output in accordance with the target style and that the output result is made diverse. Furthermore, as shown in Figures 6 and 8, the image IG as information related to the topic is output selectably, so that the user U can visually select even an image style that is difficult to verbalize. As a result, the target information can be easily set, and the output result from the model can be made more desirable.

[0059] Furthermore, since the input information is generated using a corpus in the input information generation step, it is possible to prevent the input information, i.e., the prompt, from becoming biased or from becoming an expression that is inappropriate for obtaining the desired processing result.

[0060] Furthermore, in the input information generation step, the generated multiple pieces of input information are clustered based on predetermined criteria, and in the evaluation step, evaluation is performed based on the input information for each cluster, thereby maintaining diversity in the processing results.

[0061] Furthermore, in the evaluation step, the evaluation is performed using an evaluation model that has been trained to output an evaluation result that evaluates the relationship between the input information and the processing result according to a predetermined criterion when at least one of the target information and the input information and the processing result is input, so that the evaluation step can be performed automatically, thereby realizing automation of the creation of a training data set.

[0062] (Modification) In the above embodiment, as shown in FIG. 6, multiple topics TP related to the requested information and images IG related to the topic TP are generated based on the acquired requested information, and the selected image IG is acquired as the target information. However, the process may be repeated multiple times. That is, as shown in FIG. 9, additional requested information is acquired for the selected image IG among the multiple topics TP and images IG generated based on the acquired requested information, and multiple topics TP and images IG related to the topic TP are generated based on the selected image IG, and the selected image IG is acquired as the target information. Specifically, when information is acquired that "the original prompt (topic TP) when the image IG was generated is changed little by little to create several more patterns" as an example of additional requested information for the prompt TP and image IG generated based on the initial requested information (first stage in the figure), a topic TP is further generated based on the additional requested information, and an image IG is generated based on the topic TP (second stage in the figure). The selected image IG is acquired as the target information. That is, the topic TP and the image IG related to the topic TP are generated multiple times based on the requested information. In other words, the target information can be acquired through interactive processing between the user U and the information processing system 10. This allows more precise target information to be set, making it possible to obtain more desirable output results from the model. Note that, in this modified example, the processing is performed in the first and second stages, but this is not limiting, and the target information may be acquired after repeating a number of further processing steps, such as the third stage and above.

[0063] In addition, in the above-described embodiment, desired information is acquired, and a selection from among topics related to the desired information and image data generated based on the topics is used as target information. However, this is not limited to this, and directly input text data or image data may also be used as target information.

[0064] Further, the supplemental information is a caption of the generated image data, but is not limited to this and may be other information such as log information at the time of generating the image data.

[0065] Furthermore, the topics generated based on the request information and the image data from the topics are configured to be selected by the user U, but this is not limiting and the selection may be automatic using an evaluation model or the like.

[0066] Furthermore, although image data is generated as information related to a topic, the present invention is not limited to this, and text data or other data may be generated.

[0067] In addition, the information processing system 10 is a system that automatically creates a training dataset for training a machine learning model so that the machine learning model can generate images that conform to the targeted style information, but is not limited to this, and may also be a system that creates a training dataset for training the machine learning model so that the machine learning model can generate sentences that conform to a specific writing style, or may target other information.

[0068] <Additional Notes> The present embodiment includes the following disclosure.

[0069] (Appendix 1) An information processing method executed by an information processing device, a goal information acquisition step of acquiring goal information which is information related to a goal in the processing result; an input information generation step of generating input information to be input to the trained model so that an output from the trained model becomes the processing result related to the target information; An evaluation step of evaluating the processing result output from the trained model by inputting the target information and the input information; a training data creation step of selecting the processing results that have been evaluated in a predetermined manner in the evaluation step, and creating a training data set for a machine learning model from the selected processing results; An information processing method comprising:

[0070] (Appendix 2) The training data creation step generates supplemental information about the processing result, and generates the training data set including the input information and the supplemental information. 2. An information processing method as described in Appendix 1.

[0071] (Appendix 3) The training data creation step generates supplementary information about the processing result, and generates the training data set including supplementary correction information excluding information in the supplementary information that is estimated to be highly related to the target information. 2. An information processing method as described in Appendix 1.

[0072] (Appendix 4) The training data creation step generates supplementary information for the processing result, and generates the training data set including supplementary correction information including non-achievement information when at least a part of the processing result does not reach a predetermined degree of agreement with the target information. 2. An information processing method as described in Appendix 1.

[0073] (Appendix 5) The training data creation step generates supplemental information including common predetermined information for the plurality of processing results having the same input information, and generates the training data set including the supplemental information. 2. An information processing method as described in Appendix 1.

[0074] (Appendix 6) In the target information acquisition step, request information is acquired, which is information regarding requests for a processing result, and at least one of a plurality of topics related to the request information and information related to the topics is generated and output in a selectable manner, and the selected topic is acquired as the target information. 2. An information processing method as described in Appendix 1.

[0075] (Appendix 7) In the input information generating step, the input information is generated using a corpus. 2. An information processing method as described in Appendix 1.

[0076] (Appendix 8) In the input information generating step, the generated plurality of pieces of input information are clustered based on a predetermined criterion, In the evaluation step, evaluation is performed based on the input information for each cluster. 2. An information processing method as described in Appendix 1.

[0077] (Appendix 9) In the evaluation step, the evaluation is performed using a trained model for evaluation that has been trained to output an evaluation result obtained by evaluating a relationship between the input information and the processing result according to a predetermined criterion when at least one of the target information and the input information and the processing result are input. 2. An information processing method as described in Appendix 1.

[0078] (Appendix 10) In the information processing device, a goal information acquisition step of acquiring goal information which is information related to a goal in the processing result; an input information generation step of generating input information to be input to the trained model so that the output from the trained model becomes the processing result that coincides with the target information; An evaluation step of evaluating the input information and the processing result output from the trained model by inputting the input information; a training data creation step of selecting the processing results that have been evaluated in a predetermined manner in the evaluation step, and creating a training data set for a machine learning model from the selected processing results; A program for executing an information processing method including the steps of:

[0079] (Appendix 11) An information processing system executed by an information processing device, a goal information acquisition step of acquiring goal information which is information related to a goal in the processing result; an input information generation step of generating input information to be input to the trained model so that the output from the trained model becomes the processing result that coincides with the target information; An evaluation step of evaluating the input information and the processing result output from the trained model by inputting the input information; a training data creation step of selecting the processing results that have been evaluated in a predetermined manner in the evaluation step, and creating a training data set for a machine learning model from the selected processing results; An information processing system that performs the above.

[0080] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is indicated by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. The present invention is not limited to the above-mentioned embodiments, and various modifications are possible within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0081] 10 Information Processing Systems 32 Programs 34 pre-trained models

Claims

1. An information processing method executed by an information processing device, a goal information acquisition step of acquiring goal information which is information related to a goal in the processing result; an input information generation step of generating input information to be input to the trained model so that an output from the trained model becomes the processing result related to the target information; An evaluation step of evaluating the processing result output from the trained model to which the target information and the input information are input; a training data creation step of selecting the processing results that have been evaluated in a predetermined manner in the evaluation step, and creating a training data set for a machine learning model from the selected processing results; An information processing method comprising:

2. The training data creation step generates supplemental information about the processing result, and generates the training data set including the input information and the supplemental information. The information processing method according to claim 1 .

3. The training data creation step generates supplementary information about the processing result, and generates the training data set including supplementary correction information excluding information in the supplementary information that is estimated to be highly related to the target information. The information processing method according to claim 1 .

4. The training data creation step generates supplementary information for the processing result, and generates the training data set including supplementary correction information including non-achievement information when at least a part of the processing result does not reach a predetermined degree of agreement with the target information. The information processing method according to claim 1 .

5. The training data creation step generates supplemental information including common predetermined information for the plurality of processing results having the same input information, and generates the training data set including the supplemental information. The information processing method according to claim 1 .

6. In the target information acquisition step, request information including requests for the processing result is acquired, and at least one of a plurality of topics related to the request information and information related to the topics is generated and output in a selectable manner, and at least one of the selected topic and information related to the topics is acquired as the target information. The information processing method according to claim 1 .

7. In the input information generating step, the input information is generated using a corpus. The information processing method according to claim 1 .

8. In the input information generating step, the generated plurality of pieces of input information are clustered based on a predetermined criterion, In the evaluation step, evaluation is performed based on the input information for each cluster. The information processing method according to claim 1 .

9. In the evaluation step, evaluation is performed using a trained model for evaluation that has been trained to output an evaluation result obtained by evaluating the processing result according to a predetermined criterion when at least one of the target information and the input information and the processing result are input. The information processing method according to claim 1 .

10. In the information processing device, a goal information acquisition step of acquiring goal information which is information related to a goal in the processing result; an input information generation step of generating input information to be input to the trained model so that the output from the trained model becomes the processing result that coincides with the target information; An evaluation step of evaluating the input information and the processing result output from the trained model by inputting the input information; a training data creation step of selecting the processing results that have been evaluated in a predetermined manner in the evaluation step, and creating a training data set for a machine learning model from the selected processing results; A program for executing an information processing method including the steps of:

11. An information processing system executed by an information processing device, a goal information acquisition step of acquiring goal information which is information related to a goal in the processing result; an input information generation step of generating input information to be input to the trained model so that the output from the trained model becomes the processing result that coincides with the target information; An evaluation step of evaluating the input information and the processing result output from the trained model by inputting the input information; a training data creation step of selecting the processing results that have been evaluated in a predetermined manner in the evaluation step, and creating a training data set for a machine learning model from the selected processing results; An information processing system that performs the above.

Images