Information processing device, information processing method, and program

Abstract

This information processing device is provided with: a generation unit that generates training data by inputting a prompt and input image data to one or a plurality of first machine-learning models; and a training unit that uses the training data to train a second machine-learning model having a lower processing cost than the first machine-learning model(s).

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] Techniques for reducing the processing cost of machine learning models are known. For example, Patent Literature 1 discloses a learning device that optimizes a model cascade including a first model and a second model.

[0003] International Publication No. 2021 / 176734

[0004] However, Cited Document 1 does not anticipate reducing the processing cost of a machine learning model that uses a prompt and an image as input and performs inference on a target specified in the prompt. This machine learning model is highly accurate but has a high processing cost, so there is a demand for a machine learning model that is highly accurate and has a low processing cost.

[0005] The present disclosure has been made in consideration of the above problems, and one of its objectives is to provide a machine learning model that is highly accurate and has low processing costs.

[0006] An information processing device according to an exemplary aspect of the present disclosure includes a generation means for generating training data by inputting a prompt and input image data into one or more first machine learning models, and a training means for using the training data to train a second machine learning model having a lower processing cost than the first machine learning model.

[0007] An information processing device according to an exemplary aspect of the present disclosure includes an acquisition means for acquiring image data, and an inference means for performing inference processing on the image data acquired by the acquisition means using a second machine learning model trained using learning data generated by inputting a prompt and input image data into one or more first machine learning models.

[0008] An information processing method according to an exemplary aspect of the present disclosure includes a generation process in which at least one processor generates training data by inputting a prompt and input image data into one or more first machine learning models, and a training process in which the training data is used to train a second machine learning model having a lower processing cost than the first machine learning model.

[0009] An information processing method according to an exemplary aspect of the present disclosure includes an acquisition process in which at least one processor acquires image data, and an inference process in which at least one processor performs an inference process on the image data acquired by the acquisition process using a second machine learning model trained using training data generated by inputting a prompt and input image data into one or more first machine learning models.

[0010] A program according to an exemplary aspect of the present disclosure is a program that causes a computer to function as an information processing device, and causes the computer to function as a generation means that generates training data by inputting a prompt and input image data into one or more first machine learning models, and a learning means that uses the training data to train a second machine learning model that has a lower processing cost than the first machine learning model.

[0011] A program according to an exemplary aspect of the present disclosure is a program that causes a computer to function as an information processing device, and causes the computer to function as an acquisition means that acquires image data and an inference means that performs inference processing on the image data acquired by the acquisition means using a second machine learning model trained using learning data generated by inputting a prompt and input image data into one or more first machine learning models.

[0012] According to one exemplary aspect of the present disclosure, an exemplary effect is achieved in that it is possible to provide a machine learning model that is highly accurate and has low processing costs.

[0013] FIG. 1 is a block diagram showing a configuration of an information processing device according to the present disclosure. FIG. 2 is a flow diagram showing a flow of an information processing method according to the present disclosure. FIG. 3 is a block diagram showing a configuration of an information processing device according to the present disclosure. FIG. 4 is a flow diagram showing a flow of an information processing method according to the present disclosure. FIG. 4 is a block diagram showing a configuration of an information processing device according to the present disclosure. FIG. 5 is a diagram showing an example of a flow of data in an information processing device according to the present disclosure. FIG. 6 is a diagram showing another example of a flow of data in an information processing device according to the present disclosure. FIG. 7 is a diagram showing yet another example of a flow of data in an information processing device according to the present disclosure. FIG. 8 is a diagram showing an example of an integration process by a generation unit according to the present disclosure. FIG. 9 is a diagram showing another example of an integration process by a generation unit according to the present disclosure. FIG. 10 is a flow diagram showing a flow of an information processing method according to the present disclosure. FIG. 11 is a block diagram showing a configuration of a computer that functions as information processing according to the present disclosure.

[0014] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0015] [First Exemplary Embodiment] A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is a basic form for each of the exemplary embodiments described below. Note that the scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in the drawings referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0016] (Configuration of information processing device 1) The configuration of the information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. As shown in Fig. 1, the information processing device 1 includes a generation unit 11 and a learning unit 12. In this exemplary embodiment, the generation unit 11 and the learning unit 12 respectively realize a generation means and a learning means.

[0017] The generation unit 11 generates training data by inputting the prompt and the input image data to one or more first machine learning models. The generation unit 11 supplies the generated training data to the training unit 12.

[0018] A prompt is a text that contains instructions for a machine learning model. One example of a prompt is text that contains the name of an object to be recognized. Another example of a prompt is text that contains the name of the object to be recognized and a command to recognize the object, written in natural language.

[0019] The learning unit 12 uses the training data generated by the generation unit 11 to train a second machine learning model that has a lower processing cost than the first machine learning model.

[0020] (Effects of information processing device 1) As described above, the information processing device 1 employs a configuration including a generation unit 11 that generates learning data by inputting a prompt and input image data into one or more first machine learning models, and a learning unit 12 that uses the learning data generated by the generation unit 11 to train a second machine learning model that has a lower processing cost than the first machine learning model.

[0021] Therefore, the information processing device 1 has the effect of being able to provide a machine learning model that is highly accurate and has low processing costs.

[0022] (Flow of Information Processing Method S1) The flow of information processing method S1 will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of information processing method S1. As shown in Fig. 2, information processing method S1 includes a generation process S11 and a learning process S12.

[0023] In the generation process S11, the generation unit 11 generates training data by inputting the prompt and the input image data into one or more first machine learning models. The generation unit 11 supplies the generated training data to the training unit 12.

[0024] (Learning Process S12) In the learning process S12, the learning unit 12 uses the learning data generated by the generation unit 11 in the generation process S11 to train a second machine learning model that has a lower processing cost than the first machine learning model.

[0025] (Effects of Information Processing Method S1) As described above, the information processing method S1 employs a configuration including a generation process S11 in which the generation unit 11 generates training data by inputting a prompt and input image data into one or more first machine learning models, and a training process S12 in which the training unit 12 trains a second machine learning model having a lower processing cost than the first machine learning model, using the training data generated by the generation unit 11 in the generation process S11. Therefore, the information processing method S1 can achieve the same effects as the information processing device 1 described above.

[0026] (Configuration of information processing device 2) The configuration of the information processing device 2 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 2. As shown in Fig. 3, the information processing device 2 includes an acquisition unit 21 and an inference unit 22. In this exemplary embodiment, the acquisition unit 21 and the inference unit 22 respectively realize an acquisition means and an inference means.

[0027] The acquisition unit 21 acquires image data and supplies the acquired image data to the inference unit 22.

[0028] The inference unit 22 performs inference processing on the image data acquired by the acquisition unit 21 using a second machine learning model trained using learning data generated by inputting the prompt and input image data into one or more first machine learning models.

[0029] (Effects of Information Processing Device 2) As described above, the information processing device 2 employs a configuration including an acquisition unit 21 that acquires image data, and an inference unit 22 that executes inference processing on the image data acquired by the acquisition unit 21, using a second machine learning model trained using training data generated by inputting a prompt and input image data into one or more first machine learning models. Therefore, the information processing device 2 can achieve the same effects as the information processing device 1 described above.

[0030] (Flow of Information Processing Method S2) The flow of information processing method S2 will be described with reference to Fig. 4. Fig. 4 is a flow diagram showing the flow of information processing method S2. As shown in Fig. 4, information processing method S2 includes an acquisition process S21 and an inference process S22.

[0031] (Acquisition Process S21) In the acquisition process S21, the acquisition unit 21 acquires image data. The acquisition unit 21 supplies the acquired image data to the inference unit 22.

[0032] (Inference process S22) In the inference process S22, the inference unit 22 performs inference processing on the image data acquired by the acquisition unit 21 in the acquisition process S21 using a second machine learning model trained using learning data generated by inputting the prompt and input image data into one or more first machine learning models.

[0033] (Effects of Information Processing Method S2) As described above, the information processing method S2 employs a configuration including an acquisition process S21 in which the acquisition unit 21 acquires image data, and an inference process S22 in which the inference unit 22 performs inference processing on the image data acquired by the acquisition unit 21 in the acquisition process S21, using a second machine learning model trained using training data generated by inputting the prompt and input image data into one or more first machine learning models. Therefore, the information processing method S2 can achieve the same effects as the information processing device 1 described above.

[0034] Second Exemplary Embodiment A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be denoted by the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technical means shown in each drawing referenced to describe this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0035] (Overview of Information Processing Device 3) The information processing device 3 is a device that trains a machine learning model. As an example, the information processing device 3 uses training data TD generated by a first machine learning model LM1 that receives a prompt PR and input image data IPD as input to train a second machine learning model LM2 that has a lower processing cost than the first machine learning model LM1. The prompt PR is as described above. Note that the method by which the information processing device 3 trains the second machine learning model LM2 is similar to the information processing method S1 described above, and therefore description thereof will be omitted.

[0036] The first machine learning model LM1 is a base model that has been machine-trained using a large number of sets of images and text as training data. Examples of the first machine learning model LM include, but are not limited to, Contrastive Language-Image Pre-training (CLIP), Grounding Self-Distillation with No Labels (DINO), Grounded Language-Image Pre-training (GLIP), Segment Anything Model (SAM), and Bootstrapping Language-Image Pre-training (BLIP)-2.

[0037] The second machine learning model LM2 is a machine learning model with a lower processing cost than the first machine learning model LM1, and is also referred to as a lightweight model. Examples of the second machine learning model LM2 include, but are not limited to, YOLO (You Only Look Once), DINO, and DETR (End-to-End Object Detection with Transformers).

[0038] The second machine learning model LM2 may be a pre-trained model. That is, the information processing device 3 may be configured to add a new class to the second machine learning model LM2. As an example, the information processing device 3 trains the first machine learning model LM1, which is YOLO pre-trained with COCO (Common Objects in Context), to recognize a new object.

[0039] Furthermore, the information processing device 3 executes inference processing on the image data PD. Examples of inference processing executed by the information processing device 3 include, but are not limited to, object detection, segmentation, caption generation, image classification, and region classification.

[0040] (Configuration of information processing device 3) The configuration of the information processing device 3 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the information processing device 3. As shown in Fig. 5, the information processing device 3 includes a control unit 30, a storage unit 50, a communication unit 60, and an input / output unit 70.

[0041] The storage unit 50 stores data referenced by the control unit 30. Examples of the storage unit 50 include, but are not limited to, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0042] Examples of data stored in the memory unit 50 include image data PD, input image data IPD, prompt PR, learning data TD, a first machine learning model LM1, a machine learning model LM3, and a second machine learning model LM2.

[0043] The machine learning model LM3 may be a base model like the first machine learning model LM1, or may be a lightweight model like the second machine learning model LM2. When the machine learning model LM3 is a base model, the machine learning model LM3 can also be expressed as the first machine learning model LM1.

[0044] Furthermore, the number of machine learning models LM3 is not particularly limited, and may be one, or two or more, as shown in Figure 5. When there are two or more machine learning models LM3, the two or more machine learning models LM3 may be base models, lightweight models, or a combination of base models and lightweight models.

[0045] The first machine learning model LM1 and the machine learning model LM3 are referred to as a machine learning model group LMG. That is, the machine learning model group LMG is a model group that includes at least one or more first machine learning models LM1.

[0046] Furthermore, the machine learning model being stored in the storage unit 50 means that parameters defining the machine learning model are stored in the storage unit 50. Furthermore, the machine learning model may be stored in another device that can transmit and receive data to and from the information processing device 3.

[0047] The communication unit 60 is an interface for transmitting and receiving data via a network. Examples of the communication unit 60 include, but are not limited to, communication chips for various communication standards such as Ethernet (registered trademark), Wi-Fi (Wireless Fidelity (registered trademark), and wireless communication standards for mobile data communication networks, and USB-compliant connectors.

[0048] For example, the communication unit 60 receives image data PD, input image data IPD, and prompt PR from another device, and outputs an inference result to the other device.

[0049] The input / output unit 70 is an interface that receives input of data and outputs data. Examples of the input / output unit 70 include, but are not limited to, a microphone, a camera, a gaze input device, a keyboard, a touchpad, a speaker, and a liquid crystal display.

[0050] As one example, the input / output unit 70 acquires a prompt PR input by a user and supplies the acquired prompt PR to the control unit 30. As another example, the input / output unit 70 displays an image indicated by an image signal supplied from the control unit 30.

[0051] (Control Unit 30) The control unit 30 controls each component included in the information processing device 3. As shown in Fig. 5 , the control unit 30 also includes an acquisition unit 31, a reception unit 32, a generation unit 33, a learning unit 34, and an inference unit 35. In this exemplary embodiment, the acquisition unit 31, the reception unit 32, the generation unit 33, the learning unit 34, and the inference unit 35 respectively realize an acquisition means, a reception means, a generation means, a learning means, and an inference means.

[0052] The acquisition unit 31 acquires data via the communication unit 60 or the input / output unit 70. The acquisition unit 31 stores the data acquired via the communication unit 60 or the input / output unit 70 in the storage unit 50. The acquisition unit 31 also acquires data from the storage unit 50. As an example, the acquisition unit 31 acquires image data PD.

[0053] The receiving unit 32 receives the prompt PR via the user interface. The receiving unit 32 stores the received prompt PR in the storage unit 50. With this configuration, the information processing device 3 can receive the prompt PR from the user.

[0054] As an example, the reception unit 32 outputs an image prompting the user to input a prompt PR to the input / output unit 70. The input / output unit 70 displays the image prompting the user to input a prompt PR. The reception unit 32 receives, via the input / output unit 70, the prompt PR input by the user in response to the image displayed by the input / output unit 70.

[0055] The generation unit 33 generates training data TD by inputting the prompt PR and input image data IPD to one or more first machine learning models LM1. The generation unit 33 also generates training data TD by inputting the input image data TD to a machine learning model group LMG. For example, the generation unit 33 generates training data TD by pairing images output from one or more first machine learning models LM1 or the machine learning model group LMG with labels generated with reference to the prompt PR.

[0056] The following describes an example of the processing executed by the generation unit 33. Note that, in the example of the processing executed by the generation unit 33, a configuration in which input image data IPD is input to a machine learning model group LMG will be described, but as described above, the generation unit 33 may generate training data TD using one first machine learning model LM1 instead of the machine learning model group LMG.

[0057] The learning unit 34 trains a second machine learning model LM2 using the training data TD. The learning unit 34 also trains one or more first machine learning models LM1 using the prompts PR received by the reception unit 32. With this configuration, the information processing device 3 trains the first machine learning model LM1 that generates the training data TD used to train the second machine learning model LM2, thereby making it possible to provide a highly accurate second machine learning model LM2.

[0058] The inference unit 35 uses the second machine learning model LM2 to perform inference processing on the image data PD. Examples of the inference processing are as described above. With this configuration, the information processing device 3 can provide highly accurate inference results because the second machine learning model LM2, which has been trained using the training data TD, performs the inference processing.

[0059] (Example 1 of Processing Executed by Generator 33) An example of processing executed by the generator 33 will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the flow of data in the information processing device 3.

[0060] In this example, the generation process of the training data TD by the generation unit 33 includes a process of inputting input image data IPD to a certain machine learning model included in the machine learning model group LMG, and a process of generating the training data TD by inputting at least a portion of the output from the certain machine learning model to another machine learning model included in the machine learning model group LMG. As an example, in FIG. 6 , a case will be described in which the certain machine learning model is a first machine learning model LM1 and the other machine learning model is a machine learning model LM3.

[0061] 6, the generation unit 33 supplies the input image data IPD and the prompt PR1 to the first machine learning model LM1. The first machine learning model LM1 receives the input image data IPD and the prompt PR1 and generates an inference result 1. The first machine learning model LM1 supplies at least a portion of the inference result 1 to the machine learning model LM3.

[0062] The machine learning model LM3 uses at least a portion of the inference result 1 as input to generate an inference result 2. Here, if the machine learning model LM3 is a base model, the generation unit 33 may supply a prompt PR2 to the machine learning model LM3. In this case, the machine learning model LM3 uses at least a portion of the inference result 1 and the prompt PR2 as input to generate an inference result 2. The machine learning model LM3 supplies at least a portion of the inference result 2 to the generation unit 33.

[0063] When the generation unit 33 acquires at least a portion of the inference result 2, it generates training data TD based on at least a portion of the inference result 2. The learning unit 34 acquires the training data TD generated by the generation unit 33, and trains a second machine learning model LM2 using the training data TD.

[0064] For example, assume that the learning unit 34 uses an image as input and trains the second machine learning model LM2 to detect a predetermined object contained in the image. In this case, the generation unit 33 first supplies the first machine learning model LM1 with input image data IPD containing the predetermined object and a prompt PR1 instructing the first machine learning model LM1 to extract a region containing the predetermined object.

[0065] The first machine learning model LM1 receives the input image data IPD and the prompt PR1 as input, and supplies an inference result 1 including at least an image of the detected area to the machine learning model LM3.

[0066] The machine learning model LM3 receives an image included in the inference result 1 as input and detects a predetermined object included in the image. Here, if the machine learning model LM3 is a base model, the generation unit 33 may supply the machine learning model LM3 with a prompt PR2 instructing it to detect the predetermined object. The machine learning model LM3 supplies the generation unit 33 with an inference result 2 including at least an image of the detected predetermined object.

[0067] The generation unit 33 generates learning data TD that pairs an image included in the inference result 2 with a label indicating that the image is a predetermined object.

[0068] For example, the generation unit 33 generates training data TD by combining an image included in the inference result 2 with a label generated with reference to the prompt PR1. In addition, as an example, the generation unit 33 may be configured to generate the prompt PR1 with reference to the prompt used to determine the input image data IPD.

[0069] For example, to obtain input image data IPD, the generation unit 33 first inputs a prompt such as "I want images of various excavators" into LLM (Large Language Models) having an image generation function. Then, the generation unit 33 sets the image output from the LLM to the input image data IPD.

[0070] Here, the LLM is a machine-learned model that uses the content indicated by the input information as input and generates response information indicating the response content. Examples of LLMs include Chatbot, BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer), T5 (Text-to-Text Transfer Transformer), RoBERTa (Robustly optimized BERT approach), ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately), and a learning model (e.g., ChatGPT (Chat Generative Pre-trained Transformer)) generated by transfer learning or fine tuning using a pre-trained model. Examples of LLMs with image generation capabilities include, but are not limited to, Stable Diffusion, DALL-E2, and Midjourney.

[0071] Next, the generation unit 33 analyzes the prompt "I want images of various excavators" input to the LLM and extracts the word "excavator" indicating a predetermined object. The generation unit 33 then generates a prompt PR1 (e.g., "a big excavator," "an excavator at a construction site," etc.) that includes the word "excavator" indicating the predetermined object.

[0072] The generation unit 33 also references the prompt PR1 and extracts the word "shovel" indicating a predetermined object (i.e., the predetermined object extracted by analyzing the prompt input to the LLM). The generation unit 33 generates training data TD that pairs an image included in the inference result 2 with the label "shovel."

[0073] As another example, the generation unit 33 may generate the prompt PR1 by referring to a prompt when collecting an image.

[0074] For example, the generation unit 33 searches for images on the Internet to collect images. In this case, the generation unit 33 searches for images using the prompt "yellow excavator." Then, the generation unit 33 refers to the prompt "yellow excavator" and extracts the word "excavator" indicating a predetermined object. The generation unit 33 may also analyze the prompt "yellow excavator" and extract the word "yellow excavator" indicating a predetermined object. In this case, the generation unit 33 may use LLM to extract the word "yellow excavator" indicating the predetermined object. Then, the generation unit 33 generates a prompt PR1 including the word "excavator" (or "yellow excavator") indicating the predetermined object. In the same manner as described above, the generation unit 33 generates training data TD in which an image included in the inference result 2 is paired with the label "excavator."

[0075] In the process exemplified below, labels may be generated in the same manner as in this example.

[0076] In this way, the information processing device 3 generates training data TD by inputting at least a portion of the output of a certain machine learning model to another machine learning model. For example, assume a case where the second machine learning model LM2 is trained to detect a backhoe included in an image. In this case, the information processing device 3 causes the certain machine learning model to extract a ground area, and causes the other machine learning model to detect the backhoe from the extracted ground area. In this way, the information processing device 3 causes the other machine learning model to perform an inference process based on the inference result of the certain machine learning model, thereby enabling the other machine learning model to perform a highly accurate inference process.

[0077] Furthermore, at least one of the certain machine learning model and the other machine learning model is a base model. That is, since the training data TD is generated based on the inference results of the base model, the information processing device 3 can generate highly accurate training data TD.

[0078] (Example 2 of Processing Executed by Generator 33) Another example of processing executed by the generator 33 will be described with reference to Fig. 7. Fig. 7 is a diagram showing another example of the flow of data in the information processing device 3.

[0079] In this example, the generation process of the training data TD by the generation unit 33 includes inputting at least a portion of the output of the other machine learning model into the certain machine learning model. Also in FIG. 7, the certain machine learning model is the first machine learning model LM1, and the other machine learning model is the machine learning model LM3.

[0080] 7, similar to FIG. 6, the generation unit 33 supplies the input image data IPD and the prompt PR1 to the first machine learning model LM1. The first machine learning model LM1 receives the input image data IPD and the prompt PR1 and generates an inference result 1. The first machine learning model LM1 supplies at least a portion of the inference result 1 to the machine learning model LM3.

[0081] The machine learning model LM3 takes at least a portion of the inference result 1 as input and generates an inference result 2. Here, as in FIG. 6 , if the machine learning model LM3 is a base model, the generation unit 33 may supply a prompt PR2 to the machine learning model LM3. In this case, the machine learning model LM3 takes at least a portion of the inference result 1 and the prompt PR2 as input and generates an inference result 2. The machine learning model LM3 outputs at least a portion of the inference result 2 to the first machine learning model LM1.

[0082] When the first machine learning model LM1 acquires at least a portion of the inference result 2, it uses at least a portion of the inference result 2 and the prompt PR1 as input to generate an inference result 3. The first machine learning model LM1 supplies at least a portion of the inference result 3 to the machine learning model LM3.

[0083] The machine learning model LM3 receives as input at least a portion of the inference result 3 and generates the inference result 4. As described above, the machine learning model LM3 may receive as input at least a portion of the inference result 3 and the prompt PR2 and generate the inference result 4. The machine learning model LM3 supplies at least a portion of the inference result 4 to the generation unit 33.

[0084] When the generation unit 33 acquires at least a portion of the inference result 4, it generates training data TD based on at least a portion of the inference result 4. The learning unit 34 acquires the training data TD generated by the generation unit 33, and trains a second machine learning model LM2 using the training data TD.

[0085] For example, as in the example described above, assume that the learning unit 34 uses an image as input and trains the second machine learning model LM2 to detect a predetermined object contained in the image. In this case, as in the example described above, the generation unit 33 first supplies the first machine learning model LM1 with input image data IPD including the predetermined object and a prompt PR1 instructing it to extract an area including the predetermined object.

[0086] The first machine learning model LM1 receives the input image data IPD and the prompt PR1 as input, and supplies an inference result 1 including at least an image of the detected area to the machine learning model LM3.

[0087] The machine learning model LM3 receives an image included in the inference result 1 as input and detects a predetermined object included in the image. Here, if the machine learning model LM3 is a base model, the generation unit 33 may supply the machine learning model LM3 with a prompt PR2 instructing it to detect the predetermined object. The machine learning model LM3 supplies the first machine learning model LM1 with an inference result 2 including at least an image of the detected predetermined object.

[0088] The first machine learning model LM1 supplies the machine learning model LM3 with an inference result 3 that includes at least an image of the detected area, using as input the image included in the inference result 2 and the prompt PR1. In other words, the first machine learning model LM1 supplies the machine learning model LM3 with an inference result 3 that is more accurate than the inference result 1.

[0089] The machine learning model LM3 receives an image included in the inference result 3 as input and detects a predetermined object included in the image. The machine learning model LM3 supplies the generation unit 33 with an inference result 4 that includes at least an image of the detected predetermined object. In other words, the machine learning model LM3 supplies the generation unit 33 with an inference result 4 that is more accurate than the inference result 2.

[0090] The generation unit 33 generates learning data TD that pairs an image included in the inference result 4 with a label indicating that the image is a predetermined object.

[0091] In this way, the information processing device 3 inputs at least a portion of the output from the other machine learning model into the certain machine learning model. In other words, the information processing device 3 creates a loop between the certain machine learning model and the other machine learning model. For example, assume that the second machine learning model LM2 is trained to classify the breed of a cat included in an image. In this case, the information processing device 3 causes the certain machine learning model to extract an area including an animal, and causes the other machine learning model to extract a cat from the extracted area and classify the breed of the extracted cat.

[0092] Furthermore, the information processing device 3 causes a certain machine learning model to extract a region containing an animal from a cat region extracted by another machine learning model. That is, the information processing device 3 extracts a region containing an animal with even higher accuracy. Then, the information processing device 3 causes the other machine learning model to classify the breed of the cat contained in the region containing the animal extracted with high accuracy.

[0093] With this configuration, the information processing device 3 can cause other machine learning models to execute inference processing with higher accuracy.

[0094] In this example, at least one of the certain machine learning model and the other machine learning model is a base model. That is, since the training data TD is generated based on the inference results of the base model, the information processing device 3 can generate highly accurate training data TD.

[0095] (Example 3 of Processing Executed by Generator 33) Yet another example of processing executed by generator 33 will be described with reference to Fig. 8. Fig. 8 is a diagram showing yet another example of the flow of data in information processing device 3.

[0096] In this example, the process of generating the learning data TD by the generation unit 33 includes a process of inputting input image data IPD to each of the machine learning model groups LMG, and a process of integrating the outputs from each of the machine learning model groups LMG.

[0097] 8, the generation unit 33 supplies the input image data IPD and the prompt PR1 to the first machine learning model LM1. The generation unit 33 also supplies the input image data IPD to the machine learning model LM3. Here, as in the example described above, if the machine learning model LM3 is a base model, the generation unit 33 may supply the input image data IPD and the prompt PR2 to the machine learning model LM3.

[0098] The first machine learning model LM1 receives the input image data IPD and the prompt PR1 as input and generates an inference result 1. The first machine learning model LM1 supplies at least a portion of the generated inference result 1 to the generation unit 33.

[0099] The machine learning model LM3 receives the input image data IPD (and the prompt PR2) and generates an inference result 2. The machine learning model LM3 supplies at least a portion of the generated inference result 2 to the generation unit 33.

[0100] Upon acquiring at least a portion of the inference result 1 and at least a portion of the inference result 2, the generation unit 33 generates training data TD by integrating at least a portion of the inference result 1 and at least a portion of the inference result 2. An example of the integration process will be described later. The learning unit 34 acquires the training data TD generated by the generation unit 33, and trains a second machine learning model LM2 using the training data TD.

[0101] Here, it is assumed that the machine learning model group LMG includes multiple base models. For example, it is assumed that the machine learning model group LMG includes a first machine learning model LM1 and a machine learning model LM3 that is a base model.

[0102] In this case, if the first machine learning model LM1 and the machine learning model LM3 are the same base model, it is preferable that the prompt PR1 and the prompt PR2 have different content.

[0103] Also, in this case, if the first machine learning model LM1 and the machine learning model LM3 are different base models, the prompt PR1 and the prompt PR2 may have the same content or different content.

[0104] When the prompt PR1 and the prompt PR2 have different contents, the generation unit 33 may input the prompt PR1 and the prompt PR2 corresponding to the recognition characteristics of the first machine learning model LM1 and the machine learning model LM3, respectively. In other words, the generation unit 33 may execute a process of changing or correcting the prompt PR in the process of generating the training data TD.

[0105] In one example of this configuration, the generation unit 33 first acquires a prompt PR template. Next, the generation unit 33 inputs the prompt PR template and an image including a recognition target into the first machine learning model LM1 and the machine learning model LM3, respectively. The generation unit 33 executes this process using a plurality of mutually different images and the prompt PR template. Then, the generation unit 33 acquires recognition characteristics based on the recognition rates of the first machine learning model LM1 and the machine learning model LM3.

[0106] For example, assume that the first machine learning model LM1 is a base model that is robust in recognizing large / small objects, and the machine learning model LM3 is a base model that is capable of recognizing the state of an object. In this case, the generation unit 33 inputs a prompt PR1 such as a "large excavator" or a "small bulldozer" into the first machine learning model LM1. The generation unit 33 also inputs a prompt PR2 such as a "clean excavator" or a "rusty bulldozer" into the machine learning model LM3.

[0107] As another example, in the case of a machine learning model capable of finely recognizing colors, the generation unit 33 inputs a prompt such as "yellow excavator" or "blue bulldozer" into the machine learning model.

[0108] As yet another example, in the case of a machine learning model capable of detecting groups, the generation unit 33 inputs prompts such as "excavators lined up in a row" or "bulldozers gathered in a circle" into the machine learning model.

[0109] As yet another example, in the case of a machine learning model capable of character characterization, the generation unit 33 inputs prompts such as "a bulldozer with the XX logo" or "a bulldozer with the YY logo" into the machine learning model.

[0110] As yet another example, in the case of a machine learning model capable of characterizing distance, the generator 33 inputs prompts such as "a far-away excavator" or "a nearby bulldozer" into the machine learning model.

[0111] In this way, the information processing device 3 inputs the input image data IPD to each of the machine learning model groups LMG and integrates the outputs from each of the machine learning model groups LMG. Therefore, the information processing device 3 can generate high-quality training data TD in which the recognition characteristics of each of the machine learning model groups LMG complement each other.

[0112] Furthermore, even if the outputs from each of the machine learning model groups LMG contain noise such as unrecognized or erroneous recognition results, the information processing device 3 can integrate them to generate appropriate learning data TD while suppressing the influence of noise.

[0113] (Example 1 of Integration Process) As an example, when the learning unit 34 uses an image as input and trains the second machine learning model LM2 to classify the image, the generation unit 33 acquires the output from each of the machine learning model group LMG and generates the training data TD using, for example, one of the following methods: simple average weighted average mode voting Here, the mode is, for example, a method of using the most frequent value (mode) among the outputs of the machine learning model group LMG as the correct label. For example, assume that an image is input to a machine learning model group LMG including machine learning models LM_1 to LM_3, and the output from machine learning model LM_1 is "dog," the output from machine learning model LM_2 is "cat," and the output from machine learning model LM_3 is "dog." In this case, if the learning unit 34 uses the mode, a pair of the image input to the machine learning model group LMG and a label indicating the most frequent value, "dog," is generated as the training data TD.

[0114] On the other hand, "voting" is, for example, a method of referencing the output and inference score from each machine learning model group LMG. For example, assume that an image is input to a machine learning model group LMG including machine learning models LM_1 to LM_3, and the output from machine learning model LM_1 is "dog (inference score 0.4)," the output from machine learning model LM_2 is "cat (inference score 0.8)," and the output from machine learning model LM_3 is "dog (inference score 0.4)." In this case, if the learning unit 34 uses voting, the aggregated result of the inference score for "dog" is 0.7 and the aggregated result of the inference score for "cat" is 0.8. Therefore, a pair of the image input to the machine learning model group LMG and a label indicating "cat," which has the highest aggregated inference score, is generated as training data TD.

[0115] The above processing can also be used when the learning unit 34 uses an image as input and trains the second machine learning model LM2 to classify the areas included in the image.

[0116] Furthermore, when the learning unit 34 takes an image as input and trains the second machine learning model LM2 to output text related to the image (captioning), the generation unit 33 acquires output from each of the machine learning model group LMG and generates training data TD using, for example, one of the following methods: Top N rank Summary using LLM (Example 2 of Integration Process) As another example, a case where the learning unit 34 takes an image as input and trains the second machine learning model LM2 to detect a backhoe as a predetermined object included in the image will be described with reference to FIG. 9. FIG. 9 is a diagram showing an example of the integration process performed by the generation unit 33.

[0117] 9 to each of the machine learning model group LMG. In this example, the machine learning model group LMG includes a first machine learning model LM1, a machine learning model LM3, a machine learning model LM4, and a machine learning model LM5. As described above, the machine learning model LM3, the machine learning model LM4, and the machine learning model LM5 may each be a base model or a lightweight model.

[0118] The generation unit 33 obtains inference results 1 to 4 shown in FIG. 9 from the first machine learning model LM1, the machine learning model LM3, the machine learning model LM4, and the machine learning model LM5, respectively.

[0119] Upon acquiring the inference results 1 to 4, the generation unit 33 generates the training data TD shown in Fig. 9 by, for example, NMS (Non-Maximum Suppression). Examples of NMS include, but are not limited to, soft NMS and weighted NMS based on object size.

[0120] (Example 3 of Integration Process) As yet another example, similar to Example 2, a case in which an image is used as input and the second machine learning model LM2 is trained to detect a backhoe as a predetermined object included in the image will be described with reference to Fig. 10. Fig. 10 is a diagram showing another example of integration process by the generation unit 33.

[0121] The generation unit 33 inputs the input image data IPD5 shown in the upper part of Fig. 10 to each of the machine learning model group LMG. In this example, the machine learning model group LMG includes a first machine learning model LM1, a machine learning model LM3, and a machine learning model LM4. As described above, the machine learning model LM3 and the machine learning model LM4 may each be a base model or a lightweight model.

[0122] The generation unit 33 obtains inference results 5 to 7 shown in the upper part of FIG. 10 from the first machine learning model LM1, the machine learning model LM3, and the machine learning model LM4.

[0123] In this example, the generating unit 33 generates the training data TD using the K-means method or the X-means method for MIL (multiple instance learning).

[0124] Specifically, the generation unit 33 calculates the size and center coordinates of each of the acquired inference results 5 to 7. Then, the generation unit 33 classifies the inference results 5 to 7 using the K-means method or the X-means method. As shown in the graph in the center of Fig. 10, the inference results 5 to 7 are classified as the same object because their respective distances are close in the graph of size and center coordinates.

[0125] The generation unit 33 uses the most frequent value as a representative of the inference results 5 to 7 that are classified as the same object. Then, as shown in the lower part of Fig. 10 , the generation unit 33 generates learning data TD5 to TD7, each of which is a pair of an image included in each of the inference results 5 to 7 and a label indicating that it is a backhoe, which is the most frequent value.

[0126] (Example 4 of Processing Executed by Generator 33) Yet another example of processing executed by the generator 33 will be described.

[0127] In this example, the training data TD generated with reference to at least a portion of the output of one or more first machine learning models LM1 includes an image of an object included in the input image data IPD and a soft label associated with the image. Alternatively, the training data TD generated with reference to at least a portion of the output of at least one machine learning model included in the machine learning model group LMG includes an image of an object included in the input image data IPD and a soft label associated with the image. Here, a soft label refers to, for example, a label composed of multiple non-zero probabilities for each classification class. For example, while a one-hot label for a three-class classification can be expressed as (0, 1, 0), a soft label can be expressed as (0.1, 0.7, 0.2).

[0128] In addition, in this configuration, the learning unit 34 trains the second machine learning model LM2 by referring to the image of the object and the soft label.

[0129] Below, we will explain the case where the learning data TD generated by referring to at least a portion of the output of the first machine learning model LM1 includes an image of an object contained in the input image data IPD and a soft label associated with the image.

[0130] If the categories of the classes to be classified by the second machine learning model LM2 are similar (the characteristics of the classes are similar), the generation unit 33 may generate training data TD including images and soft labels included in the output from the first machine learning model LM1. An example of a method for determining whether the categories of the classes are similar is to refer to the distance of intermediate feature vectors that reflect the language of the first machine learning model LM1. In this method, the generation unit 33 calculates the weight of the soft label based on the distance of the intermediate feature vector and applies it to the output of the first machine learning model LM1.

[0131] For example, since excavators and bulldozers are included in the category of construction heavy machinery, when the second machine learning model LM2 classifies excavators and bulldozers, this corresponds to a case where the classes are similar. In this case, if the output from the first machine learning model LM1 includes the image to be classified and the soft labels of person: 0.1, excavator: 0.6, and bulldozer: 0.2, the generation unit 33 generates training data TD that includes the images and soft labels included in the output of the first machine learning model LM1.

[0132] In this way, the information processing device 3 generates training data TD including images of objects included in the input image data IPD and soft labels associated with the images, and trains the second machine learning model LM2 by referring to the images and soft labels. With this configuration, the information processing device 3 can train the second machine learning model LM2 so that even if a classification error occurs, the second machine learning model LM2 will be categorized into a similar class.

[0133] (Example 5 of Processing Executed by Generator 33) Yet another example of processing executed by the generator 33 will be described with reference to Fig. 11. Fig. 11 is a diagram showing an example of augmentation processing by the generator 33.

[0134] The generation process of the training data TD by the generation unit 33 includes an extension process of extending the input image data IPD and a process of inputting the input image data IPD extended in the extension process to one or more first machine learning models LM1. Alternatively, the generation process of the training data TD by the generation unit 33 includes an extension process of extending the input image data IPD and a process of inputting the input image data IPD extended in the extension process to each of the machine learning models included in the machine learning model group LMG.

[0135] 11 , the generation unit 33 flips the input image data IPD8 to generate extended input image data PID8A. The generation unit 33 also rotates the input image data IPD8 to generate extended input image data PID8B. The generation unit 33 inputs at least one of the extended input image data PID8A and the input image data PID8B to one or more first machine learning models LM1 (or each of the machine learning models included in the machine learning model group LMG; the same applies hereinafter).

[0136] Furthermore, when multiple pieces of input image data IPD, such as input image data IPD8 and extended input image data IPD8A, are input to one or more first machine learning models LM1, the generation unit 33 may integrate the inference results for each of the multiple pieces of input image data IPD. The integration method is as described above.

[0137] In addition to the above-mentioned flipping and rotation, types of dilation include, but are not limited to, translation, expansion, and contraction.

[0138] In this way, the information processing device 3 inputs the expanded input image data IPD to one or more first machine learning models LM1. Therefore, if the one or more first machine learning models LM1 cannot make appropriate inference using the input image data IPD before expansion but can make appropriate inference using the expanded input image data IPD, the information processing device 3 can cause the one or more first machine learning models LM1 to make appropriate inference.

[0139] (Flow of Information Processing Method S3) The flow of information processing method S3, which is a process of training one or more first machine learning models LM1, will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the flow of information processing method S3.

[0140] (Step S31) In step S31, the reception unit 32 receives a prompt PR via the user interface and stores the received prompt PR in the storage unit 50.

[0141] (Step S32) In step S32, the learning unit 34 trains one or more first machine learning models LM1 using the prompt PR received by the receiving unit 32 in step S31.

[0142] In this way, in the information processing method S3, the first machine learning model LM1, which generates the learning data TD used to learn the second machine learning model LM2, is trained using the prompt PR received from the user, thereby making it possible to provide a highly accurate second machine learning model LM2.

[0143] (Effects of information processing device 3) As described above, in the information processing device 3, the training data TD generated by inputting the prompt PR and input image data IPD into one or more first machine learning models LM1 (or machine learning model groups LMG) is used to train a second machine learning model LM2, which has a lower processing cost than the first machine learning model LM1.

[0144] With this configuration, the information processing device 3 trains the second machine learning model LM2 using training data TD generated using the highly accurate first machine learning model LM1, thereby making it possible to provide the second machine learning model LM2 with high accuracy and low processing costs. Furthermore, because the information processing device 3 has low processing costs, for example, an edge terminal can function as the information processing device 3.

[0145] Furthermore, the information processing device 3 generates training data TD for training the second machine learning model LM2 by inputting the prompt PR and the input image data IPD to the first machine learning model LM1. Therefore, the information processing device 3 can build the second machine learning model LM2 at low construction costs.

[0146] [Software Implementation Example] Some or all of the functions of the information processing devices 1, 2, and 3 (hereinafter also referred to as "each of the above devices") may be implemented by hardware such as an integrated circuit (IC chip), or by software.

[0147] In the latter case, each of the above devices is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 13. Figure 13 is a block diagram showing the hardware configuration of computer C that functions as each of the above devices.

[0148] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to function as each of the above-mentioned devices. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing the functions of each of the above-mentioned devices.

[0149] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0150] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0151] The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0152] [Appendix 1] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0153] (Supplementary Note 1) An information processing device comprising: a generation means for generating training data by inputting a prompt and input image data into one or more first machine learning models; and a training means for using the training data to train a second machine learning model having a lower processing cost than the first machine learning model.

[0154] (Supplementary Note 2) The information processing device according to Supplementary Note 1, wherein the generating means generates the training data by inputting the input image data to a group of machine learning models including at least the one or more first machine learning models.

[0155] (Supplementary Note 3) The information processing device described in Supplementary Note 2, wherein the process of generating the training data by the generation means includes a process of inputting the input image data to a certain machine learning model included in the group of machine learning models, and a process of generating the training data by inputting at least a portion of the output from the certain machine learning model to another machine learning model included in the group of machine learning models.

[0156] (Supplementary Note 4) The information processing device according to Supplementary Note 3, wherein the process of generating the training data by the generation means includes a process of inputting at least a portion of an output from the other machine learning model into the certain machine learning model.

[0157] (Supplementary Note 5) The information processing device according to Supplementary Note 2, wherein the process of generating the learning data by the generation means includes a process of inputting the input image data to each of the machine learning models, and a process of integrating outputs from each of the machine learning models.

[0158] (Supplementary Note 6) The information processing device described in Supplementary Note 1, wherein the training data generated by referring to at least a portion of the output of the one or more first machine learning models includes an image of an object included in the input image data and a soft label associated with the image, and the training means trains the second machine learning model by referring to the image of the object and the soft label.

[0159] (Supplementary Note 7) The information processing device described in any one of Supplementary Notes 2 to 5, wherein the learning data generated by referring to at least a portion of the output of at least one machine learning model included in the group of machine learning models includes an image of an object included in the input image data and a soft label associated with the image, and the learning means trains the second machine learning model by referring to the image of the object and the soft label.

[0160] (Supplementary Note 8) The information processing device according to Supplementary Note 1, wherein the process of generating the learning data by the generation means includes an extension process of extending the input image data, and a process of inputting the input image data extended in the extension process into the one or more first machine learning models.

[0161] (Supplementary Note 9) The information processing device described in any one of Supplementary Notes 2 to 5, wherein the process of generating the learning data by the generation means includes an extension process of extending the input image data, and a process of inputting the input image data extended in the extension process into each of the machine learning models included in the machine learning model group.

[0162] (Supplementary Note 10) The information processing device according to any one of Supplementary Notes 1 to 9, further comprising: an acquisition means for acquiring image data; and an inference means for performing inference processing on the image data using the second machine learning model.

[0163] (Supplementary Note 11) An information processing device comprising: an acquisition means for acquiring image data; and an inference means for performing inference processing on the image data acquired by the acquisition means using a second machine learning model trained using training data generated by inputting a prompt and input image data into one or more first machine learning models.

[0164] (Supplementary Note 12) The information processing device according to Supplementary Note 11, further comprising: a receiving unit configured to receive, via a user interface, a prompt that can be input to the first machine learning model.

[0165] (Supplementary Note 13) The information processing device according to Supplementary Note 12, further comprising: a learning unit configured to train the one or more first machine learning models using the prompts received by the receiving unit.

[0166] 1, 2, 3 Information processing device 11, 33 Generation unit 12, 34 Learning unit 21, 31 Acquisition unit 22, 35 Inference unit 32 Reception unit IPD Input image data LM1 First machine learning model LM2 Second machine learning model TD Learning data

Claims

1. An information processing device comprising: a generation means for generating training data by inputting a prompt and input image data into one or more first machine learning models; and a training means for using the training data to train a second machine learning model that has a lower processing cost than the first machine learning model.

2. The information processing device according to claim 1, wherein the generation means generates the training data by inputting the input image data to a group of machine learning models including at least the one or more first machine learning models.

3. The information processing device of claim 2, wherein the process of generating the training data by the generation means includes a process of inputting the input image data to a certain machine learning model included in the group of machine learning models, and a process of generating the training data by inputting at least a portion of the output from the certain machine learning model to another machine learning model included in the group of machine learning models.

4. The information processing device according to claim 3, wherein the process of generating the training data by the generation means includes a process of inputting at least a portion of the output from the other machine learning model into the certain machine learning model.

5. The information processing device according to claim 2, wherein the process of generating the learning data by the generation means includes a process of inputting the input image data to each of the group of machine learning models, and a process of integrating the outputs from each of the group of machine learning models.

6. The information processing device described in claim 1, wherein the training data generated by referring to at least a portion of the output of the one or more first machine learning models includes an image of an object included in the input image data and a soft label associated with the image, and the training means trains the second machine learning model by referring to the image of the object and the soft label.

7. The information processing device described in claim 2, wherein the training data generated by referring to at least a portion of the output of at least one of the machine learning models included in the group of machine learning models includes an image of an object included in the input image data and a soft label associated with the image, and the training means trains the second machine learning model by referring to the image of the object and the soft label.

8. The information processing device according to claim 1, wherein the process of generating the learning data by the generation means includes an extension process of extending the input image data, and a process of inputting the input image data extended in the extension process into the one or more first machine learning models.

9. The information processing device according to claim 2, wherein the process of generating the learning data by the generation means includes an extension process of extending the input image data, and a process of inputting the input image data extended in the extension process into each of the machine learning models included in the group of machine learning models.

10. An information processing device described in any one of claims 1 to 9, further comprising: an acquisition means for acquiring image data; and an inference means for performing inference processing regarding the image data using the second machine learning model.

11. An information processing device comprising: an acquisition means for acquiring image data; and an inference means for performing inference processing on the image data acquired by the acquisition means using a second machine learning model trained using training data generated by inputting a prompt and input image data into one or more first machine learning models.

12. The information processing device according to claim 11, further comprising: a receiving means for receiving, via a user interface, a prompt that can be input to the first machine learning model.

13. The information processing device according to claim 12, further comprising: a learning means for training the one or more first machine learning models using the prompts received by the receiving means.

14. An information processing method including: a generation process in which at least one processor generates training data by inputting a prompt and input image data into one or more first machine learning models; and a training process in which the training data is used to train a second machine learning model that has a lower processing cost than the first machine learning model.

15. A program that causes a computer to function as an information processing device, the program causing the computer to function as: a generation means that generates training data by inputting a prompt and input image data into one or more first machine learning models; and a training means that uses the training data to train a second machine learning model that has a lower processing cost than the first machine learning model.

16. An information processing method including: an acquisition process in which at least one processor acquires image data; and an inference process in which at least one processor performs an inference process on the image data acquired by the acquisition process using a second machine learning model trained using training data generated by inputting a prompt and input image data into one or more first machine learning models.

17. A program that causes a computer to function as an information processing device, comprising: an acquisition means that acquires image data; and an inference means that performs inference processing on the image data acquired by the acquisition means using a second machine learning model trained using training data generated by inputting a prompt and input image data into one or more first machine learning models.

Images