Information processing device and information processing program

Abstract

To provide a user with a retrieval result after eliminating the ambiguity of a retrieval object desired by the user.SOLUTION: An information processing device includes: a reception unit that receives input information which is input by a user who uses a retrieval system; a generation unit that inputs the input information into a machine-learning model, causes the machine-learning model to generate question information indicating a question for specifying a retrieval object desired by the user, obtains response information indicating a response to the question information, generates a retrieval query according to the response information, and generates output information according to a retrieval result corresponding to the retrieval query; and an output control unit that outputs the output information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing program. [Background technology]

[0002] Conventionally, there are known techniques for executing a search based on a search query entered by a user and providing the user with search results. For example, there is known a technique for accepting an input of a question sentence for a natural language search from a user, accepting an instruction to execute the natural language search from the user, acquiring information indicating the circumstances under which the instruction to execute the natural language search was issued, and processing the question sentence using the acquired information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-108035 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional techniques do not necessarily provide the user with search results that complement the ambiguity of the search target desired by the user.

[0005] The present invention aims to provide users with search results that complement the ambiguity of the search target desired by the user. [Means for solving the problem]

[0006] The information processing device of the present application includes a receiving unit that receives input information entered by a user who uses a search system, a generating unit that inputs the input information into a machine learning model to cause the machine learning model to generate question information indicating a question for identifying a search target desired by the user, obtains response information indicating a response to the question information, generates a search query according to the response information, and generates output information according to search results corresponding to the search query, and an output control unit that outputs the output information. [Effects of the Invention]

[0007] According to one aspect of the embodiment, it is possible to provide the user with search results that have complemented the ambiguity of the search target desired by the user. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining an outline of a search process according to the prior art. [Figure 2] FIG. 2 is a diagram for explaining an overview of the search process according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of an information processing system according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of the information processing device according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of information processing according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a prompt according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a prompt according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a prompt according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a prompt according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a prompt according to the embodiment. [Figure 11]FIG. 11 is a flowchart showing the procedure of information processing by the information processing device according to the embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of information processing according to the modified example. [Figure 13] FIG. 13 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, a detailed description will be given of an information processing device and an information processing program according to the present application (hereinafter referred to as an "embodiment") with reference to the drawings. Note that the information processing device and the information processing program according to the present application are not limited to the embodiment. Furthermore, the same components in the following embodiments are denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0010] (Embodiment) 1. Introduction FIG. 1 is a diagram illustrating an overview of search processing according to conventional technology. In FIG. 1, a terminal device 10 of a user U1 who uses a search system 200 transmits a search query entered by the user U1 to the search system 200. The search system 200 accepts the search query from the user U1. Specifically, the search system 200 acquires the search query from the terminal device 10. When the search system 200 acquires the search query, it executes a search based on the search query and transmits the search results to the terminal device 10. The terminal device 10 receives the search results from the search system 200.

[0011] FIG. 2 is a diagram illustrating an overview of a search process according to an embodiment. FIG. 2 differs from FIG. 1 in that the information processing device 100 according to the embodiment connects a user U1 who uses the search system 200 with the search system 200 using a machine learning model M1 that connects the user U1 and the search system 200. Specifically, when the user U1 uses the search system 200, the target that the user U1 wants to search for (hereinafter, may be referred to as a "search target") may be unclear. For example, when the search target is unclear, the user may not know the name of the target that the user wants to search for. When the search target is unclear, the user may only know some of the characteristics of the target that the user wants to search for. In response to this, the information processing device 100 complements the ambiguity of the search target desired by the user U1 using the machine learning model M1 that complements the ambiguity of the search target when the user U1 uses the search system 200.

[0012] Specifically, the information processing device 100 inputs input information (e.g., a search query including ambiguous information) input by a user U1 who uses the search system 200 into the machine learning model M1, and generates information for complementing the ambiguity of a search target desired by the user U1. More specifically, the information processing device 100 generates question information indicating a question for identifying the search target desired by the user U1, as information for complementing the ambiguity of the search target desired by the user U1. In other words, the information processing device 100 generates question information indicating a question for clarifying the search target desired by the user U1, as information for complementing the ambiguity of the search target desired by the user U1. For example, the information processing device 100 inputs the input information into the machine learning model M1, and generates question information indicating a question for identifying the search target desired by the user U1. In other words, the question information is information indicating what should be asked of the user U1 in order to identify the search target desired by the user U1. In other words, the question information is information indicating the content of a question to be asked to the user U1 in order to identify a search target desired by the user U1. For example, the machine learning model M1 may be a language model that generates and outputs information according to input information. For example, the machine learning model M1 may be a large language model (LLM). For example, the information processing device 100 inputs input information to the machine learning model M1 and causes the machine learning model M1 to generate question information according to the input information. For example, the information processing device 100 inputs input information that is text to the machine learning model M1. Furthermore, the information processing device 100 causes the machine learning model M1 to generate question information that is text.

[0013] In FIG. 2, the terminal device 10 of the user U1 transmits input information input by the user U1 using the search system 200 to the information processing device 100. For example, the input information may be information input by the user U1 with some search intent. For example, the input information may be a search query including ambiguous information. The information processing device 100 accepts the input information from the user U1. The information processing device 100 acquires the input information. When the information processing device 100 acquires the input information, the information processing device 100 inputs the input information to the machine learning model M1 to cause the machine learning model M1 to generate question information indicating a question for identifying the search target desired by the user U1. In this manner, the information processing device 100 generates the question information by causing the machine learning model M1 to generate the question information. When the information processing device 100 generates the question information, it transmits the question information to the terminal device 10. In this manner, the information processing device 100 generates question information according to the input information and transmits the generated question information to the terminal device 10, thereby enabling the ambiguity of the search target desired by the user U1 to be complemented.

[0014] Furthermore, although not shown in the figures, when the terminal device 10 receives question information, it displays the received question information on a screen. Furthermore, the terminal device 10 transmits response information indicating a response to the question to the information processing device 100 as input information. The information processing device 100 acquires the response information from the terminal device 10. When the information processing device 100 acquires the response information, it inputs the response information to the machine learning model M1 and causes the machine learning model M1 to generate a search query corresponding to the response information. In this way, the information processing device 100 generates a search query by causing the machine learning model M1 to generate the search query. Furthermore, the information processing device 100 acquires response information corresponding to the question information and generates a search query corresponding to the response information, thereby being able to generate a search query that complements the ambiguity of the search target desired by the user U1.

[0015] Also, in FIG. 2, the information processing device 100 inputs a search query generated by the machine learning model M1 to the search system 200. When the search system 200 acquires the search query, it executes a search based on the search query and transmits the search results to the information processing device 100. The information processing device 100 acquires the search results from the search system 200. When the information processing device 100 acquires the search results, it inputs the search results to the machine learning model M1 and causes the machine learning model M1 to generate output information corresponding to the search results. In this way, the information processing device 100 generates output information by causing the machine learning model M1 to generate the output information. Furthermore, the information processing device 100 transmits the output information to the terminal device 10. The terminal device 10 receives the output information from the information processing device 100.

[0016] As described above, the information processing device 100 accepts input information entered by a user U1 who uses the search system 200. The information processing device 100 also inputs the input information to the machine learning model M1, causes the machine learning model M1 to generate question information indicating a question for identifying a search target desired by the user U1, acquires response information indicating a response to the question information, generates a search query according to the response information, and generates output information according to search results corresponding to the search query. The information processing device 100 also outputs the output information. This allows the information processing device 100 to provide the user U1 with search results corresponding to the search query after complementing the ambiguity of the search target desired by the user U1. Therefore, the information processing device 100 can provide the user U1 with search results after complementing the ambiguity of the search target desired by the user U1.

[0017] [2. Information Processing System Configuration] An example of the configuration of an information processing system 1 according to an embodiment will be described with reference to FIG. 3. FIG. 3 is a diagram showing an example of the configuration of the information processing system 1 according to an embodiment. As shown in FIG. 3, the information processing system 1 includes a terminal device 10, an information processing device 100, and a search system 200. The terminal device 10, the information processing device 100, and the search system 200 are connected to each other via a network N in a wired or wireless manner so as to be able to communicate with each other. The network N is, for example, a wide area network (WAN) such as the Internet. Note that the information processing system 1 shown in FIG. 3 may include a plurality of terminal devices 10, a plurality of information processing devices 100, and a plurality of search systems 200.

[0018] The terminal device 10 is an information processing device used by a user U1 who uses the search system 200. The terminal device 10 is realized, for example, by a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like. The terminal device 10 displays information received from the information processing device 100 or the like using a web browser or an application. Note that the example shown in FIG. 2 shows a case where the terminal device 10 is a smartphone.

[0019] The information processing device 100 is an information processing device that performs information processing according to an embodiment, and is realized by, for example, a server device or a cloud system. In the example of FIG. 2, the information processing device 100 accepts input information entered by a user U1 who uses the search system 200. The information processing device 100 also inputs the input information to a machine learning model M1, causes the machine learning model M1 to generate question information indicating a question for identifying a search target desired by the user U1, acquires response information indicating a response to the question information, generates a search query according to the response information, and generates output information according to search results corresponding to the search query. The information processing device 100 also outputs the output information.

[0020] The search system 200 is an information processing device that provides a search service, and is realized by, for example, a server device, a cloud system, etc. In the example of Fig. 2, when the search system 200 acquires a search query, the search system 200 executes a search based on the search query and transmits the search results to the information processing device 100.

[0021] 3. Configuration of Information Processing Device An example of the configuration of the information processing device 100 according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. The information processing device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

[0022] (Communication unit 110) The communication unit 110 is realized by a NIC (Network Interface Card), an antenna, etc. The communication unit 110 is connected to various networks via wire or wirelessly, and transmits and receives information to and from the terminal device 10 and the search system 200, for example.

[0023] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. Specifically, the storage unit 120 stores various data. For example, the storage unit 120 may store various pieces of information received by the reception unit 132. The storage unit 120 may also store various pieces of information generated by the generation unit 133. The storage unit 120 may also store various pieces of information acquired by the generation unit 133. For example, the storage unit 120 stores various programs. For example, the storage unit 120 stores an information processing program according to the embodiment. The storage unit 120 may also store information related to various machine learning models. For example, the storage unit 120 stores information related to a machine learning model M1, which is a large-scale language model or a visual language model. The storage unit 120 also stores information related to an image generation model M2. The storage unit 120 also stores information related to an image recognition model M3. The storage unit 120 also stores information about the voice recognition model M4. The storage unit 120 may also store information about machine learning models that recognize information acquired by various sensors.

[0024] (control unit 130) The control unit 130 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs stored in a storage device inside the information processing device 100 using RAM as a work area. The control unit 130 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0025] The control unit 130 has an instruction unit 131, a reception unit 132, a generation unit 133, and an output control unit 134 as functional units, and may realize or execute the information processing actions described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 4, and may be any other configuration as long as it performs the information processing described below. Furthermore, each functional unit indicates a function of the control unit 130, and does not necessarily have to be physically distinct.

[0026] (Instruction section 131) The instruction unit 131 instructs the machine learning model M1 to identify a search target desired by a user using the search system, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query. Specifically, the instruction unit 131 may instruct the machine learning model M1, which is a language model that generates and outputs information according to input information, to identify a search target desired by the user, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query. More specifically, the machine learning model M1 may be a language model trained to estimate and output the next token from an input token sequence. For example, the machine learning model M1 may be a large-scale language model (LLM). For example, the machine learning model M1 may be OpenAI's gpt-3.5 or gpt-4.

[0027] FIG. 5 is a diagram illustrating an example of information processing according to an embodiment. In FIG. 5, the instruction unit 131 instructs the machine learning model M1 to identify a search target desired by a user U1 using the search system 200, generate a search query corresponding to the identified search target, and acquire search results corresponding to the generated search query (step S11). FIG. 5 illustrates a case where the machine learning model M1 is a large-scale language model. Specifically, the instruction unit 131 inputs a prompt to the machine learning model M1 to instruct the machine learning model M1 to identify a search target desired by the user U1, generate a search query corresponding to the identified search target, and acquire search results corresponding to the generated search query. In the following, "instructing the machine learning model M1 to identify a search target desired by the user U1 using the search system 200, generate a search query corresponding to the identified search target, and acquire search results corresponding to the generated search query" may be referred to as "instructing as described above."

[0028] For example, the instructing unit 131 issues the above-described instruction to a machine learning model M1 (e.g., gpt-3.5 or gpt-4 by OpenAI, Inc.) that can input prompts divided into system prompts and user prompts. For example, the instructing unit 131 issues the above-described instruction by inputting a system prompt to the machine learning model M1 that instructs the model to identify a search target desired by the user U1, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query. For example, the instructing unit 131 may issue the above-described instruction by inputting the prompt P1 shown in FIG. 6 to the machine learning model M1. FIG. 6 is a diagram illustrating an example of a prompt according to the embodiment. Prompt P1 in Figure 6 is a system prompt that includes the following sentence: "Assist users who use the search system. Clarify the object of the user's search through conversation with the user. After reviewing and composing search information that accurately expresses the clarified object, call the search system. The following tools can be used when talking with the user and reviewing and composing search information: Image recognition tool (input: image file name, output: text describing what is shown in the image). Image generation tool (input: text, output: image based on the content of the text). If the information entered by the user to search for is correct, engage in conversation while clearly stating your own review process."

[0029] For example, the instruction unit 131 instructs the machine learning model M1 to identify a search target desired by the user U1 by inputting a prompt P1 including a sentence such as "assisting a user who uses the search system. Clarifying the target the user wants to search for by conversing with the user" shown in Fig. 6 to the machine learning model M1. Also, the instruction unit 131 instructs the machine learning model M1 to generate a search query corresponding to the identified search target and to obtain search results corresponding to the generated search query by inputting a prompt P1 including a sentence such as "considering and composing search information that accurately expresses the clarified target, and then calling the search system" shown in Fig. 6 to the machine learning model M1.

[0030] (Reception Department 132) The receiving unit 132 receives input information input by a user who uses the search system. In FIG. 5, the receiving unit 132 receives input information input by a user U1 who uses the search system 200 (step S12). For example, the receiving unit 132 may receive input information from the terminal device 10 of the user U1. For example, the receiving unit 132 may receive input text input by the user U1 as the input information. For example, the input text may be a sentence. Furthermore, when the receiving unit 132 receives the input information, it may output the input information to the generation unit 133.

[0031] (Generation unit 133) The generation unit 133 inputs input information to the machine learning model, causes the machine learning model to generate question information indicating a question for identifying a search target desired by the user, acquires response information indicating a response to the question information, generates a search query according to the response information, and generates output information according to search results corresponding to the search query. For example, the generation unit 133 may acquire the input information from the reception unit 132. When the generation unit 133 acquires the input information, the generation unit 133 generates question information corresponding to the input information based on the input information. The generation unit 133 generates question information corresponding to the input information. Specifically, the generation unit 133 inputs the input information to the machine learning model, causes the machine learning model to generate question information indicating a question for identifying a search target desired by the user. In FIG. 5, the generation unit 133 inputs the input information to the machine learning model M1, causes the machine learning model M1 to generate question information indicating a question for identifying a search target desired by the user U1 (step S13). More specifically, the generation unit 133 inputs, together with the input information, a prompt to the machine learning model M1 instructing the machine learning model M1 to generate question information indicating a question for identifying a search target desired by the user, and causes the machine learning model M1 to generate question information corresponding to the input information. For example, the generation unit 133 may input input text entered by the user U1 as the input information to the machine learning model M1, and cause the machine learning model M1 to generate question text corresponding to the input text as the question information. For example, the question text may be a sentence. Furthermore, when the generation unit 133 causes the machine learning model M1 to generate the question information, the generation unit 133 may output the question information to the output control unit 134.

[0032] For example, the generation unit 133 may input, together with the input information, a prompt P2 including a sentence such as "Is the information entered by the user sufficient to perform a search? Answer Yes or No," shown in the upper part of FIG. 7, to the machine learning model M1. FIG. 7 is a diagram illustrating an example of a prompt according to an embodiment. Furthermore, when the machine learning model M1 outputs "No" in response to the input of the prompt P2, the generation unit 133 may input, to the machine learning model M1, a prompt P3 including a sentence such as "Do you need to use a tool to ask the user again? Answer Yes or No," shown in the middle part of FIG. 7. Furthermore, when the machine learning model M1 outputs "No" in response to the input of the prompt P3, the generation unit 133 may input, to the machine learning model M1, a prompt P7 including a sentence such as "What information should be asked of the user again? Enter the wording to ask the user," shown in the upper part of FIG. 9. For example, the generation unit 133 may input the prompt P7 to the machine learning model M1 as a prompt instructing the machine learning model M1 to generate question information indicating a question for identifying a search target desired by the user. 9 is a diagram showing an example of a prompt according to the embodiment. Furthermore, the generation unit 133 may obtain, as question information, a sentence output from the machine learning model M1 in response to input of the prompt P7. In this way, the generation unit 133 may cause the machine learning model M1 to generate question information that is a sentence. In this way, the generation unit 133 may generate question information that is a sentence. Furthermore, the generation unit 133 may input prompts P2 to P11 to the machine learning model M1 as system prompts.

[0033] When the machine learning model M1 outputs "Yes" in response to the input of prompt P2, the generation unit 133 may input to the machine learning model M1 a prompt P4 including a sentence such as "Do I need to use a tool to perform a search? Answer "Yes" or "No," as shown in the lower part of FIG. 7. When the machine learning model M1 outputs "No" in response to the input of prompt P4, the generation unit 133 may input to the machine learning model M1 a prompt P8 including a sentence such as "What information do you want to search for? Output a sentence or a file name," as shown in the middle part of FIG. 9. The generation unit 133 may acquire, as a search query, the sentence or file name output from the machine learning model M1 in response to the input of prompt P8. In this manner, the generation unit 133 may cause the machine learning model M1 to generate a search query that is a sentence. In this manner, the generation unit 133 may cause the machine learning model M1 to generate a search query that is a sentence. In this manner, the generation unit 133 may cause the machine learning model M1 to generate a search query that is a sentence. In this manner, the generation unit 133 may generate a search query that is a sentence.

[0034] (output control unit 134) The output control unit 134 outputs various information. For example, the output control unit 134 may acquire various information from the generation unit 133. When the output control unit 134 acquires various information, it may output the various information. In FIG. 5, the output control unit 134 acquires question information from the generation unit 133. Furthermore, the output control unit 134 outputs the question information. In FIG. 5, when the output control unit 134 acquires question information, it outputs the question information (step S14). For example, the output control unit 134 may output the question information to the terminal device 10 of the user U1. The output control unit 134 may transmit the question information to the terminal device 10 of the user U1. When the terminal device 10 receives the question information, it may display the question information on a screen. Furthermore, the terminal device 10 may transmit information input by the user U1 to the information processing device 100 within a predetermined time after displaying the question information on a screen.

[0035] The receiving unit 132 also receives response information indicating a response to the question information. Specifically, the receiving unit 132 receives response information input by a user. In FIG. 5, the receiving unit 132 receives the response information input by the user U1 (step S15). For example, the receiving unit 132 may receive the response information from the terminal device 10 of the user U1. For example, the receiving unit 132 may receive, as the response information, information input to the terminal device 10 within a predetermined time after the terminal device 10 displays the question information on the screen. For example, the receiving unit 132 may receive, as the response information, a response text input by the user U1. Furthermore, when the receiving unit 132 receives the response information, the receiving unit 132 may output the response information to the generation unit 133.

[0036] Furthermore, the generation unit 133 acquires response information from the reception unit 132. When the generation unit 133 acquires the response information, the generation unit 133 generates a search query corresponding to the response information based on the response information. The generation unit 133 generates a search query corresponding to the response information. Specifically, the generation unit 133 inputs the response information to a machine learning model and causes the machine learning model to generate a search query. In FIG. 5, the generation unit 133 inputs the response information to the machine learning model M1 and causes the machine learning model M1 to generate a search query (step S16). More specifically, the generation unit 133 inputs, together with the response information, a prompt to the machine learning model M1 instructing the machine learning model M1 to generate a search query corresponding to the response information, and causes the machine learning model M1 to generate a search query corresponding to the response information. For example, the generation unit 133 may input response text input by the user U1 to the machine learning model M1 as the response information and cause the machine learning model M1 to generate search text corresponding to the response text as the search query. For example, the search text may be a sentence.

[0037] For example, the generation unit 133 may input, together with the response information, prompt P2, which includes a sentence such as "Is the information entered by the user sufficient to perform a search? Answer Yes or No," shown in the upper part of FIG. 7, to the machine learning model M1. Furthermore, when "Yes" is output from the machine learning model M1 in response to the input of prompt P2, the generation unit 133 may input prompt P4, which includes a sentence such as "Do I need to use a tool to perform a search? Answer Yes or No," shown in the lower part of FIG. 7, to the machine learning model M1. Furthermore, when "No" is output from the machine learning model M1 in response to the input of prompt P4, the generation unit 133 may input prompt P8, which includes a sentence such as "What information do you want to search for? Output a sentence or file name," shown in the middle part of FIG. 9, to the machine learning model M1. For example, the generation unit 133 may input prompt P8 to the machine learning model M1 as a prompt instructing the machine learning model M1 to generate a search query corresponding to the response information. Furthermore, the generation unit 133 may acquire, as a search query, a sentence or a file name output from the machine learning model M1 in response to input of the prompt P8. In this way, the generation unit 133 may cause the machine learning model M1 to generate a search query that is a sentence. In this way, the generation unit 133 may cause the machine learning model M1 to generate a search query that is a sentence. In this way, the generation unit 133 may generate a search query that is a sentence.

[0038] When the machine learning model M1 outputs "No" in response to the input of prompt P2, the generation unit 133 may input to the machine learning model M1 a prompt P3 including a sentence such as "Do you need to use a tool to ask the user again? Answer "Yes" or "No" as shown in the middle of FIG. 7. When the machine learning model M1 outputs "No" in response to the input of prompt P3, the generation unit 133 may input to the machine learning model M1 a prompt P7 including a sentence such as "What information should you ask the user again? Enter the wording to ask the user." as shown in the upper part of FIG. 9. The generation unit 133 may obtain the sentence output from the machine learning model M1 in response to the input of prompt P7 as new question information. When the generation unit 133 causes the machine learning model M1 to generate new question information, the generation unit 133 may output the new question information to the output control unit 134.

[0039] 5, when the generation unit 133 generates a search query according to the response information, the generation unit 133 inputs the generated search query to the search system 200 (step S17). For example, the generation unit 133 may transmit the generated search query to the search system 200. The generation unit 133 also acquires search results corresponding to the search query (step S18). For example, the generation unit 133 may acquire search results corresponding to the search query from the search system 200. The generation unit 133 may receive search results corresponding to the search query from the search system 200.

[0040] Furthermore, when the generation unit 133 acquires search results, it generates output information corresponding to the search results based on the search results. Specifically, the generation unit 133 inputs the search results to a machine learning model and causes the machine learning model to generate output information corresponding to the search results. In FIG. 5, the generation unit 133 inputs the search results to the machine learning model M1 and causes the machine learning model M1 to generate output information corresponding to the search results (step S19). More specifically, the generation unit 133 inputs, together with the search results, a prompt to the machine learning model M1 instructing the machine learning model M1 to generate output information corresponding to the search results, and causes the machine learning model M1 to generate the output information corresponding to the search results. For example, when the generation unit 133 acquires search results, it may input, together with the search results, a prompt P9 including a sentence such as "Do you need to use a tool to determine whether the search results are in line with the search intent? Answer with Yes or No.", as shown in the lower part of FIG. Furthermore, when the machine learning model M1 outputs "No" in response to the input of the prompt P9, the generation unit 133 may determine whether the similarity between the search query and the search results exceeds a predetermined threshold. For example, the generation unit 133 may determine whether the similarity between the search query, which is text, and the search results, which are text, is equal to or greater than a predetermined threshold. If the generation unit 133 determines that the similarity between the search query, which is text, and the search results, which are text, is equal to or greater than a predetermined threshold, the generation unit 133 may determine that the search results are in line with the search intent. Furthermore, if the generation unit 133 determines that the search results are in line with the search intent, the generation unit 133 may input a prompt P11, including a sentence shown in the lower part of FIG. 10 , to the machine learning model M1, which sentence reads, "To return the search results to the user, summarization, paraphrasing, etc. will be performed. Enter a reply to the user as the search results." FIG. 10 is a diagram illustrating an example of a prompt according to the embodiment. For example, the generation unit 133 may input the prompt P11 to the machine learning model M1 as a prompt instructing the machine learning model M1 to generate output information corresponding to the search results. The generation unit 133 may obtain, as output information, a sentence output from the machine learning model M1 in response to the input of the prompt P11. For example, the generation unit 133 may obtain, as output information, a sentence summarizing the search results, which are text. Furthermore, the generation unit 133 may obtain, as output information, a sentence that paraphrases the search result, which is a sentence.In this way, the generation unit 133 may cause the machine learning model M1 to generate output information that is a sentence. In this way, the generation unit 133 may generate output information that is a sentence.

[0041] Furthermore, if the generation unit 133 determines that the similarity between the search query (which is text) and the search results (which are text) is not equal to or greater than a predetermined threshold (is less than the predetermined threshold), the generation unit 133 may determine that the search results do not match the search intent. If the generation unit 133 determines that the search results do not match the search intent, the generation unit 133 may input a prompt P10, including a sentence such as "Modify the search information to make the search results appropriate. Do you need to use a tool? Answer Yes or No," shown in the upper part of FIG. 10, to the machine learning model M1. If the generation unit 133 outputs "No" in response to the input of the prompt P10, the generation unit 133 may generate a new search query. For example, the generation unit 133 may generate a search query different from the search query whose similarity to the search results is determined to be less than or equal to a predetermined threshold. For example, the generation unit 133 may again input a prompt P8, including a sentence such as "What information are you searching for? Output the sentence or file name," shown in the middle part of FIG. 9, to the machine learning model M1. Furthermore, the generation unit 133 may acquire, as a new search query, a sentence or a file name that is output again from the machine learning model M1 in response to the input of the prompt P8.

[0042] Furthermore, the output control unit 134 outputs output information based on the search results. In Fig. 5, the output control unit 134 may acquire the output information from the generation unit 133. When the output control unit 134 acquires the output information, it outputs the output information to the terminal device 10 of the user U1 (step S20). For example, the output control unit 134 may transmit the output information to the terminal device 10 of the user U1.

[0043] [4. Processing Procedure] FIG. 11 is a flowchart showing the procedure of information processing by the information processing device according to the embodiment. In FIG. 11, the instruction unit 131 of the information processing device 100 instructs the machine learning model to identify a search target desired by a user, generate a search query corresponding to the identified search target, and acquire search results corresponding to the generated search query (step S101). The reception unit 132 of the information processing device 100 receives input information input by a user who uses the search system (step S102). The generation unit 133 of the information processing device 100 inputs the input information to the machine learning model, causing the machine learning model to generate question information (step S103). The generation unit 133 also acquires response information (step S104). The generation unit 133 also inputs the response information to the machine learning model, causing the machine learning model to generate a search query (step S105). The generation unit 133 also inputs the search results to the machine learning model, causing the machine learning model to generate output information (step S106). Furthermore, the output control unit 134 of the information processing device 100 outputs the output information (step S107).

[0044] [5. Modifications] The processing according to the above-described embodiment may be implemented in various different forms other than the above embodiment.

[0045] In the above-described embodiment, the generation unit 133 generates a sentence. However, the information generated by the generation unit 133 is not limited to a sentence. For example, the generation unit 133 may generate an image in addition to a sentence. For example, the generation unit 133 may cause an image generation model M2, which generates an image corresponding to text from text, to generate an image. The generation unit 133 may generate an image by causing the image generation model M2 to generate an image. Specifically, the generation unit 133 causes the machine learning model M1 to generate a first input text, which is text to be input to the image generation model M2, which generates an image corresponding to the text from text, inputs the first input text to the image generation model M2, causes the image generation model M2 to generate a first generated image, which is an image corresponding to the first input text, and inputs the first generated image to the machine learning model M1 to generate question information. For example, the generation unit 133 may determine a tool to be used to generate question information, generate input information for the tool, input the generated input information to the tool, and input a prompt instructing the tool to generate question information based on output information from the tool. For example, the generation unit 133 may input a prompt instructing the user to determine a tool to be used to generate question information from among tools previously designated as available tools by the instruction unit 131. For example, the generation unit 133 may input a prompt instructing the user to determine a tool to be used to generate question information from among an image generation model and an image recognition model.

[0046] For example, the generation unit 133 may input, together with the input information, a prompt P2 including a sentence such as "Is the information entered by the user sufficient to perform a search? Answer Yes or No," shown in the upper part of FIG. 7, to the machine learning model M1. Furthermore, when "No" is output from the machine learning model M1 in response to the input of the prompt P2, the generation unit 133 may input, to the machine learning model M1, a prompt P3 including a sentence such as "Do you need to use a tool to ask the user again? Answer Yes or No," shown in the middle part of FIG. 7. Furthermore, when "Yes" is output from the machine learning model M1 in response to the input of the prompt P3, the generation unit 133 may input, to the machine learning model M1, a prompt P5 including a sentence such as "What tool should be used? Answer the tool name and input," shown in the upper part of FIG. 8. FIG. 8 is a diagram illustrating an example of a prompt according to the embodiment. For example, the generation unit 133 may determine the tool to be used to generate question information, generate input information for the tool, input the generated input information to the tool, and input prompt P5 to the machine learning model M1 as a prompt instructing the tool to generate question information based on the output information of the tool.

[0047] Furthermore, the generation unit 133 may acquire text capable of identifying the image generation model M2, which is a tool name output from the machine learning model M1 in response to the input of the prompt P5. Furthermore, the generation unit 133 may acquire a first input text output from the machine learning model M1 as an input for the image generation model M2 in response to the input of the prompt P5. Furthermore, when the generation unit 133 acquires the text capable of identifying the image generation model M2, the generation unit 133 may acquire the image generation model M2 by referring to the storage unit 120 based on the text capable of identifying the image generation model M2. Furthermore, the generation unit 133 may input the first input text to the image generation model M2 and cause the image generation model M2 to generate a first generated image that is an image corresponding to the first input text.

[0048] Furthermore, the generation unit 133 may input, together with the first generated image, a prompt P6 including a sentence such as "Do you need to use any more tools? Answer Yes or No." shown in the lower part of FIG. 8 to the machine learning model M1. Furthermore, when "No" is output from the machine learning model M1 in response to the input of the prompt P6, the generation unit 133 may input, together with the first generated image, a prompt P7 including a sentence such as "What information should you ask the user again? Enter the wording to ask the user." shown in the upper part of FIG. 9 to the machine learning model M1. Furthermore, the generation unit 133 may obtain, as question information, a question sentence output from the machine learning model M1 in response to the input of the first generated image and the prompt P7. In this way, the generation unit 133 may generate question information based on the image generated by the image generation model M2. Furthermore, the output control unit 134 may output the question sentence along with the first generated image.

[0049] FIG. 12 is a diagram illustrating an example of information processing according to a modified example. In FIG. 12, a user U1 has a search intent to find a store that accepts an unusual dog that he saw on television. However, because the user U1 does not know the breed of the unusual dog that he saw on television, he inputs an ambiguous search query, "a store that accepts unusual dogs," to the terminal device 10 of the user U1. The terminal device 10 transmits the ambiguous search query, "a store that accepts unusual dogs," to the information processing device 100. The reception unit 132 of the information processing device 100 accepts the ambiguous search query, "a store that accepts unusual dogs," as input information. Furthermore, the generation unit 133 of the information processing device 100 inputs the ambiguous search query, "a store that accepts unusual dogs," to a machine learning model M1, which is a large-scale language model, and causes the machine learning model M1 to generate text, "image of XX and XX," to be input to an image generation model M2. Here, XX and XX are the names of the breeds of the unusual dog. The generation unit 133 also inputs the text "Image of XX and XX" into the image generation model M2, causing the image generation model M2 to generate an image corresponding to XX and an image corresponding to XX. The generation unit 133 also inputs the image corresponding to XX and the image corresponding to XX generated by the image generation model M2 into the machine learning model M1, causing the machine learning model M1 to generate a question sentence, "Is it the dog in either image?" The output control unit 134 also outputs the question sentence, "Is it the dog in either image?" to the terminal device 10, along with the image corresponding to XX and the image corresponding to XX generated by the image generation model M2. The terminal device 10 also transmits a response sentence, "The image on the right!", input by the user U1 to the information processing device 100. Here, the image on the right corresponds to the image of XX. The generation unit 133 also inputs the response sentence, "The image on the right!" into the machine learning model M1, causing the machine learning model M1 to generate a search query, "Pet shops where I can buy XX." Furthermore, the generation unit 133 inputs a search query "Pet shops where you can buy XX" into the search system 200 and obtains a list of pet shops where you can buy a dog of a certain breed as a search result.Furthermore, the generation unit 133 inputs the store list information acquired as the search result into the machine learning model M1, and causes the machine learning model M1 to generate output information such as "Store ZZ in YY looks good!". The output control unit 134 of the information processing device 100 outputs the output information such as "Store ZZ in YY looks good!" to the terminal device 10.

[0050] Furthermore, in the above-described modified example, the generation unit 133 generates question information based on an image generated by the image generation model M2. However, the generation unit 133 may generate a search query based on an image generated by the image generation model M2. Specifically, the generation unit 133 causes a machine learning model to generate a second input text, which is text to be input to an image generation model that generates an image corresponding to the text from the text, inputs the second input text to the image generation model, causes the image generation model to generate a second generated image, which is an image corresponding to the second input text, and inputs the second generated image to the machine learning model to generate a search query. For example, the generation unit 133 may determine a tool to be used to generate a search query, generate input information for the tool, input the generated input information to the tool, and input a prompt instructing the machine learning model to generate a search query based on output information from the tool. For example, the generation unit 133 may input a prompt instructing the machine learning model to determine a tool to be used to generate a search query from among tools previously designated as available tools by the instruction unit 131. For example, the generation unit 133 may input a prompt to instruct the user to determine which tool should be used to generate a search query from among an image generation model and an image recognition model.

[0051] For example, the generation unit 133 may input, together with the response information, prompt P2, which includes a sentence such as "Is the information entered by the user sufficient to perform a search? Answer Yes or No," shown in the upper part of FIG. 7, to the machine learning model M1. Furthermore, when "Yes" is output from the machine learning model M1 in response to the input of prompt P2, the generation unit 133 may input prompt P4, which includes a sentence such as "Do I need to use a tool to perform a search? Answer Yes or No," shown in the lower part of FIG. 7, to the machine learning model M1. Furthermore, when "Yes" is output from the machine learning model M1 in response to the input of prompt P4, the generation unit 133 may input prompt P5, which includes a sentence such as "What tool should be used? Answer the tool name and input," shown in FIG. 8, to the machine learning model M1. For example, the generation unit 133 may determine a tool to be used to generate a search query, generate input information for the tool, input the generated input information into the tool, and input prompt P5 to the machine learning model M1 as a prompt instructing the machine learning model M1 to generate a search query based on the output information of the tool. For example, the generation unit 133 may input a prompt P5 to the machine learning model M1 as a prompt instructing the model to generate a second input text, which is a text to be input to the image generation model M2.

[0052] Furthermore, the generation unit 133 may acquire text capable of identifying the image generation model M2, which is a tool name output from the machine learning model M1 in response to the input of the prompt P5. Furthermore, the generation unit 133 may acquire second input text output from the machine learning model M1 as input to the image generation model M2 in response to the input of the prompt P5. Furthermore, when the generation unit 133 acquires text capable of identifying the image generation model M2, the generation unit 133 may acquire the image generation model M2 by referring to the storage unit 120 based on the text capable of identifying the image generation model M2. Furthermore, the generation unit 133 may input the second input text to the image generation model M2 and cause the image generation model M2 to generate a second generated image that is an image corresponding to the second input text.

[0053] The generation unit 133 may input a prompt P6 including a sentence such as "Do you need to use any other tools? Answer Yes or No," shown in the lower part of FIG. 8, together with the second generated image to the machine learning model M1. If the machine learning model M1 outputs "No" in response to the input of the prompt P6, the generation unit 133 may input a prompt P8 including a sentence such as "What information are you searching for? Output a sentence or a file name," shown in the middle part of FIG. 9, together with the second generated image to the machine learning model M1. The generation unit 133 may obtain, as a search query, a search sentence or a file name (e.g., a file name corresponding to the second generated image) output from the machine learning model M1 in response to the input of the second generated image and the prompt P8. In this way, the generation unit 133 may generate a search query based on an image generated by the image generation model M2. The generation unit 133 may input the search sentence or the file name to the search system 200.

[0054] In the above-described embodiment, the generation unit 133 inputs input text entered by the user U1 as input information into the machine learning model M1 to generate question information. However, the generation unit 133 may input information other than text as input information into the machine learning model M1. For example, the generation unit 133 may input an image as input information into the machine learning model M1. Specifically, the generation unit 133 inputs an input image, which is an image included in the input information, to an image recognition model M3 that generates a sentence explaining the content of the image from the image, causes the image recognition model M3 to generate an input sentence, which is a sentence corresponding to the input image, and inputs the input sentence into the machine learning model M1 to cause the machine learning model M1 to generate question information. For example, the image recognition model M3 may be a visual language model (VLM). For example, the generation unit 133 may determine a tool to be used to generate question information, generate input information for the tool, input the generated input information into the tool, and input a prompt instructing the tool to generate question information based on output information from the tool.

[0055] For example, the receiving unit 132 may receive an input image input by the user U1 as the input information. For example, the receiving unit 132 may receive an input image and input text as the input information. The receiving unit 132 may receive an input image, which is an image included in the input information. When the receiving unit 132 receives the input information, the receiving unit 132 may output the input information to the generation unit 133. The generation unit 133 may input, together with the input image, a prompt P2 including a sentence such as "Is the information entered by the user sufficient to perform a search? Answer Yes or No," shown in the upper part of FIG. 7, to the machine learning model M1. When the machine learning model M1 outputs "No" in response to the input of the prompt P2, the generation unit 133 may input, to the machine learning model M1, a prompt P3 including a sentence such as "Do you need to use a tool to ask the user again? Answer Yes or No," shown in the middle part of FIG. 7. Furthermore, when "Yes" is output from the machine learning model M1 in response to the input of prompt P3, the generation unit 133 may input to the machine learning model M1 a prompt P5 including a sentence such as "What tool should be used? Please answer with the tool name and input," as shown in Fig. 8. For example, the generation unit 133 may input the prompt P5 to the machine learning model M1 as a prompt that instructs the machine learning model M1 to determine the tool to be used to generate question information, generate input information for the tool, input the generated input information to the tool, and generate question information based on the output information of the tool.

[0056] Furthermore, the generation unit 133 may acquire text that can identify the image recognition model M3, which is the tool name output from the machine learning model M1 in response to the input of the prompt P5. Furthermore, the generation unit 133 may acquire the file name of the input image that was output from the machine learning model M1 as the input of the image recognition model M3 in response to the input of the prompt P5. Furthermore, when the generation unit 133 acquires the text that can identify the image recognition model M3, it may acquire the image recognition model M3 by referring to the storage unit 120 based on the text that can identify the image recognition model M3. Furthermore, when the generation unit 133 acquires the file name of the input image, it may acquire the input image by referring to the storage unit 120 based on the file name of the input image. Furthermore, the generation unit 133 may input the input image to the image recognition model M3 and cause the image recognition model M3 to generate an input sentence that is a sentence corresponding to the input image.

[0057] Furthermore, the generation unit 133 may input, together with the input sentence, a prompt P6 including a sentence with the content "Do you need to use any more tools? Answer Yes or No." shown in the lower part of FIG. 8 to the machine learning model M1. Furthermore, when "No" is output from the machine learning model M1 in response to the input of the prompt P6, the generation unit 133 may input, together with the input sentence, a prompt P7 including a sentence with the content "What information should you ask the user again? Enter the wording to ask the user." shown in the upper part of FIG. 9 to the machine learning model M1. Furthermore, the generation unit 133 may obtain, as question information, the question sentence output from the machine learning model M1 in response to the input of the input sentence and the prompt P7. In this way, the generation unit 133 may generate question information based on the input sentence generated by the image recognition model M3.

[0058] Furthermore, in the above-described modified example, the generation unit 133 generates question information based on a sentence generated by the image recognition model M3. However, the generation unit 133 may generate a search query based on a sentence generated by the image recognition model M3. Specifically, the generation unit 133 inputs a response image, which is an image included in response information, to the image recognition model M3, which generates a sentence explaining the content of an image from an image, causes the image recognition model M3 to generate a response sentence, which is a sentence corresponding to the response image, and inputs the response sentence to a machine learning model to cause the machine learning model to generate a search query. For example, the generation unit 133 may determine a tool to be used to generate a search query, generate input information for the tool, input the generated input information to the tool, and input a prompt instructing the tool to generate a search query based on output information from the tool.

[0059] For example, the receiving unit 132 may receive a response image input by the user U1 as the response information. For example, the receiving unit 132 may receive a response image and response text as the response information. The receiving unit 132 may receive a response image that is an image included in the response information. When the receiving unit 132 receives the response information, the receiving unit 132 may output the response information to the generation unit 133. When the receiving unit 132 receives the response information, the generation unit 133 may input, together with the response image, a prompt P2 including a sentence such as "Is the information input by the user sufficient to perform a search? Answer Yes or No," shown in the upper part of FIG. 7, to the machine learning model M1. When the machine learning model M1 outputs "Yes" in response to the input of the prompt P2, the generation unit 133 may input, to the machine learning model M1, a prompt P4 including a sentence such as "Do I need to use a tool to perform a search? Answer Yes or No," shown in the lower part of FIG. 7. Furthermore, when "Yes" is output from the machine learning model M1 in response to the input of prompt P4, the generation unit 133 may input to the machine learning model M1 a prompt P5 including a sentence such as "What tool should be used? Please answer with the tool name and input," as shown in Fig. 8. For example, the generation unit 133 may input the prompt P5 to the machine learning model M1 as a prompt that instructs the machine learning model M1 to determine a tool to be used to generate a search query, generate input information for the tool, input the generated input information to the tool, and generate a search query based on the output information of the tool.

[0060] Furthermore, the generation unit 133 may acquire text that can identify the image recognition model M3, which is the tool name output from the machine learning model M1 in response to the input of the prompt P5. Furthermore, the generation unit 133 may acquire the file name of the response image that was output from the machine learning model M1 as the input of the image recognition model M3 in response to the input of the prompt P5. Furthermore, when the generation unit 133 acquires the text that can identify the image recognition model M3, the generation unit 133 may acquire the image recognition model M3 by referring to the storage unit 120 based on the text that can identify the image recognition model M3. Furthermore, when the generation unit 133 acquires the file name of the response image, the generation unit 133 may acquire the response image by referring to the storage unit 120 based on the file name of the response image. Furthermore, the generation unit 133 may input the response image to the image recognition model M3 and cause the image recognition model M3 to generate a response sentence that is a sentence corresponding to the response image.

[0061] The generation unit 133 may input, together with the response sentence, a prompt P6 including a sentence such as "Do you need to use any other tools? Answer Yes or No," shown in the lower part of FIG. 8, to the machine learning model M1. When the machine learning model M1 outputs "No" in response to the input of the prompt P6, the generation unit 133 may input, together with the response sentence, a prompt P8 including a sentence such as "What information are you searching for? Output a sentence or file name," shown in the middle part of FIG. 9, to the machine learning model M1. The generation unit 133 may obtain, as a search query, a search sentence or file name (e.g., a file name corresponding to a response image) output from the machine learning model M1 in response to the input of the response sentence and the prompt P8. In this way, the generation unit 133 may generate a search query based on the response sentence generated by the image recognition model M3.

[0062] Furthermore, similarly to the image recognition model M3, the generation unit 133 may generate question information based on recognition information generated by a speech recognition model M4 that generates recognition information indicating a recognition result of speech data from speech data. For example, the generation unit 133 may generate question information based on text generated by a speech recognition model M4 that generates recognition information, which is text indicating the content of the speech data, from speech data. For example, the generation unit 133 may input input speech data, which is speech data included in input information, to the speech recognition model M4 that generates recognition information indicating a recognition result of speech data from speech data, to cause the speech recognition model M4 to generate input recognition information, which is recognition information corresponding to the input speech data, and input the input recognition information to the machine learning model M1 to cause the machine learning model M1 to generate question information. For example, the generation unit 133 may input input speech data to the speech recognition model M4 that generates recognition information, which is text indicating the content of the speech data from speech data, to cause the speech recognition model M4 to generate input speech text, which is text corresponding to the input speech data, and input the input speech text to the machine learning model M1 to cause the machine learning model M1 to generate question information.

[0063] Furthermore, similar to the case of the image recognition model M3, the generation unit 133 may generate a search query based on recognition information generated by a speech recognition model M4 that generates recognition information indicating a recognition result of speech data from speech data. For example, the generation unit 133 may generate a search query based on text generated by a speech recognition model M4 that generates recognition information, which is text indicating the content of the speech data, from speech data. The generation unit 133 inputs response speech data, which is speech data included in response information, to the speech recognition model M4 that generates recognition information indicating a recognition result of speech data from speech data, causing the speech recognition model M4 to generate response recognition information, which is recognition information corresponding to the response speech data, and inputs the response recognition information to the machine learning model M1 to generate a search query. For example, the generation unit 133 may input the response speech data to the speech recognition model M4 that generates recognition information, which is text indicating the content of the speech data, from speech data, causing the speech recognition model M4 to generate response speech text, which is text corresponding to the response speech data, and input the response speech text to the machine learning model M1 to generate a search query.

[0064] Furthermore, in the above-described embodiment, the generation unit 133 determines whether the similarity between a search query, which is text, and a search result, which is also text, is equal to or greater than a predetermined threshold. However, the generation unit 133 may determine whether the similarity between a search query and a search result, which are of different modalities, is equal to or greater than a predetermined threshold. Here, the case where the search query and the search result are of different modalities includes a case where either the search query or the search result is text, and the other is a modal other than text (e.g., image or audio data). Specifically, the storage unit 120 may store information about the multimodal model M5, which is a machine learning model that determines the similarity between text and a modal other than text. For example, when the machine learning model M1 outputs "Yes" in response to the input of the prompt P9, the generation unit 133 may input to the machine learning model M1 a prompt P5 including a sentence such as "What tool should you use? Please answer with the tool name and input," as shown in FIG. 8.

[0065] The generation unit 133 may also acquire text that can identify the multimodal model M5, which is the tool name output from the machine learning model M1 in response to the input of the prompt P5. The generation unit 133 may also acquire a search query and a file name of a search result output from the machine learning model M1 as input to the multimodal model M5 in response to the input of the prompt P5. When the generation unit 133 acquires text that can identify the multimodal model M5, the generation unit 133 may acquire the multimodal model M5 by referring to the storage unit 120 based on the text that can identify the multimodal model M5. When the generation unit 133 acquires the search query and the file name of the search result, the generation unit 133 may input the search query and the search result to the multimodal model M5 and determine the similarity between the search query and the search result. The generation unit 133 may also determine whether the similarity between the search query and the search result is equal to or greater than a predetermined threshold.

[0066] Furthermore, in the above-described embodiment, the machine learning model M1 is a large-scale language model. However, the machine learning model M1 may be a language model trained to estimate and output the next token from an input image and token sequence. For example, the machine learning model M1 may be a visual language model (VLM). For example, when the generation unit 133 receives an input image as input information, the generation unit 133 can input the input image to the machine learning model M1, which is a visual language model, without using the image recognition model M3, to generate question information corresponding to the input image. Furthermore, when the generation unit 133 receives a response image as response information, the generation unit 133 can input the response image to the machine learning model M1 without using the image recognition model M3 to generate a search query corresponding to the response information. For example, the machine learning model M1 may be CoCa (Contrastive Captioners are Image-Text Foundation Models), BLIP (Bootstrapping Language-Image Pre-training), BLIP2, GIT (Generative Image to Text Transformer), or the like.

[0067] [6. Effects] As described above, the information processing device 100 according to the embodiment includes a receiving unit 132, a generating unit 133, and an output control unit 134. The receiving unit 132 receives input information input by a user who uses the search system. The generating unit 133 inputs the input information to a machine learning model, causes the machine learning model to generate question information indicating a question for identifying a search target desired by the user, acquires response information indicating a response to the question information, generates a search query according to the response information, and generates output information according to search results corresponding to the search query. The output control unit 134 outputs the output information.

[0068] In this way, the information processing device 100 generates question information according to input information entered by a user using the search system and outputs the generated question information, thereby enabling the user to complement the ambiguity of the search target desired by the user. Furthermore, the information processing device 100 acquires response information according to the question information and generates a search query according to the response information, thereby generating a search query after complementing the ambiguity of the search target desired by the user. In this way, the information processing device 100 can provide the user with search results after complementing the ambiguity of the search target desired by the user. Furthermore, since the information processing device 100 can provide the user with search results after complementing the ambiguity of the search target desired by the user, it can contribute to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization, and build resilient technological infrastructure."

[0069] Furthermore, the output control unit 134 outputs question information. The receiving unit 132 receives response information input by a user. The generating unit 133 inputs the response information to a machine learning model and causes the machine learning model to generate a search query.

[0070] This allows the information processing device 100 to generate a search query that complements the ambiguity of the search target desired by the user, based on the response information input by the user.

[0071] Furthermore, the generation unit 133 inputs the search results into the machine learning model, causing the machine learning model to generate output information.

[0072] This allows the information processing device 100 to provide the user with search results that complement the ambiguity of the search target desired by the user.

[0073] Furthermore, the generation unit 133 causes a machine learning model to generate a first input text, which is text to be input to an image generation model that generates an image corresponding to the text from the text, inputs the first input text to the image generation model, causes the image generation model to generate a first generated image, which is an image corresponding to the first input text, and inputs the first generated image to the machine learning model, causing the machine learning model to generate question information.

[0074] As a result, the information processing device 100 can, for example, generate an image according to input information entered by a user and generate question information based on the generated image, thereby making it possible to appropriately complement the ambiguity of the search target desired by the user.

[0075] Furthermore, the generation unit 133 causes the machine learning model to generate a second input text, which is text to be input to an image generation model that generates an image corresponding to the text from the text, inputs the second input text to the image generation model, causes the image generation model to generate a second generated image, which is an image corresponding to the second input text, inputs the second generated image to the machine learning model, and causes the machine learning model to generate a search query.

[0076] This allows the information processing device 100 to, for example, generate an image according to response information input by a user and generate a search query based on the generated image, thereby making it possible to appropriately complement the ambiguity of the search target desired by the user.

[0077] In addition, the generation unit 133 inputs an input image, which is an image included in the input information, into an image recognition model that generates a sentence that explains the content of the image from the image, causes the image recognition model to generate an input sentence, which is a sentence that corresponds to the input image, and inputs the input sentence into a machine learning model to cause the machine learning model to generate question information.

[0078] This allows the information processing device 100 to recognize the content of an image input by a user and generate appropriate question information, for example.

[0079] In addition, the generation unit 133 inputs a response image, which is an image included in the response information, into an image recognition model that generates a sentence that explains the content of the image from the image, causes the image recognition model to generate a response sentence, which is a sentence that corresponds to the response image, and inputs the response sentence into a machine learning model to cause the machine learning model to generate a search query.

[0080] This allows the information processing device 100 to, for example, recognize the content of an image input by a user and generate an appropriate search query.

[0081] The machine learning model may be a large language model (LLM) or a visual language model (VLM).

[0082] This allows the information processing device 100 to generate appropriate question information and search queries by using a large-scale language model or a visual language model.

[0083] The information processing device 100 further includes an instruction unit 131. The instruction unit 131 instructs the machine learning model to identify a search target desired by a user, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query.

[0084] As a result, the information processing device 100 can assign the machine learning model the role of identifying the search target desired by the user, generating a search query corresponding to the identified search target, and obtaining search results corresponding to the generated search query.

[0085] [7. Hardware Configuration] The information processing device 100 according to the embodiment described above is realized by, for example, a computer 1000 configured as shown in Fig. 13. Fig. 13 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 100. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0086] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, programs that depend on the hardware of the computer 1000, and the like.

[0087] The HDD 1400 stores programs executed by the CPU 1100, data used by such programs, etc. The communication interface 1500 receives data from other devices via a predetermined communication network and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network.

[0088] The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input / output interface 1600.

[0089] Media interface 1700 reads a program or data stored in recording medium 1800 and provides it to CPU 1100 via RAM 1200. CPU 1100 loads the program or data from recording medium 1800 onto RAM 1200 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0090] For example, when the computer 1000 functions as the information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control unit 130. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a predetermined communication network.

[0091] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that include the embodiments described in the Disclosure of the Invention section and that have been modified and improved in various ways based on the knowledge of those skilled in the art.

[0092] [8. Other] Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0093] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0094] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content. [Explanation of symbols]

[0095] 100 Information processing device 110 Communications Department 120 Storage section 130 control section 131 Instruction section 132 Reception Department 133 Generation part 134 Output control section

Claims

1. a reception unit that receives input information input by a user who uses the search system; a generation unit that inputs the input information into a machine learning model, causes the machine learning model to generate question information indicating a question for identifying a search target desired by the user, acquires response information indicating a response to the question information, generates a search query according to the response information, and generates output information according to search results corresponding to the search query; an output control unit that outputs the output information; Equipped with The generation unit determining whether a similarity between the search query and the search results is equal to or greater than a predetermined threshold, and if it is determined that the similarity is not equal to or greater than the predetermined threshold, generating a new search query that is different from the search query; Information processing device.

2. a reception unit that receives input information input by a user who uses the search system; a generation unit that inputs the input information into a machine learning model, causes the machine learning model to generate question information indicating a question for identifying a search target desired by the user, acquires response information indicating a response to the question information, generates a search query according to the response information, and generates output information according to search results corresponding to the search query; an output control unit that outputs the output information; Equipped with The generation unit a prompt that instructs the machine learning model to determine a tool to be used to generate the question information or the search query from among tools that are presented in advance as available tools, the prompt instructing the machine learning model to generate input information for the tool, input the generated input information for the tool to the tool, and generate the question information or the search query based on output information from the tool; Information processing device.

3. The output control unit outputting the question information; The reception unit receiving the response information input by the user; The generation unit inputting the response information into the machine learning model to generate the search query; 3. The information processing device according to claim 1.

4. The generation unit inputting the search results into the machine learning model and causing the machine learning model to generate the output information; 3. The information processing device according to claim 1.

5. The generation unit causing the machine learning model to generate a first input text, which is text to be input to an image generation model that generates an image corresponding to the text from the text; inputting the first input text into the image generation model to generate a first generated image, which is an image corresponding to the first input text, from the image generation model; inputting the first generated image into the machine learning model to generate the question information from the machine learning model; 3. The information processing device according to claim 1.

6. The generation unit causing the machine learning model to generate a second input text, the second input text being a text to be input to an image generation model that generates an image corresponding to the text from the text; inputting the second input text into the image generation model to generate a second generated image, the second generated image being an image corresponding to the second input text, and inputting the second generated image into the machine learning model to generate the search query; 3. The information processing device according to claim 1.

7. The generation unit an input image that is an image included in the input information is input to an image recognition model that generates a sentence that explains the content of the image from the image, causing the image recognition model to generate an input sentence that is a sentence corresponding to the input image, and the input sentence is input to the machine learning model to cause the machine learning model to generate the question information; 3. The information processing device according to claim 1.

8. The generation unit a response image that is an image included in the response information is input to an image recognition model that generates a sentence that explains the content of the image from the image, causing the image recognition model to generate a response sentence that is a sentence corresponding to the response image, and inputting the response sentence to the machine learning model to generate the search query; 3. The information processing device according to claim 1.

9. The machine learning model is a large language model (LLM) or a visual language model (VLM), 3. The information processing device according to claim 1.

10. an instruction unit that instructs the machine learning model to identify a search target desired by the user, generate a search query corresponding to the identified search target, and obtain search results corresponding to the generated search query; 3. The information processing device according to claim 1.

11. a receiving procedure for receiving input information input by a user who uses the search system; a generation step of inputting the input information into a machine learning model, causing the machine learning model to generate question information indicating a question for identifying a search target desired by the user, obtaining response information indicating a response to the question information, generating a search query according to the response information, and generating output information according to search results corresponding to the search query; an output control procedure for outputting the output information; on the computer, The generating procedure includes: determining whether a similarity between the search query and the search results is equal to or greater than a predetermined threshold, and if it is determined that the similarity is not equal to or greater than the predetermined threshold, generating a new search query that is different from the search query; Information processing program.

12. a receiving procedure for receiving input information input by a user who uses the search system; a generation step of inputting the input information into a machine learning model, causing the machine learning model to generate question information indicating a question for identifying a search target desired by the user, obtaining response information indicating a response to the question information, generating a search query according to the response information, and generating output information according to search results corresponding to the search query; an output control procedure for outputting the output information; on the computer, The generating procedure includes: a prompt that instructs the machine learning model to determine a tool to be used to generate the question information or the search query from among tools that are presented in advance as available tools, the prompt instructing the machine learning model to generate input information for the tool, input the generated input information for the tool to the tool, and generate the question information or the search query based on output information from the tool; Information processing program.

Images