Program, information processing method, and information processing apparatus.

Abstract

To provide a program or similar tool that can obtain answers to user queries using a language generation model. [Solution] The program for one aspect obtains a user query about the device, identifies a corresponding scene from multiple scenes related to the device based on the obtained query, identifies a base prompt corresponding to the identified scene, and causes the computer to execute a process to output an answer using a language model that uses the identified base prompt.

Description

TECHNICAL FIELD

[0001] The present invention relates to a program, an information processing method and an information processing apparatus. BACKGROUND ART

[0002] In recent years, development of technologies related to troubles occurring in devices and trouble countermeasures has been actively promoted. For example, Patent Document 1 discloses an information processing system that displays information of an error (trouble) occurring in a device in association with a solution method (trouble countermeasure). PRIOR ART DOCUMENTS PATENT DOCUMENTS

[0003] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2023-091722 SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] However, the invention according to Patent Document 1 has a problem in that an answer to a user's query cannot be obtained by a language generation model.

[0005] In one aspect, an object of the present invention is to provide a program or the like that enables obtaining an answer to a user's query by a language generation model. MEANS FOR SOLVING THE PROBLEMS

[0006] A program according to one aspect causes a computer to execute processing of: acquiring a user's query about a device; identifying a corresponding scene from a plurality of scenes related to the device based on the acquired query; identifying a base prompt corresponding to the identified scene; and outputting an answer by a language model that uses the identified base prompt. EFFECTS OF THE INVENTION

[0007] In one respect, language generation models make it possible to obtain answers to user queries. [Brief explanation of the drawing]

[0008] [Figure 1] This is an explanatory diagram illustrating the overview of the automated response system. [Figure 2] This is a block diagram showing an example server configuration. [Figure 3] This is an explanatory diagram showing an example of the record layout for the Scene DB and Base Prompt DB. [Figure 4] This is an explanatory diagram showing an example of the record layout for the equipment information database and the knowledge database. [Figure 5] This is a block diagram showing an example of a terminal configuration. [Figure 6] This is a block diagram showing an example of equipment configuration. [Figure 7] This is an explanatory diagram showing an example of a log database record layout. [Figure 8] This is an explanatory diagram showing the output process for user queries regarding equipment. [Figure 9] This flowchart shows the processing steps involved in outputting responses to queries using a language model. [Figure 10] This is a flowchart showing the processing steps of the subroutine that generates a new prompt. [Figure 11] This is a flowchart showing the processing steps of the subroutine that generates the final prompt. [Modes for carrying out the invention]

[0009] The present invention will be described in detail below with reference to the drawings illustrating its embodiments.

[0010] (Embodiment 1) Embodiment 1 relates to a configuration that uses a language model to output answers to user queries regarding a device.

[0011] Conventionally, when using a language model, a user (operator) manually creates a prompt (such as a question or instruction regarding a problem, for example), or uses an example sentence (template) of a prompt registered in advance in the system. However, with prompts created by users, it is difficult to comprehensively convey necessary information. As a result, omissions are prone to occur, and instructions may become unclear. Furthermore, when such incomplete prompts are input to a language model, the generated response (for example, troubleshooting measures) does not align with the user's intention, leading to the problem of "off-target responses".

[0012] In order to solve such problems, the present embodiment automatically generates more appropriate prompts by using a scene based on a user's query and a base prompt corresponding to the scene. By inputting the generated prompt to a language model, it becomes possible to obtain highly accurate responses.

[0013] A prompt is an instruction or input sentence created in a format understandable by a language model and provided as an input to the language model. The language model interprets the input prompt and outputs an appropriate response (for example, an answer to the user's query).

[0014] As an example, to convert the input into a processable format for the language model, the prompt is split into tokens. The language model performs context understanding processing by calculating the association between each token in the prompt and other tokens.

[0015] The language model performs response generation processing on the prompt based on linguistic knowledge obtained through pre-training, fine-tuning, and other processes. For example, the language model uses generation methods such as greedy decoding, beam search, or sampling to select optimal tokens. The language model performs decoding processing on the selected tokens to convert them back into text format and generate output data.

[0016] FIG. 1 is an explanatory diagram showing an outline of an automatic answering system. The system of the present embodiment includes an information processing apparatus 1, an information processing terminal 2, and a device 3, each of which transmits and receives information via a network N such as the Internet. Note that the device 3 does not need to be connected to the network N.

[0017] The information processing apparatus 1 is an information processing apparatus that performs processing, storage, transmission and reception for various types of information. The information processing apparatus 1 is, for example, a server apparatus or a personal computer. In the present embodiment, for the sake of brevity, the information processing apparatus 1 is hereinafter referred to as the server 1.

[0018] The information processing terminal 2 is a terminal device that receives and transmits a user's query related to the device 3, and receives and displays an answer to the query. The information processing terminal 2 is an information processing device such as, for example, a smartphone, a mobile phone, a wearable device such as a smart watch, a tablet, or a personal computer terminal. Hereinafter, for the sake of brevity, the information processing terminal 2 is referred to as the terminal 2.

[0019] The device 3 is an apparatus designed to achieve a specific object, and includes, for example, manufacturing equipment or industrial equipment constituting a production line in the manufacturing industry (die casting, injection molding machines, etc.), mechanical devices (machine tools, iron cutting equipment, cutting machines, etc.), electrical equipment, medical equipment (X-ray machines, ultrasonic devices, MRI scanners, electrocardiographs, etc.), power generation devices, semiconductor manufacturing apparatuses, substrate processing apparatuses, office equipment (copiers, printers, scanners, fax machines, etc.), transportation equipment (automobiles, railways, aircraft, etc.), or bank ATMs, etc.

[0020] Note that the device 3 may be an apparatus used in fields such as medical care (cancer diagnosis, obesity risk assessment, treatment support, etc.), finance (data analysis, risk management, etc.), human resources (worker health management, work efficiency improvement, etc.), marketing (consumer behavior analysis, advertisement optimization, etc.), chemistry (new material development, process control, etc.), or logistics (automated warehouses, inventory management, etc.).

[0021] Alternatively, device 3 may be a common household appliance used daily, such as a vacuum cleaner, fan, television, microwave oven, refrigerator, coffee maker, or blender. Furthermore, device 3 may be a small device such as a clock, electric toothbrush, camera, flashlight, electric can opener, pedometer, thermometer, hygrometer, or blood pressure monitor.

[0022] In this embodiment, we describe an example where equipment 3 is an injection molding machine for manufacturing an object to be molded, but the same principles can be applied to other types of equipment.

[0023] In this embodiment, Server 1 obtains user queries regarding Device 3 from Terminal 2. Based on the obtained queries, Server 1 identifies a corresponding scene from among multiple scenes related to Device 3. Server 1 identifies a base prompt corresponding to the identified scene. Server 1 outputs a response using a language model that utilizes the identified base prompt. The language model will be described later.

[0024] Figure 2 is a block diagram showing an example configuration of Server 1. Server 1 includes a control unit 11, a storage unit 12, a communication unit 13, a reading unit 14, and a large-capacity storage unit 15. Each component is connected by bus B.

[0025] The control unit 11 includes an arithmetic processing unit such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), or quantum processor. The control unit 11 performs various information processing or control processing related to the server 1 by reading and executing the control program 1P (program product) stored in the storage unit 12. The control program 1P described in this embodiment may be provided on a recording medium or distributed from an external computer.

[0026] Furthermore, the control program 1P can be deployed on a single computer, at a single site, or distributed across multiple sites and interconnected by a communication network to run on multiple computers.

[0027] In Figure 2, the control unit 11 is described as a single processor, but it may be a multi-processor system. Furthermore, the control unit 11 may perform various information processing or control processing on the same processor within the server 1, or it may perform these processes on different processors within the server 1.

[0028] The storage unit 12 includes memory elements such as RAM (Random Access Memory) and ROM (Read Only Memory), and stores control programs 1P or data necessary for the control unit 11 to execute processing. The storage unit 12 also temporarily stores data necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing, and transmits and receives information with terminals 2, etc., via the network N.

[0029] The reading unit 14 reads a portable storage medium 1a, including a CD (Compact Disc)-ROM or DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 14 and store it in the large-capacity storage unit 15. Alternatively, the control unit 11 may download the control program 1P from another computer via a network N or the like and store it in the large-capacity storage unit 15. Furthermore, the control unit 11 may also read the control program 1P from the semiconductor memory 1b.

[0030] The large-capacity storage unit 15 includes a recording medium such as an HDD (Hard disk drive) or an SSD (Solid State Drive). The large-capacity storage unit 15 includes a language model 151, a scene database 152, a base prompt database 153, a device information database 154, and a knowledge database 155.

[0031] The language model 151 is an output device that outputs answers (responses) to user queries regarding device 3, and is a trained model generated by machine learning. The language model 151 is used as a program module that is part of the artificial intelligence software.

[0032] Language model 151 is a language generation model constructed by pre-training on a large amount of text data (dataset). For language model 151, large-scale language models (LLMs) such as Transformer, ALBERT (A Lite BERT), GPT (Generative Pre-trained Transformer)-2, GPT-3, GPT-4, LLaVA (Large Language and Vision Assistant), MiniGPT-4, or BERT (Bidirectional Encoder Representations from Transformers) can be used.

[0033] Alternatively, instead of storing the language model 151 in the large-capacity storage unit 15, the server 1 may access and read it from an external language processing server or language processing platform.

[0034] Scene DB152 stores information about multiple scenes related to device 3. Base prompt DB153 stores example prompts (templates) corresponding to each scene. Scenes and base prompts will be described later. Device information DB154 stores information about device 3. Knowledge DB155 stores product information (specifications or instruction manuals, etc.), troubleshooting information (error codes, trouble symptoms or countermeasures, etc.), or manuals, etc.

[0035] In this embodiment, the storage unit 12 and the large-capacity storage unit 15 may be configured as a single storage device. Furthermore, the large-capacity storage unit 15 may be composed of multiple storage devices. Moreover, the large-capacity storage unit 15 may be an external storage device connected to the server 1.

[0036] Server 1 may perform various information processing and control processes as a single computer, or it may be performed in a distributed manner across multiple computers. Furthermore, Server 1 may be implemented using multiple virtual machines located within a single server, or it may be implemented using a cloud server.

[0037] Figure 3 is an explanatory diagram showing an example of the record layout for Scene DB152 and Base Prompt DB153. Scene DB152 stores multiple scenes related to device 3. Scene DB152 includes columns for Scene ID, Name, Description, and Reference Image. The Scene ID column stores a uniquely identified scene ID to identify various scenes. A scene is a category classified for analyzing user instructions, questions, or requests. Examples of scenes include "Error Analysis," "Manual Reference," "Log," "Internal Equipment Usage Method," "Fault Prediction," "Model," or "Location."

[0038] The Name column stores the name of the scene (e.g., "Error Analysis"). The Description column stores a description of the scene. For example, the description for "Error Analysis" could be "The data is in JSON format. The data contains elements with the name [ModeName] whose value is either [xxx] or [xxx]. The element name [isError] exists." The Reference Image column stores image data or image features associated with the scene.

[0039] The base prompt DB153 stores the base prompt corresponding to each scene. The base prompt DB153 includes a scene ID column, a base prompt column, and a reference image column. The scene ID column stores the scene ID to identify each scene. The base prompt column stores example sentences (templates) of the prompt corresponding to each scene. The reference image column stores image data or image features related to the base prompt.

[0040] Figure 4 is an explanatory diagram showing an example of the record layout of the equipment information DB154 and the knowledge DB155. The equipment information DB154 includes columns for equipment ID, equipment name, model, and installation location.

[0041] The Equipment ID column stores the unique ID of each Equipment 3 to identify it. The Equipment ID column may also store the model number of Equipment 3. The Equipment Name column stores the name of Equipment 3. The Model column stores the type of Equipment 3 (e.g., "injection molding machine" or "die casting"). The Installation Location column stores the location (location information) where Equipment 3 is installed.

[0042] Knowledge DB155 includes columns for Knowledge ID, Model, Type, and Content. The Knowledge ID column stores a uniquely identifiable ID for each type of knowledge data. The Model column stores the type of equipment 3 to which the knowledge data corresponds. The Type column stores the type of knowledge data (e.g., manual, troubleshooting, or safety). The Content column stores the document containing the knowledge data (e.g., an operation manual related to equipment 3) or the knowledge data itself.

[0043] Figure 5 is a block diagram showing an example configuration of terminal 2. Terminal 2 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, a display unit 25, and an imaging unit 26.

[0044] The control unit 21 includes a processing unit such as a CPU or MPU, and performs various information processing and control processing related to the terminal 2 by reading and executing the control program 2P (program product) stored in the storage unit 22. The control program 2P described in this embodiment may be provided on a recording medium or distributed from an external computer.

[0045] In Figure 5, the control unit 21 is described as a single processor, but it may be a multi-processor system. Furthermore, the control unit 21 may perform various information processing or control processing using the same processor within the terminal 2, or it may perform these processes using different processors within the terminal 2.

[0046] The storage unit 22 includes memory elements such as RAM or ROM, and stores the control program 2P or data necessary for the control unit 21 to execute processing. The storage unit 22 also temporarily stores data necessary for the control unit 21 to execute arithmetic processing.

[0047] The communication unit 23 is a communication module for performing communication-related processing and sends and receives information with the server 1, etc., via the network N. The input unit 24 may be a keyboard, mouse, or a touch panel integrated with the display unit 25. The display unit 25 is a liquid crystal display or an organic EL (electroluminescence) display, etc., and displays various information according to the instructions of the control unit 21.

[0048] The imaging unit 26 is, for example, an imaging device such as a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera. The imaging unit 26 may not be built into the terminal 2, but may be configured to be externally connected directly to the terminal 2 for imaging.

[0049] Figure 6 is a block diagram showing an example configuration of device 3. Device 3 includes a control unit 31, a storage unit 32, a communication unit 33, an input unit 34, a display unit 35, and a large-capacity storage unit 36. Each component is connected by bus B.

[0050] The control unit 31 includes a processing unit such as a CPU or MPU, and performs various information processing or control processing related to the device 3 by reading and executing the control program 3P (program product) stored in the storage unit 32. The control program 3P described in this embodiment may be provided on a recording medium or distributed from an external computer.

[0051] The storage unit 32 includes memory elements such as RAM or ROM, and stores control programs 3P and other data necessary for the control unit 31 to execute processing. The storage unit 32 also temporarily stores data and other data necessary for the control unit 31 to execute arithmetic processing.

[0052] The control program 3P may also include a support agent designed to resolve potential problems that may occur with device 3. The support agent ensures the proper operation of device 3, monitors both the hardware and software of device 3, and collects trouble information when problems occur. The support agent may also be part of the hardware of device 3.

[0053] The communication unit 33 is a communication module for performing communication-related processing and transmits and receives information with the server 1, etc., via the network N. The input unit 34 includes a touch panel installed on the display screen of the display unit 35 and switches installed around the display screen of the display unit 35. The display unit 35 is a liquid crystal display or an organic EL display, etc., and displays various information according to the instructions of the control unit 31.

[0054] The large-capacity storage unit 36 ​​includes a log database 361. The log database 361 stores log data generated by the system or application. In this embodiment, the storage unit 32 and the large-capacity storage unit 36 ​​may be configured as a single integrated storage device.

[0055] Figure 7 is an explanatory diagram showing an example of the record layout of the log DB361. The log DB361 stores log data related to the operation of equipment 3. For example, if equipment 3 is an injection molding machine, the operation of equipment 3 includes injection, cooling, mold closing, or pressure fluctuations.

[0056] Log DB361 includes columns for Log ID, Device ID, Level, Operation, Log Details, and Date / Time. The Log ID column stores the Log ID of each log data, which uniquely identifies each log data. The Device ID column stores the Device ID to identify Device 3. The Level column stores the level indicating the type of log, including Normal, Error, Warning, Debug, or Information. The Operation column stores the operation of Device 3.

[0057] The log details column stores detailed log information for each operation. For example, in the case of an injection failure, the log details column will store the detailed information such as, "The resin injection did not reach the mold, resulting in an injection failure." The log details column may also store sensor data obtained from various sensors inside the device 3. Sensors include, for example, temperature sensors, humidity sensors, or light sensors. The sensor data is time-series data, and may include, for example, sensor data recorded within a time range from when the operation was performed up to 10 minutes prior.

[0058] The storage configurations described above for each database are merely examples; other storage configurations are also acceptable as long as the relationships between the data are maintained.

[0059] Figure 8 is an explanatory diagram illustrating the output processing of responses to user queries regarding device 3. Server 1 receives user queries regarding device 3. Queries are questions about device 3 (e.g., troubleshooting questions), confirmations, or instructions, and include text, image data, log data, or a combination thereof. Image data is image data related to device 3 (e.g., fault image data). Log data is log data related to the operation of device 3.

[0060] Specifically, when the user's terminal 2 receives text input from the user (for example, "What is error number 017?"), it acquires image data corresponding to the text via the imaging unit 26. Terminal 2 associates the received text and acquired image data with the device ID of device 3 and sends them to server 1. Server 1 receives the text and image data sent from terminal 2.

[0061] Note that image data is not limited to being acquired on terminal 2. For example, device 3 activates the support agent installed on it. Device 3 acquires information about the screen display when a problem occurs through the support agent. This information includes, for example, the error number or message displayed on the screen when a problem occurs, or a screenshot of that screen. Device 3 sends the acquired information to server 1.

[0062] Server 1 retrieves log data from the log DB361 of device 3, corresponding to the date and time the trouble occurred with device 3, based on the device ID of device 3. Terminal 2 may also accept log data input from the user. In this case, terminal 2 sends the received log data to server 1.

[0063] First, Server 1 identifies the corresponding scene from among multiple scenes related to Device 3 based on the acquired text, image data, and log data. These multiple scenes include, for example, "error analysis," "manual reference," "logs," "how to use internal equipment," or "fault prediction."

[0064] Specifically, Server 1 generates a prompt to identify a corresponding scene from multiple scenes. The prompt includes a query containing text, image data, and log data, as well as an instruction to select a scene from multiple scenes related to Device 3.

[0065] For example, the prompt is: For "[What is Error Number 017?]", please refer to Scene DB152 based on the image data and log data and select the appropriate scene. Image data: error.jpg The log data may be: Level [Error], Operation [Injection Failure], Details [Injection failure occurred because the resin did not reach the mold].

[0066] Server 1 inputs the generated prompt into language model 151 and outputs (identifies) the corresponding (to be selected) scene from among multiple scenes related to device 3.

[0067] Specifically, Server 1 vectorizes the input text, image data, and log data using a language model 151, for example, Word2Vec or BERT. This converts the various types of data into numerical representations that the language model 151 can understand. For example, Server 1 extracts image features from image data and predefined error numbers, error messages, or behavioral characteristics from log data using the language model 151.

[0068] Server 1 vectorizes extracted image features, error numbers, error messages, or behavioral features through the language model 151. Furthermore, Server 1 vectorizes scene-related information in the scene DB 152 through the language model 151. Then, Server 1 compares the vectors of the input text, image data, and log data with the vectors of scene-related information in the scene DB 152 using distance calculations in the vector space (e.g., cosine similarity or Euclidean distance) through the language model 151, and identifies the scene with the highest similarity. Server 1 outputs the scene identified by the language model 151. For example, the scene identified by the language model 151 is "Error Analysis".

[0069] Furthermore, the process of identifying corresponding scenes from multiple scenes is not limited to the use of language model 151. For example, Server 1 uses natural language processing techniques to extract keywords or phrases from the text included in the query. For image data, Server 1 applies an image recognition algorithm to analyze the image content and extract image features indicating trouble or failure. Server 1 also uses a known log analysis algorithm to extract information such as error codes, messages, or dates and times within the log data.

[0070] Server 1 compares the extracted keywords or phrases, image features, and log data analysis results with the scene descriptions and reference image features registered in Scene DB 152. For example, Server 1 uses a similarity calculation method such as cosine similarity or Euclidean distance to calculate the similarity between the extracted keywords or phrases, image features, and information in the log data and the scene descriptions and reference image features registered in Scene DB 152, thereby identifying the most appropriate scene.

[0071] Next, Server 1 identifies the base prompt corresponding to the identified scene. A scene corresponds to one or more base prompts. For example, a scene named "Error Analysis" corresponds to "Role: You are a support engineer who answers questions about the cause and solution for a XXX alarm..." and "Role: You are a support engineer who answers questions about the cause and solution for a hard disk failure...".

[0072] Server 1 generates a prompt to identify the base prompt corresponding to the identified scene. The prompt includes a query containing text, image data, and log data, the identified scene, and instructions for selecting a base prompt from among several base prompts corresponding to that scene.

[0073] For example, the prompt is: In the [Error Analysis] section, for the question "[What is Error Number 017?]", refer to Base Prompt DB153 based on the image data and log data to select the appropriate base prompt. Image data: error.jpg The log data may be: Level [Error], Operation [Injection Failure], Details [Injection failure occurred because the resin did not reach the mold].

[0074] Server 1 inputs the generated prompt into language model 151 and outputs (identifies) the base prompt (to be selected) corresponding to the identified scene.

[0075] Specifically, Server 1 vectorizes the input text, image data, log data, and identified scenes using, for example, Word2Vec or BERT, through the language model 151. Furthermore, Server 1 vectorizes the information related to the base prompts in the base prompt DB 153 through the language model 151. Note that the vectorization process is the same as the vectorization process described above, so a detailed explanation is omitted.

[0076] Server 1 then uses the language model 151 to perform distance calculations in a vector space to compare the input text, image data, log data, and scene vectors with the vectors of information related to base prompts in the base prompt DB 153, and identifies the base prompt with the highest similarity. Server 1 then outputs the base prompt identified by the language model 151.

[0077] As an example, in "error analysis," the identified base prompt is: "###role You are a support engineer responsible for answering questions about the cause and solution of the XXX alarm. Please limit your answer to the XXX content only. ### Requests Please make your decision in the following order: Please parse the received document in JSON format. If the analysis results contain error messages, please extract them. ××× ### Definition of items in received documents Device screen status: allMessage Device ID:××× Model name: xxx ### Output Contents Please list all possible solutions, including any branching steps, in bullet points. If you referenced any files, please output the filenames as the [source]. (This can also be written as "...")

[0078] Note that the process of identifying base prompts is not limited to the use of language model 151. For example, server 1 may calculate the similarity between the identified scene, text, image data, and log data and the image features of the base prompts and reference images registered in base prompt DB 153, and identify the base prompt corresponding to that sheet.

[0079] Furthermore, there may be cases where no base prompt exists that corresponds to the identified scene. For example, Server 1, through the language model 151, uses distance calculation in the vector space to calculate the similarity between the vectors of the input text, image data, log data, and scene and the vectors of information related to the base prompts in the base prompt DB 153. If the similarity calculated through the language model 151 is below a predetermined threshold (e.g., 40), Server 1 determines that no base prompt exists that corresponds to the identified scene and does not output the corresponding base prompt.

[0080] If Server 1 determines, through the base prompt identification process described above, that there is no base prompt corresponding to the scene, it generates a new prompt.

[0081] Specifically, Server 1 generates a prompt for generating a new prompt. This prompt includes a dialogue between multiple agents using the user's query, instructions for generating a new prompt based on that dialogue, and the format of the final prompt. The instructions for generating a new prompt are instructions (commands) for multiple agents to engage in dialogue and generate a new prompt based on the content of that dialogue.

[0082] In this embodiment, the agents are constructed based on the language model 151. The agents can perform different tasks or roles. For example, using the same language model 151, multiple agents with specific functions, such as question answering, dialogue, or reasoning, can be generated based on different prompts or instructions. This allows for the flexible use of a single language model 151 for diverse purposes, enabling the construction of a multi-agent system where agents cooperate (dialogue) with each other.

[0083] Furthermore, this is not limited to conversations between multiple agents. For example, a dialogue between multiple language models 151 is also possible.

[0084] For example, Server 1 obtains the content of a user's query by having multiple agents interact with each other. That is, each agent asks and answers questions to each other based on the information it is given, and outputs the questions and answers to Server 1 as the content of the dialogue.

[0085] First, Server 1 generates a prompt to allow multiple agents to interact. This prompt includes a user query, instructions for the agents to ask and answer questions based on that query, and instructions for outputting the content of the dialogue. As an example, the generated prompt might be: "Agent 1: You are a prompt engineer who translates people's questions into appropriate prompts for the AI. Your role is to create prompts." Second Agent: You are an auxiliary prompt engineer who answers questions to generate prompts. On device A, the user's query is "[What is error number 017?]". Discuss the user's query in a question-and-answer format and output the dialogue. (This could also be written as: ").

[0086] The prompt example described above illustrates a dialogue between two agents, but the dialogue is not limited to this; it may involve two or more agents (for example, three agents).

[0087] Server 1 inputs the generated prompt into language model 151 and retrieves the content of the dialogue between the multiple agents. As an example, the retrieved dialogue content is: "Second agent: Please write a prompt for the error number." Agent 1: In what format would you like your response? Second Agent: Please include the cause and solution for the error number in your response. Please provide the cause and solution for error code 017 on device A in the format below. Model name: Error number: Cause: It could also be "Countermeasures:".

[0088] Next, Server 1 generates a new prompt generation instruction based on the content of the acquired dialogue. For example, the generation instruction may be, "Generate a new prompt based on the content of this dialogue." Server 1 generates a prompt for generating a new prompt, which includes the content of the dialogues from multiple agents, the generated new prompt generation instruction, and the format of the final prompt.

[0089] As an example, the prompt for generating a new prompt is: "Second agent: Please write a prompt for the error number." Agent 1: In what format would you like your response? Second Agent: Please include the cause and solution for the error number in your response. Please provide the cause and solution for error code 017 on device A in the format below. Model name: Error number: Cause: countermeasure: It could also be something like, "Generate a new prompt based on the content of this dialogue."

[0090] Server 1 generates a new prompt by providing the generated prompt to the language model 151. For example, the generated new prompt is: "### role You are a support engineer for the equipment. Your role is to clearly explain the cause and solution to user inquiries regarding error codes. ### Request Please explain the cause and solution for the error number in the following order. 1. Receive queries from users regarding error numbers. 2. Based on the query, identify and explain the cause associated with the error number. 3. We will provide appropriate countermeasures for error numbers as needed. ### Error Handling - If the error number is unknown, please respond with "The error number is unknown. Please check the details." - If there are multiple solutions, please explain each one in a bulleted list. ### Output Contents - **Model Name**: (Enter the name of the device) - **Error Number**: (Enter the error number) - **Cause**: (Clearly state the cause of the error) - **Countermeasures**: (List countermeasures for the error in bullet points) ### Notes - Please make the proposed countermeasures specific and actionable. - Please explain it in a way that is easy for users to understand, avoiding technical jargon.

[0091] Server 1 stores the newly generated prompt as the base prompt in the base prompt DB153. Specifically, Server 1 associates the newly generated prompt with the scene ID of the target scene (e.g., "Error Analysis"), uses it as the base prompt, and stores the image data corresponding to the user's query as a reference image in the base prompt DB153. Note that the image data is not mandatory and can be omitted.

[0092] The above explanation described an example of a combination of text, image data, and log data, but it is not limited to this; any combination of text, image data, log data, or any other combination thereof may be used. Furthermore, image data or log data is not required; only text-based queries can be used.

[0093] Server 1 generates the final prompt by providing the identified base prompt or the newly generated prompt to the language model 151. The final prompt is the prompt generated by providing the language model 151 with the user's query, the identified base prompt or the new prompt, and the instructions and format for generating the final prompt.

[0094] Specifically, Server 1 generates a prompt to generate the final prompt. This prompt includes the user's query, the identified base prompt or new prompt, and the instructions and format for generating the final prompt. As an example, the generated prompt may be: For a user querying "[What is error number 017?]", generate a final prompt in the following format, based on the base prompt below, providing the cause and solution for error number 017 on device A. Reference document: "Troubleshooting Guide xxx.pdf" stored in Knowledge DB155 Base prompt: ×××××× format: ### role ### procedure ### Additional information ### Definition of items in received documents ### Definition of items in JSON within the received document ### Output Contents ### Output format Model name: Device ID: Alarm name: How to deal with it: Source: ### Error Handling It could also be "Notes / Precautions".

[0095] Server 1 inputs the generated prompt into language model 151 and generates the final prompt. As an example, the generated final prompt is: "### role You are a support engineer responsible for answering questions about the cause and solution of the XXX alarm. Please provide a factual answer based on the document "Troubleshooting Guide XXX.pdf" stored in Knowledge DB155. ### procedure 1. **Confirmation of the question**: - Check the question at the beginning of the received document. - Based on the content of the question, clarify what information is needed. 2. **Parsing JSON Documents**: - Parses the JSON document from the second line onwards of the received document. - Extract the necessary items (device ID, model name, device screen status). 3. **Obtaining the model name**: - Retrieves the model name from the JSON document. 4. **Reference to documentation**: - Find the information corresponding to the model name in the provided documentation. - Check how to deal with alarms and error codes related to the question. 5. **Extracting solutions**: - Extract solutions from the documentation and organize them in a bulleted list. - If there are branching steps, please describe all possible solutions without omission. 6. **Citation of source**: - The source of the extracted information will be clearly indicated. - Please specify the document name and the relevant page number. ### Additional information Please extract all possible solutions, even if the procedure branches midway through. ### Definition of items in received documents -First line: Question -From the second line onwards: JSON ### Definition of items in JSON within the received document - Device screen status: ××× - Device ID: ××× - Model name: ××× - Unused: ××× Please list all possible solutions, including any branching steps, in a bulleted list. If you referenced any files, please output the filenames as the [source]. ### Output format Model name: Device ID: Alarm name: How to deal with it: Source: ### Error Handling If there are missing items in the received document, please display the following message. "The format of the received document is incorrect. Please check again." ### Notes It would also be acceptable to say, "Please review your response to ensure that information from documents that are not relevant is not included."

[0096] The final prompt generation process may be performed using an externally trained language model or other platform, rather than utilizing language model 151. For example, server 1 may generate the final prompt using FalconAutoPrompt®, which can automatically perform everything from data analysis to the generation of an appropriate prompt.

[0097] Finally, Server 1 outputs the answer to the user's query by providing the generated final prompt to the language model 151. For example, the outputted answer is: Model name: ××× Device ID:××× Alarm name: Error number 017 How to deal with it: Error number 017 is related to a communication error. Please try the following steps. 1. Turn the device's power off and then on again to try communicating again. 2. Check the connected cables to ensure they are not loose or broken. 3. Please double-check your network settings and ensure there are no problems with your communication configuration. 4. If you are experiencing problems connecting to the server, please check the server settings or your network environment. If the problem persists after trying the above steps, please contact technical support again. It is also acceptable to cite the source as "Troubleshooting Guide xxx.pdf (Chapter 3, p. 45)."

[0098] Server 1 sends the response to the query output from language model 151 to terminal 2. Terminal 2 receives the response to the query sent from server 1 and displays the received response on its screen.

[0099] Figure 9 is a flowchart showing the processing procedure when outputting a response to a query using the language model 151. The control unit 21 of terminal 2 obtains a query from the user in the form of text and image data related to device 3 (step S201). Specifically, the control unit 21 receives text input from the user (for example, "What is error number 017?") at the input unit 24. The control unit 21 obtains image data corresponding to the text via the imaging unit 26.

[0100] The control unit 21 associates the acquired text and image data with the device ID of device 3 and transmits them to the server 1 via the communication unit 23 (step S202). Device 3 may also transmit information about the screen where a problem occurred (e.g., error number, message, or screen capture) to the server 1 via the support agent installed on device 3. The control unit 11 of the server 1 receives the device ID, text, and image data transmitted from terminal 2 via the communication unit 13 (step S101).

[0101] The control unit 11 retrieves log data corresponding to the date and time the trouble occurred in device 3 from the log DB 361 of the large-capacity storage unit 36 ​​of device 3, based on the device ID of device 3 (step S102). The control unit 21 of terminal 2 may also accept log data input from the user via the input unit 24. The control unit 21 transmits the received log data to the server 1 via the communication unit 23.

[0102] Based on the acquired text, image data, and log data, the control unit 11 identifies the corresponding scene from among multiple scenes related to the device 3 (such as "error analysis," "manual reference," "log," "how to use in-house equipment," or "fault prediction") (step S103).

[0103] Specifically, the control unit 11 generates a prompt that includes a query containing text, image data, and log data, and an instruction to select a scene from among multiple scenes relating to the device 3. The control unit 11 inputs the generated prompt into the language model 151 and outputs (identifies) the corresponding scene from among the multiple scenes relating to the device 3.

[0104] The control unit 11 identifies a base prompt corresponding to the identified scene (step S104). Specifically, the control unit 11 generates a prompt that includes the user's query, the identified scene, and an instruction to select a base prompt from a plurality of base prompts corresponding to that scene. The control unit 11 inputs the generated prompt to the language model 151 and outputs (identifies) a base prompt corresponding to the identified scene.

[0105] The control unit 11 determines whether or not a base prompt exists based on the identification result obtained from the processing in step S104 (step S105). Specifically, the control unit 11 calculates the similarity between the input text, image data, log data, and scene vectors and the vectors of information related to base prompts in the base prompt DB 153 through the language model 151. If the calculated similarity exceeds a predetermined threshold (for example, 40), the control unit 11 determines that a base prompt corresponding to the identified scene exists. Alternatively, if the calculated similarity is below the predetermined threshold, the control unit 11 determines that a base prompt corresponding to the identified scene does not exist.

[0106] If a base prompt exists (YES in step S105), the control unit 11 executes a subroutine for generating the final prompt (step S106). The subroutine for generating the final prompt will be described later.

[0107] The control unit 11 inputs the final prompt obtained through the final prompt generation process to the language model 151 (step S107) and outputs the answer to the query (step S108). The control unit 11 transmits the answer to the query output from the language model 151 to the terminal 2 via the communication unit 13 (step S109).

[0108] The control unit 21 of terminal 2 receives the response to the query sent from server 1 via the communication unit 23 (step S203). The control unit 21 displays the received response to the query on the display unit 25 (step S204). The control unit 21 then terminates processing.

[0109] If no base prompt exists (NO in step S105), the control unit 11 executes a subroutine for generating a new prompt (step S110). The subroutine for generating a new prompt will be described later. The control unit 11 stores the generated new prompt as the base prompt in the base prompt DB153 of the large-capacity storage unit 15 (step S111).

[0110] Specifically, the control unit 11 associates the generated new prompt with the scene ID of the target scene (for example, "error analysis"), uses it as the base prompt, and stores the image data corresponding to the user's query as a reference image in the base prompt DB 153. Note that the image data is not mandatory and can be omitted.

[0111] The control unit 11 proceeds to the process in step S106.

[0112] Figure 10 is a flowchart showing the processing procedure of a subroutine for generating a new prompt. The control unit 11 of server 1 generates a first prompt for causing multiple agents to interact (step S01). The first prompt includes a user query, instructions for each agent to ask and answer questions based on the query, and instructions for outputting the content of the dialogue.

[0113] The control unit 11 inputs the generated first prompt into the language model 151 that uses the user's query (step S02), and outputs the content of the dialogue between multiple agents (step S03).

[0114] The control unit 11 generates a second prompt for generating a new prompt (step S04). The second prompt includes the content of the dialogue between multiple agents, an instruction to generate a new prompt based on the content of the dialogue, and the format of the final prompt.

[0115] The control unit 11 inputs the generated second prompt into the language model 151 (step S05) and generates a new prompt (step S06). The control unit 11 finishes the subroutine for generating the new prompt and returns.

[0116] Figure 11 is a flowchart showing the processing procedure of the subroutine for generating the final prompt. The control unit 11 of server 1 generates a prompt for generating the final prompt (step S11). This prompt includes the user's query, the identified base prompt or a new prompt, and the instructions and format for generating the final prompt.

[0117] The control unit 11 inputs the generated prompt to the language model 151 (step S12) and outputs the final prompt (step S13). The control unit 11 finishes the subroutine for generating the final prompt and returns.

[0118] In the process described above, Server 1 generates a final prompt by providing the identified base prompt or the newly generated prompt to the language model 151. Server 1 outputs the answer to the user's query by providing the generated final prompt to the language model 151. Alternatively, the answer to the user's query can be output based on the identified base prompt or the newly generated prompt without using the final prompt.

[0119] First, we will explain the process of outputting a response to a user's query based on a base prompt. Server 1 outputs a response to the user's query by providing the language model 151, which uses a base prompt, with the user's query and the identified base prompt. As an example, the output response is: Model name: ××× Device ID: ××× Alarm name: Error number 017 How to deal with it: 1. Check the error message: Please check the error message "Error number 017" displayed on the screen. 2. Connection check: Please check if your network connection is working correctly, and reconnect if necessary. 3. Firmware check: Please ensure that the device's firmware is up to date. If your firmware is outdated, please update to the latest version. 4. Reset Reset the device and check if the error message reappears. 5. Check the logs: If the device logs contain detailed information about the error, review the logs to identify any additional steps. The source could also be "XXX Troubleshooting Guide, 3rd Edition".

[0120] Server 1 sends the response to the query output from language model 151 to terminal 2. Terminal 2 receives the response to the query sent from server 1 and displays the received response on its screen.

[0121] Next, we will describe the process of outputting a response to the user's query based on a new prompt. Server 1 outputs a response to the user's query by providing the newly generated prompt to the language model 151. As an example, the output response is: "**Model Name**: ×××" **Error Number**: 017 **Cause**: There may be a problem with your network connection. This error appears when the connection is unstable or the cable is loose. **countermeasure**: 1. Check that the network cable is securely connected. 2. Try restarting your router or modem to resolve the connection issue. 3. Restart the device itself and check if the error message reappears. 4. If there are problems with your network settings, please recheck them and make any necessary corrections. 5. If the problem persists, contact your network administrator to have them re-check the network status. **Note**: If error number 017 persists, it may be caused by a network equipment failure, requiring further diagnosis.

[0122] Server 1 sends the response to the query output from language model 151 to terminal 2. Terminal 2 receives the response to the query sent from server 1 and displays the received response on its screen.

[0123] According to this embodiment, it is possible to output answers to user queries regarding the device 3 using the language model 151.

[0124] According to this embodiment, it is possible to automatically generate more appropriate prompts by utilizing scenes based on user queries and base prompts corresponding to those scenes.

[0125] According to this embodiment, if a base prompt corresponding to the scene does not exist, a new prompt is generated, and the generated new prompt is provided to the language model 151, thereby enabling the output of an answer to the user's query.

[0126] According to this embodiment, it becomes possible to store the newly generated prompt as a base prompt.

[0127] According to this embodiment, a final prompt can be generated by providing the language model 151 with a specified base prompt or a new prompt.

[0128] According to this embodiment, by providing the final prompt to the language model 151, it becomes possible to obtain a highly accurate response.

[0129] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the claims, not in the sense described above, and all modifications within the sense and scope equivalent to the claims are intended.

[0130] The matters described in each embodiment can be combined with each other. Furthermore, the independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. In addition, the claims use a form in which claims referencing two or more other claims (multi-claim form), but are not limited to this. A form in which multi-claims referencing at least one multi-claim (multi-multi-claim) may also be used. [Explanation of Symbols]

[0131] 1. Information processing device (server) 11 Control Unit 12 Storage section 13 Communications Department 14 Reading section 15 Mass storage 151 Language Models 152 Scene DB 153 Base Prompt DB 154 Device information DB 155 Knowledge DB 1a Portable storage medium 1b Semiconductor memory 1P Control Program 2. Information processing terminal (terminal) 21 Control Unit 22 Memory section 23 Communications Department 24 Input section 25 Display section 26 Photography Department 2P control program 3 equipment 31 Control Unit 32 Storage section 33 Communications Department 34 Input section 35 Display section 36 Mass storage 361 Log DB 3P control program

Claims

1. Retrieve user queries regarding the device, Based on the acquired query, the corresponding scene is identified from multiple scenes relating to the device. Identify the base prompt corresponding to the identified scene, The language model, using the identified base prompt, outputs the answer. A program that instructs a computer to perform a process.

2. Identify the base prompt from multiple base prompts corresponding to the identified scene. The program according to claim 1.

3. If a base prompt corresponding to the aforementioned scene does not exist, a new prompt is generated. By providing the newly generated prompt to the language model, the response is output. The program according to claim 1 or 2.

4. The new prompt is generated through dialogue between multiple agents using the query. The program according to claim 3.

5. The final prompt is generated by providing the identified base prompt or the new prompt to the language model. By providing the generated final prompt to the language model, the answer is output. The program according to claim 3.

6. The newly generated prompt is stored as the base prompt. The program according to claim 3.

7. The aforementioned query includes, in addition to text, images or log data related to the device. Based on the aforementioned text and images or log data relating to the equipment, the scene is identified. The program according to claim 1 or 2.

8. The aforementioned query includes, in addition to text, images or log data related to the device. Based on the aforementioned text and images or log data relating to the equipment, the base prompt is identified. The program according to claim 1 or 2.

9. The prompt for generating the new prompt includes instructions for generating the new prompt based on dialogue between multiple agents using the query, and the format of the final prompt. The program according to claim 3.

10. The final prompt is a prompt generated by providing the language model with the user's query, the identified base prompt or new prompt, and the format of the final prompt. The program according to claim 5.

11. By providing the language model with a first prompt including the aforementioned query and an instruction to select a scene from a plurality of scenes relating to the device, the scene to be selected is identified. The base prompt to be selected is identified by providing the language model with the aforementioned query, a identified scene, and a second prompt including an instruction to select a base prompt from a plurality of base prompts corresponding to the scene. The program according to claim 2.

12. Retrieve user queries regarding the device, Based on the acquired query, the corresponding scene is identified from multiple scenes relating to the device. Identify the base prompt corresponding to the identified scene, The language model, using the identified base prompt, outputs the answer. Information processing methods.

13. An information processing device comprising a control unit, The control unit, Retrieve user queries regarding the device, Based on the acquired query, the corresponding scene is identified from multiple scenes relating to the device. Identify the base prompt corresponding to the identified scene, The language model, using the identified base prompt, outputs the answer. Information processing device.

Images