Information processing apparatus, information processing method, computer readable storage medium, and computer program product

Abstract

An information processing apparatus, an information processing method, a computer-readable storage medium, and a computer program product are provided. The information processing apparatus includes an acquisition unit configured to acquire text information related to an answer to an input question based on the question using a large language model, a decision unit configured to decide whether or not to present non-text information based on the text information, and a transmission unit configured to transmit the text information and the non-text information if it is decided to present the non-text information, and to transmit the text information if it is decided not to present the non-text information.

Description

Technical Field

[0001] This disclosure generally relates to information processing apparatus, information processing methods, computer-readable storage media, and computer program products, and particularly to techniques for presenting answers to questions. Background Technology

[0002] Traditionally, techniques are known that generate answer statements by inputting prompts based on user questions and related documents into a large language model, and then presenting the answer statements to the user (e.g., Japanese Patent 7538364).

[0003] However, in traditional technologies, if the important information in an answer cannot be expressed by strings alone, the presented answer statement alone is insufficient for the user to judge whether it is an appropriate answer, and a lot of labor may be needed to investigate additional information for the judgment. Summary of the Invention

[0004] This disclosure provides a technique for presenting an appropriate answer to a question.

[0005] According to a first aspect of this disclosure, an information processing apparatus is provided, comprising: an acquisition unit configured to acquire text information relating to an answer to a question based on an input question using a large language model; a decision unit configured to decide whether to present non-text information based on the text information; and a transmission unit configured to transmit the text information and the non-text information if it is decided to present the non-text information, and to transmit the text information if it is decided not to present the non-text information.

[0006] According to a second aspect of this disclosure, an information processing apparatus is provided, comprising: an acquisition unit configured to acquire text information relating to an answer to a question based on an input question using a large language model; a decision unit configured to decide whether to present non-text information based on the text information; and a presentation unit configured to present the text information and the non-text information if it is decided to present the non-text information, and to present the text information if it is decided not to present the non-text information.

[0007] According to a third aspect of this disclosure, an information processing method is provided, which is performed by an information processing device, the information processing method comprising: using a large language model to obtain text information related to an answer to an input question based on an input question; determining whether to present non-text information based on the text information; and, if it is determined that the non-text information should be presented, sending the text information and the non-text information, and if it is determined that the non-text information should not be presented, sending the text information.

[0008] According to a fourth aspect of this disclosure, an information processing method is provided, which is performed by an information processing device, the information processing method comprising: using a large language model to obtain text information related to an answer to an input question based on the input question; determining whether to present non-text information based on the text information; and presenting the text information and the non-text information if it is determined to present the non-text information, and presenting the text information if it is determined not to present the non-text information.

[0009] According to a fifth aspect of this disclosure, a computer-readable storage medium is provided storing a computer program configured to cause a computer to: use a large language model to obtain text information related to an answer to an input question based on the input question; determine whether to present non-text information based on the text information; and, if it is determined to present the non-text information, transmit the text information and the non-text information, and if it is determined not to present the non-text information, transmit the text information.

[0010] According to a sixth aspect of this disclosure, a computer-readable storage medium is provided storing a computer program configured to cause a computer to: use a large language model to obtain text information related to an answer to an input question based on the input question; determine whether to present non-text information based on the text information; and, if it is determined that the non-text information should be presented, present the text information and the non-text information, and if it is determined that the non-text information should not be presented, present the text information.

[0011] According to a seventh aspect of this disclosure, a computer program product is provided that stores computer program instructions for causing a computer to function as a unit of an information processing apparatus according to the first aspect.

[0012] According to the eighth aspect of this disclosure, a computer program product is provided that stores computer program instructions for causing a computer to function as a unit of an information processing apparatus according to the second aspect.

[0013] The features of this disclosure will become apparent from the following description of embodiments with reference to the accompanying drawings. The following description of the embodiments is given by way of example. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the specification, serve to explain the principles of the embodiments.

[0015] Figure 1 It is a diagram showing the usage of the system;

[0016] Figure 2A This is a block diagram illustrating an example of the hardware configuration of tablet PC 3;

[0017] Figure 2B This is a block diagram illustrating an example of the hardware configuration of the question-and-answer system 4;

[0018] Figure 3 This is a block diagram illustrating an example of the functional configuration of the tablet computer 3 and the question-and-answer system 4;

[0019] Figure 4 It is a flowchart of the process performed by the tablet computer 3 and the question-and-answer system 4 to create an answer to a question entered by the user and present it to the user;

[0020] Figure 5A This is a diagram showing an example of a screen display;

[0021] Figure 5B This is a diagram showing an example of a screen display;

[0022] Figure 6 This is a flowchart showing the details of the processing in step S404;

[0023] Figure 7A This is a diagram showing an example of a screen display;

[0024] Figure 7B This is a diagram showing an example of a screen display;

[0025] Figure 8 This is a diagram showing an example of a screen display;

[0026] Figure 9A This is a diagram showing an example of a screen display;

[0027] Figure 9B This is a diagram showing an example of a screen display;

[0028] Figure 10A This is a diagram showing an example of a screen display; and

[0029] Figure 10B This is a diagram showing an example of a screen display. Detailed Implementation

[0030] In the following, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claims. Several features are described in the embodiments, but not all of them are required, and several such features can be appropriately combined. Furthermore, in the drawings, the same reference numerals are given the same or similar configuration, and redundant descriptions are omitted. (First Embodiment)

[0031] First, refer to Figure 1 This describes the use of the system according to this embodiment. Figure 1 In the illustrated use case, a problem has arisen with printer 1. This problem may include image defects such as smudges at the front of the printed image. To resolve this issue, the service technician 2 (the user) who has been called to repair printer 1 uses tablet computer 3, exemplified as an information processing device, to find a solution. First, the service technician 2 inputs a question by operating tablet computer 3. After inputting the question, the service technician 2 instructs that the question be sent. Upon instructing the sending of the question, tablet computer 3 transmits the input question to a question-and-answer system 4 via a wireless network. The question-and-answer system 4 creates an answer corresponding to the question received from tablet computer 3 and sends the created answer to tablet computer 3. Upon receiving the answer from the question-and-answer system 4, tablet computer 3 presents the received answer to the service technician 2. The service technician 2 then specifies a solution to the problem with printer 1 based on the presented answer.

[0032] Next, we will refer to Figure 2A The block diagram illustrates an example of the hardware configuration of the tablet 3. Note that... Figure 2A The hardware configuration example shown is an example of a hardware configuration applicable to Tablet 3, and can be modified / changed as appropriate.

[0033] The CPU 101 uses computer programs and data stored in RAM 105 to perform various kinds of processing. Thus, the CPU 101 controls the operation of the entire tablet computer 3 and also performs or controls various kinds of processing described as being performed by the tablet computer 3.

[0034] ROM 102 stores the settings data of tablet PC 3, computer programs and data associated with the startup of tablet PC 3, and computer programs and data associated with the basic operation of tablet PC 3.

[0035] The input device 103 is a user interface such as a keyboard, mouse or touch panel, and the user can input various kinds of instructions and information to the tablet computer 3 by operating the input device 103.

[0036] The presentation device 104 is an interface for presenting the results of processing performed by the tablet computer 3 to a user. For example, the presentation device 104 includes an LCD screen or a touch panel screen, and can display the results of processing performed by the CPU 101 as images or characters. In addition, for example, the presentation device 104 includes a speaker, and can output the results of processing performed by the CPU 101 as sound.

[0037] RAM 105 includes areas for storing computer programs and data loaded from ROM 102 or storage device 106, and areas for storing computer programs and data received from external devices via communication interface 107. RAM 105 also includes working areas used by CPU 101 when performing various types of processing. RAM 105 can thus be suitably provided with various types of areas.

[0038] Storage device 106 is a non-volatile memory device such as a solid-state drive (SSD) and a hard disk drive (HDD). Storage device 106 stores the operating system, as well as computer programs and data for causing the CPU 101 to execute or control various types of processes described as being executed by the tablet computer 3.

[0039] Communication interface 107 is the interface used by the tablet PC 3 to communicate data with external devices via a network such as a LAN or the Internet. The network can be a wired network or a wireless network. The type of network is not limited to specific types such as USB or serial communication.

[0040] The CPU 101, ROM 102, input device 103, presentation device 104, RAM 105, storage device 106, and communication interface 107 are all connected to the system bus 108.

[0041] Next, we will refer to Figure 2B The block diagram illustrates an example of the hardware configuration for the question-and-answer system 4. Note that the following description focuses on the implementation of the question-and-answer system 4 using a single computer device. However, this disclosure is not limited to this, and the question-and-answer system 4 can be implemented using multiple computer devices. Figure 2B The hardware configuration example shown is an example of a hardware configuration suitable for Question and Answer System 4, and can be modified / changed as appropriate.

[0042] CPU 201 uses computer programs and data stored in RAM 205 to perform various kinds of processing. Therefore, CPU 201 controls the operation of the entire question-and-answer system 4, and also executes or controls various kinds of processing described as being performed by the question-and-answer system 4.

[0043] ROM 202 stores the setting data of the question and answer system 4, the computer programs and data associated with the startup of the question and answer system 4, and the computer programs and data associated with the basic operation of the question and answer system 4.

[0044] RAM 205 includes areas for storing computer programs and data loaded from ROM 202 or storage device 206, and areas for storing computer programs and data received from external devices via communication interface 207. RAM 205 also includes working areas used by CPU 201 when performing various types of processing. RAM 205 can thus be suitably provided with various types of areas.

[0045] Storage device 206 is a non-volatile memory device such as a solid-state drive (SSD) and a hard disk drive (HDD). Storage device 206 stores the operating system, as well as computer programs and data for causing the CPU 201 to execute or control processes described as being executed by the question-and-answer system 4.

[0046] Communication interface 207 is the interface used by the question-and-answer system 4 to communicate data with external devices via a network such as a LAN or the Internet. CPU 201, ROM 202, RAM 205, storage device 206 and communication interface 207 are all connected to system bus 208.

[0047] Figure 3 The block diagram illustrates an example of the functional configuration of the tablet computer 3 and the question-and-answer system 4, respectively. This embodiment will describe... Figure 3 The tablet computer 3 shown is implemented entirely by software (computer program). Figure 3 The functional units of the tablet computer 3 shown will sometimes be described below as the main components of processing; however, the function of each functional unit is actually implemented when the CPU 101 executes the computer program corresponding to that functional unit. Similarly, this embodiment will be described... Figure 3 The question-and-answer system 4 shown is implemented entirely by software (computer program). Figure 3 The functional units of the question-and-answer system 4 shown below will sometimes be described as the main components of the processing, but the function of each functional unit is actually implemented when the CPU 201 executes the computer program corresponding to that functional unit.

[0048] Reference Figure 4 The flowchart shown describes the process performed by the tablet computer 3 and the question-and-answer system 4 to create an answer to a question entered by the user and present it to the user.

[0049] In step S401, the question acquisition unit 301 acquires the question input based on the user's operation. For example, the question acquisition unit 301 causes the presentation device 104 to display... Figure 5AThe example shown is screen 500. The user inputs a question into input box 501 using input device 103. Upon completing the question input, the user makes a gesture on button 502 using input device 103. When gestured on button 502, question acquisition unit 301 acquires the input question. Then, question acquisition unit 301 causes presentation device 104 to display the acquired question as shown... Figure 5B The example shown is a user-side chat bubble 510a. The question acquisition unit 301 sends the acquired question to the question answering system 4 via the communication interface 107.

[0050] In step S402, generation unit 302 acquires the question sent from tablet computer 3. Generation unit 302 uses the large language model 307 stored in storage device 206 to generate text information related to the answer to the acquired question. To generate text information related to the answer to the acquired question, the method described, for example, in P. Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," Advances in Neural Information Processing Systems, vol. 33, pp. 9459-9474, 2020, may be applied.

[0051] For example, generation unit 302 uses the acquired question as a query to search in the "Answer-Related Database 306" stored in storage device 206 for the source location (e.g., URL) of the source document (e.g., an HTML document) corresponding to the question. Then, generation unit 302 generates text information related to the answer to the acquired question by inputting the acquired question and the source document identified by the source location found in database 306 into large language model 307 and performing reasoning using large language model 307. Instead of inputting the entire source document into the large language model, only relevant parts (source segments) of the source document, such as statements, paragraphs, or sections matching the query obtained through string search, can be input. By reducing the amount of information to the amount required for the large language model to provide an answer and inputting the information into the large language model, resource waste can be reduced. Large language model 307 can be implemented using any suitable large language model configured to generate text information related to the answer based on input including at least the question and the source document (or source segment). For example, this configuration can be achieved through prompts and / or training such as fine-tuning, but training is not necessarily required.

[0052] In step S403, the acquisition unit 303 uses a dictionary registered in the database 306 to acquire representative non-textual information associated with source segments (e.g., sentences, paragraphs, chapters) in the source document identified at the source location found in step S402. This embodiment will describe the case where the non-textual information is an image.

[0053] A dictionary represents the relationship between source locations in database 306 and their corresponding representative non-textual information. For example, a dictionary is a JSON file describing the addresses of source locations and their corresponding non-textual information in database 306 in key-value format and is stored in database 306. The "representative non-textual information" corresponding to a source location is, for example, non-textual information associated with that source location (e.g., non-textual information linked in a source document (HTML document)) that has a high semantic similarity to the title of the source document identified by that source location. The dictionary is pre-created and registered in database 306. Product-related knowledge information (information with a low probability of being included in the training data of a large language model) such as printer product manuals can be registered as a dictionary in the database.

[0054] The dictionary can be created manually based on user actions, or it can be created on the device. When creating a dictionary on the device, for example, the device uses a shared embedding space (cross-modal embedding) model for natural language and non-textual information to obtain embeddings for the title and non-textual information. The device then obtains the cosine similarity between the obtained embeddings as the aforementioned similarity.

[0055] In step S404, the decision unit 304 determines the presentation mode of the answer to the question. In this embodiment, the decision unit 304 determines the presentation mode of the answer to the user's question, including a presentation mode for non-textual information, from the viewpoint that the user can easily understand the content of the answer to be presented in subsequent processing. The details of the processing in step S404 will be described later.

[0056] In step S405, the presentation unit 305 causes the presentation device 104 to present the answer to the question according to the presentation mode determined by the decision unit 304. The details of the processing in step S405 will be described later.

[0057] In step S406, CPU 101 determines whether the processing termination condition is met. Various conditions can be applied as processing termination conditions. For example, if the user inputs a processing termination instruction using input device 103, CPU 101 determines that the processing termination condition is met.

[0058] As a result of the judgment, if the processing termination condition is met, then according to... Figure 4 The process of the flowchart ends, and if the process ends, the process returns to step S401.

[0059] Next, we will refer to Figure 6 The flowchart shown describes the details of the processing in step S404. In step S601, the decision unit 304 creates a text prompt. More specifically, the decision unit 304 generates a string as a text prompt by combining a preset template string, a question sent from the tablet computer 3, and "text information related to the answer to the question" generated by the generation unit 302. As an example, the following describes a situation where the template string is "In the following questions and answers, please answer 'yes' or 'no' to indicate whether you are concerned about the appearance of the object."

[0060] Note that the template string is not limited to this. For example, the decision unit 304 can obtain a string representing non-textual information obtained by the acquisition unit 303, and update the template string by inserting the obtained string into the template string. For example, by setting "In the following questions and answers, please answer 'yes' or 'no' to indicate whether you are concerned about the appearance of {non-textual information}" as the template string, replacing the part corresponding to "{non-textual information}" with "the string representing non-textual information obtained by the acquisition unit 303", and then using that template string. Note that as a method for obtaining the string representing non-textual information, methods can be provided such as using alternative text associated with non-textual information in an HTML document identified from the source location, and using an image captioning model to generate a string describing an image as non-textual information.

[0061] In step S602, the decision unit 304 inputs the text prompt generated in step S601 into the large language model 307 and uses the large language model 307 to perform reasoning, thereby obtaining "yes" or "no" as the answer corresponding to the text prompt. Note that in order to obtain the answer corresponding to the text prompt, the decision unit 304 may use a different language model than the large language model 307.

[0062] In step S603, the decision unit 304 determines whether the answer corresponding to the text prompt is "yes" or "no". As a result of the determination, if the answer corresponding to the text prompt is "yes", the process proceeds to step S604; if the answer corresponding to the text prompt is "no", the process proceeds to step S605.

[0063] In step S604, the decision unit 304 decides to "present non-text information" as the presentation mode for non-text information. In step S605, the decision unit 304 decides to "not present non-text information" as the presentation mode for non-text information.

[0064] In step S606, the decision unit 304 determines the "presentation mode of the answer" based on the presentation mode of the non-text information determined in step S604 or S605.

[0065] For example, if the decision unit 304 decides to "present non-text information" as the presentation mode for non-text information, then it decides to "present an answer including non-text information" as the "presentation mode for the answer". On the other hand, if the decision unit 304 decides to "not present non-text information" as the presentation mode for non-text information, then it decides to "present an answer without non-text information" as the "presentation mode for the answer".

[0066] Then, if the decision unit 304 decides to "present an answer including non-text information" as the "presentation mode of the answer", it sends the text information related to the answer generated in step S402 and the non-text information obtained in step S403 to the tablet computer 3 via the communication interface 207.

[0067] On the other hand, if the decision unit 304 decides to "present an answer without non-textual information" as the "answer presentation mode", it sends the text information related to the answer generated in step S402 to the tablet computer 3 via the communication interface 207.

[0068] Next, we will refer to Figure 7A and Figure 7B The description is based on the presentation mode determined by the decision unit 304 (the response presented by the presentation unit 305 using the presentation device 104 in step S405).

[0069] Figure 7A This is a diagram showing an example of the display of the answer screen on the presentation device 104 by the presentation unit 305 when the decision unit 304 decides to "present an answer including non-text information" as the "answer presentation mode".

[0070] In this case, the presentation unit 305 receives “text information related to the answer” and “non-text information” sent from the question-and-answer system 4, generates an answer screen including the received “text information related to the answer” and “non-text information”, and causes the presentation device 104 to display the answer screen.

[0071] exist Figure 7AThe answer screen shown displays a user-side chat bubble 510a representing the user's input question and a system-side chat bubble 520a representing the answer to the question. The system-side chat bubble 520a includes a header 521a and an individual case section 522a. Each individual case included in the individual case section 522a includes statements related to the source content described in the source text segment of the source document identified by the generation unit 302 in the database 306, and corresponding non-text information. The text information and images in the system-side chat bubble 520a correspond to "text information related to the answer" and "non-text information," respectively.

[0072] The strings “[1]” or “[2]” at the end of the statements in each case in the case unit 522a are linked to enable navigation to the source location in the corresponding database 306. Users can select the link using the input device 103 to learn about the details of each case.

[0073] Figure 7B This is a diagram showing an example of the display of the answer screen on the presentation device 104 by the presentation unit 305 when the decision unit 304 decides to "present an answer without non-text information" as the "answer presentation mode".

[0074] In this case, the presentation unit 305 receives the "text information related to the answer" sent from the question and answer system 4, generates an answer screen including the received "text information related to the answer", and causes the presentation device 104 to display the answer screen.

[0075] exist Figure 7B The answer screen shown displays a user-side chat bubble 510b representing the user's input question and a system-side chat bubble 520b representing the answer to the question. The system-side chat bubble 520b includes a header 521b and an individual case section 522b. Each individual case included in the individual case section 522b includes statements related to the source content described in the source document identified by the source location found by the generation unit 302 in the database 306, but not related to... Figure 7A The difference does not include the corresponding non-text information. The text information in the system-side chat bubble 520b corresponds to "text information related to the answer".

[0076] like Figure 7A As shown, if a user's question and the answer to that question focus on the appearance of an object, and the important information in the answer may not be representative of it using only strings, then the answer is presented including images as relevant non-textual information. Therefore, the user can consider the appearance of the important object that is an element of the problem they are facing (…). Figure 7AThe example shown (a stained print image) is compared with images of individual cases in the responses, and it is determined which individual case matches the user's question. Therefore, this determination is easier than when the response only includes text information. In other words, the user does not need to exert effort to make the judgment or investigate additional information for the judgment, thus reducing the user's workload.

[0077] On the other hand, such as Figure 7B As shown, if the user's question and the answer to that question do not focus on the appearance of the object, it is assumed that the important information in the answer can be sufficiently represented by strings alone, and the answer does not include images as relevant non-textual information. Therefore, the answer sufficiently includes information for determining which individual case matches the user's question, and compared to cases where images are also described as answers, the visibility of the answer is improved, thereby reducing the user's workload.

[0078] As described above, according to this embodiment, even if the important information in the answer cannot be represented by a string alone, the user can easily understand the content of the answer by presenting non-textual information (which is the important information in the answer), thereby reducing the user's effort. (Modification 1 of the first embodiment)

[0079] In the first embodiment, the decision unit 304 determines the presentation mode of the answer based on the user's question and text information related to the answer. However, the method for determining the presentation mode of the answer is not limited to a specific method.

[0080] For example, the decision unit 304 can determine the presentation mode of the answer based on the user's question and a preset template string. In this case, for example, in step S601, the decision unit 304 generates a string as a text prompt by combining the preset template string with the question sent from the tablet computer 3. The template string is, for example, "In the following question, please answer 'yes' or 'no' to indicate whether you are concerned about the appearance of the object."

[0081] Alternatively, the decision unit 304 can determine the presentation mode of the answer based on the "text information related to the answer" generated by the generation unit 302 and a preset template string. In this case, for example, in step S601, a string is generated as a text prompt by combining the preset template string and the "text information related to the answer" generated by the generation unit 302. The template string is, for example, "In the following answer, please answer 'yes' or 'no' to indicate whether you are concerned about the appearance of the object."

[0082] In cases where at least one of the text information from the user's question and answer is a long sentence, as described above, the computational cost of processing using a large language model by the decision unit 304 can be reduced by using the one with fewer characters in the question and answer and a preset template string to determine the presentation mode of the answer.

[0083] As mentioned above, regardless of which method is used, it is possible to determine whether the important information in the response can be represented by strings alone, and to appropriately decide whether to include non-textual information as the presentation mode of the response. (Modification 2 of the first embodiment)

[0084] In the first embodiment, the decision unit 304 uses a large language model to determine the presentation mode of the response. However, the method for determining the presentation mode of the response is not limited to a specific method.

[0085] For example, a learned machine learning model can be used instead of a large language model to determine the presentation pattern of the answer. Consider algorithms such as LSTM or BERT as the machine learning model to be used. In this case, hypothetical questions and answers are prepared as input, and the judgment result of the presentation pattern is prepared as output, thus being used for learning by the machine learning model.

[0086] Alternatively, it can be determined whether the input statement contains predetermined keywords. Predetermined keywords include, for example, "image," "picture," and "photo." This determination can be made by using the above methods in combination. By using the above methods, processing time can be reduced compared to processing using a large language model.

[0087] Note that regardless of which of the above methods is used, the decision unit 304 can determine whether the important information in the response can be represented by a string alone, and appropriately decide whether to include non-text information as the presentation mode of the response. (Modification 3 of the first embodiment)

[0088] In the first embodiment, the decision unit 304 determines whether to present an image as non-textual information as an answer by judging whether the user's question and the answer to the question focus on the appearance of an object, but this disclosure is not limited thereto. This modified example will describe a method in which the decision unit 304 judges whether the user's question and the answer to the question focus on the fine appearance of an object, and determines a presentation mode to change the resolution (display resolution) when displaying the image as non-textual information presented as an answer. This allows the user to more easily understand the content of the presented answer.

[0089] The decision unit 304 determines whether each of the user's question and the text information related to the answer includes a predetermined keyword related to fine appearance. Predetermined keywords include, for example, "dots," "stripes," and "fine." If at least one of the user's question and the text information related to the answer includes a predetermined keyword related to fine appearance, the decision unit 304 determines the presentation mode of the non-text information to increase the display resolution of the non-text information (the image presented as an answer) (e.g., higher than the default display resolution). On the other hand, if neither the user's question nor the text information related to the answer includes a predetermined keyword related to fine appearance, the decision unit 304 determines the presentation mode of the non-text information to decrease the display resolution of the non-text information (the image presented as an answer) (e.g., lower than the default display resolution).

[0090] In this method, if at least one of the user's question and the text information related to the answer includes predetermined keywords related to fine details, then the fine details are considered important to the user's judgment. Therefore, the tablet 3 displays the image along with the individual cases presented as answers at a higher resolution. Consequently, the user can also focus on the finer details of the presented image, and it becomes easier for the user to determine which individual case should be focused on, thus reducing effort.

[0091] On the other hand, if neither the user's question nor the text information related to the answer includes predetermined keywords related to detailed appearance, then the detailed appearance is considered unimportant to the user's judgment. Therefore, the tablet 3 displays the image along with the individual cases presented as the answer at a relatively low resolution. Compared to displaying the image at high resolution, this reduces the amount of data in the image presented as the answer and allows the answer to be presented with low latency, thereby reducing the user's response waiting time. (Modification 4 of the first embodiment)

[0092] In the first embodiment, the acquisition unit 303 uses a dictionary registered in the database 306 to acquire representative non-textual information associated with the source location, but the method for acquiring representative non-textual information is not limited to a specific method.

[0093] For example, the acquisition unit 303 can acquire the first non-text information that appears when counting from the beginning in the non-text information associated with the source location (e.g., non-text information linked in the source document (HTML document)).

[0094] If the source location represents a portion of the document data in database 306, then the acquisition unit 303 can acquire the associated non-text information in the source location, or the non-text information that is closest to the source location in terms of the number of characters or lines.

[0095] Alternatively, the acquisition unit 303 can calculate the similarity between text information related to the source location and each piece of non-text information, and then acquire the non-text information with the highest similarity. The text information related to the source location may be, for example, the text information of the entire source document or a string containing the title of the document data of the source location.

[0096] Alternatively, the acquisition unit 303 may calculate the similarity between textual information and each non-textual information related to the answer, and then acquire the non-textual information with the highest similarity. As a similarity measure, for example, cosine similarity may be used between embeddings transformed by a multimodal model utilizing the modality of non-textual information (e.g., images) and the modality of language.

[0097] If non-textual captions exist, the acquisition unit 303 can calculate the similarity between the textual information related to the answer and the captions of each non-textual information, and then acquire the non-textual information with the highest similarity. For example, this similarity can be achieved using the cosine similarity between embeddings obtained by transforming the textual information using a language model.

[0098] Regardless of which of the above methods is used, the acquisition unit 303 can appropriately acquire the non-textual information to be presented as an answer to the question, thereby achieving the same effect as in the first embodiment. (Modification 5 of the first embodiment)

[0099] The first embodiment has explained the case where the non-text information is an image, but non-text information is not limited to images. The case where the non-text information is sound will be described below. The differences from the first embodiment will be described below.

[0100] For example, suppose in step S401, the question acquisition unit 301 acquires the question "What are the possible causes and remedies for the abnormal noise from unit ABC?". In this case, in step S402, the generation unit 302, as in the first embodiment, uses the question and the source document corresponding to the question to acquire text information related to the answer. In step S403, the acquisition unit 303 acquires the audio (audio file) associated with the source text segment corresponding to the source position in the source document as non-text information.

[0101] Then, the decision unit 304 generates a text prompt by using a template string related to sound (e.g., “In the following questions and answers, please answer ‘yes’ or ‘no’ to indicate whether you are interested in sound”) and performs the same processing as in step S601.

[0102] In step S602, the decision unit 304 inputs the text prompt generated in step S601 into the large language model 307 and uses the large language model 307 to perform reasoning, thereby obtaining "yes" or "no" as the answer corresponding to the text prompt. That is, the decision unit 304 determines whether the question and answer focus on sound based on the text prompt.

[0103] In step S405, the presentation unit 305 presents the answer to the question to the user via the presentation device 104 according to the presentation mode determined by the decision unit 304. Figure 8 This is a diagram showing an example of the display of the answer screen on the presentation device 104 by the presentation unit 305 when the decision unit 304 decides to "present an answer including non-text information" as the "answer presentation mode".

[0104] In this case, the presentation unit 305 receives “text information related to the answer” and “non-text information” sent from the question-and-answer system 4, generates an answer screen including the received “text information related to the answer” and an interface for playing back the received “non-text information”, and causes the presentation device 104 to display the answer screen.

[0105] exist Figure 8 The answer screen shown displays a user-side chat bubble 510c representing the user's input question and a system-side chat bubble 520c representing the answer to the question. The system-side chat bubble 520c includes a header 521c and an individual case section 522c. Each individual case included in the individual case section 522c includes statements related to the source content described in the source text segment of the source document identified by the generation unit 302 in the database 306, and an interface for replaying the corresponding non-text information. The text information and interface in the system-side chat bubble 520c correspond to "text information related to the answer" and "interface for replaying non-text information," respectively.

[0106] When a user inputs a non-textual instruction to play back using the operation interface of the input device 103, the presentation unit 305 plays back the non-textual instruction and outputs the corresponding sound via the sound output device (such as a speaker or headphones) of the presentation device 104.

[0107] As mentioned above, if a user's question and the answer to that question focus on sound and the important information in the answer may not be able to be represented by strings alone, then the answer includes the relevant sound.

[0108] Using the above method, when sound is important information in the answer, the user can listen to the sound related to the answer, and thus the user does not need to make a judgment or investigate additional information for the judgment, thereby reducing the user's labor. (Modification 6 of the first embodiment)

[0109] The following describes the case where the non-textual information is a moving image. The differences from the first embodiment will be described below. For example, suppose that in step S401, the question acquisition unit 301 acquires the question "How to replace unit ABC?". In this case, in step S402, the generation unit 302, as in the first embodiment, uses the question and the source document corresponding to the question to acquire textual information related to the answer. In step S403, the acquisition unit 303 acquires a moving image (moving image file) associated with the source location of the source document as non-textual information.

[0110] Then, the decision unit 304 generates a text prompt by using a template string related to the motion image (e.g., "In the following questions and answers, please answer 'yes' or 'no' to indicate whether a motion image should exist to support the answer") as a template string and performing the same processing as in step S601.

[0111] In step S602, the decision unit 304 inputs the text prompt generated in step S601 into the large language model 307 and uses the large language model 307 to perform reasoning, thereby obtaining "yes" or "no" as the answer corresponding to the text prompt. That is, the decision unit 304 determines whether the question and answer focus on the moving image based on the text prompt.

[0112] In step S405, the presentation unit 305 presents the answer to the question to the user via the presentation device 104 according to the presentation mode determined by the decision unit 304. When the decision unit 304 determines "presenting an answer including non-text information" as the "answer presentation mode," the answer screen displayed by the presentation unit 305 on the presentation device 104, in addition to displaying moving images and an interface for reproducing moving images instead of images, is... Figure 7A The answer screen shown is the same.

[0113] When a user inputs a motion image reproduction instruction through the operation interface of the input device 103, the presentation unit 305 reproduces the motion image and displays it on the response screen of the presentation device 104.

[0114] As mentioned above, if a user's question and the answer to that question focus on moving images, and the key information in the answer may not be representative of a string alone, then the answer includes the relevant moving images. Therefore, the user does not need to make a judgment or investigate additional information for that judgment, thus reducing the user's workload.

[0115] Note that when the non-text information is a moving image, the decision unit 304 can use the template string "In the following questions and answers, please answer 'yes' or 'no' to indicate whether thumbnail images from the moving images used to support the answer are sufficient" as a template string for generating the text prompt. In this case, the acquisition unit 303 can acquire a thumbnail image of any frame from the acquired moving images as non-text information. This allows for displaying only the image, as in the first embodiment, thus providing a concise answer.

[0116] Similarly, as a template string for creating text prompts, the template string "In the following questions and answers, please answer 'yes' or 'no'" can be used to indicate whether sound information from the moving image alone is sufficient to support the answer. In this case, the acquisition unit 303 can acquire the sound from the moving image as non-text information. This can present an answer including sound, as in Variation 5 of the first embodiment, thereby providing a concise answer to the user. (Modification 7 of the first embodiment)

[0117] In the first embodiment, the question acquisition unit 301 acquires the question input by the user into the input box 501 on the screen 500 using the input device 103. However, the method for acquiring the question is not limited to a specific method. For example, the question acquisition unit 301 can acquire the speech emitted by the user via the microphone provided in the input device 103, and acquire the string obtained by transcribing the acquired speech as the question. In this method, the same effect as in the first embodiment can be achieved. (Modification 8 of the first embodiment)

[0118] In the first embodiment, in order to generate text information related to the answer, the generation unit 302 uses the method described in P. Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," Advances in Neural Information Processing Systems, vol. 33, pp. 9459-9474, 2020. However, the method for generating text information related to the answer is not limited to a specific method.

[0119] For example, generation unit 302 uses the user question obtained by question acquisition unit 301 (without using the text information in database 306) to generate a text prompt. Then, generation unit 302 inputs the generated text prompt into a large language model and uses the large language model to perform reasoning, thereby generating an output corresponding to the text prompt as text information related to the answer.

[0120] In this case, the generation unit 302 does not acquire the source location information in the database 306 required by the acquisition unit 303 to acquire non-textual information in the first embodiment. Therefore, the acquisition unit 303 needs to additionally search the database 306 based on the user's question and the textual information related to the answer generated by the generation unit 302 to acquire non-textual information related to the question and answer.

[0121] When the text information in database 306, which serves as the answer to a user's question, is not necessary, the method described above can be used to reduce the processing time required to retrieve data from database 306 and suppress the degradation of answer accuracy caused by inputting unnecessary data from database 306 into a large language model. The same effect as in the first embodiment can be achieved in this method. (Second Embodiment)

[0122] In the following embodiments and variations thereof, the differences from the first embodiment will be described, and unless otherwise specifically stated, the rest are the same as the first embodiment.

[0123] In the first embodiment, the decision unit 304 determines the presentation mode of the answer based on the importance of non-textual information in the content of the answer to be presented. In this embodiment, the decision unit 304 determines the presentation mode of the answer, including the presentation mode of non-textual information, from the viewpoint of whether the answer to be presented is easy to see.

[0124] In this embodiment, the processing performed by the tablet computer 3 and the question-and-answer system 4 to create an answer to a question entered by the user and present it to the user includes... Figure 4 Except for the process in step S404 of the flowchart, it is the same as in the first embodiment. The process in step S404 according to this embodiment will be described below.

[0125] The decision unit 304 counts the number of characters in the text information related to the answer generated by the generation unit 302. Then, if the counted number of characters is less than a threshold, the decision unit 304 determines the presentation mode of the non-text information so as to present the image, which is to be presented as the answer, at a larger display size (e.g., a display size larger than the default display size). On the other hand, if the counted number of characters is equal to or greater than the threshold, the decision unit 304 determines the presentation mode of the non-text information so as to present the image, which is to be presented as the answer, at a smaller display size (e.g., a display size smaller than the default display size).

[0126] Therefore, in step S405, the tablet computer 3 displays the non-text information at a display size corresponding to the presentation mode of the non-text information. (Refer to...) Figure 9A and Figure 9B The description is based on the presentation mode determined by the decision unit 304 (the response presented by the presentation unit 305 in step S405).

[0127] Figure 9A This is a diagram showing an example of a response screen displayed on the presentation device 104 by the presentation unit 305 when the decision unit 304 decides to "present an image as non-text information at a larger display size".

[0128] exist Figure 9A The answer screen shown displays a user-side chat bubble 510e representing the user's input question and a system-side chat bubble 520e representing the answer to the question. The system-side chat bubble 520e includes a header 521e and an individual case section 522e. Each individual case included in the individual case section 522e includes statements related to the source content described in the source document identified by the generation unit 302 in the database 306, and corresponding non-text information. The text information and images in the system-side chat bubble 520e correspond to "text information related to the answer" and "non-text information," respectively.

[0129] exist Figure 9A In this context, since the presentation mode for non-text information is determined to be "presenting images as non-text information at a larger display size," the images as non-text information are displayed at a larger display size on the presentation device 104. For example... Figure 9A As shown, if the number of characters in the answer is small, the characters and images in the answer can be made easier to see by increasing the display size of the image.

[0130] Figure 9BThis is a diagram showing an example of a response screen displayed on the presentation device 104 by the presentation unit 305 when the decision unit 304 decides to "present an image as non-text information in a smaller size".

[0131] exist Figure 9B The answer screen shown displays a user-side chat bubble 510f representing the user's input question and a system-side chat bubble 520f representing the answer to the question. The system-side chat bubble 520f includes a header 521f and an individual case section 522f. Each individual case included in the individual case section 522f includes statements related to the source content described in the source document identified by the generation unit 302 in the database 306, and corresponding non-text information. The text information and images in the system-side chat bubble 520f correspond to "text information related to the answer" and "non-text information," respectively.

[0132] exist Figure 9B In this context, because the presentation mode for non-text information is determined to be "presenting images as non-text information at a smaller display size," images as non-text information are displayed on the presentation device 104 at a smaller display size. For example... Figure 9B As shown, if the answer contains a large number of characters, the visibility of the entire answer is improved by reducing the display size of the image, thereby maintaining the answer's easy visibility. When the answer contains a large number of characters, if the display size of the image is large, the entire answer extends significantly beyond the screen and is difficult to visually discern, causing the user to spend time scrolling through the screen and making judgments based on the answer. In this embodiment, labor can be reduced.

[0133] In the method according to this embodiment, by changing the display size of the image, which is presented as non-text information, based on the number of characters in the answer to the question, the answer can be presented while maintaining visibility for the user. (Modification 1 of the second embodiment)

[0134] In this variation, the decision unit 304 determines whether to present non-textual information as an answer based on the number of individual cases in the individual case section 522 of the text information of the answer generated by the generation unit 302.

[0135] For example, the decision unit 304 decides not to present non-textual information as a non-textual information presentation mode for the answer if the number of individual cases is equal to or greater than a threshold (e.g., 10), and decides to present non-textual information if the number of individual cases is less than the threshold.

[0136] As mentioned above, when the number of individual cases is large, if non-textual information is listed together, it becomes difficult for users to visually distinguish the answers, and therefore non-textual information is not presented. Therefore, presenting only textual information maintains visibility for the user, thereby reducing the effort required for judgment. Note that this disclosure is not limited to this, and when the number of individual cases is equal to or greater than a threshold (e.g., 10), up to the threshold amount of non-textual information can be presented. (Modification 2 of the second embodiment)

[0137] In the second embodiment, the display size of the image presented as non-text information in the response to the question is adjusted based on the number of characters in the response. However, the display size of the image presented as non-text information in the response can be adjusted based on the amount of non-text information obtained.

[0138] For example, if the amount of non-text information acquired is equal to or greater than a threshold (e.g., four), the decision unit 304 decides to "display the image as non-text information at a smaller display size" as the presentation mode for non-text information. On the other hand, if the amount of non-text information acquired is less than a threshold, the decision unit 304 decides to "display the image as non-text information at a larger display size" as the presentation mode for non-text information.

[0139] Therefore, when the amount of non-text information is large, if the non-text information is displayed on a large screen, it will be difficult for users to visually distinguish the answer. Therefore, by presenting the non-text information on a small screen, the overall visibility of the answer can be maintained for the user, and the effort required for the user to make a judgment can be reduced. (Third Embodiment)

[0140] In this embodiment, when there are multiple cases presented as answers, the presentation mode of the answer, which includes a presentation mode of non-textual information, is determined from the viewpoint of whether it is easy to identify individual cases.

[0141] In this embodiment, the processing performed by the tablet computer 3 and the question-and-answer system 4 to create an answer to a question entered by the user and present it to the user includes... Figure 4 Except for the process in step S404 of the flowchart, it is the same as in the first embodiment. The process in step S404 according to this embodiment will be described below.

[0142] The decision unit 304 acquires text information related to the individual case from the text information related to the answer generated by the generation unit 302, and obtains the similarity between the acquired text information of the individual cases. The method for obtaining the similarity between the text information of the individual cases is not limited to a specific method. For example, the decision unit 304 acquires the cosine similarity between embeddings obtained by inputting the text information of the individual cases into a language model and using that language model for reasoning.

[0143] Then, if the highest similarity among the obtained similarities is equal to or higher than a threshold (e.g., 0.8), the decision unit 304 determines the presentation mode to present an image as non-textual information as the response. On the other hand, if the highest similarity is lower than the threshold, the decision unit 304 determines the presentation mode to not present an image as non-textual information as the response.

[0144] Reference Figure 10A and Figure 10B The description is based on the presentation mode determined by the decision unit 304 (the response presented by the presentation unit 305 in step S405). Figure 10A This is a diagram showing an example of the display of the answer screen on the presentation device 104 by the presentation unit 305 when the decision unit 304 decides to "present an answer including non-text information" as the "answer presentation mode".

[0145] exist Figure 10A The answer screen shown displays a user-side chat bubble 510g representing the user's input question and a system-side chat bubble 520g representing the answer to the question. The system-side chat bubble 520g includes a header 521g and an individual case section 522g. Each individual case included in the individual case section 522g includes statements related to the source content described in the source document identified by the generation unit 302 in the database 306, and corresponding non-text information. The text information and images in the system-side chat bubble 520g correspond to "text information related to the answer" and "non-text information," respectively.

[0146] Figure 10B This is a diagram showing an example of the display of the answer screen on the presentation device 104 by the presentation unit 305 when the decision unit 304 decides to "present an answer without non-text information" as the "answer presentation mode".

[0147] exist Figure 10BThe answer screen shown displays a user-side chat bubble 510h representing the user's input question and a system-side chat bubble 520h representing the answer to the question. The system-side chat bubble 520h includes a header 521h and an individual case section 522h. Each individual case included in the individual case section 522h includes statements related to the source content described in the source document identified by the source location found by the generation unit 302 in the database 306, but not related to... Figure 10A The difference does not include the corresponding non-text information. The text information in the system-side chat bubble 520h corresponds to "text information related to the answer".

[0148] like Figure 10A As shown, if the textual information of individual cases is highly similar, then when an image, which is presented as non-textual information, is presented, although it is difficult to identify which individual case is the desired individual case based solely on the textual information, the user can identify it by confirming the image. On the other hand, as... Figure 10B As shown, if the textual information of individual cases has low similarity, users can identify which individual case is the desired individual case based solely on the textual information without presenting an image. In this case, the visibility of the entire answer can be maintained by not presenting an image.

[0149] According to this embodiment, a decision is made on whether to present non-textual information as an answer based on the similarity between the textual information of individual cases in the answer, thereby presenting the answer while maintaining the identifiability between individual cases in the answer. (Modification 1 of the third embodiment)

[0150] In this variation, the size of the image presented as non-textual information to be used as a response is changed based on the similarity between the non-textual information. For example, the decision unit 304 obtains the similarity between the non-textual information obtained by the acquisition unit 303. Methods for obtaining the similarity between images or between sounds are known, regardless of whether the non-textual information is an image or sound, and in this embodiment, the method for obtaining the similarity between non-textual information is not limited to a specific method.

[0151] Then, if the highest similarity among the obtained similarities is equal to or higher than a threshold (e.g., 0.8), the decision unit 304 determines the presentation mode of the non-text information, thereby presenting the image as non-text information to be presented as an answer at a larger display size (e.g., a display size larger than the default display size). On the other hand, if the highest similarity among the obtained similarities is lower than the threshold, the decision unit 304 determines the presentation mode of the non-text information, thereby presenting the image as non-text information to be presented as an answer at a smaller display size (e.g., a display size smaller than the default display size). Thus, in step S405, the tablet computer 3 displays the non-text information at a display size corresponding to the presentation mode of the non-text information.

[0152] Therefore, when images that are non-textual information are highly similar, users may have difficulty recognizing them if the presented images are small. Thus, user recognizability can be maintained by increasing the size of the presented images. (Fourth Embodiment)

[0153] In the first to third embodiments, the decision unit 304 determines the presentation mode of the answer from different viewpoints. In this embodiment, the presentation mode of the answer is determined based on all viewpoints of the decision unit 304 according to the first to third embodiments. The decision unit 304 uses a machine learning model to obtain a judgment score (0 to 1) from each viewpoint of the decision unit 304 according to the first to third embodiments.

[0154] For example, the decision unit 304 obtains a score by inputting the likelihood of the answer obtained in step S602 of the first embodiment into the machine learning model, outputting the likelihood from the large language model 307, and using the machine learning model to reason, as a first judgment score corresponding to the viewpoint of the decision unit 304 of the first embodiment.

[0155] The decision unit 304 obtains a score by inputting the number of characters in the text information related to the answer generated by the generation unit 302 into the machine learning model and using the machine learning model to perform reasoning, as a second judgment score corresponding to the viewpoint of the decision unit 304 in the second embodiment.

[0156] The decision unit 304 obtains the score obtained by inputting the highest similarity into the machine learning model and using the machine learning model for reasoning, as a third judgment score corresponding to the viewpoint of the decision unit 304 in the third embodiment.

[0157] Then, the decision unit 304 obtains the sum or weighted linear sum of the first judgment score, the second judgment score, and the third judgment score as the total judgment score. If the total judgment score is equal to or greater than a threshold (e.g., 0.8), the decision unit 304 decides to present non-textual information as the answer, and if the total judgment score is less than the threshold, the decision unit 304 decides not to present non-textual information as the answer.

[0158] Furthermore, the decision unit 304 can generate a text prompt by combining a question from the user, text information related to the answer generated by the generation unit 302, and a string representing a judgment made from the perspective of the decision unit 304 according to the first to third embodiments. In this case, the decision unit 304 obtains a presentation mode for whether to present non-text information by inputting the text prompt into a large language model and using the large language model to perform reasoning.

[0159] The above method can be considered from all perspectives, including "whether the user can easily understand the content of the answer to be presented", "whether the user can easily visually identify the answer to be presented", and "whether the individual cases to be presented as answers are easy to identify", to appropriately decide whether to present non-textual information, thereby reducing the effort users need to put into judging whether the answer is the expected answer. (Fifth Embodiment)

[0160] The use cases of the systems described above in the embodiments and variations are merely examples of troubleshooting, and the systems described above can be applied to other troubleshooting scenarios. The use cases of the systems are not limited to troubleshooting.

[0161] The tablet computer 3 and the question-and-answer system 4 can be located in a different location than the damaged printer 1, for example, at the repair shop. In other words, the arrangement of the tablet computer 3 and the question-and-answer system 4 can be varied depending on the system application and other factors.

[0162] The above embodiments and variations have explained the case where the tablet computer 3 and the question-and-answer system 4 are separate devices. This disclosure is not limited thereto, and for example, an information processing device can be formed that integrates the tablet computer 3 and the question-and-answer system 4. In this case, the information processing device performs or controls various kinds of processing (other than inter-device communication processing) described as being performed by each of the tablet computer 3 and the question-and-answer system 4.

[0163] The above embodiments and modifications have explained the cases where the tablet computer 3 includes a question acquisition unit 301 and a presentation unit 305, and the question-and-answer system 4 includes a generation unit 302, an acquisition unit 303, a decision unit 304, a database 306, and a large language model 307. However, this disclosure is not limited thereto. For example, the tablet computer 3 may include a decision unit 304. The database 306 and the large language model 307 may be located in the tablet computer 3. Additionally, Figure 3 One or more of the functional units shown can be implemented in hardware.

[0164] The numerical values, processing timing, processing order, main components of processing, and data (information) configuration / acquisition methods / sending destinations / sending sources / storage locations used in the above embodiments are merely examples for detailed description and are not intended to be limited to these examples.

[0165] Some or all of the above embodiments can be used in combination as appropriate. Alternatively, some or all of the above embodiments can be used selectively. Other embodiments

[0166] Embodiments of the present invention can also be implemented by providing software (including computer program products of computer programs) that performs the functions of the above embodiments to a system or device via a network or various storage media, and the computer (central processing unit (CPU) or microprocessor unit (MPU) of the system or device) reads and executes the computer program.

[0167] While this disclosure has been described with reference to embodiments, it should be understood that this disclosure is not limited to the disclosed embodiments. The scope of the appended claims should be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

Claims

1. An information processing device, comprising: The acquisition unit is configured to use a large language model to acquire textual information related to the answer to the question based on the input question; A decision unit is configured to determine whether to present non-text information based on the text information; as well as The sending unit is configured to send the text information and the non-text information if it is determined that the non-text information should be presented, and to send the text information if it is determined that the non-text information should not be presented.

2. The device according to claim 1, wherein, The acquisition unit obtains text information related to the answer to the question based on the question and a dictionary registered in the database corresponding to the question.

3. The device according to claim 1, wherein, The decision-making unit obtains the non-text information based on the question and the text information.

4. The device according to claim 1, wherein, The decision unit generates a text prompt that includes the question and / or the text information, and decides whether to present the non-text information based on the text prompt.

5. The device according to claim 1, wherein, The decision unit generates a text prompt that includes either the question or the text information, but with fewer characters, and decides whether to present the non-text information based on the text prompt.

6. The device according to claim 1, wherein, The decision-making unit determines whether to present the non-textual information based on the number of individual cases in the text information.

7. The device according to claim 1, wherein, The decision unit determines whether to present the non-text information based on the similarity between the text information of individual cases in the text information.

8. The device according to claim 1, wherein, The decision unit determines the display resolution of the non-text information based on whether the question and / or the text information includes keywords related to fine appearance.

9. The device according to claim 1, wherein, The decision unit determines the display size of the non-text information based on the number of characters in the text information.

10. The device according to claim 1, wherein, The decision unit determines the display size of the non-text information based on the amount of non-text information.

11. The device according to claim 1, wherein, The decision unit determines the display size of the non-text information based on the similarity between the non-text information.

12. The device according to claim 1, wherein, The non-textual information includes one of the following: image, sound, moving image, thumbnail image of a frame of moving image, and sound in moving image.

13. An information processing device, comprising: The acquisition unit is configured to use a large language model to acquire textual information related to the answer to the question based on the input question; A decision unit is configured to determine whether to present non-text information based on the text information; as well as A presentation unit is configured to present both the text information and the non-text information when it is decided to present the non-text information, and to present the text information when it is decided not to present the non-text information.

14. An information processing method, executed by an information processing device, the information processing method comprising: A large language model is used to obtain textual information related to the answer to the question based on the input question; Whether to present non-text information is determined based on the text information; as well as If it is decided to present the non-text information, both the text information and the non-text information are sent; if it is decided not to present the non-text information, the text information is sent.

15. An information processing method, executed by an information processing device, the information processing method comprising: A large language model is used to obtain textual information related to the answer to the question based on the input question; Whether to present non-text information is determined based on the text information; as well as If it is decided to present the non-text information, both the text information and the non-text information are presented; if it is decided not to present the non-text information, the text information is presented.

16. A computer-readable storage medium storing a computer program that, when executed by a computer, implements the steps of the method according to claim 14 or 15.

17. A computer program product comprising a computer program that, when executed by a computer, implements the steps of the method according to claim 14 or 15.

Images