Information processing device, information processing method, and program
The information processing device and method enhance the reliability assessment of language model outputs by incorporating source information, addressing the challenge of verifying response correctness in existing technologies.
Patent Information
- Application Number
- PCT/JP2023/017860
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-21
AI Technical Summary
Existing language processing technologies, such as those described in Patent Document 1, do not provide a means to verify the correctness of responses generated by external search engines, making it difficult for users to assess the reliability of the output.
An information processing device and method that utilizes a language model to generate text, extract queries from the text, search for relevant documents, and add identifying information to the output, allowing users to easily judge the reliability of the language model's output by providing source information for the generated text.
Enables users to assess the reliability of language model outputs by providing source information, thereby improving trust in the generated responses.
Smart Images

Figure JP2023017860_21112024_PF_FP_ABST
Abstract
Description
Information processing device, information processing method, and program
[0001] The present invention relates to an information processing device, an information processing method, and a program for performing language processing on text.
[0002] A technology using a language model that performs language processing and is trained using text data is known. Patent Literature 1 discloses a question answering device that generates a response to a question based on entities extracted from the question and a perplexity value calculated from the question using a language model that assumes questions in a specific field. The question answering device divides the text that is the question into morphemes, which are the smallest meaningful units, and requests a keyword search from an external search engine using nouns among the words that are the divided morphemes as keywords. The question answering device also regards the response from the external search engine as an answer to the input question.
[0003] Japanese Patent Application Publication No. 2015-87796
[0004] The question answering device described in Patent Document 1 does not verify whether the response from the external search engine is correct, so the user cannot determine whether the response to the question is correct.
[0005] One aspect of the present invention has been made in view of the above-mentioned problems, and one example of a purpose thereof is to provide a technique that makes it easier to determine the reliability of the output of a language model.
[0006] An information processing device according to one aspect of the present invention comprises an acquisition means for acquiring target text, a generation means for generating text corresponding to the acquired target text using a language model trained to generate text based on input text, a query generation means for generating a query from the text generated by the generation means, an extraction means for extracting documents related to the query from a database by a search process using the query, and an output means for outputting a result in which information identifying the document is added to the text generated by the generation means.
[0007] An information processing method according to one aspect of the present invention includes at least one processor acquiring target text, generating text corresponding to the target text using a language model trained to generate text based on input text, generating a query from the generated text, extracting documents related to the query from a database by a search process using the query, and outputting a result in which information identifying the document is added to the generated text.
[0008] A program according to one aspect of the present invention causes a computer to execute an acquisition process for acquiring target text, a generation process for generating text corresponding to the target text using a language model trained to generate text based on input text, a query generation process for generating a query from the text generated in the generation process, an extraction process for extracting documents related to the query from a database by a search process using the query, and an output process for outputting the results of adding information identifying the document to the text generated in the generation process.
[0009] According to one aspect of the present invention, it is possible to make it easier to determine the reliability of the output of a language model.
[0010] FIG. 1 is a block diagram showing the configuration of an information processing device according to exemplary embodiment 1. FIG. 2 is a flow diagram showing the flow of an information processing method according to exemplary embodiment 1. FIG. 3 is a block diagram showing the configuration of an information processing device according to exemplary embodiment 2. FIG. 4 is a flow diagram showing the flow of an information processing method according to exemplary embodiment 2. FIG. 5 is a diagram showing examples of each process executed by an information processing device according to exemplary embodiment 2. FIG. 6 is a diagram showing a specific example of a second result output by an information processing device according to exemplary embodiment 2. FIG. 7 is a block diagram showing the configuration of an information processing system according to exemplary embodiment 3. FIG. 8 is a diagram showing an example of a document in exemplary embodiment 3 of the present invention. FIG. 9 is a diagram showing example information related to the author of a document in exemplary embodiment 3 of the present invention. FIG. 10 is a flowchart showing the flow of an information processing method according to exemplary embodiment 3 of the present invention. FIG. 11 is a diagram showing an example of an image displayed by an information processing terminal according to exemplary embodiment 3 of the present invention. FIG. 12 is a block diagram showing an example of the hardware configuration of an information processing device according to each exemplary embodiment of the present invention.
[0011] [First Exemplary Embodiment] A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.
[0012] (Configuration of Information Processing Apparatus) The configuration of an information processing apparatus 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing apparatus 1. The information processing apparatus 1 includes an acquisition unit 11 (acquisition means), a generation unit 12 (generation means), a query generation unit 13 (query generation means), an extraction unit 14 (extraction means), and an output unit 15 (output means).
[0013] The acquisition unit 11 acquires target text. The generation unit 12 generates text corresponding to the target text acquired by the acquisition unit 11 using a language model trained to generate text based on input text. The query generation unit 13 generates a query from the text generated by the generation unit 12. The extraction unit 14 extracts documents related to the query from a database by a search process using the query. The output unit 15 outputs a result in which information identifying the document is added to the text generated by the generation unit 12.
[0014] As described above, the information processing device 1 according to this exemplary embodiment has a configuration including an acquisition unit 11 that acquires target text, a generation unit 12 that generates text corresponding to the target text acquired by the acquisition unit 11 using a language model trained to generate text based on input text, a query generation unit 13 that generates a query from the text generated by the generation unit 12, an extraction unit 14 that extracts documents related to the query from a database by a search process using the query, and an output unit 15 that outputs a result in which information identifying the document is added to the text generated by the generation unit 12. Therefore, the information processing device 1 according to this exemplary embodiment has the effect of making it easier to determine the reliability of the output of the language model.
[0015] (Program) The functions of the information processing device 1 described above can also be realized by a program. The program according to this exemplary embodiment causes a computer to execute an acquisition process for acquiring target text, a generation process for generating text corresponding to the target text using a language model trained to generate text based on input text, a query generation process for generating a query from the text generated in the generation process, an extraction process for extracting documents related to the query from a database by a search process using the query, and an output process for outputting a result in which information identifying the document is added to the text generated in the generation process. This program makes it easier to determine the reliability of the output of the language model.
[0016] (Flow of Information Processing Method) The flow of the information processing method S1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the information processing method S1. Note that the execution entity of each step in this information processing method may be a processor included in the information processing device 1, or may be a processor included in another device.
[0017] In S11, at least one processor acquires target text. In S12, at least one processor generates text corresponding to the target text using a language model trained to generate text based on input text. In S13, at least one processor generates a query from the text generated in S12. In S14, at least one processor performs a search process using the query to extract documents related to the query from a database. In S15, at least one processor outputs a result in which information identifying the document is added to the text generated in S12.
[0018] As described above, the information processing method S1 according to this exemplary embodiment includes, by at least one processor, acquiring target text, generating text corresponding to the target text using a language model trained to generate text based on input text, generating a query from the generated text, extracting documents related to the query from a database by a search process using the query, and outputting a result in which information identifying the document is added to the generated text. Therefore, the information processing method S1 according to this exemplary embodiment has the effect of making it easier to determine the reliability of the output of the language model.
[0019]
[0033] A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are denoted by the same reference numerals, and their description will be omitted as appropriate.
[0020] (Overview of Information Processing Device 2) The information processing device 2 according to this exemplary embodiment is a device that acquires a target text and outputs the results (text, etc.) of performing language processing on the target text. The results output by the information processing device 2 also include information that identifies documents related to the results. Documents will be described later. Note that, hereinafter, "target text" and "text" are also referred to as "character strings."
[0021] The information processing device 2 performs language processing using a language model M trained using text data. The language model M receives a character string (text) as input and outputs the result of performing language processing on the character string. The result output from the language model M may be a character string or an image. The language processing performed by the language model M is not particularly limited, and examples include processing for generating text based on input text, text classification, sentiment analysis, information extraction, sentence summarization, text generation, image generation, and question answering.
[0022] The language model M will now be described in detail. The language model M is a model that learns the relationships between words in sentences (text data) and generates related strings related to a target string from the target string. By using a language model that has learned sentences and paragraphs from various contexts, it is possible to generate related strings with appropriate content related to the target string.
[0023] Examples of the language model M include, but are not limited to, large language models (LLMs) such as BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer), T5 (Text-to-Text Transfer Transformer), RoBERTa (Robustly optimized BERT approach), and ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately), as well as learning models generated by transfer learning or fine-tuning using a pre-trained model (e.g., ChatGPT (Chat Generative Pre-trained Transformer)).
[0024] Furthermore, the character string generated by the language model M is not limited to a natural language. For example, the language model M may output an artificial language (such as a program source code) for a character string input in a natural language. For example, the language model M may receive an input question such as "How do I retrieve data containing a specific character string from a database?" as a target character string. In response to this input, the language model M may output a program source code for performing database processing. Alternatively, the language model M may output a natural language corresponding to a character string input in an artificial language.
[0025] The character string acquired by the information processing device 2 is not particularly limited, and may be, for example, a character string including at least one word and an instruction indicating what kind of language processing to perform. For example, assume that the information processing device 2 acquires the character string "What is the main business field of XX Corporation?" In this case, the character string includes five words, "XX Corporation," "of," "main," "business field," and "is," as well as an instruction to respond to the question "What is the main business field of XX Corporation?"
[0026] (Configuration of information processing device 2) The configuration of the information processing device 2 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 2 according to this exemplary embodiment. As shown in Fig. 3, the information processing device 2 includes a control unit 20, an input unit 25, an output unit 26, and a storage unit 27.
[0027] The input unit 25 is an interface that accepts user input. For example, the input unit 25 may supply information indicating the accepted user input to the control unit 20. Examples of the input unit 25 include, but are not limited to, a mouse, a keyboard, a touchpad, and a microphone.
[0028] The output unit 26 is an interface that outputs data. As an example, the output unit 26 may be an interface that outputs data to another connected device. In this case, for example, the output unit 26 outputs the results output from the control unit 20 to the other connected device. As another example, the output unit 26 may be a display device that displays images or a speaker that outputs audio. When the output unit 26 is a display device, the output unit 26 displays an image including the results output from the control unit 20.
[0029] The storage unit 27 stores data referenced by the control unit 20. Documents are an example of data stored in the storage unit 27. Examples of the storage unit 27 include, but are not limited to, flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.
[0030] Furthermore, the storage unit 27 stores a language model M. Note that storing the language model M in the storage unit 27 means that parameters defining the language model M are stored in the storage unit 27. Furthermore, the language model M may be stored in a storage device other than the storage unit 27 (for example, an external server, etc.).
[0031] The control unit 20 controls each component included in the information processing device 2. As shown in Fig. 3 , the control unit 20 includes an acquisition unit 11, a generation unit 12, a query generation unit 13, an extraction unit 14, an output control unit 24, a calculation unit 21, a determination unit 22, and a goodness-of-fit calculation unit 23. In this exemplary embodiment, the acquisition unit 11, the generation unit 12, the query generation unit 13, the extraction unit 14, the output control unit 24, the calculation unit 21, the determination unit 22, and the goodness-of-fit calculation unit 23 are configured to realize an acquisition means, a generation means, a query generation means, an extraction means, an output means, a calculation means, a determination means, and a goodness-of-fit calculation means, respectively.
[0032] The acquisition unit 11 acquires data supplied from the input unit 25. As an example, the acquisition unit 11 may acquire a character string. The acquisition unit 11 stores the acquired character string in the storage unit 27.
[0033] The generation unit 12 generates a result of performing language processing on a character string. As an example, the generation unit 12 generates a result of performing language processing on a character string. As described above, the generation unit 12 uses a language model M that is trained to generate text based on input text. That is, the generation unit 12 inputs a character string to the language model M, and generates a result output from the language model M as a result of performing language processing on the character string. The result generated by the generation unit 12 is also referred to as a first result. The generation unit 12 associates the generated first result with the character string and stores it in the storage unit 27.
[0034] The query generation unit 13 generates a query from the first result generated by the generation unit 12. Here, a query is a character string representing an inquiry, a request, or the like. The query generation unit 13 associates the generated query with the first result and stores it in the storage unit 27. The query generation unit 13 may generate one query from one first result, or may generate multiple queries from one first result.
[0035] If the first result generated by the generation unit 12 includes a character string, the query generation unit 13 may generate a part or all of the character string as a query. Alternatively, the query generation unit 13 may, for example, extract one or more keywords from the character string and generate a query using the extracted keywords. More specifically, the query generation unit 13 may, for example, extract one or more keywords from the character string and generate a query by combining the extracted keywords with a sentence containing the keywords. Alternatively, the query generation unit 13 may directly use the keywords extracted from the character string as a query.
[0036] Furthermore, when the first result generated by the generation unit 12 includes an image, the query generation unit 13 may, for example, detect an object from the image using a method for detecting an object from an image, and generate a character string corresponding to the detected object as a query. More specifically, the query generation unit 13 may, for example, generate a character string representing the name of the object detected from the image as a query. Examples of methods for detecting an object from an image include, but are not limited to, Faster-RCNN, SSD (Single Shot MultiBox Detector), and YOLO (You Only Look Once).
[0037] As an example, the query generation unit 13 may extract one or more portions from the first result (e.g., text) generated by the generation unit 12 and generate a query for each extracted portion. Here, an example of a processing flow when the first result includes a character string will be described below. First, the query generation unit 13 extracts multiple spans from the character string included in the first result. Here, a span is a semantic block of sentences and is an example of a portion related to this specification. For example, the query generation unit 13 may extract each sentence included in the character string included in the first result as a span. For example, if the first result includes a list, the query generation unit 13 may extract each item included in the list as a span. For example, if the first result includes multiple paragraphs, the query generation unit 13 may extract each paragraph as a span. For example, the query generation unit 13 may extract each word included in the character string as a span. Information identifying the document extracted by the extraction unit 14 is added to each span by the output control unit 24, which will be described later. In other words, the span extracted by the query generating unit 13 can be said to be a unit to which information specifying a document is added.
[0038] Next, the query generation unit 13 generates a query based on the extracted span. The query generation unit 13 may generate one query for one span, or may generate multiple queries for one span. The query generation unit 13 may use the span as a query directly, or may extract a portion of a character string included in the span and use the extracted keyword to generate a query. More specifically, the query generation unit 13 may extract one or more keywords from each span and use the extracted keyword as a query. The query generation unit 13 may extract one or more keywords from each span and use the extracted keyword as a query. The query generation unit 13 may extract one or more keywords from each span and use a combination of the extracted keyword and a sentence containing the keyword as a query.
[0039] When extracting one or more keywords from each span, the query generation unit 13 may extract keywords from the spans by, for example, inputting the spans into a trained model that has been machine-learned to extract characteristic keywords from input character strings. The query generation unit 13 may also extract keywords from each span using a named entity recognition (NAR) technique. When using NAR, the query generation unit 13 may, for example, estimate the type of each word constituting the query using the NAR technique and extract words of a specific type (e.g., person's name, country name, etc.) as keywords. Here, the type of each word indicates the classification result obtained by named entity classification. The query generation unit 13 may, for example, extract words of a type other than the specific type from among the words constituting the spans as keywords.
[0040] Furthermore, when the first result output by the language model M includes an image, the query generation unit 13 may, as an example, divide the image output by the language model M into multiple regions (parts) and generate a query for each of the regions obtained by the division.
[0041] The extraction unit 14 extracts documents related to the query from a database by performing a search process using the query generated by the query generation unit 13. Examples of documents extracted by the extraction unit 14 include, but are not limited to, text data indicating a character string, files created by predetermined document creation software, PDF files, and HTML files. As an example, the extraction unit 14 may extract documents related to the query from a database stored in the storage unit 27. As another example, the extraction unit 14 may extract documents related to the query from documents stored in a database connected to the information processing device 2 via a network. Examples of databases include external databases such as online dictionaries, newspaper articles, and SNSs (social networking services). The extraction unit 14 associates the extracted documents with the query and stores them in the storage unit 27.
[0042] The method by which the extraction unit 14 extracts documents related to the query is not limited. For example, the extraction unit 14 may extract documents that include the query. Alternatively, the extraction unit 14 may extract documents related to the query using an existing search engine.
[0043] The documents extracted by the extraction unit 14 are documents searched for using a query generated from the first result, which is the output of the language model M. Therefore, it can be said that the documents extracted by the extraction unit 14 are documents that are highly relevant to the first result. In other words, it can be said that the documents extracted by the extraction unit 14 are documents that are highly likely to be the basis for the output of the language model M.
[0044] The output control unit 24 outputs the first result to the output unit 26, adding information identifying the document extracted by the extraction unit 14 to the first result. The output control unit 24 corresponds to the output unit 15 in the first exemplary embodiment described above. As an example, the output control unit 24 outputs the first result generated by the generation unit 12, adding information identifying the document extracted by the extraction unit 14 to the first result. The output control unit 24 may add information identifying some of the documents extracted by the extraction unit 14 to the first result generated by the generation unit 12. Examples of information identifying a document include the document name, author name, publication date and time, and a uniform resource locator (URL) indicating the location where the document is stored. Since the document extracted by the extraction unit 14 is a document that is likely to be the basis for the output of the language model M, the information identifying the document can also be considered source information indicating the source of the language model M. The result output by the output control unit 24 is also referred to as a second result.
[0045] Furthermore, the output control unit 24 may output the reliability calculated by the calculation unit 21, which will be described later, in addition to the second result. With this configuration, the output control unit 24 can present the reliability of the second result to the user. Furthermore, the output control unit 24 may be configured to output the second result as the optimization result when the determination unit 22, which will be described later, determines that the reliability calculated by the calculation unit 21 exceeds a threshold. With this configuration, the output control unit 24 can output a second result with a high reliability.
[0046] Furthermore, the output control unit 24 may add information specifying some of the documents to the first result, rather than adding information specifying all of the documents extracted by the extraction unit 14 to the first result. For example, the output control unit 24 may output a second result in which information specifying documents, among the documents extracted by the extraction unit 14, whose relevance calculated by the relevance calculation unit 23 described later satisfies a predetermined condition, is added to the first result (text, etc.) generated by the generation unit 12. With this configuration, it is possible to output documents that are more likely to be the basis for the output of the language model M.
[0047] Furthermore, when the query generation unit 13 generates a query for each of the portions (spans) extracted from the first result, the output control unit 24 may output each portion in association with a document extracted using the query generated for that portion. With this configuration, it is possible to output a document that is likely to be the basis for each portion (span).
[0048] The calculation unit 21 calculates the reliability of the result. As an example, the calculation unit 21 calculates the reliability of the first result (text, etc.) generated by the generation unit 12. The calculation unit 21 associates the calculated reliability with the first result and stores it in the storage unit 27. With this configuration, the calculation unit 21 can grasp how reliable the first result is.
[0049] The method by which the calculation unit 21 calculates the reliability of the first result is not limited. For example, the calculation unit 21 may calculate the reliability of the first result using an existing technique. As an example, the calculation unit 21 may calculate the reliability of the first result using the first result and the document extracted by the extraction unit 14. Examples of methods by which the calculation unit 21 calculates the reliability of the first result using the first result and the document extracted by the extraction unit 14 include (a) a method based on word distance, (b) a method based on document distance, and (c) a method based on a learning model.
[0050] When using the method based on word distance, the calculation unit 21 calculates the reliability of the first result based on the word distance between each word included in the first result and each word included in the document. Specifically, the calculation unit 21 first calculates the word distance for each combination between each word included in the first result and each word included in the document. The calculation unit 21 calculates the reliability of the first result such that the shorter the calculated word distance, the higher the reliability of the first result.
[0051] When the method based on the inter-document distance is used, the calculation unit 21 calculates the reliability of the first result based on the inter-document distance between a sentence included in the first result and a sentence included in the document. Specifically, the calculation unit 21 first calculates the inter-document distance between a sentence included in the first result and a sentence included in the document. The calculation unit 21 calculates the reliability of the first result such that the shorter the calculated inter-document distance, the higher the reliability of the first result.
[0052] When using a method based on a learning model, the calculation unit 21 uses a learning model that has been trained by machine learning to input two sentences and output the similarity between the two sentences. In this case, the calculation unit 21 inputs a sentence included in the first result and a sentence included in the document into the learning model. Then, the calculation unit 21 calculates the reliability of the first result such that the higher the similarity output from the learning model, the higher the reliability of the first result.
[0053] When the extraction unit 14 extracts a plurality of documents, the calculation unit 21 calculates the reliability of the first result for each of the plurality of documents. Then, the calculation unit 21 may calculate the arithmetic mean value of the calculated plurality of reliabilities as the reliability of the first result.
[0054] As another example, the calculation unit 21 may calculate the reliability using the first result (e.g., text), the document extracted by the extraction unit 14, and the character string (target text) acquired by the acquisition unit 11. For example, the calculation unit 21 may calculate the reliability by referring to a result in which the first result, the document extracted by the extraction unit 14, and the character string acquired by the acquisition unit 11 are input into the language model M used by the generation unit 12. As an example, the calculation unit 21 acquires, from the language model M, an index indicating how accurate the first result that references the document is as a response to the character string. The calculation unit 21 calculates the reliability of the first result such that the reliability of the first result increases as the output result becomes more positive (e.g., the larger the value of the index). With this configuration, the calculation unit 21 can preferably calculate the reliability of the first result using the language model M that generated the first result.
[0055] Furthermore, the calculation unit 21 may input the first result, the document extracted by the extraction unit 14, and the character string acquired by the acquisition unit 11 multiple times to the language model M. In this case, the calculation unit 21 may score (likelihood, positive frequency, etc.) the results output from the language model M for each of the multiple inputs, and calculate the reliability of the first result such that the higher the score, the higher the reliability of the first result.
[0056] Furthermore, instead of the character string acquired by the acquisition unit 11, the calculation unit 21 may input to the language model a character string that instructs the user to answer whether or not the first result is correct in consideration of the document identified by the information added by the output control unit 24. In this case as well, the calculation unit 21 calculates the reliability of the first result such that the more positive the output result is, the higher the reliability of the first result becomes.
[0057] The determination unit 22 determines whether the value exceeds a threshold. As an example, the determination unit 22 determines whether the reliability calculated by the calculation unit 21 exceeds a threshold. The determination unit 22 associates the determination result with the reliability and stores it in the storage unit 27.
[0058] The relevance calculation unit 23 calculates the relevance between the document extracted by the extraction unit 14 and the first result (text, etc.) generated by the generation unit 12. The relevance calculation unit 23 associates the calculated relevance with the document and stores it in the storage unit 27. If the extraction unit 14 extracts multiple documents, the relevance calculation unit 23 calculates the relevance for each of the multiple documents. Furthermore, if the query generation unit 13 generates queries for multiple portions extracted from the first result, the relevance calculation unit 23 calculates the relevance between the portions and the document extracted using the queries generated for those portions. With this configuration, it is possible to output documents that are likely to be evidence for each of the multiple portions included in the output of the language model M.
[0059] The method by which the relevance calculation unit 23 calculates the relevance is not limited. As an example, the relevance calculation unit 23 may calculate the relevance using an existing technique. Here, if the first result includes a character string, the relevance calculation unit 23 may calculate the relevance based on, for example, the degree of match between the character strings in the first result and the document extracted by the extraction unit 14. Alternatively, the relevance calculation unit 23 may perform an implication determination between the first result and the document extracted by the extraction unit 14 and calculate the relevance based on the determination result.
[0060] More specifically, if the first result includes a character string, the compatibility calculation unit 23 may calculate the compatibility using, for example, (a) a method based on word distance, (b) a method based on document distance, or (c) a method based on a learning model, which are described below.
[0061] When the method based on inter-word distance is used, the compatibility calculation unit 23 calculates the compatibility based on the inter-word distance between each word included in the character string output by the generation unit 12 and each word included in the document extracted by the extraction unit 14. Specifically, the compatibility calculation unit 23 first calculates the inter-word distance for each combination between each word included in the character string output by the generation unit 12 and each word included in the document extracted by the extraction unit 14. The compatibility calculation unit 23 calculates the compatibility so that the shorter the calculated inter-word distance, the higher the compatibility.
[0062] When the method based on the inter-document distance is used, the compatibility calculation unit 23 calculates the compatibility based on the inter-document distance between the character string output by the generation unit 12 and the document extracted by the extraction unit 14. Specifically, the compatibility calculation unit 23 first calculates the inter-document distance between the character string output by the generation unit 12 and the document extracted by the extraction unit 14. The compatibility calculation unit 23 calculates the compatibility such that the shorter the calculated inter-document distance, the higher the compatibility.
[0063] When a method based on a learning model is used, the relevance calculation unit 23 uses a learning model that has been trained by machine learning to input two sentences and output the similarity between the two sentences. In this case, the relevance calculation unit 23 inputs the character string output by the generation unit 12 and the document extracted by the extraction unit 14 into the learning model. Then, the relevance calculation unit 23 calculates the relevance such that the higher the similarity output from the learning model, the higher the relevance.
[0064] As another example, the relevance calculation unit 23 may calculate the relevance using the first result, the document extracted by the extraction unit 14, and the character string acquired by the acquisition unit 11. For example, the relevance calculation unit 23 may calculate the relevance by referring to a result of inputting the first result, the document extracted by the extraction unit 14, and the character string acquired by the acquisition unit 11 into the language model M used by the generation unit 12. As an example, the relevance calculation unit 23 acquires, from the language model M, an index indicating how accurate the first result res1, which references the document doc, is in response to the character string str1. The relevance calculation unit 23 calculates the relevance such that the more positive the output result (e.g., the larger the value of the index), the higher the relevance. With this configuration, the relevance calculation unit 23 can preferably calculate the relevance using the language model M that generated the first result.
[0065] The relevance calculation unit 23 may input the first result, the document extracted by the extraction unit 14, and the character string acquired by the acquisition unit 11 into the language model M multiple times. In this case, the relevance calculation unit 23 may score (such as likelihood or positive frequency) the results output from the language model M for each of the multiple inputs, and calculate the relevance so that the higher the score, the higher the relevance. Furthermore, instead of the character string acquired by the acquisition unit 11, the relevance calculation unit 23 may input a character string that instructs the language model to answer whether or not the first result is correct in consideration of the document identified by the information added by the output control unit 24. In this case, too, the relevance calculation unit 23 calculates the relevance so that the more positive the output result, the higher the relevance.
[0066] Furthermore, if the first result includes an image, the compatibility calculation unit 23 may, for example, generate a character string from the image and calculate the compatibility between the generated character string and the document extracted by the extraction unit 14 using the above-mentioned method (such as a method of calculating the distance between words). Examples of methods for generating a character string from an image include, but are not limited to, a method for detecting an object from an image and generating a character string corresponding to the detected object. Examples of methods for detecting an object from an image include, but are not limited to, Faster-RCNN and YOLO (You Only Look Once).
[0067] In this exemplary embodiment, the relevance calculation unit 23 may calculate the relevance using a method different from the method used by the calculation unit 21 to calculate the reliability. In other words, the reliability and the relevance may be calculated using different algorithms. For example, the calculation unit 21 may calculate the reliability using the above-mentioned (a) inter-word distance-based method using the first result and the document extracted by the extraction unit 14, while the relevance calculation unit 23 may calculate the relevance based on a result obtained by inputting a question sentence including the first result and the document extracted by the extraction unit 14 into the language model M. In this case, the question sentence may be, for example, a character string indicating content such as "When considering ${question} in light of ${related document}, is the answer ${span} logically valid?" Alternatively, the reliability and the relevance may be calculated using language models trained from different training data.
[0068] (Processing Executed by Information Processing Device 2) The information processing method executed by the information processing device 2 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of the information processing method S2 according to this exemplary embodiment. The example of Fig. 4 shows the flow of processing when the first result output by the language model M is a character string.
[0069] An example of each process executed by the information processing device 2 will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of each process executed by the information processing device 2 according to this exemplary embodiment.
[0070] (Step S21) In step S21, the acquisition unit 11 acquires a character string. The acquisition unit 11 stores the character string in the storage unit 27. The character string acquired by the acquisition unit 11 is, for example, a character string indicating the content of a question or instruction input by a user via the input unit 25. For example, as shown in FIG. 5 , the acquisition unit 11 acquires a character string str1 indicating "What are the main business activities of XX Corporation?"
[0071] (Step S22) In step S22, the generation unit 12 generates a first result corresponding to the character string obtained in step S21, using a language model M trained to generate text based on input text. The generation unit 12 associates the first result with the character string and stores it in the storage unit 27. For example, as shown in FIG. 5 , the generation unit 12 inputs a character string str1 to the language model M, and generates a first result res1 output from the language model M.
[0072] (Steps S23 and S24) In step S23, the query generation unit 13 extracts a plurality of spans from the first result generated in step S22. In step S24, the query generation unit 13 generates a query based on the extracted spans.
[0073] (Step S25) In step S25, the extraction unit 14 extracts documents related to the query through a search process using the query, and the output control unit 24 adds information identifying the documents extracted by the extraction unit 14 to the span. Step S25 includes steps S251 to S254. Steps S251 to S254 are executed for each span extracted by the query generation unit 13.
[0074] (Step S251) In step S251, the extraction unit 14 extracts documents related to the query from the database by performing a search process using the query generated by the query generation unit 13. The extraction unit 14 associates the extracted documents with the query and stores them in the storage unit 27. For example, as shown in FIG. 5 , the extraction unit 14 extracts a document doc related to the generated query from the first result res1. The document doc includes multiple documents (such as "related document 1," "related document 2," and "related document 3").
[0075] (Step S252) The process of step S252 is executed for each document extracted by the extraction unit 14. In step S252, the compatibility calculation unit 23 calculates the compatibility between a span extracted from the first result and a document extracted using a query generated for the span.
[0076] (Step S253) In step S253, the output control unit 24 selects a document based on the relevance calculated by the relevance calculation unit 23. In other words, the output control unit 24 selects a document whose relevance satisfies a predetermined condition from among the documents extracted by the extraction unit 14. Specifically, the output control unit 24 may select, for each of a plurality of spans, a document whose relevance is calculated to exceed a predetermined threshold. Alternatively, the output control unit 24 may select the top N documents (N is a natural number greater than or equal to 1) for each of a plurality of spans.
[0077] (Step S254) In step S254, the output control unit 24 adds information for identifying the document selected in step S253 for each span to the span.
[0078] (Step S26) In step S26, the calculation unit 21 calculates the reliability of the first result stored in the storage unit 27. The calculation unit 21 associates the calculated reliability with the first result and stores it in the storage unit 27. For example, in FIG. 5 , the calculation unit 21 calculates the reliability using the first result res1, document doc, and character string str1. As an example, the calculation unit 21 inputs the first result res1, document doc, and character string str1 to the language model M. In this case, the calculation unit 21 obtains from the language model M an index indicating how accurate the first result res1, which references document doc, is in response to the character string str1. The calculation unit 21 calculates the reliability by referring to the index.
[0079] 5 (${1} or ${3}) into the language model M instead of the document doc, and calculates the reliability. Furthermore, the calculation unit 21 may input, for example, a character string "In consideration of the information of the document indicated by ${1}, can it be said that the first result res1 is correct?" into the language model M instead of the character string str1, and calculate the reliability.
[0080] (Step S27) In step S27, the determination unit 22 determines whether the reliability stored in the storage unit 27 exceeds a threshold value.
[0081] (Step S28) If it is determined in step S27 that the reliability exceeds the threshold (step S27: YES), in step S28 the output control unit 24 outputs a result (second result) in which information for identifying the document stored in the storage unit 27 is added to the first result stored in the storage unit 27. For example, as shown in Fig. 5 , the output control unit 24 generates a second result res2 by adding information i_doc for identifying the document to the first result res1.
[0082] Furthermore, the output control unit 24 may generate a second result res2 by adding information that identifies only the documents referred to by the language model M when generating the first result res1, from among the documents doc extracted by the extraction unit 14. For example, as shown in FIG. 5 , when the first result res1 includes "${1}" and "${3}", the output control unit 24 adds information i_doc that identifies the documents indicated by "${1}" and "${3}" to the first result res1, and generates a second result res2.
[0083] On the other hand, if it is determined in step S27 that the reliability does not exceed the threshold (step S26: NO), the output control unit 24 does not output the second result, and the information processing device 2 terminates execution of the information processing method S2.
[0084] FIG. 6 is a diagram illustrating a specific example of the second result output by the output control unit 24. For example, the output control unit 24 outputs data indicating the second result to a display device connected to the output unit 26 and displays an image representing the second result on the display device. In the example of FIG. 6, the second result includes three spans, spn1, spn2, and spn3, and each span is associated with information identifying a document extracted using a query generated for that span. In other words, the output control unit 24 outputs the spans and the documents extracted using the query generated for that span in association with each other. The information identifying the document includes, for example, the document and link information indicating a storage location of the document. In the example of FIG. 6, for example, when a user performs an operation to select a document associated with a span using the input unit 25, the output control unit 24 may obtain the selected document using the link information and display the obtained document on the display device.
[0085] (Effects of Information Processing Device 2) In this way, the information processing device 2 according to this exemplary embodiment generates a query from a first result obtained by inputting an acquired character string into the language model M, searches for documents related to the query using the generated query, and adds information identifying the searched document to the first result and outputs the result. With this configuration, the information processing device 2 can output information identifying a document that is likely to be the basis for the output of the language model M together with the output result of the language model M.
[0086] In this way, the information processing device 2 can present to the user the output of the language model M and information about documents that are likely to be the basis for the output, so that the user can easily determine whether the output of the language model M is reliable by checking the content of the presented document and information about the document. In this way, according to this exemplary embodiment, it is possible to make it easier to determine the reliability of the output of the language model M.
[0087]
[0033] A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the above exemplary embodiment are denoted by the same reference numerals, and their description will be omitted as appropriate.
[0088] (Outline and Configuration of Information Processing System 100) The outline and configuration of the information processing system 100 according to this exemplary embodiment will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the configuration of the information processing system 100 according to this exemplary embodiment.
[0089] As shown in Fig. 7 , the information processing system 100 includes an information processing device 3 and an information processing terminal 4. The information processing device 3 and the information processing terminal 4 are communicably connected to each other via a network. An example of the information processing device 3 is a desktop computer. Examples of the information processing terminal 4 are a smartphone, a tablet terminal, and a laptop computer.
[0090] In the information processing system 100, the information processing terminal 4 acquires target text, and the information processing device 3 performs language processing on the target text using the language model M. The information processing device 3 also outputs the results of the language processing to the information processing terminal 4. Note that, hereinafter, "target text" and "text" are also referred to as "character string."
[0091] As an example, in the information processing system 100, a user of the information processing terminal 4 creates an account for using the information processing system 100 and registers basic information (such as name, age, sex, and email address). When the user inputs a character string related to a medical consultation into the information processing terminal 4, the information processing device 3 outputs a response to the medical consultation. Examples of user input include symptoms, chronic illnesses, and medication usage. In response to the medical consultation, the information processing device 3 outputs information that serves as the basis for the response in addition to the response.
[0092] (Configuration of Information Processing Device 3) As shown in FIG. 7, the information processing device 3 includes a control unit 20, a storage unit 37, and a communication unit .
[0093] The storage unit 37 has the same configuration and functions as the above-described storage unit 27. The storage unit 37 also stores information about documents and authors of the documents.
[0094] Examples of information related to documents and authors stored in the storage unit 37 are shown in Figures 8 and 9. Figure 8 is a diagram showing an example of a document in this exemplary embodiment. Figure 9 is a diagram showing an example of information related to the author of a document in this exemplary embodiment. The information related to the author of a document is an example of information that identifies a document according to this specification.
[0095] 8, the documents stored in the storage unit 37 are medical documents. As shown in Fig. 8, the documents stored in the storage unit 37 may be stored in association with the author, publication year, and paper abstract. Information representing the document author, publication year, and paper abstract is an example of information that identifies a document according to the present specification.
[0096] 9, information about the author who created the document stored in the storage unit 37 may also be stored in the storage unit 37. As shown in FIG. 9, the information about the author who created the document may include the doctor's name, place of employment, career history, and field of expertise stored in the storage unit 37. In this way, the documents in the information processing system 100 are medical documents, and the author's career history, field of expertise, etc. are clearly defined. Therefore, it can be said that the medical documents in the information processing system 100 are highly reliable documents.
[0097] The communication unit 38 is an interface that transmits and receives data to and from external devices via a network. For example, the communication unit 38 transmits data supplied from the control unit 20 to the information processing terminal 4, and supplies data received from the information processing terminal 4 to the control unit 20. Examples of the communication unit 38 include, but are not limited to, communication chips for various communication standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), and wireless communication standards for mobile data communication networks, and USB-compliant connectors.
[0098] The control unit 20 controls each component included in the information processing device 3. As shown in Fig. 7 , the control unit 20 includes an acquisition unit 11, a generation unit 12, a query generation unit 13, an extraction unit 14, an output control unit 24, a calculation unit 21, a determination unit 22, and a goodness-of-fit calculation unit 23. In this exemplary embodiment, the acquisition unit 11, the generation unit 12, the query generation unit 13, the extraction unit 14, the output control unit 24, the calculation unit 21, the determination unit 22, and the goodness-of-fit calculation unit 23 are configured to realize an acquisition means, a generation means, a query generation means, an extraction means, an output means, a calculation means, a determination means, and a goodness-of-fit calculation means, respectively.
[0099] The acquisition unit 11 acquires data supplied from the communication unit 38. As an example, the acquisition unit 11 acquires a character string. The acquisition unit 11 stores the acquired character string in the storage unit 37.
[0100] The generation unit 12 generates a first result by performing language processing on the character string. The method by which the generation unit 12 generates the first result is as described above. The generation unit 12 associates the generated first result with the character string and stores it in the storage unit 37.
[0101] The query generation unit 13 generates a query from the first result generated by the generation unit 12. The method for generating a query by the query generation unit 13 is as described above. The query generation unit 13 associates the generated query with the first result and stores it in the storage unit 37.
[0102] The extraction unit 14 extracts documents related to the query from the database by performing a search process using the query generated by the query generation unit 13. The method by which the extraction unit 14 extracts documents is as described above. The extraction unit 14 also obtains information about the authors of the extracted documents from the storage unit 37. The extraction unit 14 associates the extracted documents and the information about the authors of the extracted documents with character strings and stores them in the storage unit 37.
[0103] The output control unit 24 outputs to the communication unit 38 a second result in which information specifying the document extracted by the extraction unit 14 is added to the first result generated by the generation unit 12. Furthermore, the output control unit 24 may output a result in which information regarding the author of the document extracted by the extraction unit 14 is added.
[0104] Furthermore, as described above, the output control unit 24 may output the reliability calculated by the calculation unit 21 in addition to the second result.
[0105] Furthermore, the output control unit 24 may be configured to output the second result when the determination unit 22 determines that the reliability calculated by the calculation unit 21 exceeds a threshold value.
[0106] The calculation unit 21 calculates the reliability of the result. As an example, the calculation unit 21 calculates the reliability of the first result generated by the generation unit 12. The method by which the calculation unit 21 calculates the reliability of the first result is as described above. The calculation unit 21 stores the calculated reliability in the storage unit 37 in association with the first result.
[0107] The determination unit 22 determines whether the value exceeds a threshold. As an example, the determination unit 22 determines whether the reliability calculated by the calculation unit 21 exceeds a threshold. The determination unit 22 associates the determination result with the reliability and stores it in the storage unit 37.
[0108] (Configuration of Information Processing Terminal 4) As shown in FIG. 7, the information processing terminal 4 includes a control unit 40, an input unit 45, a display unit 46, and a communication unit 48.
[0109] The control unit 40 controls each component included in the information processing terminal 4. The control unit 40 also includes an acquisition unit 41 and an output control unit 42, as shown in FIG.
[0110] The acquisition unit 41 acquires data supplied from the input unit 45 or the communication unit 48. As one example, the acquisition unit 41 acquires a character string from the input unit 45. As another example, the acquisition unit 41 acquires a second result from the communication unit 48. The acquisition unit 41 supplies the acquired data to the output control unit 42.
[0111] The output control unit 42 outputs data to the display unit 46 or the communication unit 48. As one example, the output control unit 42 outputs the character string acquired by the acquisition unit 41 to the communication unit 48. As another example, the output control unit 42 outputs the second result acquired by the acquisition unit 41 to the display unit 46.
[0112] The input unit 45 is an interface that accepts input of data. Examples of the input unit 45 include a mouse, a keyboard, a touchpad, and a microphone. The input unit 45 supplies the accepted data to the control unit 40.
[0113] The display unit 46 is a device that displays an image. For example, the display unit 46 displays an image based on data supplied from the control unit 40. Examples of the display unit 46 include a liquid crystal display and an organic EL (Electro Luminescence) display.
[0114] The communication unit 48 is an interface that transmits and receives data to and from external devices via a network. For example, the communication unit 48 transmits data supplied from the control unit 40 to the information processing device 3, and supplies data received from the information processing device 3 to the control unit 40. Examples of the communication unit 48 include, but are not limited to, communication chips for various communication standards such as Ethernet, Wi-Fi, and wireless communication standards for mobile data communication networks, and USB-compliant connectors.
[0115] (Processing Executed in Information Processing System 100) An information processing method executed in the information processing system 100 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the flow of an information processing method S100 according to this exemplary embodiment.
[0116] (Step S101) In step S101, the acquisition unit 41 of the information processing terminal 4 acquires a character string. The acquisition unit 41 supplies the character string to the output control unit .
[0117] (Step S102 ) In step S102 , the output control unit 42 outputs the character string supplied from the acquisition unit 41 to the information processing device 3 via the communication unit 48 .
[0118] (Step S103) In step S103, the acquisition unit 11 of the information processing device 3 acquires the character string output from the information processing terminal 4 via the communication unit 38. The acquisition unit 11 stores the character string in the storage unit 37.
[0119] (Step S104) In step S104, the generation unit 12 generates a first result corresponding to the character string obtained in step S103, using the language model M trained to generate text based on the input text. The generation unit 12 stores the first result in the storage unit 37 in association with the character string.
[0120] (Steps S105 and S106) In step S105, the query generation unit 13 extracts a plurality of spans from the first result generated in step S104. In step S106, the query generation unit 13 generates a query based on the extracted spans.
[0121] (Step S107) In step S107, the extraction unit 14 extracts documents related to the query by a search process using the query. The output control unit 24 also obtains information about the authors of the extracted documents from the storage unit 37. The output control unit 24 also stores in the storage unit 37 a second result in which the information identifying the documents extracted by the extraction unit 14 and the information about the authors of the documents are added to the span.
[0122] (Step S108) In step S108, the calculation unit 21 calculates the reliability of the first result stored in the storage unit 37. The calculation unit 21 stores the calculated reliability in the storage unit 37 in association with the first result.
[0123] (Step S109) In step S109, the determination unit 22 determines whether the reliability stored in the storage unit 37 exceeds a threshold value.
[0124] (Step S110) If it is determined in step S109 that the reliability exceeds the threshold value (step S109: YES), in step S110 the output control unit 24 outputs to the information processing terminal 4 a second result in which information identifying the document stored in the memory unit 37 is added to the first result stored in the memory unit 37.
[0125] (Step S111) In step S111, the acquisition unit 41 of the information processing terminal 4 acquires the second result output from the information processing device 3 via the communication unit 48. The acquisition unit 41 supplies the acquired second result to the output control unit 42.
[0126] (Step S112 ) In step S112 , the output control unit 42 displays the second result supplied from the acquisition unit 41 on the display unit 46 .
[0127] On the other hand, if it is determined in step S109 that the reliability does not exceed the threshold (step S109: NO), the output control unit 24 does not output the second result, and the information processing system 100 terminates the execution of the information processing method S100.
[0128] Alternatively, the output control unit 24 may output information indicating that a highly reliable result could not be generated to the information processing terminal 4 via the communication unit 38. In this case, the output control unit 42 of the information processing terminal 4 may display on the display unit 46 that a highly reliable result could not be generated.
[0129] (Example 1 of an image displayed by information processing terminal 4) An example of an image displayed by the information processing terminal 4 will be described with reference to Fig. 11. Fig. 11 is a diagram showing an example of an image displayed by the information processing terminal 4 according to this exemplary embodiment.
[0130] As described above, the output control unit 42 of the information processing terminal 4 displays the second result on the display unit 46. As an example, as shown on the left side of FIG. 11 , the output control unit 42 displays the second result res "Have you also noticed either eye sensitivity to light or slurring of your speech?" in response to the input "Moderate" from the user who exhibits the symptom. In this example, the output control unit 42 displays, in addition to the second result res, information identifying a document searched using a query generated from the second result res on the display unit 46. When the user performs an operation to select information identifying a document on the screen displayed on the display unit 46, the information processing terminal 4 may acquire information on the document corresponding to the selected information and display the information.
[0131] Furthermore, when the acquiring unit 41 acquires information indicating a user's operation to select the second result res (for example, an operation of touching the display position of the second result res on the display unit 46), the output control unit 42 may display information identifying the document. Furthermore, as described above, when the output control unit 24 of the information processing device 3 outputs the second result to which information on the author of the document has been added, the output control unit 42 may display information on the author of the document in addition to the information identifying the document.
[0132] (Example 2 of image displayed by information processing terminal 4) In step S110 described above, the output control unit 24 may output information for communicating with a doctor, in addition to the second result, to the information processing terminal 4. As an example, the output control unit 24 may refer to information that identifies a document and information about the author of the document, and output contact information for a doctor who has the same specialty as the author's specialty to the information processing terminal 4 in addition to the second result.
[0133] For example, in step S110 described above, if the output control unit 24 outputs the doctor's contact information in addition to the second result to the information processing terminal 4, the information processing terminal 4 displays the doctor's contact information in addition to the second result on the display unit 46. In this configuration, if the acquisition unit 41 acquires information indicating an operation by the user to select the doctor's contact information (for example, an operation of touching the display position of the doctor's contact information on the display unit 46), the output control unit 42 makes a call to the doctor's contact information.
[0134] With this configuration, as shown in the center of FIG. 11, the information processing terminal 4 can allow the user to have a video call with a doctor.
[0135] (Example 3 of image displayed by information processing terminal 4) In step S110 described above, the output control unit 24 may output, in addition to the second result, information about an action suggested to the user to the information processing terminal 4. Examples of actions suggested to the user include going to a hospital and obtaining a prescription drug from a doctor. The following describes a configuration in which the output control unit 24 outputs, to the information processing terminal 4, information indicating a method for obtaining a prescription drug from a doctor.
[0136] If the second result includes information identifying the drug, the output control unit 24 may output the location of a pharmacy that sells the drug to the information processing terminal 4 in addition to the second result.
[0137] Alternatively, as described above, when the user communicates with a doctor, the doctor may output the user's prescription to the information processing terminal 4. When the information processing terminal 4 obtains the prescription, the information processing terminal 4 may output the location of a pharmacy that sells the drug listed in the prescription to the information processing terminal 4 in addition to the second result.
[0138] With this configuration, as shown on the right side of FIG. 11, the information processing terminal 4 can present to the user places where the medicine prescribed by the doctor can be purchased.
[0139] (Effect 1 of Information Processing System 100) As described above, in the information processing system 100 according to this exemplary embodiment, the information processing device 3 generates a query from a first result obtained by inputting a character string acquired from the information processing terminal 4 into the language model M, searches for documents related to the query using the generated query, and adds information identifying the searched document to the first result and outputs the result. With this configuration, the information processing device 3 can output information identifying a document that is likely to be the basis for the output of the language model M together with the output result of the language model M.
[0140] In this way, the information processing device 3 can present to the user the output of the language model M and information specifying a document that is likely to be the basis of the output, so that the user can easily determine the reliability of the output of the language model M by checking the presented information. In this way, according to the present exemplary embodiment, in addition to the output of the language model M, information for determining the reliability of the output can be presented to the user.
[0141] (Effect 2 of Information Processing System 100) In the information processing system 100, the information processing terminal 4 accepts a medical consultation, and the information processing device 3 generates a response to the medical consultation. The following describes the effect of the configuration in which the information processing system 100 outputs a response to a medical consultation from a user.
[0142] First, in the information processing system 100, the information processing device 3 generates a response to a medical consultation using a language model M. In this configuration, a user can receive medical consultation via an information processing terminal 4, such as a smartphone. Therefore, the information processing system 100 allows the user to easily receive medical consultation. Furthermore, the information processing system 100 allows the user to easily receive medical consultation anytime and anywhere. Furthermore, the information processing system 100 can provide the user with general information and advice regarding symptoms, for example, before the user receives a diagnosis or treatment.
[0143] Furthermore, since the information processing terminal 4 accepts medical consultations and outputs responses, the user can receive medical consultations without going to a hospital. Furthermore, with the information processing system 100, a response to a medical consultation can be obtained more quickly than if the user went to a hospital. Furthermore, with the information processing system 100, a response to a medical consultation is provided without the intervention of a doctor, so a response to a medical consultation can be provided at low cost.
[0144] [Example of Software Implementation] Some or all of the functions of the information processing devices 1, 2, and 3 may be implemented by hardware such as an integrated circuit (IC chip), or by software.
[0145] In the latter case, the information processing devices 1, 2, and 3 are realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 12. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the information processing devices 1, 2, and 3. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing devices 1, 2, and 3.
[0146] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.
[0147] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.
[0148] The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.
[0149] [Additional Note 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.
[0150] [Supplementary Note 2] Some or all of the above-described embodiments can also be described as follows. However, the present invention is not limited to the aspects described below. (Supplementary Note 1) An information processing device comprising: an acquisition means for acquiring target text; a generation means for generating text corresponding to the acquired target text using a language model trained to generate text based on input text; a query generation means for generating a query from the text generated by the generation means; an extraction means for extracting documents related to the query from a database by a search process using the query; and an output means for outputting a result in which information identifying the document is added to the text generated by the generation means.
[0151] (Supplementary Note 2) The information processing device according to Supplementary Note 1, further comprising: a calculation unit that calculates a reliability of the text generated by the generation unit.
[0152] (Appendix 3) The information processing device described in Appendix 2 further includes a determination means for determining whether the reliability exceeds a threshold, and when the determination means determines that the reliability exceeds the threshold, the output means outputs the result of adding information that identifies the document as an optimization result.
[0153] (Supplementary Note 4) The information processing device according to Supplementary Note 2 or 3, wherein the output means outputs the reliability in addition to a result to which information specifying the document is added.
[0154] (Supplementary Note 5) The information processing device according to any one of Supplementary Notes 2 to 4, wherein the calculation means calculates the reliability using the text generated by the generation means, the document extracted by the extraction means, and the target text acquired by the acquisition means.
[0155] (Appendix 6) An information processing device described in any one of Appendices 1 to 5, further comprising a compatibility calculation means for calculating the compatibility between the document extracted by the extraction means and the text generated by the generation means, and wherein the output means outputs the result of adding information identifying documents, among the documents extracted by the extraction means, whose compatibility meets a predetermined condition to the text generated by the generation means.
[0156] (Supplementary Note 7) The information processing device according to Supplementary Note 6, wherein the query generation means extracts one or more parts from the text generated by the generation means and generates a query for each extracted part, and the compatibility calculation means calculates the compatibility between the part and a document extracted using the query generated for the part.
[0157] (Supplementary Note 8) The information processing device according to Supplementary Note 7, wherein the output means outputs the portion in association with a document extracted using a query generated for the portion.
[0158] (Supplementary Note 9) An information processing method including: at least one processor acquiring target text; generating text corresponding to the target text using a language model trained to generate text based on input text; generating a query from the generated text; extracting documents related to the query from a database by a search process using the query; and outputting a result in which information identifying the document is added to the generated text.
[0159] (Supplementary Note 10) A program that causes a computer to execute an acquisition process that acquires target text, a generation process that generates text corresponding to the target text using a language model trained to generate text based on input text, a query generation process that generates a query from the text generated in the generation process, an extraction process that extracts documents related to the query from a database by a search process using the query, and an output process that outputs a result of adding information that identifies the document to the text generated in the generation process.
[0160] [Supplementary Note 3] Some or all of the above-described embodiments can also be expressed as follows: (Supplementary Note 1) An information processing device including at least one processor, the at least one processor executing an acquisition process of acquiring target text, a generation process of generating text corresponding to the acquired target text using a language model trained to generate text based on input text, a query generation process of generating a query from the text generated in the generation process, an extraction process of extracting documents related to the query from a database by a search process using the query, and an output process of adding information identifying the document to the text generated in the generation process and outputting a result.
[0161] (Supplementary Note 2) The information processing device according to Supplementary Note 1, wherein the processor further executes a calculation process of calculating a reliability of the text generated in the generation process.
[0162] (Supplementary Note 3) The information processing device described in Supplementary Note 2, wherein the processor further executes a determination process to determine whether the reliability exceeds a threshold, and if the determination process determines that the reliability exceeds a threshold in the output process, outputs the result of adding information that identifies the document as an optimization result.
[0163] (Supplementary Note 4) The information processing device according to Supplementary Note 2 or 3, wherein the processor outputs the reliability in addition to a result to which information specifying the document is added in the output process.
[0164] (Supplementary Note 5) The information processing device according to any one of Supplementary Notes 2 to 4, wherein in the calculation process, the processor calculates the reliability using the text generated in the generation process, the document extracted in the extraction process, and the target text acquired in the acquisition process.
[0165] (Appendix 6) An information processing device described in any one of Appendices 1 to 5, wherein the processor further executes a compatibility calculation process that calculates the compatibility between the documents extracted in the extraction process and the text generated in the generation process, and in the output process, the processor outputs the result of adding information that identifies documents, among the documents extracted in the extraction process, whose compatibility meets a predetermined condition to the text generated in the generation process.
[0166] (Supplementary Note 7) The information processing device described in Supplementary Note 6, wherein, in the query generation, the processor extracts one or more parts from the text generated in the generation process and generates a query for each extracted part, and in the compatibility calculation process, the processor calculates the compatibility between the part and a document extracted using the query generated for the part.
[0167] (Supplementary Note 8) The information processing device according to Supplementary Note 7, wherein the processor, in the output process, outputs the portion in association with a document extracted using a query generated for the portion.
[0168] (Supplementary Note 9) An information processing method including: at least one processor acquiring target text; generating text corresponding to the target text using a language model trained to generate text based on input text; generating a query from the generated text; extracting documents related to the query from a database by a search process using the query; and outputting a result in which information identifying the document is added to the generated text.
[0169] (Supplementary Note 10) The information processing device may further include a memory that stores a program for causing the processor to execute the acquisition process, the generation process, the query generation process, the extraction process, and the output process. The program may be recorded on a computer-readable, non-transitory, tangible recording medium.
[0170] REFERENCE SIGNS LIST 1, 2, 3 Information processing device 4 Information processing terminal 11, 41 Acquisition unit 12 Generation unit 13 Query generation unit 14 Extraction unit 15, 26 Output unit 20, 40 Control unit 21 Calculation unit 22 Determination unit 24, 42 Output control unit 23 Goodness of fit calculation unit 25, 45 Input unit 27, 37 Storage unit 38, 48 Communication unit 46 Display unit
Claims
1. An information processing device comprising: an acquisition means for acquiring a target text; a generation means for generating text corresponding to the acquired target text using a language model trained to generate text based on input text; a query generation means for generating a query from the text generated by the generation means; an extraction means for extracting documents related to the query from a database by a search process using the query; and an output means for outputting a result in which information identifying the document is added to the text generated by the generation means.
2. The information processing device according to claim 1, further comprising a calculation means for calculating a reliability of the text generated by the generation means.
3. The information processing device of claim 2, further comprising a judgment means for judging whether the reliability exceeds a threshold value, and when the judgment means judges that the reliability exceeds the threshold value, the output means outputs the result of adding information that identifies the document as an optimization result.
4. The information processing device according to claim 2 or 3, wherein the output means outputs the reliability in addition to the result to which the information specifying the document is added.
5. The information processing device according to any one of claims 2 to 4, wherein the calculation means calculates the reliability using the text generated by the generation means, the document extracted by the extraction means, and the target text acquired by the acquisition means.
6. An information processing device as claimed in any one of claims 1 to 5, further comprising a compatibility calculation means for calculating a compatibility between the document extracted by the extraction means and the text generated by the generation means, wherein the output means outputs a result of adding information identifying documents, among the documents extracted by the extraction means, whose compatibility satisfies a predetermined condition, to the text generated by the generation means.
7. The information processing device according to claim 6, wherein the query generation means extracts one or more parts from the text generated by the generation means and generates a query for each extracted part, and the compatibility calculation means calculates the compatibility between the part and a document extracted using the query generated for the part.
8. The information processing device according to claim 7, wherein the output means outputs the portion in association with a document extracted using a query generated for the portion.
9. An information processing method comprising: at least one processor: acquiring a target text; generating text corresponding to the target text using a language model trained to generate text based on input text; generating a query from the generated text; extracting documents related to the query from a database by a search process using the query; and outputting a result in which information identifying the document is added to the generated text.
10. A program that causes a computer to execute the following processes: an acquisition process that acquires target text; a generation process that generates text corresponding to the target text using a language model trained to generate text based on input text; a query generation process that generates a query from the text generated in the generation process; an extraction process that extracts documents related to the query from a database by a search process using the query; and an output process that outputs the results of adding information that identifies the document to the text generated in the generation process.
Citation Information
Patent Citations
Method and system for answering question and recording medium with recorded question answering program
JP2002132812A
Question answering device, question answering method, and program
JP2005025418A
Automatic identification and extraction of medical condition and fact from electronic medical treatment record
JP2019049964A
Information processor, diagnosis assist method, and diagnosis assist program
JP2020113003A
Method and apparatus for verifying medical fact, electronic device, computer-readable storage medium, and computer program
JP2021111411A
