Information output system

Abstract

Conventional systems are unable to generate concise and comprehensive answers to user questions and search terms. It has been difficult to provide a good balance of short summaries and detailed explanations, and to generate answers that align with the user's intent. [Solution] The present invention provides an information output system comprising an answer generation module that generates short sentences (summaries) and long sentences (detailed explanations) in response to input questions or search terms, and a data management module that stores and reuses them. The answer generation module has an answer format selection unit 292A that selects a format such as a short summary or a long explanation, and an answer generation unit 292B that generates an answer based on the selected format. This generates accurate and comprehensive answers to the user's questions and search terms.

Description

Technical Field

[0001] The present invention relates to an information output system that uses an interactive system (chatbot or search system) to generate answers and search results for questions and input of search words from a user and output information. In particular, it relates to a system that analyzes input content using natural language processing and outputs it in a predetermined answer format that combines key point answers and long sentences (detailed explanations).

Background Art

[0002] Conventional question-and-answer systems and search systems (chatbots or search engines) generally provide simple search results or answers in fixed phrases for user input, or provide them in a format that the user does not desire. There is a problem that the cases where the user asks questions in natural language and the cases where the user searches using words are mixed, and if both are handled in the same processing route, there are variations in answer accuracy and comprehensiveness. In particular, in a process that does not distinguish between a question sentence and a search term, the generated AI answers and simple search results are mixed and output, making it difficult to ensure explanatory power and reliability.

[0003] Patent Document 1 is a fixed-phrase creation support device that analyzes a sentence input by a user and outputs a fixed answer sentence for it. This device analyzes the input sentence by keywords, selects the most appropriate one from prepared answer samples, and displays it, and has a function of automatically determining the type of sentence sample selected by the user. It is possible to call sentence samples more easily without the user instructing the type of sentence sample to be the answer.

[0004] Patent Document 2 stores past questions and corresponding answers in a database, calculates the similarity for a new question from a user, and extracts an appropriate answer based on the past questions. It is possible to output an answer candidate sentence suitable for a new question sentence.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Patent Application Publication No. 7-73180 [Patent Document 2] Japanese Patent Publication No. 2023-31791 [Overview of the project] [Problems that the invention aims to solve]

[0006] The present invention aims to provide an information output system that can determine whether the input content is a question or a search term using natural language processing, establish the optimal processing route for each, and then output a unified format combining a key response (short sentence) and a longer sentence (detailed explanation).

[0007] In other words, a problem with conventional technology is the lack of output formats for generating answers that align with the user's intent, or the fact that even for similar questions, only inappropriate answers that do not align with the user's intent may be obtained. Specifically, simply providing standardized answers, as in Patent Document 1, presents the following problems.

[0008] <It is difficult to provide answers that accurately reflect the user's intent.> Traditionally, systems simply return information that matches the user's question, without attempting to deeply understand the background or intent of the question or generate appropriate answers.

[0009] <The level of detail and comprehensiveness of the responses is insufficiently adjusted.> Some users request concise summaries, while others seek more detailed explanations and supporting evidence. Traditional systems struggle to accommodate both, making it difficult to provide the necessary information without excess or deficiency. Furthermore, while the supporting papers for user responses are constantly updated, the core points of the user's response rarely need updating, resulting in significant management effort.

[0010] <Insufficient use of past answers and search results> Since users rarely reuse questions they've asked or answers they've received in the past for subsequent searches or questions, and new answers need to be generated every time, the system is inefficient. It would be better to output the answers and search results the user wants based on past data.

[0011] <The information lacks reliability and sufficient evidence.> The search results and answers lack credibility because they do not indicate the source of information, particularly whether they are based on academic papers or reliable data.

[0012] To address these challenges, the present invention aims to provide an advanced question-answering system that enhances its suitability to user needs. [Means for solving the problem]

[0013] This invention has been made in view of these problems, and provides a more flexible and reliable information output system (program, method) by providing key answers to questions and searches, along with detailed explanations and justifications related thereto.

[0014] (1A) An information output system that, in response to a question or search term input from a user, outputs one or more of either a response or a search result generated in a predetermined format. By outputting in a "predetermined format," the content and structure of the answers are consistent, allowing users to obtain stable and consistent answers even when asking multiple questions. In particular, by combining a key point response (summary) with a longer text (detailed explanation), users can easily understand the main points and access detailed information as needed.

[0015] To classify the input as either a "question" or a "search term," a natural language processing model analyzes multiple elements of the input sentence, including its syntactic structure, sentence-final expressions, vocabulary's parts of speech, dependencies, and the presence or absence of interrogative words. Specifically, if the sentence ends with an interrogative form such as "ka," "desu ka," or "deshou ka," or contains an interrogative word (such as "nani," "doko," or "naze"), it is determined to be a "question." On the other hand, if verbs and particles are missing and a sequence of nouns forms the main structure, or if it is a search engine-type word input, it is determined to be a "search term." This classification is performed automatically based on sentence vector similarity and syntactic score calculated by a neural language model (such as BERT).

[0016] For inputs classified in this way, if it is a "question," the answer generation unit 292B operates based on the semantic analysis results, and the generation AI generates an answer that includes a key response and its supporting explanation. On the other hand, if it is a "search term," the search unit 292C operates, extracting similar data using cosine similarity, Word2Vec, BERT, etc., and formatting the extracted results to generate a key response and related information. Although the generation methods for the two are different, they are both integrated from S304 onwards and formatted into a unified format of short sentences (key response) and long sentences (explanatory text) by a common answer format selection unit 292A. Subsequently, the data management module saves and reuses them in the same structure, so although "answers" and "search results" are generated through different paths, they are handled uniformly at the output stage. In other words, although the question route and the search route are processed by different analysis modules, both are ultimately formatted by a common response format selection unit and output in a unified format of key responses and long sentences. Therefore, even if different processing is performed on the two routes, the output format is unified.

[0017] (1B) The predetermined response format includes a response generation module that generates a key response obtained based on the content of the question or the search terms, and a long sentence explaining the key response. A summary response includes short information such as a conclusion or summary (key points), while a longer text includes detailed information, evidence, and links to papers that supplement the summary (key points). Either one or both of these texts make it easier for users to grasp the information step by step. Furthermore, generating a longer text based on the summary response ensures that users receive information that is neither too much nor too little.

[0018] (1C) The response of the response generation module generates an explanation of the main points response as the long sentence. Specifically, the response generation module analyzes the input content and performs the following processing. (a) The input analysis unit analyzes the structure of the input sentence using a natural language processing model and determines the question intent based on sentence endings, particle structure, interrogative words, dependency relationships, etc. If a question intent is recognized, the input is classified as a "question," and if the input is a list of words or mainly consists of noun phrases and no question intent is not recognized, it is classified as a "search term." (b) In the case of a question input, the answer generation unit generates a key response based on the semantic analysis results, obtains supporting information from external sources (paper database, knowledge database, etc.), and generates a detailed explanation. (c) When a search term is entered, the search unit retrieves relevant information based on similarity calculation (cosine similarity, Jacquard index, Word2Vec, BERT, etc.) and generates a summary response and related explanatory text. (d) Both the question route and the search route are integrated from step S304 onward and output in a predetermined answer format consisting of a summary response plus a longer text. The longer text can include additional explanations based on the information provided in the summary response, as well as data and supporting evidence, to enhance the reliability of the answer for the user.

[0019] Furthermore, it is possible to use a large-scale language model to generate longer texts based on key responses and related text and abstract information from relevant papers. In this case, if the quality of the key responses is evaluated and saved in advance by users, users can view the text with confidence that the overall gist is correct.

[0020] It includes a data management module for saving the generated answer, and the data management module searches for the saved answer for a new input and outputs this answer, or generates a new answer using the saved answer. The saved answer can be searched and the past answer can be reused for similar new questions. Or, for the same question or similar questions, a new answer can be generated based on the saved answer, enabling consistent answer provision and reducing the risk that the user receives different answers each time.

[0021] Furthermore, the data management module assigns an approval flag for content confirmation by medical staff and saves the answer with the approval flag as a pair with a long text. The answer with the approval flag skips the regeneration process and is immediately output as is when the same or similar input occurs next time. This can reduce processing time and computing resources.

[0022] Here, the summary answer stored in the data management module can also be automatically generated using a summarization algorithm. It is also possible to combine methods such as generating a patterned answer based on question category and template mapping, summarization generation by keyword extraction, using an existing database, template-based answer generation, and natural language generation by machine learning.

[0023] Note that the "answer" refers to the output generated based on semantic analysis and generative AI processing by a natural language processing model for a natural sentence containing the question intention in the user input. On the other hand, the "search result" refers to the information obtained using a similarity calculation algorithm (such as cosine similarity, Word2Vec, BERT, etc.) for a word or keyword sequence that does not contain the question intention. However, the output formats of both are unified, and in either case, they are presented in a two-layer structure of a summary answer and a long text (detailed explanation text), and are saved and reused in the same format by the data management module.

[0024] The relative criteria for distinguishing between "straightforward responses" and "long texts" are the number of characters or words, the depth of information (straightforward responses generally convey superficial points that concisely summarize or conclude, while long texts include detailed background information, evidence, logical explanations, and specific examples), and the purpose of the information (straightforward responses prioritize conciseness, while long texts focus on specialized knowledge and detailed analysis). Thus, regarding "short" (straightforward responses) and "long," a straightforward response provides concise content that shows the main points, such as conclusions or summaries, so that the user can quickly understand it, by limiting the amount of information. A long text goes beyond a straightforward response, providing detailed background information, evidence, and specific examples to allow the user to understand it more deeply.

[0025] The new answers generated by the data management module can either reuse past answers and use the stored answer content as is, or maintain the answer format while making slight modifications to create a "new answer." Alternatively, it can combine multiple elements from past answers to generate an answer that is optimal for the new question, update information in real time based on new search results, generate individually optimized answers that received high ratings and satisfaction based on the user's previous questions and answer history, generate the most appropriate answer from stored data using tags and metadata, provide answers in different formats, or generate answers by comparing and contrasting multiple answers to similar questions, thereby generating answers that analyze multiple answers from a new perspective. In other words, the form of the "new answer" can be adaptively generated using various methods, such as reusing past answers, acquiring new data, personalization based on user history, classification using tags, and even combining different expression formats and references to external information. Specifically, the accuracy of short sentences is ensured by saving the key points of the responses and having experts such as doctors evaluate their comprehensiveness and accuracy. Subsequently, it is possible to generate longer sentences by combining the key points with related research paper information and providing it to the generating AI as a prompt.

[0026] (2) The data management module saves the generated responses with one or more search terms or search tags for the categories of the responses, and uses the search results of the search using the response search terms or search tags stored in the data management module as the output for the new input. By using search terms or tags, you can quickly retrieve answers tagged with relevant terms or tags, easily search and extract previously generated answers, and provide appropriate answers to new questions. Furthermore, using tags makes it possible to suggest highly relevant answers to new questions. For example, an answer tagged with "symptoms of the disease" will also be referenced in response to another question, "How can this disease be prevented?", enabling the provision of more comprehensive information. In addition, tagging allows for a precise understanding of the content of answers, enabling the provision of consistent and accurate answers to similar questions.

[0027] Tagging is automatically assigned based on keywords automatically extracted using natural language processing. Tags have a hierarchical structure, classifying responses by subject and category, improving the efficiency of answer searches. The use of multiple tags allows for complex searches and the extraction of related answers. Search tags are keywords, categories, or related search terms that concisely represent the content of the answer, and are either automatically extracted by the answer generation module or manually assigned by the system administrator.

[0028] Specifically, the tagging mechanism assigns keywords and metadata (tags) to generated answers. Tags are automatically generated through keyword extraction using natural language processing (NLP). Furthermore, the tags have a hierarchical structure, organized according to subject and category (disease name, symptom name, treatment method, etc.), making it easy to identify answers with multiple related tags. When a user enters a new question, the question content is analyzed and matched with related tags. This matches the tags assigned to saved answers with the input content, allowing the most relevant answers to be output. Tags can also include information about the answer format (e.g., "summary," "detailed explanation," "case study," "academic paper link," etc.), allowing control over the answer format and ensuring appropriate output. Multiple tags can be assigned to a single answer, and combinations of tags enable detailed and complex searches. Tag filtering allows users to narrow down the answers that best match their intent.

[0029] Tagging can be done manually by selecting terms from a list, or it can be done using a generative AI. Specifically, tags are automatically generated by extracting keywords using natural language processing based on questions and answers. Furthermore, tags can be given a hierarchical structure, organized according to subject or category, making it easy to identify answers with multiple related tags. When a user enters a new question, the question content is analyzed and compared with related tags, matching the input content with the tags attached to saved answers, and the most relevant answer can be output. In addition, information about the answer format (e.g., "summary," "detailed explanation," "case study," "academic paper link") can also be attached to the tags, allowing control over the type of response and output in the appropriate format. Multiple tags can be attached to a single answer, and combinations of multiple tags enable detailed and complex searches. By filtering tags, the answer that best matches the user's intent can be narrowed down. Tag types include disease names, symptom names, treatment names, types of diagnosis / treatment, treatment names, and research types.

[0030] (3) The summary response must include at least one of the following: summary information or key points. The longer text must include at least one of the following: explanation, commentary, evidence, a link to a supporting paper, or case information.

[0031] A "keyword response" provides concise information that allows users to quickly grasp the main points, while a "longer text" adds detailed explanations to support that summary, providing evidence and credibility. This allows users to meet both or both of their needs: wanting to know the conclusion quickly and wanting a deeper understanding. Furthermore, including evidence, links to academic papers, and case information in "longer texts" can enhance the accuracy and reliability of the answers. Users can not only obtain conclusions but also understand the data and research on which those conclusions were derived, using this information to judge the appropriateness of the answers. Especially for specialized questions, citing relevant research and case studies in "longer texts" provides detailed information useful not only to general users but also to experts. Combining the two formats—simple answers and "longer texts"—can meet the diverse needs of users. Summaries are provided for users who want to know the main points in a short time, while evidence and links are provided for users who want detailed information, enabling flexible and effective information delivery.

[0032] (4) Having a research paper search database and storing one or more of the titles, full texts, or abstracts of research papers in the said research paper search database, The response generation module performs a search of the aforementioned research paper database in response to a question or search term, obtains research paper information, and uses it as RAG to generate responses and search results. Using research paper information, the system generates long texts based on RAG (Retrieval-Augmented Generation) technology, while also obtaining research paper information from a research paper search database. RAG is a technology that improves the quality of responses generated by utilizing the information obtained through searches, helping AI to directly utilize search results to provide users with more accurate and reliable answers. According to the present invention, the reliability of the answers is increased because academic papers and reliable data are directly referenced based on the search terms, and because answers are always generated based on the latest papers and research data, the latest information can be provided to the user. Furthermore, by dynamically generating appropriate search terms according to the content of the question and searching the paper search database, it has the flexibility to handle a wide range of topics.

[0033] Furthermore, RAG technology can generate highly accurate and reliable answers by combining generative models and search functions. In the medical field, RAG technology can be used by doctors and healthcare professionals to search for the most relevant and up-to-date research papers and guidelines based on a patient's symptoms, generating information useful for diagnosis and treatment. This enables medical care that incorporates the latest research findings.

[0034] (5) The system generates answers in accordance with PREP based on the paper information obtained through the search. Specifically, based on the paper information obtained through the search of papers using the aforementioned paper search database, the answer generation module performs prompt control in accordance with the PREP method, instructing the answer generation AI to generate answers in accordance with points, explanations, evidence, and conclusions, and outputs the results. PREP is a framework consisting of four elements: Point (conclusion), Reason (reason), Example (example), and Point (reiterated conclusion). Because it follows this framework, it provides answers with a logical and clear structure. The flow of conclusion, reason, example, and reaffirmation organizes information and increases persuasiveness. By controlling the response generation module using prompts that utilize the PREP framework, it can generate logical and persuasive answers, providing reliable answers based on research paper information. The response generation module generates PREP so that the first P corresponds to a short sentence, and REP corresponds to a longer sentence. In medical papers, the order of summary, explanation / explanation, evidence / basis, and summary is also acceptable.

[0035] The prompts that generate answers according to the points, explanations, evidence, and conclusions implement instructional commands (prompts) for the answer generation module to operate based on the PREP method and apply a structured PREP template. Each element of Point, Reason, Example, and Point in the PREP method is controlled as an output block, with Point corresponding to a short sentence (summary response) and Reason, Example, and Point (reiteration) being generated as a longer sentence (detailed explanation).

[0036] (6) The search engine for the text of the aforementioned papers will organize the data of the aforementioned papers by tagging it. Tagged research paper data is tagged with keywords and themes based on the content of the papers, allowing search engines to efficiently organize information and enable users to quickly access the information they are looking for. This makes it possible to accurately extract highly relevant papers from a large amount of data. Furthermore, by assigning tags, search engines can provide more relevant results based on the content of the papers, rather than simply matching keywords.

[0037] (7) Users evaluate the comprehensiveness of the answers and the appropriateness of the order in which they are written, and save the results of their evaluation. The evaluation results are saved in the data management module and fed back into the prompt selection and answer template selection in subsequent answer generation. This enables autonomous improvement of answers based on user evaluations. According to this invention, by increasing the comprehensiveness of the key points response, the main points and information related to the input are covered without omission. In the key points response that enhances comprehensiveness, ambiguous expressions such as "some..." and "in some cases..." are avoided, and clear answers are provided to the user. Furthermore, in terms of comprehensiveness of information, it is possible to display a list of tests in the case of tests, and a list of drugs in the case of drugs, in order of importance and frequency, and to add a note that specialists have confirmed the comprehensiveness and guaranteed the quality of the information. It is generally known that AI-generated answers often have insufficient comprehensiveness. Specifically, there is a report that about 30% of the responses to questions about cardiovascular medicine treatments had problems with comprehensiveness. In this invention, it is possible to generate highly comprehensive texts by preparing short sentences that have been confirmed to guarantee comprehensiveness, and then having the AI-generated text create longer sentences based on these. In fact, the comprehensiveness error has been reduced to less than one-third.

[0038] (8) The input and output are performed using a chatbot system or a search system. By using a chatbot system, it is possible to generate and provide answers to user input quickly and in real time. Furthermore, the chatbot enables interactive, conversational responses by providing additional questions and information through dialogue with the user. If the user requests a more detailed answer, the chatbot will generate an appropriate response by asking further questions. Furthermore, using a search system makes it possible to accurately retrieve and output the most relevant information from a large database. Chatbots enable interactive responses, while search systems provide high-speed search output. Furthermore, chatbots and search systems can flexibly adapt to the format of user input, enabling them to send and receive information in various forms, such as natural language questions, specific search keywords, and voice input, thus broadly addressing user needs.

[0039] (9) The data management module assigns an approval flag to the generated summary response to indicate that the content has been reviewed by a healthcare professional, saves the summary response with the approval flag and the long text associated with that summary response as a pair, and when there is a new input that is identical or of a predetermined similarity or higher, it retrieves the pair and outputs it as is. The approval flag serves to ensure the reliability of information. In particular, for highly specialized and accuracy-demanding information such as medical information, the approval flag indicates that the validity of its content has been verified by experts. For example, a healthcare professional (doctor, pharmacist, etc.) can review the content of a key point response via a system administrator interface. If they determine that the content is accurate and appropriate, they can assign an "approved" flag to that response. This approval flag is stored in the data management database, linked to the corresponding key point response data. Key point responses with the approval flag are stored as a pair with their associated longer text and are preferentially reused as a highly reliable set of responses. Furthermore, the approval flag can have multiple statuses, such as "unverified," "under review," "approved," and "revision requested," allowing for a systematic information quality control process. (10) The data management module reduces the computational resources or processing time required to generate long sentences by reusing the long sentences corresponding to the key points responses to which the approval flag has been assigned, without regenerating them. (11) The preparation of the summary response may involve human intervention, including that of healthcare professionals or system administrators.

[0040] In fields such as healthcare, healthcare professionals can add an approval flag to the generated summary responses to indicate they have reviewed the content. This flagged data is reused without being regenerated, ensuring both stable response quality and reduced computational load. Initial creation of the summary responses can also be done manually by healthcare professionals or system administrators. [Effects of the Invention]

[0041] According to the present invention, more appropriate and detailed answers are provided to user questions or search terms, specifically by providing answer formats consisting of a summary response and a longer text (detailed explanation and evidence), thereby providing information that is appropriate to the user's needs. In particular, by providing a concise answer to the input in the form of a short summary response, along with a longer answer (long text) that further explains the content, the user can first grasp the overall picture before understanding the details.

[0042] Furthermore, by automatically distinguishing between questions and search terms during the input analysis stage and applying different algorithms to each, processing branching can be clarified. In addition, by integrating the outputs of both and presenting them in a common format of key points + detailed explanation, the consistency and explainability of the answers can be improved. [Brief explanation of the drawing]

[0043] [Figure 1] A diagram showing the overall configuration of an information output system according to an embodiment of the present invention. [Figure 2] A block diagram showing an example of the functional configuration of a server. [Figure 3] A flowchart illustrating the processing steps of an information output system program. [Figure 4] An explanatory diagram of an embodiment of the present invention. [Figure 5] A diagram illustrating an example of a question and an answer generated in a predetermined format. [Figure 6] A diagram to explain a comprehensive answer. [Figure 7] A diagram illustrating an example answer to a question. [Figure 8]Examples of instructions and specifications for the format of user responses. [Figure 9] A diagram illustrating an example of a question and an answer generated in a predetermined format. [Modes for carrying out the invention]

[0044] Figure 1 shows the overall configuration of an information output system 1 according to an embodiment of the present invention. In this system 1, a PC 10 or portable device 30, which serves as a user terminal, and a system server 20 are connected via a wired / wireless network 80 such as the Internet. The server 20 is a server that functions as a web server (including a cloud server) and exchanges information with terminals 10 and 30 via web pages. In addition, while terminals 10 and 30 have a web browser installed for viewing web pages, a dedicated application for enjoying the services of the server 20 may also be installed, and the system may be configured so that web pages can be viewed using the dedicated application. This system 1 is a system that outputs information in a predetermined answer format that combines a key point answer and a long sentence in response to a question or search term input from the user. In particular, it generates and provides useful information in fields where the accuracy, basis, and comprehensiveness of information are considered important, such as the medical and academic fields.

[0045] The user terminal 10 is connected to the server 20 via the network 80 so as to be able to communicate with it. The user inputs questions or search terms, and the output content of the answers or search results is provided via a visual screen or audio. The user terminal 10 is connected to the network 80 by communicating with communication devices such as a wireless base station 81 that supports various communication standards such as LTE, and a wireless LAN router that supports IEEE and wireless LAN standards. The user terminal 10 includes a communication IF 12, an input device 13, an output device 14, a memory 15, a storage unit 16, and a processor 19. The terminal 10 as a user terminal is a desktop or laptop PC, while the terminal 30 is a mobile terminal such as a tablet or smartphone. The user terminal 10 has a web browser and application software dedicated to this system installed. The user inputs questions or search terms to the information output server 100 using a user interface provided through this web browser or dedicated application, such as a chatbot-style dialogue screen or a search screen with a search box, and receives and displays the answers sent from the information output server 100.

[0046] The communication interface 12 is an interface for the user terminal 10 to communicate with the server 20 and input / output signals. The input device 13 is an input device (such as a keyboard, touch panel, touchpad, mouse, or other pointing device) for receiving input operations from the user. The output device 14 is an output device (such as a display or speaker) for presenting information to the user. The memory 15 is for temporarily storing programs, data processed by programs, etc., and is a volatile memory such as DRAM. The storage unit 16 is a storage device for saving data, such as flash memory or an HDD. The processor 19 is hardware for executing the instruction set written in the program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0047] Server 20 receives input of questions or search terms from user terminals 10 and 30, generates answers or search results in a predetermined format, and transmits the answers or search results to user terminal 10. Server 20 is a computer connected to a network 80 such as the Internet, and includes a communication IF 22, an input / output IF 23, memory 25, storage 26, and a processor 29.

[0048] Communication IF22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices. Input / Output IF23 functions as an interface to an input device for receiving input operations from the user and an output device for presenting information to the user. Memory 25 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM. Storage 26 is a storage device for saving data, such as flash memory or an HDD. Processor 29 is hardware for executing the instruction set written in the program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0049] <Functional configuration of Server 20> Figure 2 is a block diagram showing an example of the functional configuration of server 20. Server 20 is electrically connected by a bus to a communication means 220, an input device 230, an output device 240, a storage means 280, a control means 290, a response generation means 292, a display control means 293, and a user IF 294.

[0050] The communication means 220 performs modulation and demodulation processing for the server 20 to communicate with user terminals 10 and 30, processes the signal calculated by the control means 290 for transmission, and transmits it to external devices and equipment. The communication means 220 processes the signal received from the outside and outputs it to the control means 290. In this way, the communication means 220 interprets commands or input content and provides them to each means, and also functions as an interface that interprets various display commands issued from the storage means 280 and performs output control.

[0051] The input device 230 is a device used by an administrator to input instructions or information to operate the server 20 as needed, and may be a keyboard, mouse, reader, or touch-sensitive device. The input device 230 also converts the instructions input by the administrator into electrical signals and outputs the electrical signals to the control means 290. The input device 230 also includes a receiving port that accepts electrical signals input from external input devices. The output device 240 is a display device 241 such as an LCD or organic EL that provides information to an administrator who operates the server 20 as needed. The display 241 can display data corresponding to the control content of the control means 290 and can check the communication status between the server 20 and other external devices 10, 30.

[0052] The storage means 280 is implemented by memory (RAM) 25 and storage 26 such as a disk device (floppy disk, hard disk, or magneto-optical disk, etc.) and stores data, programs, etc. used by the server 20. The storage means 280 stores the application program 282 of this system, as well as data for the work area 281, data storage area 283, and screen definition storage area 284.

[0053] The work area 281 is a region that is secured when the system is started and where various data input and output by the system are temporarily stored. The data storage area 283 is a region where data temporarily stored in the work area 281 is semi-permanently stored through write control when a save request is made. The screen definition area 284 is a region where screen definition information for various screens to be output and displayed to user terminals 10,30 is stored in advance, and includes format information for screen settings to be displayed by the display control means 293.

[0054] The answer format DB283A is a structured database that stores a predetermined answer format. The database stores information such as key points in answers, long sentences, links to papers and articles, case information, reliable data and other supporting information, and answers organized in formats such as PREP (Point, Reason, Example, Point) for each question. Furthermore, to enhance the comprehensiveness of the answers, an answer template has been created that lists specific examples in key points answers, and a function to insert specific examples is incorporated into the answer generation module. It is also possible to set a flag during answer generation to automatically insert specific examples into key points answers.

[0055] Answer DB283B is a database that stores generated answers or search results. Answers and search results generated based on user questions or search terms are automatically saved in Answer DB283B. If a similar question is asked later, already generated answers or search results can be reused. Also, if new user input is similar to past input, existing answers can be retrieved and presented from Answer DB283B. Furthermore, the response database 283B contains responses stored in multiple formats (e.g., summary responses, longer texts, and texts with specific examples). Users can select the appropriate response format according to their needs and input type.

[0056] Furthermore, the answer database (DB283B) tags the stored answers, allowing for efficient searching of answers related to specific keywords or content. Tagging enables the selection of the most suitable format for the user's question.

[0057] User DB283C can provide personalized services by storing personal and identifying information such as the user's name, age, gender, and contact information. By storing information on questions and search terms previously entered by the user, it can track what questions were asked in the past and provide appropriate answers to similar questions. By saving answers and search results previously provided to the user, answers can be efficiently reused and their content improved. By storing information such as the user's preferred answer format, frequently used keywords, and topics of interest, it is possible to adjust the answer format to suit the user. Paper search DB283D stores one or more of the following: paper title, full text, or abstract, or information obtained from a paper search system on an external cloud.

[0058] The display control means 293 controls how the system displays the answers and search results generated by the system to the user terminals 10 and 30. It performs processing such as displaying a summary response followed by a longer text (detailed explanation), determining which format of answer to display in response to the user's question or search term input, and dynamically updating the screen display when the user asks a new question or when displaying search results based on saved answers.

[0059] User interface 294 performs functions such as accepting questions and search terms from user terminals 10 and 30 via keyboard input or voice input, creating a layout that is easy for the user to view based on the screen format information in screen definition area 284, and providing buttons or links for the user to ask further questions or request details regarding the displayed answers.

[0060] Figure 3 is a flowchart of an information output system that shows the processing steps of a program that generates and outputs a response to user input.

[0061] The input means 291 of this system is in a waiting state for input from user terminals 10,30 (step S301) and determines whether a question or search term has been entered (step S302). If there is no input, it continues in the waiting state, but if it is determined that there is input, it analyzes the input content (step S303). If the input content is a free-text question, it processes to output an answer, and if the input content is a free-text search term, it processes to output search results. The analysis of the input content analyzes the structure and lexical information of the input sentence using a natural language processing (NLP) model and determines whether there is a question intent. The intent of the question is analyzed by the appearance of question words ("what," "where," "why," etc.) and sentence-ending expressions ("is it," "is it?", etc.) to identify the category and type of question. If a question intent is detected, the input is classified as a "question."

[0062] On the other hand, if the structure of the input sentence is noun phrase-centric and the intent of the question is unclear, or if it consists only of words or keyword sequences, it is classified as a "search term." The response to the input of a search term is analyzed by evaluating the word match rate or similarity. Similarity can be measured not only by exact matches, but also by using methods such as cosine similarity, Jacquard index, or word embedding models such as Word2Vec or BERT to measure the similarity between words in the query and the target data.

[0063] Next, the system's storage means 280 is referenced to search the answer DB 283B and check whether answers to similar questions have been stored in the past (step S305).

[0064] If no existing answers exist, the answer generation means 292 first selects an answer format to be generated by the answer format selection unit 292A (step S304). Based on the input content and the user's past preferences, it selects an answer format stored in the answer format DB 283A (e.g., summary response, long text or both, pyramidal type, inverted pyramidal type, short essay type, syllogistic type, bulleted list type, Q&A type, storytelling type, comparative type, chronological type, conclusion-first type, etc.). If based on the user's past history, it refers to the user DB 283C and refers to the evaluation results of the comprehensiveness of the answer and the appropriateness of the order of presentation as a predetermined format, which were previously evaluated by the user.

[0065] In other words, the "data management module" implements the function of selecting and providing answer formats through functions executed by the answer format selection unit 292A, answer format DB 283A, answer DB 283B, and user DB 283C. The "data management module" uses the answer format selection unit 292A to determine the optimal answer format based on the user's input and preferences, and selects an appropriate format from the answer format DB 283A. By referring to the answer DB 283B, it is possible to reuse previously generated answers for new questions, and based on the information accumulated in the user DB 283C, it provides the optimal answer format and content for each individual user. The user DB 283C stores the evaluation results of user assessments regarding the comprehensiveness of the answers and the validity of the order in which they are written as a predetermined format. By saving the output for input with tags and using these tags to generate output for new inputs, answers can be reused efficiently, and if there are answers with similar tags, output can be generated based on them, providing consistent answers for similar questions. Tags can be assigned to specific keywords, topics, categories, metadata related to the intent of the question, etc. For example, you can pre-define tags such as "disease," "treatment," and "topic."

[0066] Furthermore, the "data management module" has the function of accumulating past answers and presenting the most appropriate answer in response to new input from the user. This ensures that consistent answers are provided even when the user repeatedly asks similar questions, improving the efficiency of the system.

[0067] Next, the response generation means 292 generates a response based on the selected response format (step S306). It may reuse past responses, or create a new response if there are no past responses. The response content is generated as a response by the response generation unit 292B or as a search result by the search unit 292C, and the created response content is saved in the response DB 283B (step S307). The response generation unit 292B or the search unit 292C uses a database search system or a knowledge database system to obtain response information related to the question from a database or a pre-trained knowledge base. Additional information can also be collected by using external APIs or search engines that utilize external resources or databases.

[0068] The search engine has the functionality to search a database of research papers. In response to requests from the evidence retrieval function, the search engine efficiently searches for the most relevant research papers based on the entered keywords or natural language queries, and returns the results to the evidence retrieval function. The search engine offers a variety of search functions, including full-text search, keyword search, author name search, and filtered search by publication year. Furthermore, the search engine pre-indexes research paper data to achieve high-speed searches. In addition, the search engine has the functionality to analyze research paper data and automatically tag it based on its content. For example, it assigns tags indicating the subject of the paper (e.g., "lung cancer," "immunotherapy"), the type of research (e.g., "clinical trial," "review"), and the methods used, structuring and organizing the research paper data. This tagging improves search accuracy and recall, enabling the acquisition of more appropriate evidence.

[0069] In step S306, based on the classification of the answer or search result (step S303), the answer generation unit 292B is activated if a question is entered, and the search unit 292C is activated if a search word is entered. Based on the NLP analysis results, the question route obtains key answers, searches the paper search DB 283D as needed to obtain supporting information, and generates a long explanatory text based on this. In the search route, similar information is searched for using word embedding models such as cosine similarity, Jacquard exponent, Word2Vec, and BERT for the input words, and relevant key answers are obtained to generate an explanatory text.

[0070] The generated response is sent to user terminals 10 and 30 to output it to the user (step S308). After the above processing, this subroutine terminates.

[0071] This section details the functions of the "Answer Generation Module." The "Answer Generation Module" is primarily implemented by the answer format selection unit 292A, the answer generation unit 292B, and the search unit 292C, and provides appropriate answers to questions. It implements functions such as generating a summary answer based on user input as a key-point response, generating long texts including detailed explanations and examples, generating search terms based on the question to collect information from external databases, a RAG module function that generates reliable answers based on the retrieved information, and a source information generation function that attaches links to papers and external information to the answer. Note that RAG (Retrieval-Augmented Generation) technology is used for generating search terms. The "Answer Generation Module" generates summary information as a key response and provides detailed explanations and supporting information as a longer text. This module dynamically generates search terms and refers to a reliable external database to always present the user with the latest information. In addition, answer pairs (key response + long text) that have been flagged as approved have a control flag set to skip the regeneration process. Therefore, the next time the same or similar question is entered, the regeneration will be omitted in the approval flag confirmation step, and the saved output data will be output immediately.

[0072] The "Data Management Module" refers to the answer DB283B to determine if there are past answers to the same or similar questions / search terms (S305). If the similarity is above a predetermined value, past answers can be reused, saving computational resources. On the other hand, if a new response is generated (S306), it is saved in the response DB283B as a set of a summary response and a longer text (S307). When saving, if the summary response has been reviewed by a medical professional, a "reviewed" flag can be added. Summary responses and longer texts with the "reviewed" flag are reused without being regenerated thereafter.

[0073] Furthermore, after input analysis (step S303), the response format selection unit 292A selects the response format best suited to the user's needs, and the response generation unit 292B generates a specific response. The generated response can also be output as a highly reliable response to the user by the search unit 292C, which may incorporate search results as needed.

[0074] The flowchart shown in Figure 3 illustrates the close collaboration between the answer generation module and the data management module. The answer generation module generates appropriate answers based on the input of questions and search terms, while the data management module stores and manages the generated answers. If the same question or search terms are entered again, that is, if the system searches the answer DB283B to see if the "same question" exists, and if the "same question" is found, the data management module searches for past answers, saving the effort of regenerating by the answer generation module and allowing for the selection of answer formats based on past answers for the same user.

[0075] Figure 4 is an explanatory diagram of an embodiment illustrating examples of "tagging" and "answerable questions," with examples of tags shown in the upper section and examples of answerable questions in the lower section. Sub-tags are set for each of the "disease," "treatment," and "topic" tags, and "answerable questions" are defined according to each "tag." Pulmonary isopulsive pressure and pulmonary artery resistance are set as search terms or search words to associate "tags" with "answerable questions."

[0076] Figure 5 is an explanatory diagram of an embodiment illustrating the question content and response (answer) shown in Figure 4, and shows an example of a question and an answer generated in a predetermined format. In response to a question input from the user, the information output system generates the answer content according to the response and answer format and outputs and displays it on the user terminal. The answer format is divided into a summary response and a long sentence. The summary response is a summary of information, and the long sentence is an explanation related to the summary response, with the long sentence showing (explanation) and (conclusion). The short sentence of the answer is controlled by prompts to include numbers in the sentence and to generate content that enhances the comprehensiveness of the answer. In other words, the summary response is defined in the prompts to enhance comprehensiveness and generated by the generation AI. The long sentence is defined in the prompts to include numbers and other evidence and generated by the generation AI.

[0077] Figure 6 illustrates a comprehensive response; before the system's improvement, users were presented with incomplete responses, while after the improvement, users are provided with comprehensive and concise responses. Furthermore, the system allows for evaluation of responses by doctors and other professionals, and automatically improves the system to produce higher-rated responses. Evaluation points include comprehensiveness checks, conciseness checks, and evidence quality checks.

[0078] Figure 7 shows an example of an answer to a question; although the answer is detailed, it does not meet the user's requirements. Therefore, by allowing the user to specify or select an answer format as shown in Figure 8, the user can obtain an answer in a format that meets their requirements, such as points, reasons (examples of papers or case reports), and points, as shown in Figure 9. In the details of the reasons, examples of papers or case reports are embedded, or links such as papers 1 to 5 (paper 4) are attached to generate the answer. [Industrial applicability]

[0079] The present invention can be used in search systems such as chatbot systems that handle inquiries in a chat format as an information output system, search engines, and knowledge base systems. In particular, as a tool for providing medical information, it can provide detailed evidence based on reliable academic papers and case information, along with concise symptom descriptions and summaries of treatment plans, and is useful as a tool that contributes to improving the efficiency of medical treatment.

[0080] Furthermore, the reuse configuration of flagged answers improves the efficiency of computing resources and stabilizes the quality of answers, making the information output system of the present invention applicable to a wide range of fields, including healthcare, education, and automated FAQ responses. [Explanation of symbols]

[0081] 10: User terminal (13: Input device, 14: Output device, 15: Memory, 16: Storage unit, 19: Processor) 20: Server (25: Memory, 26: Storage, 29: Processor) 80: Internet network, 81: Wireless base station

[0082] The program of the present invention can be installed in computer terminals having computer functions such as CPU, memory, and storage, as well as mobile devices such as smartphones, tablets, and wearable devices, digital home appliances such as smart TVs, smart speakers, and smart home appliances, recording media such as USB memory, SD cards, hard disk drives (HDDs) / solid state drives (SSDs), dedicated equipment and terminals such as POS terminals, vending machines, ATMs, and medical equipment, and game consoles (home and portable). When this program is installed in medical equipment, it may be linked with the hospital's electronic health record (EHR / EMR) system.

[0083] [Note B1] A program for operating a computer that includes a processor and memory, The processor, according to the program, The system analyzes user input (questions or search terms), generates a response combining a key point and a longer explanation of that key point, and outputs it. This program is (a) A step of obtaining a key response corresponding to the question or search term from the data management module, (b) The steps of searching for external sources to obtain supporting information and generating a longer text that explains the main points response based on said supporting information, (c) A data management step which involves saving the generated response in association with the input, and reusing the saved response to output the result if the same or a predetermined degree of similarity or higher input is received again. A program that includes this. [Note B2] The data management step includes the program described in [Appendix B1], which stores search tags representing search words or categories related to the generated answers, and searches for and uses the stored answers based on the search tags in the output for a new input. [Note B3] The aforementioned summary response includes at least one summary or point, The aforementioned long text is a program described in [Appendix B1] that includes at least one of the following: a summary of the information or a description, explanation, evidence, a link to a supporting paper, or case information relating to the aforementioned points. [Note B4] It has a paper search database that stores at least one of the paper's title, abstract, or full text, In the above response generation step, the above paper search database is searched to obtain paper information, The program described in [Appendix B1] generates the above response using the acquired research paper information as the basis for RAG. [Note B5] The response generation step is a program described in [Appendix B4] that performs prompt control to generate a response that conforms to at least one of the following categories: points, explanations, evidence, and conclusions, based on the paper information. [Note B6] The search engine for searching the aforementioned paper database is the program described in [Appendix B4] that tags and organizes the data of the aforementioned papers. [Note B7] The user evaluates the comprehensiveness of the aforementioned answers and the appropriateness of the order of the answer format. The data management module is a program described in [Appendix B1] that stores the evaluation results and feeds them back to the response generation module's generation process. [Note B8] The programs described in [Appendix B1] to [Appendix B7], characterized in that the input and output are performed using a chatbot system or a search system. [Note B9] The data management module adds an approval flag to the generated summary response, indicating that the content has been reviewed by a healthcare professional. The key points response to which the approval flag is assigned and the corresponding longer text are saved as a pair. The program described in one of the [Appendix B1] to [Appendix B8] reuses the pair without regenerating it when re-inputting. [Note B10] The summary response stored in the aforementioned data management module is a program described in any of [Appendix B1] to [Appendix B9], which is registered through confirmation or creation by a healthcare professional or system administrator.

[0084] [Note C1] An information output method that generates and outputs an answer combining a key response and a longer sentence explaining the key response in response to a question or search term input from a user, (a) A step of obtaining a key response corresponding to the question or search term from the data management module, (b) The steps of searching for external sources to obtain supporting information and generating a longer text that explains the main points response based on said supporting information, (c) A data management step that stores the generated response in association with the input, and reuses the stored response when the same or a predetermined level of similarity or higher input is received again. A method for outputting information that includes this information. [Note C2] The data management step saves search tags that represent search words or categories related to the generated responses. The information output method described in [Appendix C1], which searches for and uses saved answers based on the search tag when outputting to a new input. [Note C3] The aforementioned summary response includes at least one summary or point, The information output method described in [Appendix C1] includes at least one of the following: the long text, the summary information or the point, the explanation, the rationale, the link to the supporting paper, or the case information. [Note C4] It has a paper search database that stores at least one of the paper's title, abstract, or full text, In the above response generation step, the above paper search database is searched to obtain paper information, The information output method described in [Appendix C1] generates the above answer using the acquired paper information as the basis for RAG. [Note C5] The response generation step is an information output method described in [Appendix C4] that performs prompt control to generate a response that conforms to at least one of the categories of points, explanation, evidence, and conclusion, based on the paper information. [Appendix C6] The search engine that searches the aforementioned paper database uses the information output method described in [Appendix C4] to tag and organize the data of the aforementioned papers. [Note C7] The information output method described in [Appendix C1], wherein the user evaluates the comprehensiveness of the answers and the appropriateness of the order of the answer format, the data management module saves the evaluation results, and feeds them back to the generation process of the answer generation module. [Note C8] The input and output are performed via a chatbot system or a search system using one of the information output methods described in [Appendix C1] to [Appendix C7]. [Note C9] The data management module adds an approval flag to the generated summary response, indicating that the content has been reviewed by a healthcare professional. The key points response to which the approval flag is assigned and the corresponding longer text are saved as a pair. The information output method described in any of the [Appendix C1] to [Appendix C8], which reuses the pair without regenerating it when re-inputting. [Note C10] The summary response stored in the data management module is registered via confirmation or creation by a healthcare professional or system administrator using one of the information output methods described in [Appendix C1] to [Appendix C9].

[0085] A system for managing the quality of responses is implemented as one of the functions of an information processing system. Specifically, the comprehensiveness of the answers and the appropriateness (satisfaction level) of the order in which they are presented according to the prescribed format are evaluated by users, and these evaluation results are stored in a data management module. Since each user's judgment of appropriateness will differ, it is preferable to save the evaluations for each user and automatically improve the AI ​​used in the response generation module to produce answers that receive high ratings. In addition, the quality of the answers is managed by having doctors evaluate short sentences through checks.

Claims

1. In an information output system that outputs information in a predetermined response format combining a concise answer and a longer sentence in response to a user's question or search term input, It comprises a response generation module and a data management module, The aforementioned data management module stores key answers to questions or search terms. The response generation module analyzes the input and - Obtain the key points response corresponding to the input from the data management module, - Search external sources to obtain supporting information, Based on the aforementioned supporting information, generate the aforementioned long text explaining the main points of the response. - Output the response including the above-mentioned key points and the above-mentioned long text. The data management module stores the responses output by the response generation module in association with the inputs, and when a new input is found that is identical to or has a predetermined degree of similarity to a stored input, the data management module uses the stored responses to output the new input.

2. The data management module stores at least one of the search terms related to the generated response, or a search tag representing the category of the response. The information output system according to claim 1, wherein the output for the new input utilizes the search results based on the search word or the search tag.

3. The aforementioned summary response includes at least one summary or point, The information output system according to claim 1, characterized in that the long text includes at least one of the summary information or the points, such as an explanation, commentary, evidence, a link to a supporting paper, or case information.

4. It has a paper search database that stores at least one of the paper's title, abstract, or full text, The aforementioned external information source includes the aforementioned paper search database, The information output system according to claim 1, characterized in that the response generation module, in response to the input of a question or search term, performs a search of the paper search database to obtain paper information, and uses the obtained paper information as supporting information to generate the long sentence.

5. The information output system according to claim 4, characterized in that the response generation module uses prompts to generate a response that follows at least one of the categories of points, explanation, evidence, and conclusion, based on the paper information obtained from the paper search database, and outputs a response configured according to the said categories.

6. The system has a search engine that searches the aforementioned paper search database, The information output system according to claim 4, wherein the search engine tags and organizes the data of the aforementioned papers.

7. Regarding the above-mentioned answers, the user evaluates the comprehensiveness of the answers and the appropriateness of the order in which they are written in the predetermined answer format. The data management module stores the results of the evaluation, The information output system according to claim 1, characterized in that it provides feedback to the response generation by the response generation module.

8. The information output system according to claim 1, characterized in that the input and output are performed using a chatbot system or a search system.

9. The data management module assigns an approval flag to the summary response stored in the data management module, indicating that the content has been confirmed by a medical professional. The key points response to which the approval flag has been assigned, and the long text associated with that key points response, are saved as a pair. The information output system according to any one of claims 1 to 8, characterized in that when there is a new input that is identical or of a predetermined degree of similarity or higher, it acquires and outputs the corresponding summary response and the long text.

10. In the information output system described in claim 9, The data management module is characterized by reducing the computational resources or processing time required to generate the long text by reusing the key points response and the corresponding long text that have been given the approval flag without regenerating them.

11. In the information output system according to claim 1, The information output system is characterized in that the summary response stored in the data management module is either created through human intervention, including by a medical professional or system administrator, or automatically generated and then reviewed by a medical professional.

Images