Search system, search method, and search program

Abstract

This search system comprises an acquisition unit, a relevance degree estimation unit, a determination unit, and an output unit. The acquisition unit acquires a query. The relevance degree estimation unit estimates the degree of relevance between the acquired query and the sentence of an answer candidate included in a document having a hierarchical structure. The determination unit determines question text for narrowing down answer candidates for the query on the basis of the hierarchical structure of the document and the estimated degree of relevance. The output unit outputs the determined question text . Accordingly, it is possible to efficiently search for information requested by a user, and thus this search system can be used for assisting in decision making, for example. The search system also has a function as an optimization AI that optimizes a question for use in narrowing down answer candidates according to the situation of a user.

Description

Search system, search method, and recording medium

[0001] The present disclosure relates to a search system and the like.

[0002] The information search support system assists users in their information search by, for example, displaying items that are likely to be related to the information the user desires to obtain. The information search by the user is performed, for example, via a chatbot.

[0003] The information processing device of Patent Document 1 acquires a request to view content via a chatbot, and then outputs search results for content that meets the request.

[0004] Japanese Patent Application Laid-Open No. 2022-165365

[0005] In the information processing device of Patent Document 1, the process for obtaining the information requested by the user may be lengthy.

[0006] In order to solve the above-mentioned problems, the present disclosure aims to provide a search system etc. that can efficiently search for information requested by a user.

[0007] In order to solve the above problems, the search system of the present disclosure includes an acquisition means for acquiring a query, a relevance estimation means for estimating the relevance between the acquired query and a candidate answer sentence contained in a document having a hierarchical structure, a determination means for determining a question sentence that narrows down the candidate answers to the query based on the hierarchical structure of the document and the estimated relevance, and an output means for outputting the determined question sentence.

[0008] In the search method disclosed herein, a search system acquires a query, estimates the relevance between the acquired query and candidate answer sentences contained in a hierarchically structured document, determines a question sentence that narrows down the candidate answers to the query based on the hierarchical structure of the document and the estimated relevance, and outputs the determined question sentence.

[0009] The recording medium of the present disclosure is a search system that non-temporarily records a search program that causes a computer to execute the following processes: a process of acquiring a query; a process of estimating the relevance between the acquired query and candidate answer sentences contained in a document having a hierarchical structure; a process of determining a question sentence that narrows down candidate answers to the query based on the hierarchical structure of the document and the estimated relevance; and a process of outputting the determined question sentence.

[0010] According to the present disclosure, information requested by a user can be efficiently searched for.

[0011] 1 is a diagram illustrating an example of a configuration of an information search system according to the present disclosure. FIG. 2 is a diagram illustrating an example of a configuration of a search system according to the present disclosure. FIG. 3 is a diagram illustrating an example of a structural analysis of a document according to the present disclosure. FIG. 4 is a diagram illustrating an example of a structural analysis of a document according to the present disclosure. FIG. 5 is a diagram illustrating an example of a generation of a question according to the present disclosure. FIG. 6 is a diagram illustrating an example of a relevance between a query and an answer candidate according to the present disclosure. FIG. 7 is a diagram illustrating an example of a generation of a question according to the present disclosure. FIG. 8 is a diagram illustrating an example of a display screen according to the present disclosure. FIG. 9 is a diagram illustrating an example of a display screen according to the present disclosure. FIG. 10 is a diagram illustrating an example of structural data according to the present disclosure. FIG. 11 is a diagram illustrating an example of an operation flow of a search system according to the present disclosure. FIG. 12 is a diagram illustrating an example of an operation flow of a search system according to the present disclosure. FIG. 13 is a diagram illustrating an example of a hardware configuration according to the present disclosure.

[0012] Embodiments of the present disclosure will be described in detail with reference to the drawings. FIG. 1 is a diagram illustrating an overview of the configuration of an information search system. The information search system includes, for example, a search system 10 and a terminal device 20. The search system 10 is connected to the terminal device 20 via, for example, a network. There may be multiple terminal devices 20. The number of terminal devices 20 can be set as appropriate.

[0013] An information retrieval system is, for example, a system that searches for information in response to a user query. A user query is, for example, a search request for information that the user wishes to obtain. For example, when searching for a necessary procedure, the user inputs the information about the procedure that the user wishes to obtain as a query. For example, when wanting to know how to reimburse travel expenses, the user inputs the sentence "How to reimburse travel expenses" as a query. A user's query may be input in the form of a question or a request.

[0014] The search target may be, for example, a document with a hierarchical structure. A document with a hierarchical structure may be, for example, a document divided into sections, each of which is hierarchically organized. A document with a hierarchical structure may be, for example, a document written so that the information becomes more detailed as you go from the top to the bottom of the hierarchy. A document with a hierarchical structure may be, for example, a document divided into chapters. A document divided into chapters may be, for example, a document divided into sections, each of which is hierarchically organized. A document divided into chapters may be, for example, a document written in chapters, sections, and paragraphs. A document with a hierarchical structure may be a web article that contains a link to a more detailed article. The search target may also include information contained in a document that contains a link in a structured document. A link in a document may be, for example, information that identifies the document. A link may be information that indicates at least one of the following: the document name, the location of the document, the location of the document in the document, and the uniform resource locator (URL) of the web where the document is posted. Links in a document are not limited to the above. The hierarchical structure of a text may also be a hierarchical structure between multiple documents.

[0015] The document to be searched is, for example, a document that contains information that a user wishes to obtain. The document to be searched may include a document that contains information that a user may wish to obtain. The document to be searched is, for example, at least one of a manual, an instruction manual, a specification, a collection of questions and answers, and an article posted on the web. The document to be searched may also be a presentation file. The document to be searched is not limited to the above.

[0016] Acquisition of a user's query and presentation of an answer to the query are performed, for example, via a chatbot. For example, a user inputs information they wish to acquire as a query to a chatbot that performs procedures in an interactive format. The search system 10 searches the chatbot for information corresponding to the acquired query. The search system 10 then outputs the search results to the chatbot as an answer to the query. The user acquires the answer to the query from the chatbot.

[0017] The configuration of the search system 10 will now be described. FIG. 2 is a diagram showing an example of the configuration of the search system 10. The search system 10 basically includes a question generation unit 103, an acquisition unit 104, a relevance estimation unit 105, a determination unit 106, and an output unit 107. The search system 10 also includes, for example, a document acquisition unit 101, a structured data generation unit 102, a structured data storage unit 108, a question storage unit 109, and a storage unit 110.

[0018] The document acquisition unit 101 acquires, for example, a document to be searched. The document to be searched is, for example, a document that describes information that may be information that a user wants to obtain. The document to be searched is, for example, a document having a hierarchical structure. A hierarchical structure means, for example, when there is a first sentence and a second sentence, the second sentence is a subordinate concept of the first sentence. In other words, a hierarchical structure means, for example, when there is a first sentence and a second sentence, the second sentence contains more detailed information than the first sentence. The document to be searched is, for example, a document in which information that may be information that a user wants to obtain is described at any level among documents in which information is described hierarchically.

[0019] The document acquisition unit 101 acquires the documents to be searched from, for example, a document providing server (not shown) connected via a network. The document providing server is, for example, a server that provides at least one of instruction manuals, manuals, user manuals, and specifications. The document acquisition unit 101 may acquire the documents to be searched from a document management system within an organization. The document acquisition unit 101 may also acquire the documents to be searched from a web page. The documents to be searched may also be input into the search system 10 by an administrator of the search system 10. How the documents to be searched are acquired is not limited to the above.

[0020] The structure data generation unit 102 generates structure data for a document having a hierarchical structure acquired by the document acquisition unit 101, for example. The structure data generation unit 102 analyzes the hierarchical structure of the document, for example. Then, it generates structure data based on the analysis results. The structure data is, for example, data indicating the relationships between sentences between hierarchies. In other words, the structure data is data that can determine the relationships between sentences included in the document. The structure data generation unit 102 generates, for example, a tree indicating the relationships between sentences included in the document as structure data.

[0021] When the structure data has a tree structure, the structure data generation unit 102 generates, for example, a tree as data indicating a hierarchical structure, with the sentence at the highest level as the root, answer candidates for the query as leaf nodes, and sentences at each level between the highest level and the level of the answer candidates for the query as internal nodes. Answer candidates are, for example, sentences described in the most detail in a document, and therefore are written at the lowest level in a hierarchically structured document. Therefore, the structure data generation unit 102 generates, for example, a tree in which answer candidates are leaf nodes. The structure data generation unit 102 also generates a tree in which sentences described at levels higher than the answer candidates are internal nodes. Internal nodes are nodes located between the root and leaf nodes in the tree. The structure data generation unit 102 may also generate a tree in which internal nodes are answer candidates. When an internal node is an answer candidate, the structure data generation unit 102 generates a tree by, for example, creating a copy of the internal node as a leaf node and connecting it directly below the internal node.

[0022] The structure data generation unit 102 generates hierarchical structure data for a document based on, for example, at least one of descriptions regarding chapters and descriptions regarding titles in a hierarchically structured document. In the case of a chaptered document, the structure data generation unit 102 analyzes the hierarchical structure of the document based on, for example, the numbers assigned to each chapter, section, and paragraph. The structure data generation unit 102 generates structure data for the document by, for example, representing the relationships between chapters, sections, and paragraphs using a tree. The structure data generation unit 102 may also analyze the structure of the document using at least one of symbols corresponding to sections, indents, consecutive line breaks, blank lines, and font modifications. Font modifications include, for example, character size, character thickness, and underlining or color applied to characters. Font modifications are not limited to those described above. Furthermore, the notation in a text used to generate hierarchical structure data is not limited to those described above.

[0023] FIG. 3 is a diagram schematically illustrating an example of generating tree structure data from a document. In the example of FIG. 3, the document indicated by (1) describes a method for reimbursing expenses for a domestic business trip. In the example of FIG. 3, the document has a hierarchical structure. For example, the document has a hierarchical structure such as "1. Domestic Business Trip," "1.1 Use of A," and "1.1.1 Required Documents." In the example of FIG. 3, the structure data indicated by (2) is a tree representing the structure of the document. In the example of structure data of FIG. 3, the tree has, for example, "1. Domestic Business Trip," "1.1 Use of A," and "1.1.1 Required Documents" as nodes. In the example of structure data of FIG. 3, the tree has, for example, "1. Domestic Business Trip," "1.2 Use of B," "1.2.1 Required Documents," and "1.2.2 Required Documents" as nodes. For example, by generating structure data such as the example of FIG. 3, it is possible to determine the relationships between sentences contained in the document.

[0024] When the document is an article published on the web, the structure data generation unit 102 identifies the relationship between pages, for example, based on links. Then, the structure data generation unit 102 generates structure data for the document by, for example, using a tree to represent the relationship between pages, where each link leads to a lower level. The phrase "each link leads to a lower level" means, for example, that the linked page contains more detailed information than the link source page. In such a case, the structure data generation unit 102 generates structure data with a tree structure, where each link leads to a lower level.

[0025] When a sentence contains a link to another document and the linked document has a hierarchical structure, the structure data generating unit 102 may generate structure data for the linked document. The answer candidates included in the linked document are also called divided answer candidates.

[0026] The structural data generation unit 102 may also generate structural data of a document using the results of analyzing the hierarchical structure using a structural analysis model. The structural analysis model is, for example, a learning model that extracts the hierarchical structure of a document from the document. The structural analysis model is generated, for example, by learning the relationship between information indicating a document and sections within the document, and a division flag and a hierarchical level. The information indicating a section is, for example, at least one of a number, a symbol, an indent, consecutive line breaks, a blank line, and a font modification. The information indicating a section is not limited to the above. The division flag is information indicating whether a section is divided. The hierarchical level is information indicating the hierarchical level to which a sentence belongs in a document. The structural analysis model can be generated, for example, using a recurrent neural network (RNN), a long short-term memory (LSTM), or a transformer. The learning algorithm for generating the structural analysis model is not limited to the above.

[0027] Fig. 4 is a diagram schematically showing an example of generating structured data using a structural analysis model. In the example of Fig. 4, the document shown in (1) is the document for which structured data is to be generated. The structural analysis result shown in (2) in the example of Fig. 4 is the result of analyzing the hierarchical structure of the document using the structural analysis model. In the example of the structural analysis result of Fig. 4, divided sentences indicate each sentence included in the document. A sentence may be a sentence unit, or may be multiple sentences.

[0028] In the example of the structural analysis result of FIG. 4 , when the division flag is "1", it indicates that the sentence is in a separate section from the previous sentence. Also, in the example of the structural analysis result of FIG. 4 , when the division flag is 0, it indicates that the sentence is in the same section as the previous sentence. In the example of the structural analysis result of FIG. 4 , the division flag of sentence B is 0, so sentences A and B are sentences in the same section. Also, in the example of the structural analysis result of FIG. 4 , sentence A is the first sentence of the document, so its hierarchical level is 1. Also, in the example of the structural analysis result of FIG. 4 , sentence B is in the same section as sentence A, so its hierarchical level is not displayed. On the other hand, in the example of the structural analysis result of FIG. 4 , sentence C has a flag of 0, so it is in a different section from sentences A and B. Also, in the example of the structural analysis result of FIG. 4 , sentence C is a sentence lower in level than sentences A and B, so its hierarchical level is 2. Also, in the example of the structural analysis result of FIG. 4 , sentence D has a flag of 1, so it is a sentence in a different section from sentence C. 4, sentence D is in the same layer as sentence C and is a lower layer than sentences A and B, so its hierarchical level is 2. In the example of the structural analysis result in FIG. 4, sentences other than sentences A, B, C, and D are also assigned split flags and hierarchical levels in the same way.

[0029] In the example of Fig. 4, the structure data (C) is a tree of the hierarchical structure of the document. The example of the structure data in Fig. 4 is a tree in which each section is a node. In the example of the structure data in Fig. 4, the node of sentence C and the node of sentence D are connected as edge nodes of sentence A and sentence B. In other words, the example of the structure data in Fig. 4 indicates that sentences C and D are lower in the hierarchy than sentences A and B, respectively.

[0030] The structured data generation unit 102 may also generate structured data using a language model that employs natural language processing. The structured data generation unit 102 generates structured data by inputting a prompt, created in the form of a command statement such as, "Please output the sections of the following document in a tree structure," into the language model. The prompt may also include examples of division flags and hierarchical levels. The language model outputs information indicating a path from the root to each section, for example, using an identification number assigned to each section. The structured data generation unit 102 then generates a tree using, for example, the information indicating a path from the root to each section.

[0031] Here, the definition of a language model will be explained. A language model is a model that learns the relationships between words in a sentence and generates related strings related to a target string from the target string. By using a language model that has learned sentences and paragraphs from various contexts, it is possible to generate related strings with appropriate content related to the target string. For example, a case where a language model is used in question answering will be explained. The language model receives an input question such as "What kind of country is Japan?" as the target string. The language model generates a string such as "Japan is an island country in the Northern Hemisphere..." as an answer to the question.

[0032] The learning method of the language model is not particularly limited, and as an example, the language model may be one that is learned to output at least one sentence including an input string. Specific examples of the language model include GPT-2 (Generative Pre-Training-2), GPT-3, GPT-3.5, and GPT-4, which output sentences including an input string by predicting a string that is likely to follow the input string. Other examples of language models include T5 (Text-to-Text Transfer Transformer), BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly optimized BERT approach), and ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately).

[0033] The question generation unit 103 generates, for example, a question sentence for narrowing down answer candidates. For example, the question generation unit 103 generates a question sentence based on sentences included in each level of a hierarchically structured document. For example, the question generation unit 103 generates a question for narrowing down answer candidates by determining which sentence of answer candidate sentences included in a document is closest to a user's query for each level of the hierarchically structured document. For example, the question generation unit 103 imagines a question sentence for narrowing down answer candidates by using sentences higher in the hierarchy than the answer candidate sentences included in the document.

[0034] When the structure data has a tree structure, the question generation unit 103 generates a question to narrow down answer candidates based on, for example, sentences corresponding to each internal node of the tree. The question generation unit 103 generates, for example, a question to determine a path along the tree from the root to the answer candidate. When, for example, there are multiple child nodes on the path along the tree from the root to the answer candidate, the question generation unit 103 generates a question to be used to select a child node. That is, the question generation unit 103 generates, for example, a question to be used to select a branch in the tree. The question generation unit 103 generates, for example, a question to confirm the user's situation. The question to confirm the user's situation is, for example, a question to confirm what information the user needs in order to narrow down the answers. The question to select an edge is used, for example, to ask the user a question and determine which edge of the tree to select based on the answer to the question.

[0035] The question generation unit 103 generates a question by, for example, converting a sentence corresponding to a child node into a question format. The question generation unit 103 generates a question based on, for example, a predetermined rule. The predetermined rule is, for example, a rule that defines the relationship between a sentence extraction method and a word to be added to the extracted sentence. The question generation unit 103 generates a question by, for example, adding a word indicating a question to a chapter title. The word indicating a question is, for example, a word used at the end of a question. For example, when the question generation unit 103 extracts a sentence "Using A," the question generation unit 103 adds the word "Is it about?" as a word indicating a question. Then, the question generation unit 103 generates a sentence "Regarding the use of A?" as a question. For example, in this case, "Is it about?" is a word indicating a question. The question generation unit 103 may also generate a question that allows a choice of sentences of a child node. When generating a question that requires selecting one of the sentences of the child node, the question generator 103 generates, for example, a sentence such as "A or B?" as the question sentence.

[0036] The question generation unit 103 may extract a predetermined number of characters or a predetermined position from the beginning of a sentence. The position is, for example, a position from the beginning of the sentence that is set based on at least one of the number of lines from the beginning of the sentence, the number of periods, a line break symbol, and an indent. The setting of the position is not limited to the above. For example, the question generation unit 103 generates a question by adding a word indicating a question to the extracted sentence. The question generation unit 103 may generate a question by adding a word indicating a question to a keyword included in the sentence. The keyword is, for example, a term that is highly related to an answer candidate. The keyword may be set in a dictionary. The rules based on which the question generation unit 103 generates a question are not limited to those described above.

[0037] When a tree has internal nodes as answer candidates and a tree has been generated in which the internal nodes are copied to leaf nodes, the question generator 103 generates, for example, a question sentence that connects the internal node corresponding to the answer candidate to the leaf node on the tree. For example, the question generator 103 generates a question such as, "Would you like to know more about this chapter?"

[0038] The question generation unit 103 may use a question generation model to generate a question to check the user's situation. The question generation model is, for example, a machine learning model that takes a sentence as input and outputs a question. The question generation model is, for example, generated by learning the relationship between a sentence and a question. The sentence is, for example, a chapter title or the first sentence of a chapter. The question generation model is, for example, generated by deep learning using a neural network. The algorithm for generating the question generation model is not limited to the above.

[0039] FIG. 5 is a diagram illustrating an example of a question for narrowing down answer candidates. In the example of FIG. 5, questions corresponding to each child node on the tree are generated. That is, in the example of FIG. 5, a question can be used, for example, as a question for selecting a child node at a certain node. The tree in the example of FIG. 5 has, for example, nodes corresponding to "1.1 Use of A" and "1.2 Use of B" as child nodes of the node corresponding to "1. Domestic Business Trip." In this case, the question generation unit 103 generates, for example, a question, "Are you asking about the use of A?" using a sentence included in "1.1 Use of A." Furthermore, the question generation unit 103 generates, for example, a question, "Are you asking about the use of B?" using a sentence included in "1.2 Use of B." In the example of FIG. 5, the question generation unit 103 generates questions for other nodes in the same manner.

[0040] The acquisition unit 104 acquires a query. The acquisition unit 104 acquires, for example, a user query regarding information that the user wishes to acquire. The acquisition unit 104 acquires, for example, a user query from the terminal device 20. The query is, for example, input to the terminal device 20 by the user's operation. The query is, for example, a statement indicating the information that the user wishes to acquire. For example, when a user wishes to acquire information regarding how to apply for travel expenses, the query may be, for example, "How do I apply for travel expenses?" The query may also be a question. For example, when a user wishes to acquire information regarding how to apply for travel expenses, the query may be, for example, "How do I apply for travel expenses?" The query is not limited to the above.

[0041] The acquiring unit 104 acquires, for example, a user's answer to a question for narrowing down answer candidates. The acquiring unit 104 acquires, for example, a user's answer to a question for narrowing down answer candidates from the terminal device 20. The user's answer is input, for example, to the terminal device 20 by the user's operation.

[0042] Furthermore, when information is searched for via a chatbot, the acquisition unit 104 may acquire answers to queries and questions from a server (not shown) that controls the chatbot, and the output unit 107 may acquire answers to queries and questions from an information search server (not shown).

[0043] The acquiring unit 104 may acquire an instruction to cancel an answer from a user to a question. The acquiring unit 104 acquires an instruction to cancel an answer from a user to a question, for example, from the terminal device 20. The information to cancel an answer to a question is issued, for example, when an answer to a question has been given, but a further question or a presented answer differs from the question or answer expected by the user. The determining unit 106 may determine a question corresponding to a node in the next higher hierarchy as the question for narrowing down answer candidates. The instruction to cancel the user's answer to a question may also include information specifying a question to return to.

[0044] The relevance estimation unit 105 estimates the relevance between the query acquired by the acquisition unit 104 and a sentence of an answer candidate included in a document having a hierarchical structure. The relevance estimation unit 105 estimates, for example, the relevance between the query and the answer candidate. The relevance between the query and the answer candidate is, for example, an index indicating the degree of relationship between the answer candidate and the query. The relevance estimation unit 105 estimates the relevance based on, for example, the similarity of the words included and the similarity of the meanings. The relevance between the query and the answer candidate can be, for example, an index indicating the degree to which the answer candidate is suitable as an answer to the query.

[0045] The relevance estimation unit 105 converts the query and the answer candidate into embedding vectors, for example, and estimates the distance between the embedding vector of the query and the embedding vector of the answer candidate as the relevance.

[0046] Here, we will explain embedding vectors. Embedding vectors (or simply referred to as "vector representations") are a method for converting text data (such as words, sentences, and documents) into a numerical vector space. Using an embedding vector representation makes it possible to obtain features of the text data. There are no particular limitations on the method for defining or calculating an embedding vector. For example, an embedding vector may be a vector calculated using techniques such as Word2Vec, GloVe (Global Vectors for Word Representation), FastText, or Universal Sentence Encoder. For example, Word2Vec is a method for capturing the meaning of words. Specifically, there are two models: Skip-gram and CBOW (Continuous Bag-of-Words). These techniques calculate embedding vectors that represent the meaning of words by learning distributed representations (vectors) of words while taking into account the context surrounding the words ("context").

[0047] When a sentence of an answer candidate includes a link to another document, the relevance estimation unit 105 may further estimate the relevance between the user's query and the sentence of the answer candidate included in the other document. That is, the relevance estimation unit 105 estimates the relevance between the user's query and the divided answer candidate.

[0048] FIG. 6 is a table showing an example of the relevance between a user's query and answer candidates and split answer candidates. The example relevance table in FIG. 6 is a table in which IDs, answer candidates, split answer candidates, the relevance of the answer candidates, and the relevance of the split answer candidates are associated. In the relevance table in FIG. 6, IDs are identifiers for each sentence included in a document. In the relevance table in FIG. 6, answer candidates are sentences of answer candidates included in the document to be searched. In the relevance table in FIG. 6, split answer candidates are sentences of answer candidates included in a document to which a link is provided to the answer candidate. In the relevance table in FIG. 6, the answer candidate relevance is the relevance between the query and the answer candidate. Furthermore, in the relevance table in FIG. 6, the split answer candidate relevance is the relevance between the query and the split answer candidate. In the example relevance table in FIG. 6, the relevance between a sentence of an answer candidate in the travel expense settlement manual and the query is shown as the relevance of the split answer. In the example of the relevance table in FIG. 6, for example, the relevance of "1.1.1 Necessary Documents" with ID 403 and "1.1.2 Necessary Documents" with ID 406 is higher than the other items.

[0049] The relevance estimation unit 105 may estimate the relevance by further using user attributes. For example, the relevance estimation unit 105 estimates the relevance by weighting based on the user attributes. For example, the relevance estimation unit 105 estimates the relevance by weighting sentence information so that the sentence information is more likely to be desired based on the user attributes. For example, the relevance estimation unit 105 estimates the relevance by weighting so that the relevance of sentence information that is expected to be used frequently based on the user attributes is higher. For example, the user attributes are information about the user that has a high correlation with the information the user desires to obtain. For example, the user attributes are information about at least one of the user's gender, age, place of residence, affiliation, occupation, position, work location, years of service, and family composition. The user attributes are not limited to the above. For example, the relevance estimation unit 105 estimates the relevance between the query and the answer candidate by multiplying the relevance estimated from the distance between the embedding vector of the query and the embedding vector of the answer candidate by a coefficient based on the user attributes.

[0050] The coefficients are set as table-format data that defines the relationship between the user's attributes, keywords included in the answer, and the coefficients. The coefficients are set based on the relationship between the department to which the user belongs and the content of the answer. For example, if the keyword "business trip expense settlement" is included, and the user belongs to a department that makes business trips, the coefficient is set to be higher than if the user belongs to a department that does not make business trips.

[0051] The determining unit 106 determines a question sentence that narrows down answer candidates for the query based on the hierarchical structure of the document and the estimated relevance. Furthermore, the determining unit 106 determines whether to ask a question that further narrows down answer candidates or to present answer candidates for the query based on, for example, an answer from the user, the hierarchical structure of the document, and the estimated relevance. When it is determined to ask a question that further narrows down answer candidates, the determining unit 106 determines a question sentence that further narrows down answer candidates.

[0052] When the structure data has a tree structure, the determination unit 106 determines a question sentence that narrows down the answer candidates based on, for example, the tree and the relevance between the query and the answer candidates. When there are multiple answer candidates whose relevance to the query is equal to or greater than a predetermined standard, the determination unit 106 determines, as the question sentence, a question sentence corresponding to an internal node that is commonly linked to leaf nodes corresponding to answer candidates whose relevance to the query is equal to or greater than the predetermined standard. Furthermore, the determination unit 106 determines, for example, based on the answer from the user, the document tree, and the estimated relevance, whether to ask a question that further narrows down the answer candidates or to present answer candidates for the query. Then, when it is determined to ask a question that further narrows down the answer candidates, the determination unit 106 determines, based on the tree, a question sentence that further narrows down the answer candidates. Among question sentences that have not been used in a question that narrows down the answer candidates, the determination unit 106 determines, as the question sentence that further narrows down the answer candidates, a question sentence that corresponds to the highest internal node.

[0053] The determination unit 106 determines an optimized tree structure as a tree to be used to determine an answer to a query, for example, based on the relevance between the query and the answer candidates. Optimization refers to, for example, generating a tree from which branches of answer candidates whose relevance to the query is lower than a predetermined standard are removed. In other words, the optimized tree has a structure that includes only branches related to answer candidates whose relevance to the query is equal to or higher than a predetermined standard. Removing branches is also referred to as pruning. Based on the tree structure from which branches of answer candidates that are unlikely to be suitable as answers to the query are removed, questions can be efficiently asked to narrow down the answer candidates.

[0054] The determination unit 106 determines answer candidates for a query based on, for example, the relevance between the query and the answer candidates. For example, the determination unit 106 determines, as answer candidates, sentences included in a hierarchically structured document that have a relevance equal to or greater than a predetermined standard. Then, when there are multiple answer candidates whose relevance to the query is equal to or greater than a predetermined standard, the determination unit 106 determines, for example, a question sentence in a higher hierarchy that is common to the answer candidates whose relevance is equal to or greater than the predetermined standard, to narrow down the answer candidates to be presented to the user. The predetermined standard is an index indicating the degree to which the answer candidate sentence is suitable as an answer to the query. For example, if the relevance to the query is equal to or greater than a predetermined standard, the answer candidate sentence is suitable as an answer to the query. The predetermined standard is set, for example, by an administrator or a user of the search system 10.

[0055] When the structure data has a tree structure, for example, when there are multiple answer candidates whose relevance meets a criterion, the determination unit 106 determines a question at a node common to the multiple answer candidates as a question to be asked to the user. For example, the determination unit 106 may determine, among the answer candidates whose relevance meets a criterion, a sentence whose relevance reaches a predetermined rank as an answer candidate. By determining a question at a node common to the answer candidates as a question to be asked to the user, questions at branches linked to answer candidates whose relevance does not meet the criterion are excluded from the candidates when determining a question to be asked to the user. Therefore, among the multiple branches included in the tree, branches linked to answer candidates whose relevance does not meet the criterion are pruned in the process of determining a question to narrow down the answer candidates. Therefore, the tree structure is optimized, for example, as a tree used to determine an answer to a query.

[0056] The predetermined ranking is set, for example, so that a user can efficiently search for information to obtain. For example, if the number of answer candidates is increased, the number of questions required to narrow down the answer candidates may also increase. On the other hand, if the number of answer candidates is reduced, the information the user wants to obtain may not be included in the answer candidates, and the user may have to start the search again. The predetermined ranking of the answer candidates is set, for example, by an administrator or a user of the search system 10.

[0057] When there are multiple nodes common to multiple answer candidates, the determining unit 106 determines, for example, a question corresponding to a node that is closer to the root among nodes that have not asked a question, as a question that narrows down the answer candidates.The determining unit 106 then repeats the questions that narrow down the answer candidates along the tree until, for example, it reaches the node of an answer candidate.

[0058] The determination unit 106 determines answer candidates for a query by, for example, asking questions to narrow down the answer candidates along the tree until it reaches a node of an answer candidate. The determination unit 106 can determine answer candidates according to the user's situation by asking questions at nodes common to multiple answer candidates as questions to narrow down the answer candidates. The user's situation is, for example, confirmation of the type of information the user needs.

[0059] When an answer candidate includes a link to a document having a hierarchical structure, the determination unit 106 may determine the answer candidate for the query as a split answer candidate based on a tree that indicates the structure of the linked document with the answer candidate as the root node. The determination unit 106, for example, sets a node corresponding to a sentence containing the link as the root of the tree of the linked document. Then, the determination unit 106 determines an answer to the question or query that narrows down the split answer candidates based on, for example, the tree structure of the linked document and the relevance between the query and the split answer candidate.

[0060] When an instruction to cancel an answer to a question is acquired from a user, the determining unit 106 may determine a question corresponding to a node one level higher as a question for narrowing down answer candidates. Also, when the information to cancel an answer to a question specifies a question to return to in the tree, the determining unit 106 may determine the specified question as a question for narrowing down answer candidates.

[0061] The structure data may also be something other than a tree. For example, the structure data may be data that shows a hierarchical structure using a directed graph. Furthermore, when there are multiple split answer candidates, a partial tree corresponding to each split answer candidate may be generated. In this case, for example, a shortest path tree technique may be used to determine questions that narrow down the answer candidates.

[0062] FIG. 7 is a diagram schematically illustrating a flow path on a tree when determining questions to narrow down answer candidates using the tree. In the example of FIG. 7 , for example, assume that the relevance levels of "1.1.1 Required Documents," "1.2.2 Required Documents," and "2.1.1 Submitted Documents" are equal to or greater than a certain standard. In this case, the determination unit 106 determines questions to narrow down which of "1.1.1 Required Documents," "1.2.2 Required Documents," and "2.1.1 Submitted Documents" is suitable as an answer. The determination unit 106 determines questions along the thick arrows in the example of FIG. 7 . That is, the determination unit 106 determines questions by excluding questions related to the thin arrows in the example of FIG. 7 . In this way, by removing questions on the path whose relevance levels do not meet the standard from the candidates, answer candidates can be efficiently narrowed down.

[0063] The output unit 107 outputs the question sentence determined by the determination unit 106 to narrow down the answer candidates. For example, when the determination unit 106 determines to ask a question to further narrow down the answer candidates, the output unit 107 outputs the question sentence. When the determination unit 106 determines to present answer candidates for a query, the output unit 107 outputs the answer candidates. For example, the output unit 107 outputs the answer to the question sentence or query determined by the determination unit 106 to the terminal device 20. The output unit 107 may output the answer to the question or query determined by the determination unit 106 via a chatbot. When outputting via the chatbot, the output unit 107 may output the answer to the question or query determined by the determination unit 106 to a server (not shown) that controls the chatbot. Furthermore, the output unit 107 may output the answer to the question or query determined by the determination unit 106 to an information search server (not shown).

[0064] The output unit 107 may output the answer to the query together with sentences written before and after the answer in the document. For example, the output unit 107 outputs, as the answer to the query, a sentence in a section in which the answer to the query is written and sentences in sections before and after the section in the search target document. When the hierarchies of the sections before and after the section in which the answer to the query is written are different from the hierarchical level of the section in which the answer is written, the output unit 107 may output a sentence included in the same hierarchical level as the hierarchical level of the section in which the answer is written. In other words, the output unit 107 may exclude a sentence in a hierarchical level different from the hierarchical level of the section in which the answer is written from the output targets. When outputting the answer to the query together with sentences written before and after the answer in the document, the output unit 107 may output the answer sentence to the query in a highlighted manner. For example, the output unit 107 outputs the answer sentence to the query by highlighting it using at least one of a character size, a character thickness, a font type, a background color, and an added line. Furthermore, when outputting sentences before and after an answer to a query, the output unit 107 may output an image used for an operation to change the display range. The image used for an operation to change the display range is, for example, a button or slider that changes the display range when operated on the screen. The image used for an operation to change the display range is not limited to the above.

[0065] The output unit 107 may output a list of questions for narrowing down answer candidates and answers from users to the questions. The output unit 107 outputs the questions for narrowing down answer candidates and a history of answers from users to the questions in a chronological order so that the user can check them. Furthermore, the output unit 107 may output, for example, questions for narrowing down answer candidates and answers from users to the questions so that, by operating a slider, the user can view answers other than the questions for narrowing down the displayed answer candidates and the answers from users to the questions. The output unit 107 may output structure data together with the questions for narrowing down answer candidates. Furthermore, the output unit 107 may output answer candidates together with the questions for narrowing down answer candidates.

[0066] FIG. 8 is a diagram showing an example of a display screen for an answer to a query. The example of the display screen in FIG. 8 is an example in which a query is input on a chatbot, a question is displayed, an answer to the question is input, and the answer to the query is displayed. In the example of the display screen in FIG. 8, a query "Business Trip Settlement Certificate" is input by a user. In the example of the display screen in FIG. 8, a question to narrow down answer candidates is displayed, and the user's answer to the question is input. In the example of the display screen in FIG. 8, the sentence "Please see below" and the sentence of the answer to the query are displayed. In addition, in the example of the display screen in FIG. 8, it is possible to display the sentences before and after the sentence of the answer to the query by operating a slider.

[0067] The structure data storage unit 108 stores, for example, structure data. The structure data storage unit 108 stores, for example, structure data generated by the structure data generation unit 102. The structure data storage unit 108 may store structure data generated in a system external to the search system 10. Furthermore, the structure data may be stored in a storage means other than the structure data storage unit 108.

[0068] The question storage unit 109 stores, for example, questions that narrow down answer candidates. The question storage unit 109 stores, for example, questions that narrow down answer candidates generated by the question generation unit 103. The question storage unit 109 may also store questions that narrow down answer candidates generated in a system external to the search system 10. Furthermore, questions that narrow down answer candidates may be stored in a storage unit other than the question storage unit 109.

[0069] The memory unit 110 stores, for example, data related to the process of searching for answers to a query. The memory unit 110 stores, for example, criteria for determining candidate answer sentences for a query from sentences included in documents to be searched. The memory unit 110 stores, for example, a structural analysis model. The memory unit 110 stores, for example, a question generation model. The memory unit 110 stores, for example, a language model. The memory unit 110 may be a storage device external to the search system 10. The memory unit 110 may be, for example, a storage device connected to an information processing device connected via a network. The arrangement of the memory unit 110 may be designed as appropriate.

[0070] (Specific Example of Use of Information Providing System) The information providing system of the present disclosure can be used, for example, in the medical healthcare field. As an example of use in the medical healthcare field, an example in which a user searches for information in a medical information system will be described. FIG. 9 is an example of a display screen when a user searches for information about vaccination in the medical information system. FIG. 10 is a diagram showing an example of structured data of a document related to vaccination. A user inputs information they wish to obtain as a query by operating the terminal device 20. For example, suppose the user wants to know the requirements for receiving a vaccination against coronavirus, a new type of virus. The user operates the terminal device 20 and inputs, for example, the sentence, "Please tell me about the guidelines for coronavirus vaccination." as a query. When the user inputs a query, the terminal device 20 outputs the input query to the acquisition unit 104 of the search system 10.

[0071] The acquisition unit 104 of the search system 10 acquires a user's query from the terminal device 20. Once the user's query is acquired, the relevance estimation unit 105 estimates the relevance between the user's query and answer candidates. Then, the determination unit 106 determines a question to narrow down the answer candidates. In the example of FIG. 9 , the determination unit 106 refers to the example tree of FIG. 10 and determines to ask the user the question, "Are you looking for general vaccination guidelines or information about a specific age group?" In this case, once the question to the user has been determined, the output unit 107 outputs the question to the user to the terminal device 20.

[0072] When a question for the user is acquired, the terminal device 20 outputs the question to the user on a display device (not shown). After viewing the question, the user operates the terminal device 20 to input an answer to the question. In the example of FIG. 9, the sentence "I'd like some general guidelines." is input as the answer to the question.

[0073] The terminal device 20 outputs the answer to the question to the acquisition unit 104 of the search system 10. Once the answer to the question has been acquired, the relevance estimation unit 105 refers to the structure data and determines whether to ask a question to further narrow down the answer candidates or to provide an answer. In the example of FIG. 9 , the determination unit 106 determines to ask the user the question, "Do you need information about vaccination eligibility and recommended vaccine types?" Once the question to the user has been determined, the output unit 107 outputs the question to the user to the terminal device 20.

[0074] The terminal device 20 outputs a question to the user on a display device (not shown). After viewing the question, the user operates the terminal device 20 to further input an answer to the question. In the example of Figure 9, the sentence "Eligibility to receive the vaccine" is input as the answer to the question.

[0075] In the example of Figure 9, in response to the user's response of "vaccination eligibility," a further question is asked: "Are you looking for information on necessary conditions and vaccination intervals?" Also in the example of Figure 9, the user further responds with "vaccination intervals."

[0076] When the answer "vaccination interval" is obtained from the user, the determination unit 106 refers to the structure data and determines that the answers to the query have been narrowed down. When it is determined that the answers to the query have been narrowed down, the determination unit 106 determines that the answer candidates remaining as a result of the narrowing down should be used as the answer to the query. When the answer to the query has been determined, the output unit 107 outputs the answer to the query to the terminal device 20.

[0077] When the answer to the query is acquired, the terminal device 20 outputs the answer to the query on a display device (not shown).

[0078] In this way, by asking a user a question based on the structured data, the search system 10 can efficiently present an answer to the query. The information providing system may also be used, for example, to search for desired information from medical papers, with medical papers being the documents to be searched. The documents to be searched may also be selected by the user. When a document to be searched is selected by the user, structured data for the document to be searched and a question to narrow down the answers may also be generated.

[0079] Furthermore, when the search target is medical papers, the citation relationships of the medical papers may be used as structured data. In this case, the structured data may be, for example, data that combines a tree in which medical papers are assigned to each node with a tree that shows the hierarchical structure of each medical paper. Furthermore, the papers may be papers in fields other than medical fields.

[0080] When searching for pharmaceutical patent publications, the citation relationships of the patent publications may be used as structure data. The patent publications may also be patent publications in fields other than pharmaceuticals. Electronic medical record data may also be used as documents to be searched.

[0081] The following describes the process of structuring a document by the search system 10. Fig. 11 is a diagram showing an example of an operational flow when the search system 10 performs the process of structuring a document.

[0082] The document acquisition unit 101 acquires, for example, a document to be searched (step S11).

[0083] When a document to be searched is acquired, the structure data generation unit 102 analyzes the hierarchical structure of the document to be searched (step S12).The structure data generation unit 102 then generates structure data of the document based on the analysis result of the hierarchical structure (step S13).The structure data generation unit 102 structures the hierarchical document using, for example, a tree.

[0084] After generating the structure data of the document, the structure data generating unit 102 stores the structure data of the document in, for example, the structure data storage unit 108 (step S14).

[0085] When the structure data of the document is saved, the question generation unit 103 generates a question sentence for narrowing down which sentence among the answer candidate sentences included in the document is closest to the user's query, for example, for each layer of the hierarchically structured document (step S15). After generating the question sentence, the question generation unit 103 stores the generated question sentence, for example, in the question storage unit 109 (step S16).

[0086] The following describes the process of outputting an answer to a user's query by the search system 10. Fig. 12 is a diagram showing an example of an operational flow when the search system 10 performs the process of outputting an answer to a user's query.

[0087] The acquiring unit 104 acquires a query relating to information that the user wishes to acquire (step S21). The acquiring unit 104 acquires a query relating to information that the user wishes to acquire from the terminal device 20, for example.

[0088] When a query relating to information desired by the user is acquired, the relevance estimation unit 105 estimates the relevance between the acquired query and answer candidates included in the hierarchically structured sentence (step S22).

[0089] If there are multiple answer candidates whose relevance meets the criteria (Yes in step S23), the determining unit 106 determines a question to be asked to the user to narrow down the answer candidates (step S24).

[0090] When the question for narrowing down the answer candidates is determined, the output unit 107 outputs the determined question (step S25). The output unit 107 outputs the determined question to the terminal device 20, for example.

[0091] When the question is output, the acquisition unit 104 acquires, for example, the user's answer to the question (step S26). The acquisition unit 104 acquires the user's answer to the question, for example, from the terminal device 20. The user's answer to the question is input, for example, to the terminal device 20 by the user's operation.

[0092] When the user's answer to the question is acquired, the determination unit 106 determines an answer to the question or query that narrows down answer candidates to be presented to the user based on the user's answer to the question, the tree structure, and the relevance of the answer candidates (step S27). When the answer to the query is determined in step S27 (Yes in step S28), the output unit 107 outputs the determined answer (step S29). The output unit 107 outputs the determined answer to, for example, the terminal device 20.

[0093] In step S28, when a question for narrowing down the candidates is determined (No in step S28), the search system 10 returns to step S25 and performs the processes from step S25 onwards.

[0094] In step S23, if there is only one answer candidate that satisfies the criteria (No in step S23), the determination unit 106 determines the answer candidate that satisfies the criteria as the answer candidate to the query (step S30). Once the answer candidate is determined, the output unit 107 outputs the determined answer as the answer to the query (step S29).

[0095] The search system 10 acquires a query related to information desired by a user. The search system 10 also estimates the relevance between the query and answer candidates. The search system 10 then determines a question sentence for narrowing down answers based on the query, the document structure, and the estimated relevance. In this way, by determining a question sentence for narrowing down answers based on the relevance between the query and answer candidates, the search system 10 can, for example, exclude answer candidates with low relevance to narrow down answers. Therefore, by using the search system 10, it is possible to efficiently search for information requested by a user.

[0096] The search system 10 generates, for example, tree-structured data indicating the hierarchy of each sentence contained in a hierarchically structured document. When there are multiple answer candidates that meet a search criterion, the search system 10 determines questions to narrow down the answer candidates based on, for example, the tree structure and the relevance between the query and the answer candidates. This allows the search system 10 to exclude questions related to branches in the tree that are linked to answer candidates with low relevance from the questions to be asked to narrow down the answer candidates. Therefore, for example, by structuring a document using a tree, answers to a query can be narrowed down along the tree, thereby reducing the number of questions required of a user. Therefore, by asking questions along the tree, answers to a query can be efficiently determined. In this way, by asking questions based on the document structure and narrowing down the split answer candidates, answers to a query can be efficiently searched for.

[0097] Furthermore, the search system 10, for example, having the configuration described above, can efficiently search for information requested by a user. Therefore, the search system 10 can be used, for example, for decision-making support. The search system also functions as an optimization AI (artificial intelligence) that optimizes questions to narrow down answer candidates according to the user's situation.

[0098] Each process in the search system 10 may be distributed and executed among multiple information processing devices connected via a network. For example, the processes in the document acquisition unit 101, structured data generation unit 102, and question generation unit 103 may be executed in different information processing devices, and the processes in the acquisition unit 104, relevance estimation unit 105, determination unit 106, and output unit 107 may be executed in different information processing devices. It may be appropriately set which of the multiple information processing devices executes each process in the search system 10.

[0099] Each process in the search system 10 can be realized by executing a computer program on a computer. Fig. 13 shows an example of the configuration of a computer 200 that executes a computer program that performs each process in the search system 10. The computer 200 includes a CPU (Central Processing Unit) 201, a memory 202, a storage device 203, an input / output I / F (Interface) 204, and a communication I / F 205.

[0100] The CPU 201 reads and executes computer programs for performing each process from the storage device 203. The CPU 201 may be configured with a combination of multiple CPUs. The CPU 201 may also be configured with a combination of a CPU and another type of processor. For example, the CPU 201 may be configured with a combination of a CPU, a graphics processing unit (GPU), and a tensor processing unit (TPU). The memory 202 is configured with a dynamic random access memory (DRAM) or the like, and temporarily stores the computer programs executed by the CPU 201 and data being processed. The storage device 203 stores the computer programs executed by the CPU 201. The storage device 203 is configured with, for example, a non-volatile semiconductor storage device. Other storage devices such as a hard disk drive may also be used for the storage device 203. The input / output I / F 204 is an interface that receives input from an operator and outputs display data, etc. The communication I / F 205 is an interface that transmits and receives data to and from a terminal device or another information processing device. The terminal device 20 may also have a similar configuration.

[0101] The computer program used to execute each process can also be stored and distributed on a computer-readable recording medium that non-temporarily stores data. Examples of recording media that can be used include magnetic tapes for recording data and magnetic disks such as hard disks. Optical disks such as CD-ROMs (Compact Disc Read Only Memory) can also be used as recording media. Non-volatile semiconductor storage devices can also be used as recording media.

[0102] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0103] [Supplementary Note 1] A search system comprising: an acquisition means for acquiring a query; a relevance estimation means for estimating a relevance between the acquired query and a sentence of an answer candidate included in a document having a hierarchical structure; a determination means for determining a question sentence that narrows down answer candidates for the query based on the hierarchical structure of the document and the estimated relevance; and an output means for outputting the determined question sentence.

[0104] [Supplementary Note 2] The search system described in Supplementary Note 1, wherein the acquisition means acquires an answer from a user to a question that narrows down the answer candidates; the decision means decides whether to ask a question that further narrows down the answer candidates or to present answer candidates to the query based on the answer from the user, the hierarchical structure of the document, and the estimated relevance; and the output means outputs the question sentence when it is decided to ask a question that further narrows down the answer candidates, and outputs the answer candidates when it is decided to present answer candidates to the query.

[0105] [Supplementary Note 3] The search system according to Supplementary Note 1 or 2, wherein when there are multiple answer candidates whose relevance to the query is equal to or greater than a predetermined standard, the determination means determines a question sentence that narrows down the answer candidates from the question sentences in a higher hierarchy that are common to the answer candidates whose relevance to the query is equal to or greater than the predetermined standard.

[0106] [Supplementary Note 4] The search system according to any one of Supplementary Note 1 to Supplementary Note 3, further comprising a question generation unit configured to generate the question sentence based on a sentence included in each layer of a document having a hierarchical structure.

[0107] [Supplementary Note 5] The search system according to either Supplementary Note 1 or 4, further comprising a structure data generation means for generating data on the hierarchical structure of a document based on at least one of a description of chapters and a description of titles in a document having a hierarchical structure.

[0108] [Supplementary Note 6] The search system according to Supplementary Note 5, wherein the structure data generation means generates, as data indicating the hierarchical structure, a tree in which a sentence at the highest level is the root, answer candidates to the query are leaf nodes, and sentences at each level between the highest level and the level of the answer candidates to the query are internal nodes.

[0109] [Supplementary Note 7] The search system according to Supplementary Note 6, wherein when there are multiple answer candidates whose relevance to the query is equal to or greater than a predetermined standard, the determination means determines, as the question sentence, a question sentence corresponding to an internal node that is commonly linked to a leaf node corresponding to the answer candidates whose relevance to the query is equal to or greater than the predetermined standard.

[0110] [Supplementary Note 8] The search system according to Supplementary Note 7, wherein the determining means determines, from among the question sentences that are not used in the question that narrows down the answer candidates, a question sentence that corresponds to a top-level internal node, as a question sentence for the question that further narrows down the answer candidates.

[0111] [Supplementary Note 9] The search system according to any one of Supplementary Notes 6 to 8, further comprising a question generation unit that generates the question sentence based on a sentence corresponding to each internal node of the tree.

[0112] [Supplementary Note 10] The search system according to any one of Supplementary Notes 6 to 9, wherein the determining means determines an optimized tree structure as a tree to be used for determining an answer to the query based on the relevance.

[0113] [Supplementary Note 11] The search system described in any one of Supplementary Notes 6 to 9, wherein, when an answer candidate includes a link to a document having a hierarchical structure, the determination means determines split answer candidates that are answer candidates to the query in the linked document based on a tree that indicates the structure of the linked document with the answer candidate as a root node.

[0114] [Supplementary Note 12] The search system according to Supplementary Note 9, wherein the question generation means generates a question that narrows down answer candidates using a machine learning model that generates questions from sentences based on sentences at each level of a document having a hierarchical structure.

[0115] [Supplementary Note 13] The search system according to any one of Supplementary Notes 1 to 12, wherein the relevance estimation means estimates the relevance based on an embedding vector of the query and an embedding vector of the answer candidate.

[0116] [Supplementary Note 14] The search system according to any one of Supplementary Notes 1 to 13, wherein the acquisition means acquires the query from a chatbot, and the output means outputs an answer to the question or query that narrows down the answer candidates to the chatbot.

[0117] [Supplementary Note 15] The search system according to any one of Supplementary Notes 1 to 14, wherein the relevance estimation means estimates the relevance by weighting the answer candidates based on a frequency of use of the answer candidates according to attributes of the user.

[0118] [Supplementary Note 16] A search method comprising: acquiring a query; estimating a degree of relevance between the acquired query and sentences of answer candidates contained in a document having a hierarchical structure; determining a question sentence that narrows down answer candidates to the query based on the hierarchical structure of the document and the estimated degree of relevance; and outputting the determined question sentence.

[0119] [Supplementary Note 17] A recording medium that non-temporarily records a search program that causes a computer to execute the following processes: a process of acquiring a query; a process of estimating the relevance between the acquired query and sentences of answer candidates included in a document having a hierarchical structure; a process of determining a question sentence that narrows down answer candidates to the query based on the hierarchical structure of the document and the estimated relevance; and a process of outputting the determined question sentence.

[0120] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0121] REFERENCE SIGNS LIST 10 Search system 101 Document acquisition unit 102 Structured data generation unit 103 Question generation unit 104 Acquisition unit 105 Relevance estimation unit 106 Determination unit 107 Output unit 108 Structured data storage unit 109 Question storage unit 110 Storage unit 20 Terminal device 200 Computer 201 CPU 202 Memory 203 Storage device 204 Input / output I / F 205 Communication I / F

Claims

1. The means of obtaining the query, A correlation estimation means for estimating the correlation between the acquired query and the candidate answer sentences contained in a document having a hierarchical structure, A decision means for determining a question that narrows down the candidate answers to the query based on the hierarchical structure of the document and the estimated degree of relevance, An output means for outputting the determined question text, A search system equipped with the following features.

2. The acquisition means acquires the user's response to the question that narrows down the answer candidates, The decision means determines, based on the user's response, the hierarchical structure of the document, and the estimated degree of relevance, whether to ask a question to further narrow down the candidate answers or to present candidate answers to the query. The output means outputs the question when it is decided to ask a question to further narrow down the answer candidates, and outputs the answer candidates when it is decided to present answer candidates for the query. The search system according to claim 1.

3. The determination means, when there are multiple answer candidates whose relevance to the query is above a predetermined standard, determines a question that narrows down the answer candidates from the question sentences at a higher level that are common to the answer candidates whose relevance is above the predetermined standard. The search system according to claim 1 or 2.

4. The document further comprises a question generation means that generates the question sentence based on the sentences included in each level of the hierarchical structure of the document. The search system according to claim 1 or 2.

5. The system further comprises a structure data generation means for generating data on the hierarchical structure of a document based on at least one of the descriptions regarding the chapter structure and the descriptions regarding the titles in a document having a hierarchical structure. The search system according to claim 1 or 2.

6. The structured data generation means generates a tree as data representing the hierarchical structure, in which the sentence at the highest level is the root, the candidate answers to the query are leaf nodes, and the sentences in each level between the highest level and the level of the candidate answers to the query are internal nodes. The search system according to claim 5.

7. The determination means, when there are multiple answer candidates whose relevance to the query is above a predetermined standard, determines the question statement to be the question statement corresponding to the internal node that is commonly linked to the leaf nodes corresponding to the answer candidates whose relevance is above the predetermined standard. The search system according to claim 6.

8. The determination means determines, among the question sentences not used in the question for narrowing down the answer candidates, the question sentence corresponding to the highest-level internal node, as the question sentence for the question for further narrowing down the answer candidates. The search system according to claim 7.

9. Get the query, The degree of relevance between the obtained query and the candidate answer sentences contained in the hierarchical document is estimated. Based on the hierarchical structure of the aforementioned documents and the estimated degree of relevance, a question is determined to narrow down the candidate answers to the query. Output the determined question text. How to search.

10. The process of retrieving the query, A process to estimate the degree of relevance between the obtained query and the candidate answer sentences contained in a document having a hierarchical structure, A process to determine a question that narrows down the candidate answers to the query based on the hierarchical structure of the document and the estimated degree of relevance, The process of outputting the determined question text and A search program that causes a computer to execute a command.