Information processing device, information processing method, and information processing program

Abstract

To improve retrieval performance for creating an appropriate answer to a question of a user.SOLUTION: An information processing apparatus according to the present application includes an evaluation unit and an extraction unit. The evaluation unit evaluates degree of association between each of a plurality of retrieval results retrieved based on a question of a user and the question. The extraction unit extracts, from the retrieval results, a retrieval result having the high degree of association evaluated by the evaluation unit, where the retrieval result is used for creating an answer to the question.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] Conventionally, automatic response services such as chatbots have been becoming widespread. For example, there is known a technique of creating an answer while referring to a sentence corresponding to a user's question using a search model when the user inputs a question.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology, for example, when the user's question is abstract, when there are many similar sentences in referring to sentences when creating an answer to the question, or when the source of the answer is unclear and it is difficult to judge whether it is correct or not, there are cases where an appropriate answer to the user's question cannot be provided.

[0005] The present application has been made in view of the above, and an object thereof is to improve the search performance for creating an appropriate answer to a user's question. ?

Means for Solving the Problems

[0006] The information processing apparatus according to the present application includes an evaluation unit that evaluates the relevance of a plurality of search results retrieved based on a user's question and the question, respectively, and an extraction unit that extracts, from the search results, search results having a high relevance evaluated by the evaluation unit and that are for creating an answer to the question.

Effects of the Invention

[0007] According to one embodiment, the search performance for generating appropriate answers to user questions can be improved. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is an explanatory diagram illustrating the overview of the information processing according to the embodiment. [Figure 2] Figure 2 shows an example of the configuration of an information processing system according to the embodiment. [Figure 3] Figure 3 shows an example of information processing according to the embodiment. [Figure 4] Figure 4 shows an example of information processing (question conversion) according to the embodiment. [Figure 5] Figure 5 shows an example of information processing (improved search) according to this embodiment. [Figure 6A] Figure 6A is an explanatory diagram (1) illustrating the relationship between the first-generation target and the second-generation target according to the embodiment. [Figure 6B] Figure 6B is an explanatory diagram (2) illustrating the relationship between the first-generation target and the second-generation target according to the embodiment. [Figure 7] Figure 7 shows an example of information processing (reranking) according to the embodiment. [Figure 8] Figure 8 is an explanatory diagram illustrating the rerank according to the embodiment. [Figure 9] Figure 9 shows an example of the configuration of a terminal device according to this embodiment. [Figure 10] Figure 10 shows an example of the configuration of an information processing device according to an embodiment. [Figure 11] Figure 11 shows an example of a first-generation target storage unit according to this embodiment. [Figure 12] Figure 12 shows an example of a second-generation target storage unit according to this embodiment. [Figure 13] Figure 13 is a flowchart showing an example of the question conversion process according to the embodiment. [Figure 14] FIG. 14 is a flowchart showing an example of the search improvement process according to the embodiment. [Figure 15] FIG. 15 is a flowchart showing an example of the rerank process according to the embodiment. [Figure 16] FIG. 16 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing apparatus.

Embodiment for Carrying Out the Invention

[0009] Hereinafter, embodiments (hereinafter referred to as "embodiments") for implementing the information processing apparatus, information processing method, and information processing program according to the present application will be described in detail with reference to the drawings. Note that the information processing apparatus, information processing method, and information processing program according to the present application are not limited by this embodiment. In the following embodiments, the same parts are denoted by the same reference numerals, and duplicate descriptions are omitted.

[0010] (Embodiment) [1. Outline of Information Processing] Conventionally, automatic response services such as chatbots have been spreading. For example, there is a known technique in which when a user inputs a question, an answer is created while referring to sentences corresponding to the user's question using a search model. However, in the conventional technique, for example, when the user's question is abstract, when there are many similar sentences in the reference of sentences when creating an answer to the question, or when the source of the answer is unclear and it is difficult to judge whether it is correct or not, it may not be possible to provide an appropriate answer to the user's question.

[0011] For example, with respect to a user's question such as "Tell me about the welfare within the company", when the question is abstract and no search hits are obtained, or when relevant sentences cannot be searched, there may be cases where an answer cannot be given. Also, for example, when there are many similar sentences and it is impossible to judge which one to use, all sentences may be used in the answer regardless of their relevance. Also, for example, when the source of the answer is unclear, sentences without direct descriptions such as translations may be used in the answer.

[0012] FIG. 1 is a diagram showing an example of information processing according to the prior art. When user U1 inputs a question Q regarding the company's internal regulations (step S1), a plurality of sentences with a high degree of similarity to question Q are retrieved (step S2) and extracted from the sentences of the internal regulations (step S3), and an answer to question Q is generated and provided from those sentences (step S4).

[0013] However, in the information processing of FIG. 1, as described above, there were cases where an appropriate answer to the user's question could not be provided. In particular, text generation models such as GPT and so-called generative AI such as Stable Diffusion perform learning using information belonging to many domains in order to generate and output responses (text or images) for various inputs. As a result of such learning, it can be said that generative AI can output general answers to general questions, but it cannot be said that it can generate appropriate answers to questions that require information in a specific domain such as internal regulations. On the other hand, a model that has learned only information in a specific domain has not learned general responses, so it is considered that the user needs to input appropriate inputs. Therefore, there is a need for a process to appropriately provide specialized information such as internal regulations as an answer from the question input by the user without considering the input for obtaining an appropriate answer. The present application has been made in view of the above, and aims to improve the search performance for creating an appropriate answer to the user's question.

[0014] 〔2. Configuration of Information Processing System〕 The information processing system 1 shown in FIG. 2 will be described. As shown in FIG. 2, the information processing system 1 includes a terminal device 10 and an information processing device 100. The terminal device 10 and the information processing device 100 are communicably connected by wire or wirelessly via a predetermined communication network (network N). FIG. 2 is a diagram showing a configuration example of the information processing system 1 according to the embodiment.

[0015] Terminal device 10 is an information processing device used by a user who asks a question. The user is, for example, a user who asks questions about the contents of regulations, laws, rules, manuals, etc. For example, a user who asks a question such as "Please tell me about the company's employee benefits" in order to check the company's employee benefits. The user enters the question using, for example, a designated chat tool. Terminal device 10 can be any device as long as it can realize the processing in the embodiment. Also, terminal device 10 may be a smartphone, tablet device, notebook PC, desktop PC, mobile phone, PDA, etc. Figure 2 shows the case where terminal device 10 is a smartphone.

[0016] Terminal device 10 is, for example, a smart device such as a smartphone or tablet, and is a mobile terminal device that can communicate with any server device via a wireless communication network such as 4G-5G (Generation) or LTE (Long Term Evolution). Terminal device 10 also has a screen, such as an LCD display, which has touch panel functionality and may accept various operations on displayed data such as content from the user, such as tapping, sliding, and scrolling, using a finger or stylus. In Figure 2, terminal device 10 is used by user U1.

[0017] The information processing device 100 is an information processing device aimed at improving search performance for creating appropriate answers to user questions, and can be any device as long as it can realize the processing in the embodiment. For example, it aims to provide appropriate answers to user questions even when the user's question is abstract, when there are many similar sentences when referencing sentences to create answers to questions, or when the source of the answer is unknown and it is difficult to determine whether it is correct or incorrect. The information processing device 100 performs at least one of the following three main processes (question transformation, search improvement, and reranking). For example, the information processing device 100 may perform at least one of the following processes, and other information processing devices may perform the other processes and provide the processing results to the information processing device 100.

[0018] In the first process (question transformation), the information processing device 100 provides the user's question to the generating AI to make the question more specific. This is expected to enable appropriate search hits even when the question is abstract, and to enable the appropriate retrieval of related texts. In the second process (search improvement), the information processing device 100 uses the result of the first process to search for information with a high degree of similarity to the first process result and identifies the information that will serve as the basis for creating an answer to the question. This is expected to enable the acquisition of an answer using texts that are truly relevant to the answer, even when there are many similar texts. In the third process (reranking), the information processing device 100 uses the result of the second process to evaluate the degree of relevance between the second process result and the question and reranks the second process result. This is expected to enable the acquisition of an answer using an appropriate source even when the source is unknown and it is difficult to determine whether it is correct or incorrect. Finally, the information processing device 100 provides the result of the third process to the generating AI to generate and provide an answer to the question.

[0019] Generative AI is a generative model that has been trained using data publicly available on the internet, for example. A GPT (Generative Pre-trained Transformer) model is one such model that generates answers to questions. In such models, answers are generated from publicly available data on the internet. Because new data is published daily on the internet, the model's answers are updated daily. This means that even if the same question is asked, the data the model references when generating the answer will differ depending on when the question is asked, resulting in different answers. Furthermore, because the model can provide detailed answers to questions across a wide range of fields, users ask the model a diverse array of questions.

[0020] Although Figure 2 shows a case where the terminal device 10 and the information processing device 100 are separate devices, the terminal device 10 and the information processing device 100 may be integrated into a single unit.

[0021] [3. An example of information processing] Figure 3 is a diagram illustrating an example of information processing according to the embodiment. It is a diagram that summarizes the three processes described above and shows the flow of a series of processes. Details of the three processes will be described later for each process. When user U1 inputs a question Q regarding the company's internal regulations (step S11), the question Q is provided to the generating AI (step S12) to perform question conversion (concretization of the question) (step S13). Then, the similarity (cosine similarity) with the information after the question is compared to search for multiple sentences with high similarity (step S14), and each sentence associated with each of those sentences is identified (step S15). Then, the identified sentences are provided to the generating AI (step S16) and reranked by evaluating their relevance to question Q (step S17). Then, sentences with high relevance evaluations are extracted based on the reranking results (step S18), and an answer to question Q is generated and provided from the extracted sentences (step S19). Details of each process will be explained below.

[0022] Figure 4 shows an example of information processing (question conversion) according to the embodiment. The information processing device 100 acquires a question from user U1 transmitted from terminal device 10 when user U1 inputs a question (step S101). In Figure 4, question Q is acquired. The information processing device 100 provides question Q to the generation AI along with prompts to concretize question Q (step S102), and acquires the generation result from the generation AI (step S103). This generation result is information used to narrow down the target that will serve as the basis for creating an answer to question Q in the second process. That is, the information processing device 100 uses the generation result in step S103 to search for information that will serve as the basis for creating an answer to question Q (step S104). Note that this question conversion process may be executed when the level of abstraction of the question is high. For example, the information processing device 100 may evaluate the level of abstraction of question Q and perform question conversion if the level of abstraction of question Q is evaluated as high, and if the level of abstraction of question Q is not evaluated as high, it may perform the second process using question Q.

[0023] Here, we will explain the details of the question transformation according to the embodiment. There are mainly two types of question transformation according to the embodiment. The first type of question transformation is transformation into a provisional answer. For example, if question Q is "Tell me about employee benefits," then question Q is transformed into a provisional answer such as, "At our company, we focus on employee benefits. The following are our main employee benefit programs. Health insurance: We provide health insurance to all employees. This allows employees to cover medical expenses in case of illness or injury. These are our main employee benefit programs. We are enhancing these programs to improve the ease of work and satisfaction of our employees."

[0024] The second type of question transformation is the transformation of a question into a question similar to another question. For example, it can be transformed into one or more questions. For instance, if question Q is "Tell me about employee benefits," it can be transformed into one or more similar questions such as "What types and contents of employee benefits are there?", "Tell me about the company's employee benefits system," "What are the benefits of employee benefits?", and "Tell me about the methods of implementing employee benefits and specific examples."

[0025] The information processing device 100 may perform question conversion by providing the generating AI with a prompt for converting question Q into a provisional answer along with question Q, or by providing the generating AI with a prompt for converting question Q into a similar question along with question Q. The prompt may also include an instruction sentence indicating that question Q should be converted into a provisional answer, along with an instruction sentence that expresses at least one of the following in natural language: role, task content, task objective, premise, and constraints. Here, role is a sentence that indicates the position in which the generating AI will provide the answer when generating the answer, for example, a sentence that sets a role such as "You are a professional in prompt generation." Task content is a sentence that indicates the content of the task that the generating AI will perform when generating the answer, for example, a sentence that gives a task instruction such as "Your task is to write an answer to the user's question." Task objective is a sentence that indicates the purpose for which the generating AI will perform the task when generating the answer, for example, a sentence that gives a task objective such as "To help the user search for related articles." Furthermore, a premise is a sentence that indicates the assumptions under which the generating AI will perform the task when generating an answer, for example, a sentence that gives the premise of the task, such as "This concerns procedures related to company regulations." Furthermore, a constraint is a sentence that indicates the constraints under which the generating AI will perform the task when generating an answer, for example, a sentence that gives the constraint of the task, such as "It must be written within XX characters." Based on instructions such as role, task content, task objective, premise, and constraint, the conversion of question Q into a provisional answer may be performed. Furthermore, a prompt may include an instruction sentence indicating that question Q should be converted into a similar question, along with an instruction sentence that expresses at least one of the roles, task content, task objective, premise, and constraint in natural language. Based on instructions such as role, task content, task objective, premise, and constraint, the conversion of question Q into a similar question may be performed.

[0026] As a result, while vector searches are difficult with abstract questions, search accuracy can be improved by making the questions more specific.

[0027] Figure 5 shows an example of information processing (improved search) according to the embodiment. In the following embodiment, the target that serves as the basis for creating the answer to question Q is referred to as the first generation target, and the target generated from the first generation target is referred to as the second generation target. The first generation target and the second generation target have a parent-child chunk relationship. For example, the second generation target is a generation target generated by dividing the first generation target into predetermined sentence units. Alternatively, for example, the second generation target is a generation target generated by dividing the first generation target into sentence units such that the number of characters is less than or equal to the maximum number of characters. Furthermore, the first generation target is a generation target generated from predetermined source data, for example, on the premise that it will be used to answer a question. In Figure 5, the second generation target is searched, the first generation target is identified from the second generation target, and the data of the first generation target is used to create the answer to question Q.

[0028] In Figure 5, in order to determine candidate information to be used to create an answer to question Q, candidate information to be used to create an answer to question Q is searched directly from the information type "first-generation target," and information to identify candidate information to be used to create an answer to question Q is searched from the information type "second-generation target." Here, as a result of searching for information from the information types "first-generation target" and "second-generation target," if candidate information to be used to create an answer to question Q can be found directly from the information type "first-generation target," that information is designated as candidate information to be used to create an answer to question Q. On the other hand, if candidate information to be used to create an answer to question Q cannot be found directly from the information type "first-generation target," it is determined whether information to identify candidate information to be used to create an answer to question Q can be found from the information type "second-generation target." If it is determined that information can be found, the information identified in association with that information is designated as candidate information to be used to create an answer to question Q. In the former case, where a direct search is performed from the information type "first-generation target," it is, for example, when information with a high degree of similarity to the first processing result can be found from the information type "first-generation target." For example, when information with a higher degree of similarity to the first processing result can be found from the information type "first-generation target" than information that can be found from the information type "second-generation target." On the other hand, in the latter case, where a search is performed from the information type "second-generation target," it is, for example, when information with a high degree of similarity to the first processing result cannot be found from the information type "first-generation target." For example, when information with a higher degree of similarity to the first processing result cannot be found from the information type "first-generation target" than information that can be found from the information type "second-generation target."

[0029] Furthermore, while Figure 5 illustrates the case where information is searched from both the "first-generation target" and "second-generation target" information types to determine candidate information to be used in creating an answer to question Q, it is also possible to use only the "second-generation target" information type. In other words, it is not limited to searching for both the information that can be searched from the "first-generation target" information type and the information that can be searched from the "second-generation target" information type; it is also possible to search for only the information that can be searched from the "second-generation target" information type. In this case, candidate information to be used in creating an answer to question Q is not searched from the "first-generation target" information type, but information to identify candidate information to be used in creating an answer to question Q is searched from the "second-generation target" information type. The information identified in conjunction with the information thus searched is then designated as candidate information to be used in creating an answer to question Q.

[0030] Figure 6A is an explanatory diagram illustrating the relationship between the first-generation target and the second-generation target according to the embodiment. In Figure 6A, parent data D2 and parent data D3, etc., which are the first-generation targets, are generated from source data D1; child data D4 and child data 5, which are the second-generation targets, are generated from parent data D2; child data (grandchild data) D8 and child data (grandchild data) D9, which are the second-generation targets, are generated from child data D4; and child data (grandchild data) D10 and child data (grandchild data) D11, which are the second-generation targets, are generated from child data D5. Note that if source data D1 is the entire text of the company regulations, then parent data D2 and parent data D3, etc., are part of the company regulations, child data D4 and child data 5 are further part of that, and child data (grandchild data) D8, child data (grandchild data) D9, child data (grandchild data) D10 and child data (grandchild data) D11 are further part of that.

[0031] Figure 6B provides a concrete example of Figure 6A. In Figure 6B, parent data D2 and parent data D3 in Figure 6A correspond to parent chunks, child data D4 and child data 5 in Figure 6A correspond to child chunk 1, and child data (grandchild data) D8, child data (grandchild data) D9, child data (grandchild data) D10 and child data (grandchild data) D11 in Figure 6A correspond to child chunk 2. In Figures 6A and 6B, further child data (great-grandchild data from the perspective of the first generation target) may be generated from each of the child data (grandchild data) D8 to child data (grandchild data) D11, which are targets of the second generation. The great-grandchild data generated in this way corresponds to child chunk 3 (not shown). In the embodiment below, with respect to the parent chunk which is the target of the first generation, all child chunks from child chunk 1 onwards (child chunk 2, child chunk 3, ...) are targets of the second generation.

[0032] The information processing device 100 obtains the first processing result (step S201). That is, it obtains the transformed information obtained by transforming question Q. The information processing device 100 searches for information with a high similarity to the first processing result (step S202). In this case, the information processing device 100 searches using vector search together with the original question, question Q. For example, the information processing device 100 searches based on the similarity of vector features. Then, the information processing device 100 searches for information from the information type "first-generation target" and the information type "second-generation target," and if information is searched from the information type "second-generation target," it identifies the information associated with the searched information (information corresponding to the information type "first-generation target") (step S203). In this case, for example, if multiple pieces of information corresponding to the information type "second-generation target" are found, the information processing device 100 identifies multiple pieces of information corresponding to the information type "first-generation target" by identifying the information corresponding to the information type "first-generation target" associated with each piece of information. In step S203, the information processing device 100 searches for information based on the information type "first-generation target" and the information type "second-generation target." If it searches for information based on the information type "first-generation target," it identifies the searched information (information corresponding to the information type "first-generation target"). In this case, for example, if multiple pieces of information corresponding to the information type "first-generation target" are found, the information processing device 100 identifies multiple pieces of information corresponding to the information type "first-generation target."

[0033] The information processing device 100 may, after searching for information from the information type "first-generation target" and the information type "second-generation target," determine whether or not information with a high degree of similarity to the first processing result can be found from the information type "first-generation target." If the information processing device 100 determines that information with a high degree of similarity to the first processing result can be found from the information type "first-generation target," it may directly search for information corresponding to the information type "first-generation target" from the information type "first-generation target" and identify the information found in that manner. If the information processing device 100 determines that information with a high degree of similarity to the first processing result cannot be found from the information type "first-generation target," it may search for information corresponding to the information type "second-generation target" from the information type "second-generation target" and identify the information corresponding to the information type "first-generation target" from the information found in that manner.

[0034] Furthermore, in steps S202 and S203, if there are multiple second-generation targets, the information processing device 100 may identify the generation target one level higher than the retrieved second-generation target. For example, using the examples in Figures 6A and 6B, if the retrieved second-generation target is child chunk 1, the information processing device 100 may identify the parent chunk associated with child chunk 1; if the retrieved second-generation target is child chunk 2, it may identify child chunk 1 associated with child chunk 2; and if the retrieved second-generation target is child chunk 3, it may identify child chunk 2 associated with child chunk 3. The information processing device 100 may then use the generation targets identified in this way as a second processing result. That is, they may be used as candidates for answering the question. Furthermore, even in such cases, the information processing device 100 may identify the first-generation target of the retrieved second-generation target. For example, using the examples in Figures 6A and 6B, if the retrieved second-generation target is child chunk 1, the device may identify the parent chunk associated with child chunk 1; if the retrieved second-generation target is child chunk 2, the device may identify the parent chunk associated with child chunk 1, which is associated with child chunk 2; and if the retrieved second-generation target is child chunk 3, the device may identify the parent chunk associated with child chunk 1, which is associated with child chunk 2, which is associated with child chunk 3, and this may be the second processing result. In addition, if a parent chunk is retrieved directly, the information processing device 100 may identify the parent chunk that was retrieved in that manner and this may be the second processing result. That is, the parent chunk identified in this manner may be used as a candidate for answering the question.

[0035] In this case, the source data, the first-generation target, and the second-generation target are not necessarily limited to text data, but may be multimodal data such as image data. For example, the information processing device 100 may use an image with reduced resolution as the first generation, and an image with further reduced resolution as the second generation, and may generate images of each generation by pixelating the image. By vectorizing these images of each generation together with the text, multimodal searching can be realized. For example, the information processing device 100 may use various models that recognize images and objects captured in the images and output the recognized objects in text format to convert the image into text, link the text with the text surrounding the image to create metadata, and identify the second processing result on a metadata basis. For example, when text is searched, the information processing device 100 may identify the data of the generation target associated with the second-generation target, which is metadata associated with the searched text, as the second processing result. Alternatively, for example, the information processing device 100 may directly search the metadata and identify the searched metadata as the second processing result.

[0036] As a result, while conventional techniques sometimes failed to perform appropriate searches due to reduced search accuracy even with correct answers when there was a lot of noise, this new approach allows for the search of highly relevant information by improving the embedding accuracy of "children," and by referencing "parents," it becomes possible to create comprehensive and detailed answers from a wide range of contexts.

[0037] Figure 7 shows an example of information processing (reranking) according to the embodiment. The information processing device 100 obtains a second processing result (step S301). That is, it obtains multiple search results retrieved based on question Q. The information processing device 100 provides question Q and the multiple search results to the generating AI along with a prompt for evaluating the degree of relevance between question Q and the multiple search results (step S302), and obtains the evaluation result from the generating AI (step S303). This evaluation result is information for reranking the multiple search results, which are the second processing result, in order to create an answer to question Q. That is, the information processing device 100 uses the evaluation result from step S303 to rerank the multiple search results, which are the second processing result, and extracts the search results to be used to create an answer to question Q (step S304). In this extraction process, the multiple search results are ranked in descending order of evaluation, and a predetermined number of top-ranking search results with high evaluations are extracted.

[0038] The information processing device 100 then provides the question Q and the extraction results to the generating AI along with a prompt for generating an answer to the question Q (step S305), and obtains the answer result from the generating AI (step S306). The information processing device 100 then provides the obtained answer result to the user U1 as the answer to the question Q (step S307).

[0039] As shown in Figure 7, the information processing device 100 evaluates the degree of relevance between multiple search results and question Q, extracts the search results that are evaluated as having a high degree of relevance, generates an answer to question Q using the extracted search results, and provides the generated answer to user U1.

[0040] Figure 8 is an explanatory diagram illustrating the reranking according to the embodiment. In Figure 8, documents A to E are candidates as multiple search results based on question Q. Documents A to E are first-generation targets. The relevance of each of documents A to E to question Q is evaluated by the generating AI, and the top search results with the highest relevance are extracted. In Figure 8, document D, which had the highest relevance, document B, which had the second highest relevance, and document C, which had the third highest relevance, are extracted. Then, an answer is created using the top search results with the highest relevance. In Figure 8, an answer to question Q is created using document D, which had the highest relevance, document B, which had the second highest relevance, and document C, which had the third highest relevance.

[0041] In this case, documents A through E are not necessarily limited to text data, but may be multimodal data such as image data. The information processing device 100 may generate an answer to question Q based on multiple search results, such as image data. In this case, the final answer to question Q may be text data or image data.

[0042] As a result, by re-evaluating information that was judged to have a high similarity in vector search based on its relevance to the question and narrowing it down to the top-ranking information, it becomes possible to create highly accurate answers.

[0043] [4. Configuration of terminal equipment] Next, the configuration of the terminal device 10 according to the embodiment will be described using Figure 9. Figure 9 is a diagram showing an example of the configuration of the terminal device 10 according to the embodiment. As shown in Figure 9, the terminal device 10 has a communication unit 11, an input unit 12, an output unit 13, and a control unit 14.

[0044] (Communications Section 11) The communication unit 11 is implemented, for example, by a NIC (Network Interface Card). The communication unit 11 is connected to a predetermined network N by wire or wireless connection and sends and receives information to and from the information processing device 100 via the predetermined network N.

[0045] (Input section 12) The input unit 12 accepts various operations from the user. In Figure 4, for example, it accepts various operations from user U1. For example, the input unit 12 may accept various operations from the user via a display surface using a touch panel function. Alternatively, the input unit 12 may accept various operations from buttons provided on the terminal device 10, or from a keyboard or mouse connected to the terminal device 10. For example, the input unit 12 accepts an operation to input a question.

[0046] (Output section 13) The output unit 13 is a display screen for a tablet terminal, for example, which is implemented using a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display device for displaying various types of information. For example, the output unit 13 displays information transmitted from the information processing device 100. For example, the output unit 13 displays the answer to a user question generated by the information processing device 100.

[0047] (Control Unit 14) The control unit 14 is, for example, a controller, and is implemented by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs stored in the internal memory of the terminal device 10 using RAM (Random Access Memory) as the working area. For example, these various programs include application programs installed on the terminal device 10. For example, these various programs include application programs that display information transmitted from the information processing device 100 (such as answers to user questions). The control unit 14 is also implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0048] As shown in Figure 9, the control unit 14 has a receiving unit 141 and a transmitting unit 142, and realizes or executes the information processing operations described below.

[0049] (Receiver 141) The receiving unit 141 receives information transmitted from, for example, the information processing device 100. For example, the receiving unit 141 receives answers to user questions.

[0050] (Transmitter 142) The transmitting unit 142 transmits, for example, user operation information. For example, the transmitting unit 142 transmits a question received from the user. For example, the transmitting unit 142 transmits a question entered, specified, or selected by the user.

[0051] [5. Configuration of the Information Processing Device] Next, the configuration of the information processing device 100 according to the embodiment will be described using Figure 10. Figure 10 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. As shown in Figure 10, the information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130. The information processing device 100 may also have an input unit (for example, a keyboard or mouse) that receives various operations from the administrator of the information processing device 100, and a display unit (for example, a liquid crystal display) for displaying various information.

[0052] (Communications Department 110) The communication unit 110 is implemented, for example, by a NIC. The communication unit 110 is connected to the network N by wire or wireless connection and sends and receives information to and from terminal devices 10, etc., via the network N.

[0053] (Storage unit 120) The storage unit 120 is implemented by, for example, semiconductor memory elements such as RAM and flash memory, or storage devices such as hard disks and optical discs. As shown in Figure 10, the storage unit 120 has a first-generation target storage unit 121 and a second-generation target storage unit 122.

[0054] The first-generation target storage unit 121 stores the first-generation targets. Here, Figure 11 shows an example of the first-generation target storage unit 121 according to this embodiment. The information stored in the first-generation target storage unit 121 is used, for example, to evaluate the degree of relevance to the user's question and to create an answer to the user's question. The information stored in the first-generation target storage unit 121 is also used, for example, to evaluate the degree of similarity to the user's question and to identify the first-generation targets. As shown in Figure 11, the first-generation target storage unit 121 has items such as "first-generation target ID", "source data ID", and "first-generation target".

[0055] The "First Generation Target ID" indicates identification information for identifying the first generation target. The "Source Data ID" indicates identification information for identifying the source data associated with the first generation target. The "First Generation Target" indicates the first generation target. In the example shown in Figure 11, conceptual information such as "First Generation Target #1" and "First Generation Target #2" is shown to be stored in "First Generation Target," but in reality, text, images, etc., that represent the first generation target are stored there.

[0056] The second-generation target storage unit 122 stores the second-generation targets. Here, Figure 12 shows an example of the second-generation target storage unit 122 according to this embodiment. The information stored in the second-generation target storage unit 122 is used, for example, to evaluate the similarity to the user's question and to identify the first-generation target. As shown in Figure 12, the second-generation target storage unit 122 has items such as "second-generation target ID", "first-generation target ID (or second-generation target ID)", and "second-generation target".

[0057] The "Second Generation Target ID" indicates identification information for identifying the second generation target. The "First Generation Target ID (or Second Generation Target ID)" indicates identification information for identifying the first generation target associated with the second generation target of the target. Furthermore, if the second generation target of the target is a "grandchild," it indicates identification information for identifying the second generation target that is its "child." The "Second Generation Target" indicates the second generation target. In the example shown in Figure 12, conceptual information such as "Second Generation Target #1" and "Second Generation Target #2" is shown as being stored in "Second Generation Target," but in reality, text, images, etc., that represent the second generation target are stored there.

[0058] (Control unit 130) The control unit 130 is a controller, and is implemented, for example, by a CPU or MPU executing various programs stored in the memory device inside the information processing device 100 using RAM as the working area. Alternatively, the control unit 130 can be implemented by an integrated circuit such as an ASIC or FPGA.

[0059] As shown in Figure 10, the control unit 130 includes an acquisition unit 131, a conversion unit 132, a search unit 133, a specific unit 134, an evaluation unit 135, an extraction unit 136, a generation unit 137, and a provision unit 138, and realizes or executes the information processing operations described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in Figure 10, and other configurations are also acceptable as long as they perform the information processing described later.

[0060] (Acquisition part 131) The acquisition unit 131 acquires various information from the storage unit 120. The acquisition unit 131 also stores the acquired information in the storage unit 120.

[0061] The acquisition unit 131 acquires various information from external information processing devices. The acquisition unit 131 also acquires various information from other information processing devices such as the terminal device 10.

[0062] The acquisition unit 131 acquires, for example, a user's question. For example, the acquisition unit 131 acquires a question that has been entered, specified, or selected by the user.

[0063] (Conversion unit 132) The conversion unit 132 converts the user's question into concrete information, for example. For example, the conversion unit 132 converts the user's question into concrete information in order to create an answer to the user's question using a generation AI. That is, for example, the conversion unit 132 converts it into information for creating an answer to the user's question.

[0064] The conversion unit 132 converts the data into information on which similarity can be determined in order to narrow down the target (corresponding to the search target) that will serve as the basis for creating the answer. For example, the conversion unit 132 converts the data into information on which similarity can be determined in a vector search, along with the user's question.

[0065] The conversion unit 132 performs a conversion process, for example, when the user's question meets predetermined conditions. For example, the conversion unit 132 performs a conversion process when the user's question is not excessively short (e.g., it is longer than a predetermined number of characters).

[0066] The conversion unit 132 converts the user's question into information that is a provisional answer to the question, for example, as an example of information that embodies the user's question. The conversion unit 132 also converts the user's question into information that is one or more questions similar to the question, for example, as an example of information that embodies the user's question.

[0067] The conversion unit 132 converts the user's question into concrete information by, for example, providing the generating AI with an instruction sentence indicating that the question should be converted. The conversion unit 132 also converts the user's question into concrete information by providing the generating AI with an instruction sentence that expresses at least one of the following in natural language: role, task content, task objective, premise, and constraints.

[0068] (Search section 133) The search unit 133 searches based on information converted by the conversion unit 132, for example. For example, the search unit 133 searches for information with a high similarity (cosine similarity) based on a vector search with the information converted by the conversion unit 132. For example, the search unit 133 searches for information corresponding to the type of information called "second-generation target" that is generated based on information corresponding to the type of information called "first-generation target". For example, the search unit 133 searches for information corresponding to the type of information called "second-generation target" that is generated by dividing information corresponding to the type of information called "first-generation target" into predetermined sentence units, or information corresponding to the type of information called "second-generation target" that is generated by dividing information corresponding to the type of information called "first-generation target" into sentence units so that the number of characters is less than or equal to the maximum number of characters. Also, for example, the search unit 133 searches for information corresponding to the type of information called "first-generation target". For example, the search unit 133 searches for information based on the information type "first-generation target" and the information type "second-generation target," and then determines whether information with a high degree of similarity to the first processing result can be found from the information type "first-generation target." If it determines that information with a high degree of similarity to the first processing result can be found from the information type "first-generation target," it searches for information corresponding to the "first-generation target" type. In this case, for example, if the search unit 133 determines that information with a high degree of similarity to the first processing result cannot be found from the information type "first-generation target," it searches for information corresponding to the "second-generation target" type. Furthermore, for example, the search unit 133 searches for multiple pieces of information with a high degree of similarity (information corresponding to a predetermined generation target) based on a vector search with the information converted by the conversion unit 132. For example, the search unit 133 searches for a predetermined number of pieces of information with a high degree of similarity (information corresponding to a predetermined generation target).

[0069] (Specific Section 134) The identification unit 134 identifies information corresponding to the type of information called a "first-generation target" that will serve as the basis for creating an answer to a question, based on information corresponding to the type of information called a "second-generation target" retrieved by the search unit 133. For example, the identification unit 134 identifies information corresponding to the type of information called a "first-generation target" that is linked to information corresponding to the type of information called a "second-generation target" retrieved by the search unit 133. For example, the identification unit 134 identifies information corresponding to the type of information called a "first-generation target" that is linked to multiple pieces of information corresponding to the type of information called a "second-generation target" retrieved by the search unit 133. Furthermore, the identification unit 134 identifies information corresponding to the type of information called a "first-generation target" retrieved by the search unit 133 as the basis for creating an answer to a question. For example, the identification unit 134 identifies information corresponding to multiple pieces of information called a "first-generation target" retrieved by the search unit 133.

[0070] (Evaluation Section 135) The evaluation unit 135 evaluates, for example, the degree of relevance between the first-generation target (corresponding to the search result) identified by the identification unit 134 and the user's question. For example, the evaluation unit 135 evaluates the degree of relevance between multiple first-generation targets and the user's question. For example, the evaluation unit 135 performs the evaluation by providing the generating AI with an instruction document indicating that it will evaluate the degree of relevance between multiple first-generation targets and the user's question. For example, the evaluation unit 135 performs the evaluation by providing the generating AI with multiple first-generation targets and the user's question, along with an instruction document indicating that it will evaluate the degree of relevance between multiple first-generation targets and the user's question.

[0071] (Extraction part 136) The extraction unit 136 extracts, for example, first-generation targets that have been evaluated as highly relevant by the evaluation unit 135 from among the first-generation targets identified by the identification unit 134. For example, the extraction unit 136 extracts a predetermined number of first-generation targets with high relevance by ranking the first-generation targets identified by the identification unit 134 in descending order of relevance.

[0072] (Generation unit 137) The generation unit 137 generates an answer to the user's question using, for example, one or more first-generation targets extracted by the extraction unit 136. For example, the generation unit 137 generates an answer to the user's question by providing the generation AI with an instruction sentence indicating that it will generate an answer to the question using first-generation targets. For example, the generation unit 137 generates an answer to the user's question by providing the generation AI with an instruction sentence indicating that it will generate an answer to the question using first-generation targets, along with one or more first-generation targets and the user's question.

[0073] (Provider 138) The providing unit 138, for example, provides the user with the answer generated by the generation unit 137 as the answer to the user's question.

[0074] [6. Information Processing Flow] Next, the information processing procedure by the information processing system 1 according to the embodiment will be described using Figures 13 to 15. Figure 13 is a flowchart showing the procedure for the question conversion process according to the embodiment, Figure 14 is a flowchart showing the procedure for the search improvement process according to the embodiment, and Figure 15 is a flowchart showing the procedure for the reranking process according to the embodiment.

[0075] As shown in Figure 13, the information processing device 100 acquires a user's question (step S401). The information processing device 100 determines whether the user's question satisfies predetermined conditions (step S402). If the information processing device 100 determines that the user's question satisfies predetermined conditions (step S402; YES), it provides the question to the generating AI along with an instruction message indicating that the question should be converted (step S403). The information processing device 100 then converts the user's question into concrete information (step S404). On the other hand, if the information processing device 100 determines that the user's question does not satisfy predetermined conditions (step S402; NO), it terminates the information processing. In this case, the information processing device 100 may notify the user to ask the question again (the content of the question may be the same) or to change the question.

[0076] As shown in Figure 14, the information processing device 100 acquires information that embodies the user's question (step S501). The information processing device 100 searches for multiple first-generation targets or second-generation targets that have a high similarity to the information that embodies the user's question by comparing the feature quantities of the information that embodies the user's question with the features of the first-generation target candidates and the second-generation target candidates (step S502). The information processing device 100 identifies the first-generation targets associated with the multiple first-generation targets or the multiple second-generation targets that have been found, thereby identifying the information that will serve as the basis for creating an answer to the user's question (step S503).

[0077] Furthermore, in steps S502 and S503, the information processing device 100 may determine the similarity between the information that embodies the user's question and the candidate first-generation target by comparing the feature quantities of the information that embodies the user's question and the candidate first-generation target. If there is a candidate first-generation target with a high degree of similarity, the device may search for multiple first-generation targets with a high degree of similarity and identify the multiple first-generation targets found to identify the information that forms the basis for creating an answer to the user's question. Alternatively, in steps S502 and S503, the information processing device 100 may determine the similarity between the information that embodies the user's question and the candidate first-generation target by comparing the feature quantities of the information that embodies the user's question and the candidate first-generation target. If there is no candidate first-generation target with a high degree of similarity, the device may search for multiple second-generation targets with a high degree of similarity and identify the first-generation targets associated with the multiple second-generation targets found to identify the information that forms the basis for creating an answer to the user's question. In this case, if there are no candidates for a second-generation target with a high degree of similarity, the information processing device 100 may notify the user that it cannot provide a highly accurate answer result, or notify the user that it is requesting that the question be changed (of course, the notification is not limited to such examples). Furthermore, in steps S502 and S503, the information processing device 100 may determine the similarity with both the first-generation target and the second-generation target, search for multiple generations with a higher degree of similarity, and identify multiple first-generation targets based on the search results to identify information that will serve as the basis for creating an answer to the user's question.

[0078] As shown in Figure 15, the information processing device 100 acquires a plurality of first-generation targets that satisfy predetermined conditions (step S601). The information processing device 100 provides the plurality of first-generation targets and the question to the generating AI along with an instruction document indicating that the degree of relevance between each of the plurality of first-generation targets and the user's question should be evaluated (step S602). The information processing device 100 evaluates the degree of relevance between each of the plurality of first-generation targets and the user's question and reranks the first-generation targets in descending order of relevance according to the degree of relevance (step S603). Based on the reranking results, the information processing device 100 extracts a predetermined number of first-generation targets with high relevance (step S604). Then, the information processing device 100 generates an answer to the user's question based on the extraction results (step S605).

[0079] [7. Effects] As described above, the information processing device 100 according to this embodiment includes an evaluation unit 135 and an extraction unit 136. The evaluation unit 135 evaluates the relevance of multiple search results retrieved based on the user's question to the question. The extraction unit 136 extracts from the search results the search results with a high degree of relevance evaluated by the evaluation unit 135, which are necessary for creating an answer to the question.

[0080] As a result, the information processing device 100 according to the embodiment can, for example, promote improved search performance for creating appropriate answers to user questions. Furthermore, the information processing device 100 according to the embodiment can, for example, enable the generation of answers using higher-level information by re-evaluating the relevance of the questions.

[0081] Furthermore, the information processing device 100 according to this embodiment has a generation unit 137 that generates an answer by providing the search results and questions extracted by the extraction unit 136 to a generation AI capable of generating answers to predetermined questions.

[0082] As a result, the information processing device 100 according to the embodiment can, for example, generate more accurate answers using information whose relevance has been re-evaluated.

[0083] Furthermore, the extraction unit 136 extracts a predetermined number of highly relevant search results by ranking the search results in order of relevance.

[0084] As a result, the information processing device 100 according to the embodiment can, for example, use only information with a higher relevance level in its re-evaluation to generate answers, thereby enabling the generation of answers that are more accurate and aligned with the user's questions.

[0085] Furthermore, the evaluation unit 135 evaluates the degree of relevance of each search result to a question by providing the search results and the question to a generating AI capable of evaluating the degree of relevance to a predetermined question.

[0086] As a result, the information processing device 100 according to the embodiment can, for example, accurately evaluate the degree of relevance to the question for each search result, thereby enabling the generation of more accurate answers that are in line with the user's questions.

[0087] Furthermore, the evaluation unit 135 evaluates the degree of relevance between the search results, which are the search results that form the basis of the search results linked to the search results retrieved based on the similarity with the information based on the question, and the search results themselves.

[0088] As a result, the information processing device 100 according to the embodiment can evaluate relevance based on search results that take similarity into account, thereby promoting improvement in search performance in two stages: similarity and relevance.

[0089] Furthermore, the search results are part of internal company regulations created for internal use.

[0090] As a result, the information processing device 100 according to this embodiment can, for example, improve the search performance for creating appropriate answers to user questions regarding internal company regulations.

[0091] [8. Hardware Configuration] Furthermore, the information processing device 100 according to the above-described embodiment is realized by a computer 1000 having the configuration shown in Figure 16. Figure 16 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 100. The computer 1000 has a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I / F) 1500, input / output interface (I / F) 1600, and media interface (I / F) 1700.

[0092] The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400, and controls various parts. The ROM 1300 stores boot programs executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0093] The HDD1400 stores programs executed by the CPU1100, as well as data used by such programs. The communication interface1500 acquires data from other devices via a predetermined communication network and sends it to the CPU1100, and transmits data generated by the CPU1100 to other devices via the predetermined communication network.

[0094] The CPU 1100 controls output devices such as displays and printers, and input devices such as keyboards and mice, via the input / output interface 1600. The CPU 1100 acquires data from input devices via the input / output interface 1600. The CPU 1100 also outputs the generated data to output devices via the input / output interface 1600.

[0095] The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 can be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), tape media, magnetic recording medium, or semiconductor memory.

[0096] For example, when the computer 1000 functions as an information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 130 by executing a program loaded on the RAM 1200. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, these programs may be obtained from other devices via a predetermined communication network.

[0097] [9. Other] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0098] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0099] Furthermore, the embodiments described above can be combined as appropriate, as long as the processing content is not contradictory.

[0100] Although some embodiments of the present invention have been described in detail above with reference to the drawings, these are illustrative examples, and the present invention can be implemented in various other forms with modifications and improvements based on the knowledge of those skilled in the art, starting with the embodiments described in the disclosure section of the invention.

[0101] Furthermore, the terms "section, module, unit" mentioned above can be replaced with "means" or "circuit," etc. For example, the acquisition unit can be replaced with acquisition means or acquisition circuit. [Explanation of symbols]

[0102] 1. Information Processing System 10 Terminal devices 11 Communications Department 12 Input section 13 Output section 14 Control Unit 100 Information Processing Devices 110 Communications Department 120 Storage section 121 First Generation Target Memory Unit 122 Second Generation Target Memory Unit 130 Control Unit 131 Acquisition Department 132 Conversion section 133 Search Section 134 Specific part 135 Evaluation Department 136 Extraction part 137 Generation part 138 Provision Department 141 Receiving Unit 142 Transmitter N Network

Claims

1. A system that evaluates the relevance of multiple first search results, generated by dividing predetermined source information based on a user's question, to the question, and an evaluation unit that evaluates the relevance of each of these results to the question. An extraction unit extracts from the first search results a predetermined number of second search results with a high degree of relevance as evaluated by the evaluation unit, for the purpose of creating an answer to the question. It has, The aforementioned first search result is the first piece of information that falls under the category of information targeting the first generation. The first search result is the first information identified by searching for second information that corresponds to multiple types of second-generation target information that have different relationships with the type of first-generation target information, based on the above question, and is linked to the said second information. An information processing device characterized by the following:

2. A generation unit generates the answer by providing the second search result extracted by the extraction unit and the question to a generation AI capable of generating answers to a predetermined question. The information processing apparatus according to claim 1, further comprising the above.

3. The extraction unit is By ranking the first search results in order of their relevance, a predetermined number of the top-ranking second search results with high relevance are extracted. The information processing apparatus according to feature 1.

4. The evaluation unit described above, The first search result and the question are provided to a generating AI capable of evaluating the degree of relevance to a predetermined question, thereby evaluating the degree of relevance to the question for each of the first search results. The information processing apparatus according to feature 1.

5. The aforementioned first search result is part of an internal company regulation created for internal use. The information processing apparatus according to feature 1.

6. A method of information processing performed by a computer, An evaluation process that evaluates the relevance of multiple first search results, which are generated by dividing predetermined source information and searching based on the user's question, to the question, An extraction step is performed to extract from the first search results a predetermined number of second search results that have the highest relevance as evaluated in the evaluation step, and to extract second search results for creating an answer to the question. Includes, The aforementioned first search result is the first piece of information that falls under the category of information targeting the first generation. The first search result is the first information identified by searching for second information that corresponds to multiple types of second-generation target information that have different relationships with the type of first-generation target information, based on the above question, and is linked to the said second information. An information processing method characterized by the following:

7. An evaluation procedure for evaluating the relevance of multiple first search results, which are generated by dividing predetermined source information and searched based on a user's question, to that question, An extraction procedure for extracting a predetermined number of second search results from the first search results that have the highest relevance as evaluated by the above evaluation procedure, and for creating an answer to the above question; Have the computer run it, The aforementioned first search result is the first piece of information that falls under the category of information targeting the first generation. The first search result is the first information identified by searching for second information that corresponds to multiple types of second-generation target information that have different relationships with the type of first-generation target information, based on the above question, and is linked to the said second information. An information processing program characterized by the following features.

Images