Information processing systems, information processing methods, and programs

Abstract

We provide an information processing system that can easily offer a mechanism for flexibly answering user questions. [Solution] According to one aspect of the present invention, an information processing system is provided comprising one or more processors, wherein the processors, in a reception step, receive a user question from a user; in an extraction step, extract one or more Q&A data that are more relevant to the user question than other Q&A data from a pre-prepared group of Q&A data; in a generation step, generate an answer to the user question as a user-facing answer based on the extracted one or more Q&A data; and in a presentation step, present the generated user-facing answer to the user.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Patent Document 1 discloses a technique including an execution unit that receives message information from a terminal device, a scenario that leads from question information to answer information, and transmits, to the terminal device, message information that responds to the message information received from the terminal device based on setting information that associates the question information and the answer information, and a display control unit that displays a correction unit based on a first evaluation value for the question information and the answer information and a second evaluation value for the scenario.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a demand for providing a mechanism that can simply and flexibly present an answer to a user's question.

[0005] In view of the above circumstances, the present invention aims to provide an information processing system or the like that can easily provide a mechanism for flexibly answering user questions.

Means for Solving the Problems

[0006] According to one aspect of the present invention, an information processing system is provided comprising one or more processors, wherein the processors, in a reception step, receive a user question from a user; in an extraction step, extract one or more Q&A data that are more relevant to the user question than other Q&A data from a pre-prepared group of Q&A data; in a generation step, generate an answer to the user question as a user-facing answer based on the extracted one or more Q&A data; and in a presentation step, present the generated user-facing answer to the user.

[0007] This configuration makes it easy to provide a mechanism that can flexibly answer user questions. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the overall configuration of Question Answering System 1. [Figure 2] This figure shows an example of the hardware configuration of the response server 10. [Figure 3] This figure shows an example of the hardware configuration of user terminal 30. [Figure 4] This is an activity diagram showing an example of response service processing. [Figure 5] This figure shows an example of a user question that was entered. [Figure 6] This figure shows an example of a Q&A data set. [Figure 7] This figure shows an example of a displayed response. [Figure 8] This figure shows an example of the displayed weight setting screen. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below can be combined with each other.

[0010] Incidentally, the program for implementing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or it may be provided as a downloadable medium from an external server, or it may be provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] Furthermore, in various information processing according to one embodiment, an input and an output corresponding to the input can be realized. Here, as long as an output is obtained as a result of the input, the form of the information referenced in such information processing (hereinafter referred to as "reference information") is not limited. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression equation constructed by a statistical method), or a trained model that has been pre-trained to learn the correlation between input and output, or a generative AI such as a large-scale language model that can output a desired result by inputting a prompt (these models include parameters that construct the correlation relationship between input and output) or a visual language model.

[0012] Furthermore, in one embodiment, "part" may include, for example, hardware resources implemented by a circuit in a broad sense, and the information processing of software that can be specifically realized by these hardware resources. Also, in one embodiment, various types of information are handled, and this information can be represented, for example, by the physical values ​​of signal values ​​representing voltage and current, the high or low values ​​of signal values ​​as a set of binary bits composed of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on a circuit in a broad sense.

[0013] Furthermore, a circuit in a broad sense is a circuit realized by combining at least a suitable combination of circuits, circuits, processors, and memory. The processor may be a general-purpose processor or a dedicated circuit. In other words, it includes application-specific integrated circuits (ASICs), programmable logic devices (for example, simple programmable logic devices (SPLDs), complex programmable logic devices (CPLDs), and field programmable gate arrays (FPGAs)), etc.

[0014] <Embodiment> 1. System Configuration The system configuration according to the embodiment will be described below. Figure 1 shows an example of the overall configuration of Question Answering System 1. Question Answering System 1 is a system that provides a response service that returns answers to questions from users in natural language. Figure 1 shows an overview of each device included in Question Answering System 1 and an overview of the users who use those devices. Each overview will be explained as needed, with reference to other figures.

[0015] The question answering system 1 comprises a communication line 2, a response server 10, a chat server 20, and a user terminal 30. Note that the configuration shown in Figure 1 is an example; the response server 10 and the chat server 20 may each be distributed across two or more devices, and may be provided in the form of SaaS (Software as a Service) or a cloud computing system. Furthermore, there may be two or more user terminals 30.

[0016] The communication line 2 is not particularly limited, but for example, it is constituted by the Internet network. Further, the communication line 2 may include a local area network, a mobile communication network, a VPN (Virtual Private Network), and the like. The communication line 2 mediates the exchange of data between devices connected to its own line. In the example of FIG. 1, a response server 10 and a chat server 20 are wired-connected to the communication line 2, and a user terminal 30 is wirelessly connected. Note that the connection of each device to the communication line 2 may be wired or wireless.

[0017] The user terminal 30 is a terminal used by a user who uses the question-and-answer system 1, and for example, is a notebook personal computer or a tablet terminal. The user terminal 30 has display means and input means and functions as a UI (User Interface) for work by the user. The chat server 20 is an information processing device that executes information processing for providing a chat service. The response server 10 is an information processing device that executes information processing for providing the above-described response service.

[0018] The response server 10 includes a Q&A information DB3 (DB: Data Base). A plurality of pre-prepared Q&A data is stored in the Q&A information DB3. The Q&A data is data configured by associating a question with its answer, and these plurality of Q&A data are collectively referred to as a Q&A data group. The Q&A data represents, for example, questions and answers regarding a predetermined field. Examples of the predetermined field may include fields related to business (such as legal affairs, accounting, and general affairs), fields related to products, or fields related to the usage method of services.

[0019] In addition, the response server 10 executes, for example, an authentication process for authenticating a user and a display process for causing a client terminal such as the user terminal 30 to display an image. The response server 10 stores authentication information (e.g., user ID, password, etc.) for authenticating the user, and authenticates the user who has input the authentication information as a user of the question-and-answer system 1. By authenticating the user, the response server 10 determines the data that the user can access, or assigns identification information to the data input by the user to make it identifiable.

[0020] For example, the response server 10 performs processes such as generation and transmission of HTML (Hyper Text Markup Language) files by a web program as display processes, generates a web page showing the system screen, and causes the generated system screen to be displayed on the client terminal using the function of the browser. Note that an application program for the client terminal to use the question-and-answer system 1 may be introduced, and the response server 10 may generate and display a system screen that can be displayed by the application. By performing these display processes, the response server 10 controls the display of the client terminal.

[0021] The above-described authentication process and display control process are executed not only by the response server 10 but also by the chat server 20. The authentication information of the user used in each server may be common or different. In the case of being different, a correspondence table showing the correspondence relationship between these authentication information is used to manage the users in each server as the same user.

[0022] 2. Hardware Configuration Hereinafter, the hardware configuration according to the embodiment will be described. FIG. 2 is a diagram showing an example of the hardware configuration of the response server 10. The response server 10 includes a control unit 101, a storage unit 102, a communication unit 103, and a bus 104. The bus 104 electrically connects each part included in the response server 10.

[0023] (Control Unit 101) The control unit 101 has at least one processor. The at least one processor may consist of, for example, a central processing unit (CPU), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), one or more integrated circuits, one or more discrete circuits, or a combination thereof.

[0024] The control unit 101 is a computer that realizes various functions related to the question answering system 1 by reading a predetermined program stored in the memory unit 102. In other words, information processing by the software stored in the memory unit 102 is concretely realized by the control unit 101, which is an example of hardware, and can be executed as each functional unit included in the control unit 101. Note that the control unit 101 is not limited to being a single unit, and may be implemented with multiple control units 101 for each function, or a combination thereof.

[0025] (Storage unit 102) The memory unit 102 stores various types of information as defined above. This can be done, for example, as a storage device such as a solid-state drive (SSD) or hard disk drive (HDD) that stores various programs related to the question answering system 1 executed by the control unit 101, or as memory such as random access memory (RAM) that stores temporarily necessary information (arguments, arrays, etc.) related to program calculations. The memory unit 102 stores various programs and variables related to the question answering system 1 executed by the control unit 101.

[0026] (Communications Department 103) The communication unit 103 is composed of a communication module. The communication module may be a wireless communication module compliant with standards such as IEEE 802.11a / b / g / n / ac / ax, LTE, 5G, 6G, or a wired communication module compliant with standards such as IEEE 802.3. The communication unit 103 is configured to transmit various electrical signals from the response server 10 to external components. The communication unit 103 is also configured to receive various electrical signals from external components to the response server 10. More preferably, the communication unit 103 has a network communication function, thereby enabling the communication of various information between the response server 10 and external devices via the communication line 2.

[0027] The chat server 20 has the same hardware configuration as the response server 10. For the chat server 20, only the control unit 201 will be described using a different reference numeral than the control unit 101 of the response server 10.

[0028] Figure 3 shows an example of the hardware configuration of a user terminal 30. The user terminal 30 comprises a control unit 301, a storage unit 302, a communication unit 303, an input unit 304, an output unit 305, and a bus 306. The bus 306 electrically connects each part of the user terminal 30. The control unit 301, storage unit 302, and communication unit 303 are similar hardware to the control unit 101, storage unit 102, and communication unit 103 shown in Figure 3, although their specifications and models may differ.

[0029] (Input section 304) The input unit 304 has input receiving means such as keys, buttons, touchscreens, and mice, and accepts input from the user. The input unit 304 may also have sound receiving means such as a microphone, and may have a function to collect the user's voice and accept the input of the collected voice.

[0030] (Output section 305) The output unit 305 has display means such as a display and sound emission means such as a speaker, and outputs visual information and auditory information. For example, the output unit 305 displays visual information such as screens, images, icons, and text on the display surface of the display unit in a manner that is visible to the user. The output unit 305 also outputs audible sound such as speech or synthesized sound from the speaker.

[0031] 3. Information Processing The following describes the information processing according to the embodiment. In the following description, the response server 10, the chat server 20, and the user terminal 30 are described as the main entities of each information processing, but these information processing are executed by at least one processor provided in the question answering system 1, that is, the processors in the control units 101, 201, and 301 of each device. The following describes the response service processing that is performed when a response service is provided.

[0032] Figure 4 is an activity diagram showing an example of response service processing. Response service processing is initiated, for example, when a user operates the user terminal 30 to use the chat service provided by the chat server 20. First, the chat server 20 generates a chat screen (activity A11) and displays the generated chat screen on the user terminal 30 (activity A12). The user terminal 30 accepts user questions in threads prepared for each field (activity A13). Questions entered by the user in this way will be referred to as "user questions" below.

[0033] Figure 5 shows an example of a user question that has been entered. In the chat screen G1 shown in Figure 5, user question Q1 has been entered in the thread for "Field A" among the threads for "Field A" through "Field D". When the confirmation button B11 is pressed on the user terminal 30, the entered user question Q1 is confirmed and sent to the chat server 20. The chat server 20 receives the sent user question Q1 as a user question related to Field A (Activity A14).

[0034] In parallel, the response server 10 periodically monitors the chat server 20 for any user questions (Activity A15). When the response server 10 receives a user question, it requests the chat server 20 to send the user question along with field information indicating the field of the question, and accepts the user question and field information sent in response to the request. When the response server 10 receives user question Q1 in field A, it reads the Q&A data set for field A from the Q&A information DB3 (Activity A21).

[0035] Figure 6 shows an example of a Q&A data set. In the Q&A information DB3 shown in Figure 6, fields, question numbers, questions, and answers are stored in a corresponding manner. Fields include "Field A" and "Field B," etc. Question numbers include numbers such as "QA001," "QA002," and "QA003" corresponding to Field A, and numbers such as "QB001" and "QB002" corresponding to Field B. When the response server 10 receives a user question Q1 in Field A, it reads the Q&A data set corresponding to Field A.

[0036] The response server 10 calculates a first metric for each Q&A data based on the retrieved Q&A data set and the acquired user question Q1 (Activity A22). For example, the response server 10 calculates the magnitude of the similarity between the vectors of user question Q1 and the Q&A data as the first metric for that Q&A data. The response server 10 converts each text representing user question Q1 and each Q&A data into a numerical vector using a predetermined vectorization model (Word2Vec, FastText, or BERT (Bidirectional Encoder Representations from Transformers), etc.).

[0037] Specifically, the response server 10 generates a first vector representing user question Q1 and a second vector representing each Q&A data. The first and second vectors are represented, for example, as N-dimensional real-valued vectors. Based on the generated first and second vectors, the response server 10 calculates the similarity between these vectors. For example, cosine similarity can be used to calculate the similarity. The response server 10 calculates cosine similarity by dividing the dot product of the first and second vectors by the product of the vector length of the first vector and the vector length of the second vector. The response server 10 uses the calculated cosine similarity value as the first index.

[0038] In addition, other methods such as Euclidean distance, Manhattan distance, or Pearson correlation coefficient may be used to calculate the similarity between vectors. Furthermore, the response server 10 may, if necessary, perform normalization processing so that the similarity values ​​between vectors fall within a predetermined range of 0 to 1. In this way, a score indicating the semantic closeness between the texts represented by the first and second vectors can be obtained as the first indicator for each Q&A data.

[0039] Next, the response server 10 performs preprocessing on the user question Q1 obtained in A15 (activity A23). Specifically, the response server 10 performs preprocessing by removing question suffixes and function words from the user question Q1. Question suffixes are words used at the end of a sentence to convey the intention of a question or inquiry, and include words such as "desu ka," "masu ka," and "oshite kudasai." Function words are words that play a role in indicating grammatical relationships in a sentence, and include particles and auxiliary verbs.

[0040] Next, the response server 10 calculates a second metric for each Q&A data based on the pre-processed user question Q1 and the Q&A data set read in A21 (Activity A24). First, the response server 10 extracts keywords contained in the user question Q1. Specifically, for example, the response server 10 extracts a continuous section consisting of Japanese characters from the user question Q1 based on the Unicode Script attribute, generates a string of 2 to 4 characters from that continuous section as an N-gram, and extracts the generated string as a keyword.

[0041] The Script attribute is classification information assigned to each character in the Unicode standard, indicating the character system (script) to which the character belongs. For example, Japanese characters such as kanji (Han), hiragana (Hiragana), and katakana (Katakana) each have different Script attributes. The response server 10 identifies Japanese characters based on these Script attributes and enables the extraction of consecutive Japanese strings.

[0042] Furthermore, N-gram is a technique for extracting a subsequence consisting of n consecutive elements from a string or word sequence that makes up text data. For example, when n=2, it is called a 2-character N-gram (bigram), and when n=3, it is called a 3-character N-gram (trigram). By using N-gram, it is possible to capture partial features of text without using morphological analysis, and keywords can be extracted flexibly even when unknown words or proper nouns are included.

[0043] The response server 10 calculates the inclusion rate of the extracted keywords in the Q&A data as a second indicator of the Q&A data. Specifically, for each extracted keyword, the response server 10 obtains the number of times that keyword appears in each Q&A data. The response server 10 calculates the keyword inclusion rate by dividing the sum of these occurrence counts by a predetermined standard value such as the total number of keywords extracted from user question Q1 or the total number of characters in the Q&A data.

[0044] Furthermore, the response server 10 may perform normalization processing so that the calculated inclusion rate is in the range of 0 or more and 1 or less. For example, the response server 10 can divide the number of keywords that match the Q&A data among the extracted keywords by the total number of extracted keywords and use this ratio as a second indicator. In this way, the response server 10 calculates a score indicating the similarity at the word level between user question Q1 and the Q&A data as a second indicator.

[0045] Next, the response server 10 narrows down a predetermined number of Q&A data from the Q&A data set based on the calculated first and second indicators (Activity A25). For example, the response server 10 narrows down a predetermined number of Q&A data from those with the highest sum of the first and second indicators. The predetermined number can be 10, 15, 20, 25, 30, 35, 40, 45, or 50. Here, we will explain the case where the number is narrowed down to 30.

[0046] Next, the response server 10 calculates a third metric, the semantic relevance of the Q&A data to user question Q1, from a predetermined number of filtered Q&A data (Activity A26). The response server 10 calculates the third metric using an AI module that has been trained to output the semantic similarity between two sentence data when input. Specifically, the response server 10 inputs user question Q1 and the filtered Q&A data into the AI ​​module, and calculates the semantic relevance of the Q&A data to user question Q1 output by the AI ​​module as the third metric.

[0047] Next, the response server 10 extracts one or more Q&A data entries that are highly relevant to user question Q1 based on the calculated third indicator (Activity A27). For example, the response server 10 extracts a predetermined number of Q&A data entries that are highly relevant to user question Q1, starting with those with the larger calculated third indicators. The predetermined number is, for example, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, or 20. Here, we will explain the case where 10 Q&A data entries are extracted.

[0048] Next, the response server 10 calculates a composite score for each extracted Q&A data (Activity A31). Specifically, the response server 10 calculates a composite score by weighting and adding the first, second, and third indicators calculated for each extracted Q&A data. The weighting varies depending on the field of the user question. For example, in field A, the weight of the first indicator is 0.45, the weight of the second indicator is 0.2, and the weight of the third indicator is 0.35.

[0049] Next, the response server 10 extracts one or more Q&A data entries that are highly relevant to user question Q1 based on the calculated composite score (Activity A32). For example, the response server 10 extracts Q&A data entries whose calculated composite score is equal to or greater than a predetermined value as Q&A data entries that are highly relevant to user question Q1. The response server 10 may also set a minimum number of Q&A data entries to extract. In that case, if the number of extracted Q&A data entries falls below the minimum number, the response server 10 extracts additional Q&A data entries in descending order of composite score until the minimum number is reached.

[0050] Then, the response server 10 generates an answer to user question Q1 (hereinafter referred to as the "user-facing answer") based on the user question Q1 and the Q&A data extracted in A32 (Activity A33). The response server 10 generates the user-facing answer to user question Q1 using an AI module that has been trained to output an answer to a question when it is input with a question and one or more Q&A data points. Specifically, the response server 10 inputs user question Q1 and the Q&A data extracted in A32 into the AI ​​module, and generates the answer output by the AI ​​module as the user-facing answer.

[0051] The response server 10 sends the generated user response to the chat server 20. When the chat server 20 receives the sent user response (Activity A34), it displays the received user response on the user terminal 30 as the answer to user question Q1 (Activity A35).

[0052] Figure 7 shows an example of a displayed response. In the chat screen G2 shown in Figure 7, the user response A1 is displayed following the user question Q1. The user response A1 includes the question number of the Q&A data used to generate the user response A1. In the example in Figure 7, "QA006", "QA028", "QA044", "QA049", and "QA076" are included in the user response A1. These question numbers function as links L1 to the respective Q&A data. When link L1 is operated, the response server 10 displays the Q&A data for the corresponding question number on the user terminal 30.

[0053] <Question and answer> As described above, the response server 10 receives user questions from the user. In the example above, the response server 10 receives user question Q1 at A15 in Figure 4. A15 is an example of a reception step that the response server 10 executes as an example of a reception unit.

[0054] Next, the response server 10 extracts one or more Q&A data points from the pre-prepared Q&A data set that are more relevant to the user question than other Q&A data points. In the example above, the response server 10 extracts one or more Q&A data points from the Q&A data set from A21 to A32 in Figure 4. A21 to A32 is an example of an extraction step executed by the response server 10 as an example of the extraction unit.

[0055] Next, the response server 10 generates a user-facing answer to the user question based on the extracted Q&A data. In the example above, the response server 10 generates a user-facing answer to user question Q1 at A33 in Figure 4. A33 is an example of a generation step executed by the response server 10 as an example of a generation unit.

[0056] The response server 10 then presents the generated user response to the user. In the example above, the response server 10 presents the sent user response to the user by sending the generated user response to the chat server 20 at A33 in Figure 4. A33 is an example of a presentation step executed by the response server 10 as an example of a presentation unit.

[0057] This configuration makes it easier to provide a mechanism that flexibly answers user questions based on a set of Q&A data, compared to a case where the generation and extraction units described above are not implemented. Furthermore, because the system can select appropriate Q&A data from the set of Q&A data according to the content of the user question and generate an answer based on that content, it can respond with high versatility even if the content of the questions is diverse. In addition, since the answer generation process can be automated, operators no longer need to create answer texts individually, significantly reducing the burden required for answer creation.

[0058] Furthermore, the response server 10 (an example of an extraction unit) extracts one or more Q&A data that are highly relevant to the user question based on one or more of the following indicators: a first indicator, which is the magnitude of the similarity between the vectors of the user question and the Q&A data; a second indicator, which is the inclusion rate of keywords contained in the user question in the Q&A data; and a third indicator, which is the semantic relevance of the Q&A data to the user question output by the AI ​​module.

[0059] In the example above, the response server 10 uses all three indicators—the first, second, and third—to extract one or more Q&A data entries that are highly relevant to the user's question. This configuration allows for the generation of appropriate answers to user questions. Furthermore, it is possible to extract Q&A data considering not only the degree of word matching but also the semantic similarity of the entire sentence. Therefore, even if there are differences in expression or variations in terminology, Q&A data that substantially matches the user's question can be selected preferentially, thereby improving the accuracy and relevance of the resulting answers.

[0060] <Second indicator> Furthermore, if a second indicator is used in the extraction step, the response server 10 performs preprocessing to remove question suffixes and function words from the user question prior to calculating the second indicator. In the example above, the response server 10 performs this preprocessing on the acquired user question Q1 at A23 in Figure 4. A23 is an example of a preprocessing step executed by the response server 10 as an example of a preprocessing unit.

[0061] In this configuration, since noisy words such as question suffixes and function words are removed in advance, keywords can be recognized more accurately compared to cases where no preprocessing is performed. Furthermore, since the number of keyword candidates can be reduced by removing unnecessary words in advance, the load on the extraction process can be reduced, and the overall response processing can be sped up.

[0062] Furthermore, in calculating the second index, the response server 10 (an example of an extraction unit) extracts a continuous section consisting of Japanese characters based on the Unicode Script attribute, generates a string of 2 to 4 characters from that continuous section as an N-gram, and uses that string as a keyword. In the above example, at A24 in Figure 4, the response server 10 uses the string generated as an N-gram as a keyword without using morphological analysis (a process that divides a Japanese sentence into "morphemes," which are the smallest meaningful units, and identifies the part of speech (noun, verb, particle, etc.) of each).

[0063] In this configuration, by using the Unicode Script attribute, Japanese characters, alphanumeric characters, symbols, etc. can be reliably distinguished, preventing the extraction of unnecessary strings as keywords and improving keyword accuracy. Furthermore, keywords can be generated with lightweight processing without morphological analysis, while appropriately extracting phrases that characterize user questions. In addition, by using N-grams of 2 to 4 characters, keywords can be flexibly extracted even when unknown words or proper nouns are included, thereby increasing the reliability of the content rate calculated as a second indicator.

[0064] <Weighting> Furthermore, the response server 10 (an example of an extraction unit) extracts one or more Q&A data points that are highly relevant to the user's question based on a score obtained by weighting and adding scores to two or more of the first, second, and third indicators. In the example above, the response server 10 extracts Q&A data based on a composite score calculated by weighting the three indicators, as shown in A31 and A32 of Figure 4.

[0065] This approach allows for the generation of more appropriate user-facing answers to user questions. Furthermore, by weighting and integrating multiple metrics, the impact of bias in specific metrics can be suppressed, and the relationship between user questions and Q&A data can be comprehensively evaluated. As a result, the stability of the extraction results for user questions is improved, and appropriate Q&A data can be selected even when there are variations in wording or phrasing in the question text.

[0066] In the example above, the weight of the first metric was 0.45, the weight of the second metric was 0.2, and the weight of the third metric was 0.35, but the weights of each metric are not limited to these. For example, the weight of the first metric could be between 0.35 and 0.5, the weight of the second metric could be between 0.15 and 0.25, and the weight of the third metric could be between 0.3 and 0.45, with the sum of the weights of each metric being 1. According to this configuration, it is possible to generate more appropriate user-facing answers to user questions.

[0067] Furthermore, this configuration allows for appropriate adjustment of the relative importance of the three indicators, enabling flexible modification of the extraction process characteristics depending on the type of user question and the usage environment. As a result, the relevance can be comprehensively evaluated without bias towards any particular indicator, and appropriate Q&A data can be selected even if there are variations in the expression or vocabulary of user questions. Consequently, the quality and stability of the final user-facing answers can be improved.

[0068] <Filter> Furthermore, the response server 10 (an example of an extraction unit) uses one or more of the first and second indicators to narrow down a predetermined number of Q&A data from the Q&A data group, and then uses a third indicator to extract one or more Q&A data that are highly relevant to the user's question from the narrowed-down predetermined number of Q&A data. In the above example, the response server 10 narrows down the Q&A data based on the first and second indicators at A25 in Figure 4, and extracts one or more Q&A data based on the third indicator at A27.

[0069] According to such an aspect, by performing preliminary filtering based on the first index and the second index, it is possible to appropriately limit the object for performing precise evaluation based on the third index. Therefore, while suppressing the processing load of the AI module, it is possible to accurately extract Q&A data having semantic relevance. As a result, the overall response processing can be speeded up, and it is possible to reduce the cost required for using the AI module and improve the accuracy and stability of the extraction result. Furthermore, even when the number of Q&A data groups increases, since the target data can be reduced by lightweight filtering processing based on the first and second indexes, it also scales well for a large-scale database. For this reason, it can be flexibly operated according to an increase in the number of users or an expansion of the Q&A data group.

[0070] Note that when performing filtering, weighting may be performed on each of the first index and the second index. Also, in the above example, the response server 10 performs filtering on a predetermined number of Q&A data, but is not limited thereto, and filtering may be performed on Q&A data in which the total value of the first index and the second index is a predetermined value or more. Also, the response server 10 may perform filtering only based on the first index, or may perform filtering only based on the second index. In any case, while suppressing the processing load of the AI module, it is possible to accurately extract Q&A data having semantic relevance.

[0071] <Identification information of Q&A data> Also, the response server 10 (an example of a presentation unit) presents to the user for the user-oriented answer by associating the identification information of one or more Q&A data used when generating the user-oriented answer. In the above example, the response server 10 presents the question number (an example of identification information) of the Q&A data used when generating the answer to the user in the user-oriented answer A1 shown in FIG. 7.

[0072] This approach makes it easier to verify the appropriateness of the Q&A data used to generate user-facing answers compared to not providing users with identifying information about the Q&A data. Furthermore, clearly indicating the Q&A data that forms the basis of the answer enhances the transparency of the answer content. This allows users to easily judge the reliability of the answer, and even if an incorrect answer is provided, the Q&A data causing the error can be quickly identified and corrected. As a result, the overall answer quality of the system can be continuously improved.

[0073] Furthermore, the response server 10 (an example of a presentation unit) presents the user with identification information for the Q&A data as a link to the relevant Q&A data in a list of Q&A data. In the example above, the response server 10 presents the user with a link L1 to each Q&A data in the user response A1 shown in Figure 7.

[0074] This approach makes it easier for users to access the Q&A data used to generate their answers compared to not providing links. Furthermore, users can ask additional questions after reviewing the Q&A data, improving the efficiency of the interaction. Additionally, operators can easily understand which Q&A data is frequently accessed, enabling efficient improvement and maintenance of the Q&A database. As a result, the overall quality of answers within the system can be continuously improved.

[0075] <Variation example: Forced extraction of technical terms> The method of keyword extraction is not limited to the method described above. The response server 10 (an example of an extraction unit) may prioritize including predefined corporate domain-specific technical terms as keywords if they are included in the user query. Technical terms may include, for example, "Subcontracting Act," "Parent Company," "Approval Process," "Business Discussion," "NDA," or "Confidentiality." The response server 10 stores these technical terms in advance and determines whether or not they are included as keywords in the user query.

[0076] This approach allows for more reliable recognition of technical terms. Furthermore, this approach enables the intentional recognition of compound words and technical terms that would be split in general N-gram extraction, thereby improving the accuracy of relevance assessment for specialized questions. In addition, by prioritizing the extraction of technical terms, it is possible to appropriately address questions specific to the business context and specialized areas of a company, thus maintaining a high level of quality in answer generation.

[0077] Furthermore, the response server 10 (an example of an extraction unit) may, based on the extracted technical terms, refer to a predefined relational dictionary and dynamically add synonyms or related words corresponding to those technical terms as keywords. A relational dictionary is a dictionary that associates technical terms with synonyms and related words. Related words include, for example, "approval," "approval," and "decision-making" for "approval process," and "parent company," "subcontractor," and "Article 3 document" for "subcontracting law."

[0078] This approach allows for more reliable recognition of technical terms compared to cases where keywords are not dynamically added. Furthermore, by dynamically adding synonyms and related terms, relevance can be evaluated based on conceptually equivalent phrases, even if technical terms are not directly included in the user's question. Therefore, it becomes possible to appropriately extract Q&A data even when variations in expression or paraphrasing exist, thereby improving the accuracy of answer generation.

[0079] <Example: Weighting settings> The user may freely set the weights described above. In that case, the response server 10 displays a settings screen on the user terminal 30 for setting weights for two or more indicators. This display process is an example of a settings display step executed by the response server 10 as an example of a settings display unit.

[0080] Figure 8 shows an example of the displayed weight setting screen. In the weight setting screen G3 shown in Figure 8, the text "Please enter the weights," an input field C31, and a confirmation button B31 are displayed. Input field C31 displays fields for entering the weights of the "first indicator," "second indicator," and "third indicator." The weights are entered as percentages, and the total is displayed. When the total reaches 100%, the confirmation button B31 becomes operable.

[0081] When the confirmation button B31 is pressed, the response server 10 calculates a composite score in A31 shown in Figure 4 using the input weights. Then, the response server 10 (an example of an extraction unit) extracts one or more Q&A data points that are highly relevant to the user's question in A32 shown in Figure 4, based on the composite score obtained when the weights set on the settings screen are applied.

[0082] This configuration allows for adjustment of response trends. Furthermore, users can adjust the importance of indicators according to their usage and needs, enabling them to obtain optimal extraction results tailored to the type of question and business field. For example, it becomes possible to increase the weight of a second indicator for questions containing many technical terms, further improving the relevance and flexibility of the responses. Moreover, the ability to dynamically change weights allows for flexible adaptation to changes in the operating environment and business policies. This optimizes the balance between extraction accuracy and processing load compared to fixed weighting, improving the scalability and operability of this information processing system.

[0083] <Variation example: Number of items to filter> The predetermined number of criteria used to narrow down the Q&A data as described above may be variable. For example, the predetermined number may be determined according to the total number of Q&A data included in the Q&A data group. Specifically, the predetermined number may be increased as the total number of Q&A data included in the Q&A data group increases. This is because the larger the number of data included in the Q&A data group, the higher the probability that it will contain a large amount of data highly relevant to user questions. By increasing the predetermined number and acquiring a wider range of candidate data, it becomes possible to appropriately secure the target for precise evaluation using the third indicator, and it becomes easier to select highly relevant Q&A data even for specialized or diverse questions.

[0084] Furthermore, the predetermined number may be determined according to the field of questions and answers represented by the Q&A data. This is because the diversity of question content and the degree of variation in terminology differ depending on the field to which the Q&A data belongs. For example, in fields such as legal affairs and human resources, where there are many specialized terms and a wide range of question expressions, it is necessary to obtain more candidate data in order to accurately evaluate the correspondence between user questions and Q&A data. On the other hand, in fields that use simple procedural explanations or limited terminology, a small number of candidate data may be sufficient to determine the relevance.

[0085] This approach allows for the optimization of the amount of candidate data according to the characteristics of each field, thereby maintaining an appropriate balance between extraction accuracy and processing load. In particular, it reduces the likelihood of missing highly relevant Q&A data even in specialized fields with significant variations in terminology, improving the suitability of answer generation. Furthermore, by adjusting the predetermined number of data points according to the field, processing efficiency can be increased while avoiding excessive data acquisition, enabling the provision of high-quality answers without reducing the overall response speed.

[0086] Furthermore, the predetermined number may be determined according to the number of keywords included in the user question. This is because the more keywords included in a user question, the more likely it is that the question content is complex, involving multiple perspectives and elements, and the more widely relevant Q&A data will be dispersed. In questions with many keywords, there is a risk that appropriate Q&A data may be missed when initial narrowing is performed using only one or two indicators. For this reason, when there are many keywords, it is reasonable to secure a larger predetermined number of candidate Q&A data.

[0087] This configuration allows for the acquisition of a wide range of relevant Q&A data, even when user questions contain multiple elements. This increases the likelihood of selecting appropriate Q&A data for precise evaluation using a third indicator. Consequently, it becomes possible to generate highly relevant answers while maintaining extraction accuracy, even for complex or multifaceted questions. Furthermore, by automatically adjusting a predetermined number of items according to the number of keywords, the balance between processing load and answer quality can be appropriately optimized.

[0088] Furthermore, the predetermined number may be determined according to the types of keywords included in the user question. This is because the distribution of potentially relevant Q&A data differs greatly depending on whether the keywords included in the user question are technical terms or general terms, or conceptually broad or specific terms. For example, if technical terms or industry-specific terms are included, Q&A data related to those terms tends to be concentrated in specific fields, making it possible to extract appropriate data with a relatively small number of candidates. On the other hand, if general terms or terms representing broad concepts are included, the range of relevant Q&A data becomes wider, so it is necessary to secure a larger predetermined number of candidates to prevent missing any.

[0089] This configuration allows for securing a suitable range of candidate Q&A data depending on the nature of the keywords included in the user's question, thereby improving the accuracy of extraction in the initial filtering stage. In particular, for questions containing general or abstract terms, increasing the number of candidates prevents missing highly relevant Q&A data, while for questions containing technical terms, reducing unnecessary candidates suppresses the processing load. As a result, it is possible to achieve both the efficiency of the extraction process and the suitability of the answer generation.

[0090] As described above, by making the predetermined number of filters variable, the balance between processing load and answer accuracy can be dynamically optimized compared to when the predetermined number is fixed. Furthermore, with this configuration, the number of candidates can be flexibly changed according to the content of the user question and the state of the Q&A data group, so stable extraction accuracy can be maintained for various question formats. In addition, by increasing or decreasing the number of candidates as needed, it is possible to avoid evaluating unnecessarily large amounts of data while also suppressing the omission of highly relevant Q&A data. Therefore, this information processing system is easily adaptable to the expansion of the Q&A data group and changes in operational conditions, and has high scalability and flexibility even in long-term operation.

[0091] <Variation example: Skip answer generation> The generation of user-facing answers may be skipped. For example, if the response server 10 (an example of a presentation unit) determines that the degree of relevance between the user question and a pre-prepared pre-question included in the Q&A data set is above a certain level, it may present a pre-prepared pre-answer to the user as a user-facing answer corresponding to that pre-question. The degree of relevance may be determined after extracting the Q&A data in A27 or A32 as shown in Figure 4, after narrowing down the Q&A data in A25, or immediately after receiving the user question in A15.

[0092] This is because, when the degree of matching between a user question and a pre-prepared question included in the Q&A data set is sufficiently high, the user question is likely to correspond to a pre-anticipated, standardized content, and the user's request can be adequately met with only the pre-prepared answers without the need for generation processing. In particular, generation processing places a high computational burden on the AI ​​module and does not necessarily need to be performed for all questions. Therefore, in cases where the degree of matching is high, there are reasonable situations where omitting the answer generation process does not cause practical problems.

[0093] In this configuration, since the AI ​​module does not perform the response generation process when it is not needed, the processing load is reduced and the response speed is improved compared to when the generation of user-facing responses is not skipped. Furthermore, by directly providing pre-answered responses based on existing Q&A data, consistency in response content can be ensured, and inconsistencies and errors in generation that occur during the generation process can be prevented. In addition, since fast and stable answers can be provided for routine questions, the overall efficiency and reliability of the system can be improved.

[0094] <Example of change: indicator> In the example above, the response server 10 extracted one or more Q&A data using all three indicators: the first, second, and third indicators. However, it is not limited to this. The response server 10 may also extract one or more Q&A data using only two or one of the three indicators. In either case, a mechanism for flexibly answering user questions based on the Q&A data set can be easily provided.

[0095] <Example of change: composite score> In the example above, the response server 10 calculated a composite score using all three indicators: the first, second, and third indicators. However, it is not limited to this. The response server 10 may also calculate a composite score using a combination of two indicators: the first and second indicators, the first and third indicators, or the second and third indicators. In this case, each indicator may be weighted. In any case, the influence of bias in a particular indicator can be suppressed, and the relationship between user questions and Q&A data can be evaluated comprehensively.

[0096] <Example of variation: Variations in configuration> The configuration shown in Figure 1, etc. (overall configuration, hardware configuration, and functional configuration, etc.) is just an example, and other configurations can be used as long as they do not cause inconvenience in implementation. For example, a client terminal such as the user terminal 30 may perform some of the processing that a server device such as the response server 10 performs, or a server device may perform some of the processing that a client terminal performs. Also, processing that was performed in one activity may be performed in two or more activities, or processing that was performed in two or more activities may be performed in one activity. In short, as long as the necessary information processing is performed throughout the question answering system 1, the devices that perform that information processing are not limited.

[0097] The output destination for information or data (hereinafter referred to as "information, etc.") may be other devices, displays, storage units (including built-in and external storage units), email addresses, or accounts in other systems. Acquisition of information, etc. includes not only acquiring information, etc. transmitted from other devices, but also acquiring information, etc. generated by the device itself or information, etc. stored by the device itself. The database may be stored by an information processing device such as the response server 10, or it may be stored in an external device. Furthermore, the information stored in the database may be used in a manner that temporarily stores it in memory.

[0098] The embodiments described above included an information processing system (question answering system 1) comprising a server device equipped with a processor (response server 10, etc.) and a terminal (user terminal 30) that can access the server device, but are not limited to this. The embodiments described above may also be, for example, an information processing method. The information processing method comprises steps executed by one or more processors in the information processing system. Furthermore, the embodiments described above may also be a program. The program causes a computer to execute the same steps as those executed by the information processing system.

[0099] <Note> Furthermore, they may be provided in the following embodiments.

[0100] (1) An information processing system comprising one or more processors, wherein the processors: in a reception step, receive a user question from a user; in an extraction step, extract one or more Q&A data that is more relevant to the user question than other Q&A data from a pre-prepared group of Q&A data; in a generation step, generate an answer to the user question as a user-facing answer based on the extracted one or more Q&A data; and in a presentation step, present the generated user-facing answer to the user.

[0101] This configuration makes it easy to provide a mechanism that can flexibly answer user questions.

[0102] (2) An information processing system as described in (1) above, wherein the processor, in the extraction step, extracts one or more Q&A data that are highly relevant to the user question based on one or more of the following indicators: a first indicator, which is the magnitude of the similarity between the vectors of the user question and the Q&A data; a second indicator, which is the inclusion rate of keywords contained in the user question in the Q&A data; and a third indicator, which is the semantic relevance of the Q&A data to the user question output by the AI ​​module.

[0103] In this configuration, it is possible to generate appropriate answers to user questions.

[0104] (3) An information processing system as described in (2) above, wherein the processor, in the preprocessing step, if the second indicator is used in the extraction step, performs preprocessing to remove question suffixes and function words from the user question prior to calculating the second indicator.

[0105] This approach allows for more accurate recognition of keywords.

[0106] (4) An information processing system as described in (3) above, wherein the processor, in the extraction step, in calculating the second index, extracts a continuous interval consisting of Japanese characters based on the Unicode Script attribute, generates a string of 2 to 4 characters from the continuous interval as an N-gram, and uses the string as the keyword.

[0107] This configuration prevents false positives caused by the inclusion of non-Japanese characters and improves keyword accuracy.

[0108] (5) An information processing system as described in (4) above, wherein the processor, in the extraction step, if a predefined corporate domain-specific technical term is included in the user question, preferentially includes the technical term in the keyword.

[0109] This approach allows for more reliable recognition of technical terms.

[0110] (6) An information processing system as described in (5) above, wherein the processor, in the extraction step, refers to a predefined relational dictionary based on the extracted technical term and dynamically adds synonyms or related words corresponding to the technical term to the keyword.

[0111] This approach allows for more reliable recognition of technical terms.

[0112] (7) An information processing system according to any one of (2) to (6) above, wherein the processor extracts one or more Q&A data that is highly relevant to the user question in the extraction step, based on scores obtained by adding weights to two or more of the first indicator, the second indicator, and the third indicator.

[0113] This configuration makes it possible to generate more appropriate answers to user questions.

[0114] (8) An information processing system as described in (7) above, wherein the weight of the first index is 0.35 or more and 0.5 or less, the weight of the second index is 0.15 or more and 0.25 or less, the weight of the third index is 0.3 or more and 0.45 or less, and the sum of the weights of each index is 1.

[0115] This configuration makes it possible to generate more appropriate answers to user questions.

[0116] (9) An information processing system as described in (7) above, wherein the processor displays a setting screen in the setting display step for setting weights for the two or more indicators, and in the extraction step for extracting one or more Q&A data that is highly relevant to the user question based on the score when the weights set on the setting screen are applied.

[0117] This approach allows for adjustment of the response trends.

[0118] (10) An information processing system according to any one of (2) to (9) above, wherein the processor, in the extraction step, narrows down a predetermined number of Q&A data from the Q&A data group using one or more of the first indicator and the second indicator, and extracts one or more Q&A data that are highly relevant to the user question from the narrowed predetermined number of Q&A data using the third indicator.

[0119] This configuration allows for faster response processing and improved accuracy and stability of the extraction results.

[0120] (11) An information processing system as described in (10) above, wherein the predetermined number of processors is determined according to the total number of Q&A data included in the Q&A data group, the fields of questions and answers indicated by the Q&A data, the number of keywords included in the user questions, or the types of such keywords.

[0121] This configuration allows for the dynamic optimization of the balance between processing load and response accuracy.

[0122] (12) An information processing system according to any one of (1) to (11) above, wherein in the presentation step, if the degree of matching between the user question and the pre-prepared pre-questions included in the Q&A data group is above a certain level, the processor presents to the user a pre-prepared pre-answer corresponding to the pre-question as the answer to the user.

[0123] This configuration can reduce the processing load.

[0124] (13) An information processing system according to any one of (1) to (12) above, wherein in the presentation step, the processor presents to the user the user answer by associating the user answer with the identification information of the one or more Q&A data used when generating the user answer.

[0125] This configuration makes it easy to verify the appropriateness of the Q&A data used to generate user-facing answers.

[0126] (14) An information processing system as described in (13) above, wherein the processor presents to the user, in the presentation step, the identification information of the Q&A data as a link to the Q&A data in the list of the Q&A data group.

[0127] In this configuration, the Q&A data used to generate user-facing answers can be easily verified.

[0128] (15) An information processing system according to any one of (1) to (14) above, comprising a server device equipped with the processor and a terminal that can access the server device.

[0129] According to this configuration, the information processing system can be implemented in various ways.

[0130] (16) An information processing method comprising each of the steps described in any one of (1) to (14) above, which is performed by a processor.

[0131] According to this embodiment, one embodiment of the information processing method can be provided.

[0132] (17) A program that causes a computer to perform any one of the steps described in (1) to (14) above.

[0133] According to this embodiment, one embodiment can be provided in the form of a program. Of course, this is not always the case. Furthermore, the embodiments and modifications described above may be implemented in any combination.

[0134] Finally, various embodiments of the present invention have been described, but these are presented as examples only and are not intended to limit the scope of the invention. Novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. Embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0135] 1: Question Answering System 2: Communication lines 10: Response Server 20: Chat Server 30: User terminal 101: Control Unit 201: Control Unit 301: Control Unit

Claims

1. An information processing system comprising one or more processors, wherein the processors In the registration step, we accept user questions from users. In the extraction step, from a pre-prepared set of Q&A data, each representing a question and answer, two or more Q&A data that have a higher relevance to the user question compared to other Q&A data are extracted. In the generation step, based on the two or more extracted Q&A data, the answers to the user questions are generated as user-facing answers. In the presentation step, the generated user-facing response is presented to the user. In the extraction step, two or more Q&A data sets with a high relevance to the user question are extracted based on a first indicator, which is the magnitude of the similarity between the vectors of the user question and the Q&A data; a second indicator, which is the inclusion rate of keywords contained in the user question in the Q&A data; and a third indicator, which is the semantic relevance of the Q&A data to the user question output by the AI ​​module. Information processing system.

2. In the information processing system according to claim 1, the processor is In the preprocessing step, if the second indicator is used in the extraction step, preprocessing is performed to remove question suffixes and function words from the user question prior to calculating the second indicator. In the extraction step, in calculating the second index, a continuous interval consisting of Japanese characters is extracted based on the Unicode's Script attribute, a string of 2 to 4 characters is generated from the continuous interval as an N-gram, and this string is used as the keyword. Information processing system.

3. In the information processing system according to claim 1, the processor is In the extraction step, two or more Q&A data sets highly relevant to the user question are extracted based on scores obtained by weighting and adding scores to two or more of the first, second, and third indicators. Information processing system.

4. In the information processing system described in claim 3, The weight of the first index is 0.35 or more and 0.5 or less, the weight of the second index is 0.15 or more and 0.25 or less, the weight of the third index is 0.3 or more and 0.45 or less, and the sum of the weights of each index is 1. Information processing system.

5. In the information processing system according to claim 1, the processor is In the extraction step, one or more of the first and second indicators are used to narrow down a predetermined number of Q&A data from the Q&A data set, and from the narrowed predetermined number of Q&A data, two or more that are highly relevant to the user question are selected using the third indicator. Extract Q&A data. Information processing system.

6. In the information processing system according to claim 5, the processor is The predetermined number is determined according to the total number of Q&A data included in the Q&A data group. Information processing system.

7. In the information processing system according to claim 5, the processor is The predetermined number is determined according to the fields of questions and answers indicated by the Q&A data included in the Q&A data group. Information processing system.

8. In the information processing system according to claim 5, the processor is The predetermined number is determined according to the number of keywords included in the user question. Information processing system.

9. In the information processing system according to claim 5, the processor is The predetermined number is determined according to the type of keyword included in the user question. Information processing system.

10. In the information processing system according to claim 1, the processor is In the presentation step, if the degree of matching between the user question and the pre-prepared pre-questions included in the Q&A data set is above a certain level, the pre-prepared pre-answers corresponding to the pre-questions are presented to the user as the user-specific answer. Information processing system.

11. In the information processing system according to claim 1, the processor is In the presentation step, the user's response is presented to the user, along with the identification information of the two or more Q&A data used to generate the user's response. Information processing system.

12. In the information processing system according to claim 11, the processor is In the presentation step, the identification information of the Q&A data is presented to the user as a link to the Q&A data in the list of the Q&A data group. Information processing system.

13. In the information processing system according to any one of claims 1 to 12, A server device equipped with the aforementioned processor, An information processing system comprising a terminal capable of accessing the aforementioned server device.

14. Information processing method, The processor comprises each step according to any one of claims 1 to 12, Information processing methods.

15. It is a program, Cause the computer to perform each of the steps described in any one of claims 1 to 12. program.

Images