Information provision system, information provision method, and program

Abstract

This enables the generation of more accurate output information for a variety of inputs. [Solution] The information provision system (100) includes a CPU (11). The CPU (11) identifies the group to which the user belongs based on the user's characteristic information. The CPU (11) determines an external knowledge database (51) corresponding to the identified group. The CPU (11) obtains the response content to the user's input from the determined external knowledge database (51). The CPU (11) inputs the obtained response content into a machine learning model (1311) to generate a response to the user.

Description

Technical Field

[0001] This invention relates to an information providing system, an information providing method, and a program.

Background Art

[0002] In an information providing system that outputs advice or answers to questions for users who execute various activities, the use of artificial intelligence (AI), which is a machine learning model, is progressing. As an example of an information providing system using this AI, Patent Document 1 discloses a technique of preparing a plurality of AIs, selecting an optimal AI based on user-related information, and generating and outputting an answer.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, even when selecting the optimal AI for the content of a question from a plurality of AIs and answering and outputting, if the input question does not correspond to that AI, there is a problem that the optimal answer required by the user cannot be obtained.

[0005] An object of this invention is to enable the generation of output information with higher accuracy for various input contents.

Means for Solving the Problems

[0006] To achieve the above object, this invention identifies the group to which the user belongs based on the user's characteristic information, determines external knowledge corresponding to the identified group, The response content to the user's input is obtained from the determined external knowledge, The acquired response content is input into a machine learning model to generate a response to the user. This is an information provision system characterized by having a control unit. [Effects of the Invention]

[0007] According to the present invention, output information can be generated with greater accuracy for a variety of input contents. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the functional configuration of the information provision system of this embodiment. [Figure 2] This is a diagram illustrating the procedure for generating the answer. [Figure 3] This flowchart shows the control procedure for the information generation process. [Figure 4] This diagram illustrates examples of how to adjust the content of your response. [Figure 5] This flowchart shows the control procedure for the cluster configuration process. [Figure 6] This is a flowchart showing the control procedure for the response evaluation process. [Figure 7] This flowchart shows the control procedure for the weighting adjustment process. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. As shown in the block diagram of Figure 1, the information provision system 100 includes a server device 10, a performance information server 30, an external knowledge group 50, and a user terminal group 70. The information provision system 100 provides content transmitted from the user, particularly answers to questions, for the purpose of resolving questions and improving abilities and performance related to learning a specific subject or a specific type of activity. Examples of specific types of activities include sports such as running and intellectual competitions such as Go and Shogi. Multiple users may each have a user terminal 71 belonging to the user terminal group 70. Alternatively, the user terminal 71 may be a shared terminal owned by a school or training facility, etc., as long as the user can be identified by signing in or the like. Here, as an example, M user terminals 71 are shown, but M is arbitrary.

[0010] The server device 10 generates answers (responses) to questions received from each user terminal 71 of the user terminal group 70 and outputs the answers to the user terminals. The performance information server 30 is a database that stores the history of performance and ability scores related to the above subjects or categories by the users of each user terminal. This performance information is used for evaluation related to the use of the external knowledge database 51 used for answers, as described later. Performance may include results obtained from regular exams, qualification exams and certification exams, competitions, and individual training. Results of regular exams may be entered into the performance information server 30 separately from the users by the person administering the regular exam. Individual results obtained from training may be obtained automatically from each user terminal group 70 or through user reporting operations.

[0011] The external knowledge group 50 stores specialized information used in the responses generated by the server device 10. In one embodiment, the external knowledge group 50 may include multiple external knowledge databases 51. Alternatively, the external knowledge may be integrated and stored in a single external database device and managed individually by documents, chapters, sections, etc.

[0012] The server device 10 includes a CPU 11 (Central Processing Unit), RAM 12 (Random Access Memory), a storage unit 13, an operation reception unit 14, a display unit 15, a communication unit 16, and the like. The CPU 11, as the control unit of this embodiment, is a processor that performs arithmetic processing and comprehensively controls the operation of the server device 10. The CPU 11 may have a single processor, or it may have multiple processors that are operated in parallel, or operated independently depending on the application. At least the CPU 11 is included in the computer of this embodiment. The RAM 12 provides the CPU 11 with a working memory space and stores temporary data. The RAM 12 may be DRAM, other RAM, or a combination of multiple types of RAM. The storage unit 13 is a non-volatile memory that stores the program 131 and various setting information. The non-volatile memory is a non-volatile memory, and the non-volatile memory may be flash memory or an HDD (Hard Disk Drive), etc. The program 131 includes a program related to the process of generating answers to questions entered by a user or the like. The configuration information includes cluster information 132 and user information 133. Cluster information 132 and user information 133 will be described later.

[0013] The operation reception unit 14 receives input operations from a user or the like and outputs a signal to the CPU 11 according to the received content. The operation reception unit 14 may include, for example, a pointing device such as a mouse and a keyboard. The pointing device may include a touch sensor. The display unit 15 displays information based on the control of the CPU 11. The display unit 15 may have, for example, a digital display screen such as a liquid crystal display (LCD). The operation reception unit 14 and the display unit 15 may be peripheral devices connected to the main body of the server device 10. The communication unit 16 controls communication with the outside world. The communication unit 16 may have, for example, a network card and is capable of sending and receiving data with electronic devices, database devices, etc. on an external network via the Internet, etc., according to a predetermined communication standard. These electronic devices and database devices include the performance information server 30, the user terminals 71 of the user terminal group 70, and the external knowledge database 51 of the external knowledge group 50. The communication standard may be limited to wired communication or may include wireless communication.

[0014] Next, the method of providing information in this embodiment, which is the generation of answers, will be described. When the server device 10 receives input such as a question from a user, it generates an answer to that question. At this time, even if the gist of the question is the same, the generated answer may differ in terms of the detail and simplicity of the content depending on the characteristics of the user (characteristic information) and the characteristics of the description of the specific question. The generation of answers is carried out according to the procedure shown in Figure 2.

[0015] User characteristics include at least the user's performance and achievements, and other ability scores related to the area of ​​the question. Furthermore, user characteristics may include some or all of the information declared by the user during user registration. Declared information may include, for example, favorite subjects, performance scores such as target scores and completion times, desired academic fields, desired employment fields, desired qualifications, or target examinations. Performance scores may include scores or the number of errors, the speed or time required to complete tasks, etc. If the information provided relates to activities involving physical movement, such as sports, biometric information such as the user's height, weight, body fat percentage, and pulse rate may also be included in user characteristics. Additionally, the history of past inputs by each user may be included in user characteristics. These user characteristics are stored and retained in the storage unit 13 as user information 133.

[0016] On the one hand, the question field and the like are specified (P2) by, for example, syntactic analysis of the input question. Also, the question content is digitized by multi-dimensional vectorization. The multi-dimensional vectorization may be performed via query transformation or the like. The query transformation used may include, for example, multi-query generation that generates multiple paraphrases of the question, Hypothetical Document Embeddings (HyDE) that generates a hypothetical answer, and the like. The target to be digitized may include not only the question content but also its specificity, the degree of development from previous related questions, and the like. That is, the multi-dimensional vector may include understanding degree information indicating how much prior knowledge of the question the user has and how deeply the user grasps the key points of the question. When the question field for which the server device 10 generates an answer is single, the specification of the question field may not be performed. Also, the user information 133 may be referred to, and a multi-dimensional vector obtained from the history information of questions input by the same user within a recent predetermined number of times or a predetermined period may be acquired. This multi-dimensional vector may be compared with the multi-dimensional vector obtained from the current input content, and the degree of similarity between the two may be determined. When the degree of similarity is high, it may be determined that the same content or highly relevant content has been repeatedly questioned. In this case, further considering the result of the above syntactic analysis and the like, it may be determined whether the user is stuck on the same content or whether relevant questions about more in-depth content have been made as the understanding progresses. Alternatively, the degree of progress of understanding may be obtained by inputting the question content multiple times into a large language model or the like and determining the degree of progress of understanding.

[0017] Multiple users are grouped into one of several clusters based on multiple characteristics included in this characteristic information. Cluster division may utilize various clustering methods using multiple indicators. These clustering methods may include hierarchical and non-hierarchical clustering. Examples of hierarchical clustering include Ward's method, while examples of non-hierarchical clustering include k-means, k-means+, and k-medoids. This clustering is performed before the question is submitted. Therefore, when the server device 10 obtains the user's characteristic information based on the user's identification information (P1), it can determine the cluster to which the user belongs according to the characteristic information (P3). If there are multiple subjects, fields, and / or progress levels for which the server device 10 can generate answers, the cluster, or group, to which the user belongs may be determined separately for each subject, field, or progress level. Progress levels may include mechanical classifications such as grade level or age. Based on the question field identified in P2 above, an appropriate cluster is selected.

[0018] The substantial content (response content) required for generating a response to the input content is obtained from the external knowledge group 50 (P4). The external knowledge databases 51 (external knowledge) belonging to the external knowledge group 50 are each information sources with guaranteed content accuracy and reliability, are appropriately maintained and managed, and their content is updated as necessary. Particularly in cutting-edge technology fields where knowledge updates and technological innovations continue, the content of the external knowledge databases 51 is updated at an appropriate frequency. Each external knowledge database 51 contains a database that holds documents including elementary or basic descriptions and a database that holds documents including more detailed and advanced descriptions. Also, even for documents at the same level, the databases can differ depending on whether the document is suitable for a user who is dealing with the question content for the first time with expertise in the subject matter or issue, or for a user who is not good at the subject matter or issue and is having trouble with the part related to the question content. The entity that maintains and manages the external knowledge database 51 may be separate from the entity that manages the server device 10. Also, the entities that maintain and manage the multiple external knowledge databases 51 belonging to the external knowledge group 50 may each be different. For example, the entity that manages the server device 10 may enter into information usage contracts with the entities that manage the respective external knowledge databases 51, and the server device 10 may be able to use the external knowledge databases 51.

[0019] The external knowledge database 51 from which the response content is obtained is selected according to the cluster to which the user belongs. The information obtained from the external knowledge database 51, that is, the scope of the text, is a scope that generally corresponds to the field of the question content. For example, chapters, sections, or paragraphs of a document may be used as units. The relationship between the cluster and the selected external knowledge database 51 is stored in the cluster information 132.

[0020] Multiple external knowledge databases 51 may be selected depending on the cluster. The descriptive content (information) of the multiple external knowledge databases 51 that have been selected and acquired may be further weighted according to the weights defined for each cluster. That is, the weighting may determine how much weight is given to the descriptive content acquired from each external knowledge database 51 when acquiring the response content (P5). The response content may be acquired by a vector search corresponding to the question content for the acquired descriptive content (P6). For example, the server device 10 may extract as the response content the portion of the acquired descriptive content whose cosine similarity to the multidimensionally vectorized question content as described above satisfies a criterion. As a condition for satisfying the criterion, the criterion value may be adjusted for each external knowledge database 51 according to the weighting described above.

[0021] The server device 10 inputs the response content obtained and extracted from external knowledge and the user's question into the machine learning model 1311, that is, uses / based on it, to generate an answer (response) (P7). The machine learning model 1311 may be a large-scale language model (LLM). In particular, the machine learning model 1311 may use transformers. The machine learning model 1311 may be a model trained for the purpose of analyzing a question and generating an answer to that question. By using search-enhanced generation (RAG) technology, in which information from external knowledge is added to a single LLM in this way, the cost of preparing multiple LLMs for each knowledge domain is reduced.

[0022] By generating answers based on the wording of the question, the answer can be individually reflected in how well the user grasps the key points of the question and what background knowledge they possess. Specifically, if the user has not narrowed down the main points of the question, or if it is a subject they are not good at, outputting a detailed answer from the start is unlikely to be understood by the user and will not improve their motivation to learn. On the other hand, if the user asks a specific question based on basic knowledge, it can be assumed that the user already understands to some extent, so a detailed answer that fills in any gaps in understanding and improves the level of understanding is preferable. The machine learning model 1311 may be trained to receive the input of the question content and generate an answer that reflects the user's level of understanding. The answer generated by the machine learning model 1311 in this way is output to the user terminal 71 from which the question was sent (P8). In addition, information on the response content is added to the user information 133 (P9).

[0023] The above weighting may be readjusted at an appropriate frequency. At a predetermined frequency, the performance and ability scores of users before and after a question in which an answer has been generated and output are obtained from the performance information server 30 as the user's status corresponding to the answer (P11). The appropriate frequency may be determined according to the period during which answers to a predetermined number of questions are generated, or the interval at which the results of a test taken by multiple users are obtained by the performance information server 30. Based on the cluster to which the user who asked the question and received the answer belongs, the weighting of each external knowledge database 51 is obtained. The validity of the answer is evaluated based on the correspondence between the weighting and the degree of change (improvement) in performance (P12). In other words, the validity of the weighting of each external knowledge database 51 is evaluated by evaluating its contribution to the degree of change. The server device 10 adjusts the weighting so that the contribution and the weighting correspond more closely (P13). The server device 10 updates the weighting settings stored in the cluster information 132 with the adjusted weighting information (P14).

[0024] The CPU 11 of the server device 10 executes the process of generating information to be provided to the user according to the control procedure shown in Figure 3. The CPU 11 obtains the user's question from the user terminal 71 (S1). The CPU 11 obtains the user's characteristic information by referring to the user information 133 (S2). Based on the user's characteristic information and, if necessary, information on the question field, the CPU 11 identifies the cluster to which the user belongs (S3; identification means). The CPU 11 selects an external knowledge database 51 corresponding to the identified cluster and obtains the description content corresponding to the question content from the selected external knowledge database 51 (S4; determination means). From each of the obtained description contents, the CPU 11 extracts descriptions that satisfy the conditions as response content according to the weighting according to the cluster (S5; acquisition means). Vector search may be used for extraction as described above. The CPU 11 inputs the extracted response content and question content into the machine learning model 1311 and generates an answer (S6; generation means). The CPU 11 outputs the generated answer to the user terminal 71 that obtained the question (S7). CPU 11 adds the information related to the response to user information 133 (S8). Then, CPU 11 terminates the information generation process.

[0025] For example, as shown in Figure 4(a), the system may not only return a general answer to the question, but also, as shown in Figure 4(b), adjust the output to be closer to basic content and more intuitive, depending on the cluster to which the questioner belongs. Here, in response to an input requesting an explanation of Ohm's law, in the initial stages, the system may encourage a basic understanding by explaining it using examples rather than mathematical formulas.

[0026] As described above, clustering settings are configured in advance before answer generation. Even if the characteristics of individual users change, the cluster to which the user belongs can be quickly identified by applying the user's characteristics to the configured clustering criteria. In other words, the cluster to which a user belongs may change as the user's abilities improve. Clustering settings are configured by the cluster configuration process shown in Figure 5. Here, the cluster configuration process is performed by the CPU 11 of the server device 10, but it may also be performed by an external device capable of acquiring user information 133, and may be executed when external knowledge is updated or added, or when a large amount of user information is updated or added.

[0027] The CPU 11 sets the external knowledge database 51 to be acquired when generating answers (S11). The CPU 11 initializes the weighting data to be set for each cluster (S12). Initialization may mean setting an equal proportion for all external knowledge databases 51. The CPU 11 acquires user information for each user to be used for clustering (S13). The user information includes parameters related to multiple characteristics as described above. The CPU 11 clusters the users using the clustering method described above based on the parameters of the multiple users (S14). Clustering may be performed by dividing into a predetermined number of clusters.

[0028] The CPU 11 sets the external knowledge database 51 to be selected for each of the above-specified number of clusters (S15). This selection may be made based on the input settings of the administrator who reviewed the clustering results. Alternatively, knowledge levels, capability levels, etc., may be set in advance for the external knowledge database 51, and the CPU 11 may determine the selection according to these levels. The CPU 11 sets the weights so that the weights of the external knowledge databases 51 other than the selected external knowledge database 51 become zero. The CPU 11 may also set equal weights for the selected external knowledge databases 51. That is, in the initial stage of clustering, there does not need to be any difference in the external knowledge databases 51 selected between clusters. Alternatively, the CPU 11 may set initial weights for each of the selected external knowledge databases 51. Then, the CPU 11 terminates the cluster setting process.

[0029] After the server device 10 receives a response from the user, it evaluates the effectiveness of the response according to the response evaluation process shown in Figure 6 at an appropriate time. As described above, the appropriate time may be determined based on the number of responses, or it may be determined in conjunction with the registration of scoring results for periodic exams, etc. The CPU 11 obtains the history of questions and answers with the user conducted within the evaluation period from the user information 133 (S21).

[0030] The CPU 11 obtains performance information immediately before the question and performance information immediately after the answer (S22). The performance information obtained includes at least the performance in the subject area corresponding to the question. Answers for which performance information immediately after the answer is not available should be excluded from evaluation. Based on the performance information obtained before and after the question answer, the CPU 11 obtains the change in performance as an evaluation value of the response result (S23). The CPU 11 stores the evaluation value of the response result in user information 133, associating it with the above question answer history (S24). Then, the CPU 11 terminates the answer evaluation process.

[0031] The weighting of the external knowledge database 51, which is initially set for each cluster, is adjusted based on the results of the response evaluation described above, by the weighting adjustment process shown in Figure 7. The CPU 11 obtains the response results for each user's question (S31). The CPU 11 obtains the weighting information for each cluster (S32).

[0032] The CPU 11 calculates the correlation between the matching result and the weight for each cluster (S33). The CPU 11 adjusts the weight amount set for each cluster to improve the matching result (S34). The CPU 11 updates and registers the adjusted weight amount in the cluster information 132 (S35). Then, the CPU 11 terminates the weight adjustment process.

[0033] As described above, the information provision system 100 of this embodiment includes a CPU 11. The CPU 11 identifies the cluster to which the user belongs based on the user's characteristic information. The CPU 11 determines an external knowledge database 51 corresponding to the identified cluster. The CPU 11 obtains the response content to the user's input from the external knowledge database 51. The CPU 11 inputs the obtained response content into a machine learning model to generate an answer for the user. Such an information provision system 100 can selectively use external knowledge that is suited to the user's characteristics when generating output text for the user. Therefore, the information provision system 100 is able to generate more accurate output information that is more tailored to the user. Furthermore, natural language processing using a machine learning model is currently preferred for generating and outputting natural-sounding response text. In addition, by obtaining appropriate knowledge content from an external source as described above, the information provision system 100 does not need to train the machine learning model with the knowledge itself. Therefore, the information provision system 100 can output an appropriate level of accurate answer to the user while reducing the unnaturalness of the text.

[0034] Furthermore, there may be multiple external knowledge databases 51 selected depending on the cluster. The CPU 11 may weight the information obtained from the multiple selected external knowledge databases 51 and extract the response content. In other words, the information provision system 100 may not only simply select an external knowledge database 51, but may also combine information obtained from multiple external knowledge databases 51 in appropriate proportions. This makes it possible to generate more accurate and appropriate answers not only for classification by the cluster to which the user belongs, but also for each of the user's questions.

[0035] Furthermore, the CPU 11 may acquire the user's status in response to the answer, i.e., changes in performance before and after the answer. The CPU 11 may adjust the weighting based on the relationship between the status and the weighting. It is difficult to accurately determine from the outset which external knowledge databases 51 should be associated with each configured cluster and in what proportion. In this way, by evaluating what impact it actually had on the user, the information provision system 100 can adjust the selection and weighting of the external knowledge databases 51 with greater accuracy.

[0036] Furthermore, the characteristic information may include the user's input history. By considering not only the user's performance, achievements, and status, but also the characteristics of the text input by each individual user, it becomes possible to generate responses that better reflect the user's level of understanding and motivation for improvement.

[0037] Furthermore, the characteristic information may include the user's ability scores in the field corresponding to the input content, such as test or competition results. By setting up clusters according to the user's abilities, the information provision system 100 can appropriately provide information at the level necessary for the user, that is, to the extent that they can understand and put into practice.

[0038] Furthermore, the user's input may be a question, and the response may be an answer based on the answer content obtained from determined external knowledge. In other words, the information provision system 100 may generate an answer to a question. By obtaining the answer content from external knowledge selected based on the user's characteristics and generating an answer at the level the user expects, the information provision system 100 can respond more appropriately to the user's requests and contribute to improving the user's abilities and motivation.

[0039] Furthermore, the CPU 11 may acquire an evaluation value representing the change from the performance information in the subject area corresponding to the user's question before the question is entered to the performance information in the subject area corresponding to the user's question after the answer to the question is generated. The CPU 11 may associate the acquired evaluation value with the user's user information. This allows the information provision system 100 to quantitatively evaluate to what extent the answer contributes to improving performance. By considering such evaluations when creating answers, the information provision system 100 can generate answers that are more suitable for the user with greater accuracy.

[0040] Furthermore, the information provision method of this embodiment includes the following steps: (1) Identifying the cluster to which the user belongs based on the user's characteristic information. (2) Determining an external knowledge database 51 corresponding to the identified cluster. (3) Obtaining a response to the user's input from the determined external knowledge database 51. (4) Generating a response to the user using the obtained response. This information provision method makes it possible to generate a response tailored to the user with greater accuracy. Therefore, this information provision method can improve the user's understanding of and desire to improve in the field related to the response. In addition, by installing and running the program 131 related to the above information provision method on a computer, it becomes possible to easily provide more accurate information without requiring a special hardware configuration.

[0041] It should be noted that the present invention is not limited to the above embodiments, and various modifications are possible. For example, in the above, text portions related to the question content were extracted from the external knowledge database 51 by vector search, but this is not the only method. Text portions related to the question content may be extracted by other methods such as keyword search or semantic search. Furthermore, text portions may be extracted by a hybrid search that uses both keyword search and vector search.

[0042] Furthermore, while the above example considered the input history of questions when generating answers, this is not necessarily required. As long as it is understood that performance and ability in the same field remain at the same level, the content of the answers to questions can be adjusted each time. Alternatively, changes in performance and ability do not need to be considered for individual questions.

[0043] Furthermore, while the weighting adjustments described above were based on objective numerical changes such as performance before and after the user's response to the question, this is not the only way to do so. The user's subjective evaluation of the response may also be reflected in the weighting adjustments. That is, the degree to which the user found the response easy to understand and the degree to which the user was satisfied with the response content may be received as numerical evaluations through user operation from the user terminal 71. Also, while the generated responses described above were limited to text data, this is not the only way to do so. The information provision system 100 may also output diagrams and charts that are necessary for explanation or that can further promote understanding. Furthermore, while the above described receiving questions from the user, the content received does not necessarily have to be a question. For example, even if content such as the user's impressions is entered, advice or other responses may be generated as response text in response to the input.

[0044] Furthermore, in the above description, the setting of a cluster according to the user, the selection of the external knowledge database 51, the processing of extracting response content from the information obtained from the external knowledge database 51, and the generation of an answer by the machine learning model 1311 were all performed by a single server device 10. However, some or all of these processes may be performed by different information processing devices. That is, the server device 10, upon receiving a question from a user, may request the execution of each processing device along with the necessary information and obtain the processing results. In addition, in the above description, a storage unit 13 consisting of an HDD, flash memory, or other non-volatile memory was used as an example of a computer-readable medium for storing the program 131 related to the generation control of the information provided by the present invention, but it is not limited to these. Other computer-readable media such as other non-volatile memory such as MRAM, or portable recording media such as CD-ROMs and DVD discs can be applied. Furthermore, a carrier wave can also be applied to the present invention as a medium for providing the program data according to the present invention via a communication line. In addition, the specific configuration, processing operation content, and procedures shown in the above embodiments can be appropriately modified without departing from the spirit of the present invention. The scope of this invention includes the scope of the invention as described in the claims and its equivalents. [Explanation of Symbols]

[0045] 11 CPUs, 51 external knowledge databases, 100 information provision systems, 1311 machine learning models, 132 cluster information, 133 user information

Claims

1. Identify the group to which the user belongs based on the user's characteristic information. Determine the external knowledge corresponding to the identified group, The response content to the user's input is obtained from the determined external knowledge, The acquired response content is input into a machine learning model to generate a response to the user. An information provision system characterized by comprising a control unit.

2. The aforementioned external knowledge varies depending on the group, The control unit weights the information obtained from the determined plurality of external knowledge and extracts the response content. The information provision system according to claim 1, characterized in that

3. The information provision system according to claim 2, characterized in that the control unit acquires the user's status in response to the response and adjusts the weights based on the relationship between the status and the weights.

4. The information provision system according to claim 1, characterized in that the characteristic information includes the history of input by the user.

5. The information provision system according to claim 1, characterized in that the characteristic information includes the user's ability score in a field corresponding to the input content.

6. The information provision system according to claim 1, characterized in that the input content by the user is a question, and the response is an answer based on the answer content obtained from the determined external knowledge.

7. The control unit, The evaluation value is obtained from the user's performance information in the field corresponding to the question before the question is entered, to the change in the user's performance information in the field corresponding to the question after the answer to the question is generated. The acquired evaluation values ​​are associated with the user's characteristic information. The information provision system according to claim 6, characterized in that

8. Identify the group to which the user belongs based on the user's characteristic information. Determine the external knowledge corresponding to the identified group, The response content to the user's input is obtained from the determined external knowledge, The acquired response content is input into a machine learning model to generate a response to the user. A method for providing information characterized by the following features.

9. Computers, A means for identifying the group to which a user belongs based on the user's characteristic information. A decision means for determining external knowledge corresponding to the identified group, Acquisition means for obtaining the response content to the user's input from the determined external knowledge, A generation means that inputs the acquired response content into a machine learning model to generate a response to the user. A program designed to function as such.

Images