Information processing device, program, information processing system, and information processing method

Abstract

To provide an information processor capable of outputting appropriate knowledge from log information on messages.SOLUTION: An information processor includes means of: extracting log information on messages written by users for communication; performing first classification for classifying messages included in the log information into at least question messages and answer messages and second classification for classifying fields for each message, using predetermined conditions to perform the first classification and the second classification on messages; assigning the answer message to the question message on the basis of a classification result; determining information on accuracy of the answer message assigned to the question message on the basis of a field of the answer message and a field of a user who has written the answer; and outputting the question message and the answer message as a pair of messages with the information on the accuracy of the answer message.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a program, an information processing system, and an information processing method.

Background Art

[0002] A chat system in which a plurality of users exchange information and consultations by messages has been conventionally known. In the chat system, messages exchanged between a plurality of users are stored as a chat log.

[0003] Patent Document 1 discloses a technique for creating a knowledge database in a chatbot system that generates an answer to a question text using a knowledge database that stores the question text and its answer in advance when receiving the question text from a user.

Summary of the Invention

Problems to be Solved by the Invention

[0004] At present, questions such as information exchange and consultations, and answers thereto, have to be carried out by individuals with knowledge. In addition, the chat log stored in the conventional chat system stores messages of questions and answers exchanged by a plurality of users. Therefore, it is considered that the chat log is manually compiled and utilized as a knowledge database. However, there is a problem that the content of the answer message included in the chat log may not be appropriate.

[0005] Note that Patent Document 1 does not solve the above problems.

[0006] An object of one embodiment of the present invention is to provide an information processing apparatus that can output appropriate knowledge from log information of messages.

Means for Solving the Problems

[0007] To achieve the above objectives, one embodiment of the present invention includes: an extraction means for extracting log information of messages written by multiple users for communication; a classification means for performing a first classification and a second classification of the messages using predetermined conditions, classifying the messages contained in the log information into at least question messages and answer messages, and a second classification for classifying the field of each message; an assignment means for assigning the answer messages to the question messages based on the classification results; a determination means for determining the likelihood of the answer messages assigned to the question messages based on the field of the answer messages and the field of the user who wrote the answer; and an output means for outputting the question messages and the answer messages as a pair of messages along with the likelihood of the answer messages. An assignment result update means updates the pair of messages according to an operation received from an administrator terminal displaying information on the certainty of the response message and the pair of messages, It is an information processing device. [Effects of the Invention]

[0008] According to one embodiment of the present invention, appropriate knowledge can be output from message log information. [Brief explanation of the drawing]

[0009] [Figure 1] This is a diagram illustrating an example of an information processing system according to this embodiment. [Figure 2] This is a hardware configuration diagram of an example of a computer according to this embodiment. [Figure 3] This is a functional configuration diagram of an example of an information processing system according to this embodiment. [Figure 4] This is a flowchart showing an example of the processing procedure of the information processing device according to this embodiment. [Figure 5] This is an explanatory diagram of an example chat log. [Figure 6] This is an explanatory diagram of an example of a message included in a chat log. [Figure 7] This is a flowchart illustrating an example of the message classification process. [Figure 8]It is an explanatory diagram of an example of the process of message classification. [Figure 9] It is an explanatory diagram of an example of a message after message classification. [Figure 10] It is a flowchart of an example of the processing procedure of message processing. [Figure 11] It is an explanatory diagram of an example of a message after personal information removal. [Figure 12] It is an explanatory diagram of an example of a message after redundant text removal. [Figure 13] It is a flowchart of an example of the processing procedure of message assignment. [Figure 14] It is a flowchart of an example of the processing procedure of step S62. [Figure 15] It is an explanatory diagram of an example of a semantic space. [Figure 16] It is a flowchart of an example of the processing procedure of step S62. [Figure 17] It is an explanatory diagram of an example of user information. [Figure 18] It is an explanatory diagram of an example of user information. [Figure 19] It is a configuration diagram of an example of the information processing system according to the present embodiment. [Figure 20] It is a functional configuration diagram of an example of the information processing system according to the present embodiment.

Mode for Carrying Out the Invention

[0010] <00,00087>Hereinafter, each embodiment of the present invention will be described with reference to the accompanying drawings.

[0011] [First Embodiment]<00,00091>[System Configuration] FIG. 1 is a configuration diagram of an example of the information processing system according to the present embodiment. The information processing system 1 according to the present embodiment is configured such that an information processing apparatus 10, an administrator terminal 12, and a user terminal 14 are communicably connected via a network 18 such as the Internet or a LAN (Local Area Network).

[0012] The user terminal 14 is a computer operated by a user who uses the chat function. The information processing device 10 provides the chat function to a plurality of users who operate the user terminal 14. The information processing device 10 stores a chat log (although it is referred to as a chat log for convenience in this embodiment, it is information that stores the exchange of messages such as questions and answers, and is not limited to the name, and may be referred to as history information, dialogue information, etc.).) of a plurality of users who operate the user terminal 14. The chat log is an example of log information of messages exchanged by a plurality of users for communication.

[0013] The administrator terminal 12 is a computer operated by an administrator. The information processing device 10 provides a function described later for outputting appropriate knowledge from the chat log to the administrator who operates the administrator terminal 12.

[0014] When the information processing device 10 receives a message specifying a chat group from the user terminal 14, it provides the chat function to the user terminals 14 of a plurality of users by distributing the message to the plurality of user terminals 14 participating in the chat group. The information processing device 10 stores the messages exchanged in a certain chat group as a chat log.

[0015] The information processing device 10 transmits and receives data to and from the administrator terminal 12, executes information processing according to the operation of the administrator received by the administrator terminal 12, and provides the execution result to the administrator terminal 12. The administrator terminal 12 displays the execution result provided from the information processing device 10.

[0016] Note that the configuration of the information processing system 1 shown in Figure 1 is just one example. The information processing device 10 may be implemented using a single computer or multiple computers, or it may be implemented using a cloud service. Furthermore, the information processing device 10, the administrator terminal 12, and the user terminal 14 may be, for example, output devices such as projectors, display devices with electronic whiteboard functionality, digital signage, HUD (Head Up Display) devices, industrial machinery, imaging devices, sound collection devices, medical equipment, networked home appliances, automobiles (Connected Car), notebook PCs (Personal Computers), mobile phones, smartphones, tablet terminals, game consoles, PDAs, digital cameras, wearable PCs, or desktop PCs.

[0017] <Hardware Configuration> "computer" The information processing device 10, administrator terminal 12, and user terminal 14 in Figure 1 are implemented by a computer 500 with the hardware configuration shown in Figure 2, for example. Figure 2 is a hardware configuration diagram of an example of a computer according to this embodiment.

[0018] Computer 500 is equipped with a CPU (Central Processing Unit) 501, ROM (Read Only Memory) 502, RAM (Random Access Memory) 503, HD 504, HDD (Hard Disk Drive) controller 505, display 506, external device connection I / F (Interface) 508, network I / F 509, data bus 510, keyboard 511, pointing device 512, DVD-RW (Digital Versatile Disk Rewritable) drive 514, and media I / F 516.

[0019] Of these components, the CPU 501 controls the operation of the entire computer 500 according to the program. The ROM 502 stores programs used to drive the CPU 501, such as the IPL. The RAM 503 is used as the work area for the CPU 501. The HD 504 stores various data, such as programs. The HDD controller 505 controls the reading or writing of various data to the HD 504 according to the control of the CPU 501.

[0020] The display 506 displays various information such as cursors, menus, windows, characters, or images. The external device connection interface 508 is an interface for connecting various external devices. In this case, external devices include, for example, USB (Universal Serial Bus) memory. The network interface 509 is an interface for data communication using the network 18. The data bus 510 is an address bus, data bus, etc., for electrically connecting various components such as the CPU 501.

[0021] The keyboard 511 is a type of input means equipped with multiple keys for inputting characters, numbers, and various instructions. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting processing targets, and moving the cursor. The DVD-RW drive 514 controls the reading or writing of various data to the DVD-RW 513, which is an example of a removable recording medium. Note that it is not limited to DVD-RW, but may also be DVD-R, etc. The media interface 516 controls the reading or writing (storage) of data to the recording medium 515, such as flash memory.

[0022] Note that the hardware configuration shown in Figure 2 is just one example, and it is not necessary to include all of the components shown in Figure 2, or to include components other than those shown in Figure 2.

[0023] <Functional Configuration> Figure 3 is a functional configuration diagram of an example of an information processing system according to this embodiment. In the following, we will explain the processing after the chat log of messages exchanged by multiple users operating the user terminal 14 is saved in the information processing device 10, so the illustration of the user terminal 14 is omitted.

[0024] Furthermore, the functional configuration shown in Figure 3 is just one example. For example, at least a portion of the functional configuration of the information processing device 10 in Figure 3 may be provided on the administrator terminal 12 or other information processing device, and they may be configured to perform processing in cooperation with each other. Note that the functional configuration diagram in Figure 3 omits components that are not necessary for the explanation of this embodiment.

[0025] For example, the information processing device 10 realizes the functional configuration shown in Figure 3 by executing an OS and a program. The information processing device 10 in Figure 3 includes a chat processing unit 30, a log information storage unit 32, an initial setting unit 34, an extraction unit 36, a classification unit 38, a processing unit 40, an assignment unit 42, an output unit 44, an assignment result storage unit 46, an operation receiving unit 48, an assignment result update unit 50, a determination unit 200, and a user information storage unit 202. The administrator terminal 12 also realizes the functional configuration shown in Figure 3 by executing an OS and a program. The administrator terminal 12 includes a display unit 60 and an operation receiving unit 62.

[0026] The chat processing unit 30 provides a chat function to multiple users operating the user terminal 14, allowing them to exchange messages written in text or other formats. The log information storage unit 32 saves the messages exchanged by the multiple users in text or other formats as a chat log.

[0027] The initial setup unit 34 receives the settings for the chat logs to be monitored from, for example, an administrator operating the administrator terminal 12. The settings for the chat logs to be monitored may be made, for example, by specifying a chat group. The chat logs to be monitored are assumed to include messages from both the questioner and the responder.

[0028] The extraction unit 36 ​​extracts the monitored chat logs from the log information storage unit 32. For example, the extraction unit 36 ​​extracts the chat logs in a way that identifies the message speaker and the order in which the messages were exchanged. The extraction unit 36 ​​may also extract the chat logs in a way that identifies the parent-child relationships between messages.

[0029] The classification unit 38 analyzes and classifies the messages contained in the chat logs extracted by the extraction unit 36. For example, the classification unit 38 uses predetermined conditions, as described later, i.e., a machine learning-based classification model, to classify the messages contained in the chat logs from multiple perspectives.

[0030] The classification unit 38 classifies messages into at least question messages and answer messages. The classification unit 38 also classifies messages into fields related to the content of those messages. Details of the message classification by the classification unit 38 will be described later.

[0031] The processing unit 40 performs processing on the messages contained in the extracted chat logs, including removing personal information, removing unnecessary text, and shaping the text. For example, the processing unit 40 uses various machine learning models to perform processing on the messages contained in the extracted chat logs, including removing personal information, removing unnecessary text, and shaping the text.

[0032] The assignment unit 42 assigns answer messages to question messages based on the classification results of the classification unit 38. For example, if the chat log has been extracted in a way that allows for the identification of parent-child relationships between messages, the assignment unit 42 assigns answer messages to question messages based on the parent-child relationship information. The process of assigning answer messages to question messages may also be described as matching question messages and answer messages.

[0033] Furthermore, if the chat log extracted in a way that allows for the identification of the order of message exchanges, the assignment unit 42 may assign answer messages to question messages based on the order of message exchanges. The assignment unit 42 may also assign answer messages that are close in position in the semantic space, as described later, to question messages based on the similarity of the messages.

[0034] Furthermore, even if the chat log has been extracted in a way that allows for the identification of parent-child relationships between messages, the assignment unit 42 may prioritize assigning response messages that are closer in semantic space, as described later, to the question message based on the similarity of the messages. The assignment of response messages to question messages by the assignment unit 42 may be done based on one or more of the following: the parent-child relationship between messages, the order of the exchange, and the similarity of the messages.

[0035] The determination unit 200 determines information about the likelihood of the answer message assigned to the question message, based on the field of the answer message classified by the classification unit 38 and the field of expertise of the speaker of that message. The user information storage unit 202 stores information about the field of expertise of the user. Details of how the determination unit 200 determines the likelihood of the answer message will be described later.

[0036] The output unit 44 outputs the question message and answer message assigned by the assignment unit 42 as a pair of messages. The output unit 44 also outputs information on the likelihood of the answer message. The output unit 44 sends the information on the likelihood of the answer message and the data of the question message and answer message pair to the administrator terminal 12, causing the administrator terminal 12 to display the information on the likelihood of the answer message and the question message and answer message pair.

[0037] The display unit 60 of the administrator terminal 12 displays the question message and answer message pair along with information on the plausibility of the answer message output by the information processing device 10. Therefore, the administrator can easily check the question message and answer message pair included in the monitored chat log, as well as the information on the plausibility of the answer message.

[0038] Here, "likelihood information" refers to an indicator or degree that shows the content of the response message is more appropriate, and information with a high degree of likelihood indicates that the content is more appropriate.

[0039] For example, an administrator can determine whether a pair of question and answer messages included in a monitored chat log is appropriate, based on information about the plausibility of the answer message, and then use the pair of messages as knowledge. The Q&A (Question and Answer) format, where question and answer messages are paired, can be used for FAQs (Frequently Asked Questions), which collect frequently asked questions and their answers.

[0040] Furthermore, administrators may use the question and answer message pairs as training data for application programs such as chatbots. A chatbot is a term combining "chat" and "bot," and is an automated conversation program that utilizes artificial intelligence. The question and answer message pairs included in the monitored chat logs can be used, for example, as training data necessary for a chatbot to learn how to converse.

[0041] Furthermore, the operation reception unit 62 of the administrator terminal 12 receives requests from the administrator to update the question and answer message pairs displayed on the display unit 60. If the administrator needs to make corrections or other updates to the question and answer message pairs displayed on the display unit 60, they perform the operation via the operation reception unit 62. The operation reception unit 62 transmits the operation details received from the administrator to the information processing device 10.

[0042] The operation receiving unit 48 of the information processing device 10 receives the operation content received from the administrator terminal 12. The assignment result storage unit 46 stores the pairs of question messages and answer messages assigned by the assignment unit 42 as assignment results. The assignment result update unit 50 updates the pairs of question messages and answer messages stored in the assignment result storage unit 46 according to the operation content received from the administrator terminal 12.

[0043] In this way, the administrator can check the question and answer message pairs output by the information processing device 10 after it has classified and processed the monitored chat logs, along with information on the likelihood of the answer messages, on the administrator terminal 12, and use the message pairs with a high likelihood of the answer messages as appropriate knowledge.

[0044] <Processing> Figure 4 is a flowchart of an example of the processing procedure of the information processing device according to this embodiment. In step S10, the initial setup unit 34 of the information processing device 10 receives the settings for the chat log to be monitored from, for example, an administrator operating the administrator terminal 12.

[0045] In step S12, the extraction unit 36 ​​of the information processing device 10 extracts the chat log to be monitored from the log information storage unit 32, for example, as shown in Figure 5.

[0046] Figure 5 is an explanatory diagram of an example of a chat log. The chat log shown in Figure 5 includes the following items: date and time, speaker, message, and reply message. The "date and time" item represents the date and time the message was registered. The "speaker" item is an example of the identification information of the user who registered the message. The "message" item is an example of the text data representing the message that was made. The "reply message" item is an example of the text data representing the reply message to the made message.

[0047] The chat log in Figure 5 is an example of a chat log in which the speaker of a message, the parent-child relationship between messages, and the order in which messages were exchanged can be identified. In the chat log in Figure 5, the message that was spoken and the message that was replied to that message have a parent-child relationship. For example, the message that was spoken is the "parent," and the message that was replied to is the "child." The chat log extracted in step S12 may also be a chat log in which the parent-child relationship between messages is not identified.

[0048] In step S14, the classification unit 38 analyzes and classifies the messages included in the chat log extracted in step S12, such as the one in Figure 6, as shown in Figures 7 and 8. Figure 6 is an explanatory diagram of an example of a message included in the chat log. Figure 7 is a flowchart of an example of the message classification processing procedure. Figure 8 is an explanatory diagram of an example of the message classification process.

[0049] Figure 6 illustrates the correspondence between the messages included in the chat log in Figure 5 and the item "Message No.". In step S30 of Figure 7, the classification unit 38 extracts one unprocessed message from the messages like those in Figure 6 included in the chat log extracted in step S12.

[0050] In step S32, the classification unit 38 classifies the messages extracted in step S30 using a pre-trained classification model, such as the one shown in Figure 8. Figure 8 shows an example in which the classification unit 38 uses three types of pre-trained classification models 100A to 100C.

[0051] The machine learning-prepared classification model 100A has been trained to recognize the characteristics of question messages, answer messages, and other messages. When a message extracted in step S30 is input, the model classifies the input message into either a question message, an answer message, or another message. In Figure 8, the classification performed by the machine learning-prepared classification model 100A is labeled as Classification A.

[0052] The machine learning-prepared classification model 100B has been trained to recognize the characteristics of claim messages, urgent messages, and ordinary messages. When a message extracted in step S30 is input, the model classifies the input message as either a claim message, an urgent message, or an ordinary message. In Figure 8, the classification performed by the machine learning-prepared classification model 100B is labeled as Classification B.

[0053] Furthermore, the machine learning-prepared classification model 100C has already learned the characteristics of messages such as laws and regulations, specifications, administrative procedures, and others. When a message extracted in step S30 is input, the model classifies the fields related to the content of the input message into laws and regulations, specifications, administrative procedures, and others. In Figure 8, the classification by the machine learning-prepared classification model 100C is labeled as classification C.

[0054] In step S34, the classification unit 38 outputs the result of classifying the one message extracted in step S30. For example, as shown in Figure 8, the classification unit 38 classifies the one message extracted in step S30 from three different perspectives and outputs the classification result.

[0055] The processing in steps S30 to S36 is repeated until there are no unprocessed messages in the messages in Figure 6. The classification unit 38 classifies the messages in Figure 6 that were included in the chat log extracted in step S12 from one or more perspectives, and outputs the classification results in association with the messages, for example, as shown in Figure 9.

[0056] Figure 9 is an explanatory diagram of an example of a message after message classification. In Figure 9, the results of the classification by the classification unit 38 in Figure 8 are associated with the message. For example, for a message with message No. "1", the classification unit 38 classifies Category A as "Question", Category B as "Ordinary", and Category C as "Laws and Regulations".

[0057] Returning to step S16 in Figure 4, the processing unit 40 targets the message in Figure 6, for example, which was included in the chat log extracted in step S12, and performs processing to remove personal information, remove unnecessary text, and reshape the text.

[0058] Figure 10 is a flowchart of an example of the message processing procedure. In step S40, the processing unit 40 extracts one unprocessed message from the messages shown in Figure 6 that were included in the chat log extracted in step S12.

[0059] In step S42, the processing unit 40 removes personal information from the message extracted in step S40. The removal of personal information may be performed using a machine learning model that has been trained to recognize the characteristics of personal information. Alternatively, the removal of personal information may be performed using proper name extraction technology. For example, the processing unit 40 may list the named entities (proper names, etc.) contained in the message, select the search target and search method for reference information for each classification of named entities (place names, personal names, etc.), and search for reference information for named entities. Based on the searched reference information, it may identify and remove personal information from the listed named entities. Figure 11 is an explanatory diagram of an example of a message after personal information has been removed. The message in Figure 11 has had personal information removed from the message in Figure 9.

[0060] In step S44, the processing unit 40 removes extraneous text from the message in Figure 11 from which personal information was removed in step S42. The removal of extraneous text may be performed by identifying the characteristics of the extraneous text using a machine learning model. Alternatively, the removal of extraneous text may be performed using language analysis such as morphological analysis. For example, the processing unit 40 classifies the message text into parts such as the main text, questions, supplementary information, greetings, thanks, and others, and removes the greetings, thanks, and others as extraneous text. Figure 12 is an explanatory diagram of an example of a message after the removal of extraneous text. Note that morphological analysis is just one example, and instead of morphological analysis, or in conjunction with morphological analysis, kana-kanji conversion, proofreading support, dependency parsing, key phrase extraction, and natural language understanding may be used.

[0061] In step S46, the processing unit 40 modifies the message in Figure 12 from which the extraneous text was removed in step S44. This text modification may be performed using, for example, deep learning technology. For example, the processing unit 40 modifies the message in Figure 12 from which the extraneous text was removed in step S44 by replacing each word with a word that appears more frequently.

[0062] Returning to step S18 in Figure 4, the assignment unit 42 performs the process of assigning messages classified as "Questions" to messages classified as "Answers" based on the results of the classification unit 38 in step S14.

[0063] Figure 13 is a flowchart of an example of the message assignment process. In step S60, the assignment unit 42 extracts one message with classification A "question" from the unprocessed messages processed in step S16.

[0064] In step S62, the assignment unit 42 identifies the messages with classification A that are "answers" to be assigned to the messages with classification A that are "questions" extracted in step S60. For example, if the messages were extracted from a chat log where parent-child relationships can be identified, the assignment unit 42 can identify the messages with classification A that are "answers" and have a parent-child relationship with the "questions" extracted in step S60.

[0065] On the other hand, if the messages are extracted from chat logs where parent-child relationships cannot be identified, the assignment unit 42 may use natural language processing technology that utilizes artificial intelligence to automatically associate documents containing similar content (https: / / jp.ricoh.com / technology / tech / 115_deep_alignment) in the processing of step S62.

[0066] By utilizing natural language processing technology that leverages artificial intelligence to automatically match documents containing similar content, the assignment unit 42 can identify messages classified as "answers" that contain similar content to the "question" messages extracted in step S60.

[0067] Furthermore, if the messages are extracted from a chat log that allows for the identification of the order of message exchanges, the assignment unit 42 can identify the messages with classification A that are "answers" to be assigned to the "question" messages extracted in step S60, for example, as shown in Figures 14 and 15.

[0068] Figure 14 is a flowchart of an example of the processing procedure in step S62. Figure 15 is an explanatory diagram of an example of a semantic space. In step S100, the assignment unit 42 extracts messages that are in a predetermined range (for example, the first 10 and the last 20) based on the order of the "question" messages extracted in step S60 from the messages processed in step S16.

[0069] In step S102, the assignment unit 42 removes messages whose classification A is "Question" from the messages extracted in step S100. In step S104, the assignment unit 42 places the messages whose classification A is "Answer" that were not removed in step S102 into a semantic space such as that shown in Figure 15.

[0070] The semantic space in Figure 15 contains example messages from a question document and an example message from an answer document. Messages from a question document and an answer document that are located close together in the semantic space of Figure 15 can be recognized as messages containing similar content.

[0071] In step S106, the assignment unit 42 identifies an "answer" message that is close to the position of the "question" message extracted in step S60 in the semantic space of Figure 15. If there is no "answer" message within a predetermined range from the position of the "question" message extracted in step S60 in the semantic space of Figure 15, the assignment unit 42 does not need to identify an "answer" message.

[0072] Furthermore, the assignment unit 42 may use classification B when identifying "answer" messages that are close to the location of a "question" message. Classification B represents the urgency of the message. The assignment unit 42 may use classification B as a weighting coefficient for distance in the semantic space to adjust the system so that messages with high urgency are more easily characterized.

[0073] Furthermore, the assignment unit 42 may use classification C when identifying "answer" messages that are close to the location of a "question" message. Classification C represents the domain of the message. The assignment unit 42 may use classification C as a weighting coefficient for distance in the semantic space to make it easier to characterize messages that belong to the same domain.

[0074] When an "answer" message is identified in step S62, the assignment unit 42 proceeds to step S64 and assigns the "answer" message identified in step S62 to the "question" message extracted in step S60. The assignment unit 42 can assign "answer" messages to "question" messages by repeating the processes in steps S60 to S66 until there are no unprocessed "question" messages in the messages shown in Figure 12.

[0075] Figures 13 to 15 illustrate the process of identifying "answer" messages containing similar content to the "question" messages extracted in step S60 in step S62. However, as shown in Figure 16, it is also possible to identify "question" messages that are similar to the "question" messages (similar question messages).

[0076] Figure 16 is a flowchart of an example of the processing procedure in step S62. In step S120, the assignment unit 42 extracts messages from the processed messages that are in a predetermined range (for example, the first 10 and the last 20) based on the order of the "question" messages extracted in step S60.

[0077] In step S122, the assignment unit 42 places the messages extracted in step S120 into a semantic space as shown in Figure 15. In step S124, the assignment unit 42 identifies the "answer" message located near the "question" message extracted in step S60 within the semantic space of Figure 15.

[0078] In step S126, the assignment unit 42 can identify a "question" message located close to the "question" message extracted in step S60 in the semantic space of Figure 15 as a similar question message.

[0079] Thus, according to the process in step S62 shown in Figure 16, pairs of messages, each with a question message and an answer message assigned to it, can be presented to the administrator together with similar question messages.

[0080] Returning to step S19 in Figure 4, the determination unit 200 determines, for each pair of messages assigned in step S18, the likelihood information of the "answer" message assigned to the "question" message contained in that pair of messages.

[0081] For example, the decision unit 200 can determine information about the likelihood of the "answer" included in a pair of messages by comparing the field of classification C of the "answer" message included in the pair of messages with the field in which the speaker of that "answer" message has expertise.

[0082] Information on the speaker's area of ​​expertise can be obtained, for example, from the user information stored in the user information storage unit 202, as shown in Figure 17 or Figure 18. Figures 17 and 18 are explanatory diagrams illustrating an example of user information. The user information in Figures 17 and 18 may also be obtained from a database that records user skills, such as an employee database.

[0083] The user information in Figure 17 stores information about the user's area of ​​expertise. For example, the user information in Figure 17 shows that user "Tanaka" has expertise in the field of "laws and regulations, specifications."

[0084] Therefore, in the case of user information as shown in Figure 17, if the field of classification C of the "answer" message included in the paired message is remembered as a field related to the speaker's expertise, then the reliability of that "answer" message can be judged as high. Conversely, if the field of classification C of the "answer" message included in the paired message is not remembered as a field related to the speaker's expertise, then the reliability of that "answer" message can be judged as low.

[0085] The user information in Figure 18 stores the user's expertise numerically for each field. For example, the user information in Figure 18 shows that user "Sato" has high expertise in the field of "Clerical Processing" and lower expertise in the field of "Specifications" compared to "Clerical Processing".

[0086] Therefore, in the case of user information as shown in Figure 18, if the field of classification C of the "answer" message included in the paired message is remembered as a field in which the speaker has high expertise, then the reliability of that "answer" message can be judged to be high. Conversely, if the field of classification C of the "answer" message included in the paired message is not remembered as a field in which the speaker has low expertise, then the reliability of that "answer" message can be judged to be low.

[0087] Messages with a low probability of being a "response" can be automatically deleted, or they can be required to be reviewed by an administrator.

[0088] Returning to step S20 in Figure 4, the output unit 44 outputs the "question" message and the "answer" message assigned by the assignment unit 42 as a pair of messages. The output unit 44 also outputs information about the likelihood of the "answer" message included in the pair of messages.

[0089] According to the information processing system 1 of this embodiment, appropriate knowledge can be output from the chat log of a chat service in which questioners can easily register question messages and respondents can respond to questions with answer messages.

[0090] Furthermore, the information processing system 1 according to this embodiment can reduce the work required by administrators to utilize chat logs as knowledge. In addition, the information processing system 1 according to this embodiment can reduce the message processing work required to disclose pairs of question and answer messages exchanged in chat as knowledge.

[0091] [Other embodiments] Figure 1 illustrates an example where the information processing device 10 provides the chat function, but a configuration with a chat service device 16 providing the chat function is also possible, as shown in the information processing system 1A in Figure 19.

[0092] Figure 19 is a configuration diagram of an example of an information processing system according to this embodiment. The chat service device 16 in Figure 19 provides chat functionality to multiple users operating the user terminal 14 and stores chat logs. The chat service device 16 also provides the chat logs to the information processing device 10. The chat service device 16 may be implemented using a single computer or multiple computers, or it may be implemented using a cloud service.

[0093] Furthermore, the information processing device 10 uses the chat log provided by the chat service device 16 to perform the same processing as in the first embodiment.

[0094] Figure 20 is a functional configuration diagram of an example of an information processing system according to this embodiment. The functional configuration of Figure 20 is the same as that of Figure 3, with some exceptions, so explanations of the same parts will be omitted as appropriate.

[0095] For example, the chat service device 16 realizes the functional configuration of Figure 20 by executing an OS and a program. The chat service device 16 in Figure 20 has a chat processing unit 80, a log information storage unit 82, and a log information provision unit 84. Similarly, the information processing device 10 realizes the functional configuration of Figure 20 by executing an OS and a program. The information processing device 10 in Figure 20 has a log information storage unit 32, an initial setting unit 34, an extraction unit 36, a classification unit 38, a processing unit 40, an assignment unit 42, an output unit 44, an assignment result storage unit 46, an operation receiving unit 48, an assignment result update unit 50, a log information acquisition unit 90, a determination unit 200, and a user information storage unit 202.

[0096] The chat processing unit 80 provides a chat function to multiple users operating the user terminal 14, allowing them to exchange messages written in text or other formats. The log information storage unit 82 saves the messages exchanged by the multiple users in text or other formats as a chat log. The log information provision unit 84 provides the chat log stored by the log information storage unit 82 to the information processing device 10.

[0097] The log information acquisition unit 90 of the information processing device 10 acquires chat logs from the chat service device 16 and stores them in the log information storage unit 32. The other configurations are the same as the functional configuration in Figure 3, so their explanation is omitted.

[0098] In summary, according to this embodiment, appropriate knowledge can be output more efficiently from chat logs.

[0099] Each of the functions of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules designed to execute each of the functions described above.

[0100] The apparatus described in the examples represents only one of several computing environments for carrying out the embodiments disclosed herein. The present invention is not limited by these embodiments, and the components in these embodiments include those readily conceivable to those skilled in the art, those substantially identical, and those within the scope of so-called equivalents. Furthermore, various omissions, substitutions, modifications, and combinations of components can be made without departing from the spirit of these embodiments.

[0101] Examples of the present invention are as follows: <1> An extraction method for extracting log information of messages written by multiple users for communication purposes, A classification means that performs a first classification of the messages contained in the log information using predetermined conditions, classifying the messages into at least question messages and answer messages, and a second classification of the fields of each message. Assignment means for assigning the answer message to the question message based on the classification result, A determination means for determining information on the likelihood of the answer message assigned to the question message, based on the field of the answer message and the field of the user who provided the answer. An output means that outputs the question message and the answer message as a pair of messages, along with information on the likelihood of the answer message. An information processing device having <2> A user information storage unit that stores information in the field in which the user has expertise, It further possesses, The determination means determines information on the likelihood of the answer message assigned to the question message, based on the field in which the user has expertise, which is stored in the user information storage unit, and the field of the answer message. The aforementioned <1> The information processing device described. <3> A user information storage unit that stores information on the likelihood of the user in each of the aforementioned fields, It further possesses, The determination means determines the likelihood information of the answer message assigned to the question message, based on the likelihood information of the user for each of the fields stored in the user information storage unit and the field of the answer message. The aforementioned <1> The information processing device described. <4> The output means outputs information on the likelihood of the response message and the pair of messages as training data for the application program. The aforementioned <1> ~ <3> An information processing device as described in any one of the following items. <5> Assignment result update means updates the pair of messages according to an operation received from an administrator terminal displaying information on the certainty of the response message and the pair of messages. The aforementioned further having <1> ~ <4> An information processing device as described in any one of the following items. <6> The aforementioned predetermined condition is a trained classification model. The aforementioned <1> ~ <4> An information processing device as described in any one of the following items. <7> In an information processing device, A procedure for extracting log information of messages written by multiple users for communication purposes. A classification procedure that uses predetermined conditions to perform a first classification on the messages included in the log information, classifying them into at least question messages and answer messages, and a second classification on the field of each message. An assignment procedure for assigning the answer message to the question message based on the classification results, A determination means for determining information on the likelihood of the answer message assigned to the question message, based on the field of the answer message and the field of the user who provided the answer. An output procedure that outputs the question message and the answer message as a pair of messages, along with information on the likelihood of the answer message. A program to execute. <8> An information processing system in which an information processing device and an administrator terminal are connected in a manner that enables communication, The aforementioned information processing device is An extraction method for extracting log information of messages written by multiple users for communication purposes, A classification means that performs a first classification of the messages contained in the log information using predetermined conditions, classifying the messages into at least question messages and answer messages, and a second classification of the fields of each message. Assignment means for assigning the answer message to the question message based on the classification result, A determination means for determining information on the likelihood of the answer message assigned to the question message, based on the field of the answer message and the field of the user who provided the answer. An output means that outputs the question message and the answer message as a pair of messages, along with information on the likelihood of the answer message. It has, The aforementioned administrator terminal is Display means for displaying the pair of messages along with information on the likelihood of the response message. An information processing system having [a certain feature]. <9> An information processing method performed by an information processing device, An extraction step to extract log information of messages written by multiple users for communication, A classification step in which, using predetermined conditions, the messages included in the log information are classified into at least question messages and answer messages, and the messages are classified into a first classification and a second classification, which classifies each message into a field. An assignment step of assigning the answer message to the question message based on the classification results, A decision step to determine information on the likelihood of the answer message assigned to the question message, based on the field of the answer message and the field of the user who provided the answer; An output step that outputs the question message and the answer message as a pair of messages, along with information on the certainty of the answer message. An information processing method having <10> A user information storage step involves storing information in the user information storage unit that the user has expertise in. It further possesses, The aforementioned decision step determines information on the likelihood of the answer message assigned to the question message, based on the field in which the user has expertise, which is stored in the user information storage unit, and the field of the answer message. The aforementioned <9> The information processing method described. <11> A user information storage step involves storing information about the likelihood of the user in each of the aforementioned fields in a user information storage unit. It further possesses, The aforementioned decision step determines the likelihood information of the answer message assigned to the question message, based on the likelihood information of the user for each of the fields stored in the user information storage unit and the field of the answer message. The aforementioned <9> The information processing method described. <12> The output step outputs information on the likelihood of the response message and the paired messages as training data for the application program. The aforementioned <9> ~ <11> The information processing method described in any one of the following items. <13> Assignment result update step, which updates the pair of messages according to an operation received from an administrator terminal displaying information on the certainty of the response message and the pair of messages, The aforementioned further having <9> ~ <12> The information processing method described in any one of the following items. <14> The aforementioned predetermined condition is a trained classification model. The information processing method according to any one of claims 9 to 13. [Explanation of Symbols]

[0102] 1. Information Processing System 10 Information Processing Devices 12 Administrator terminal 14 User terminals 18 Network 36 Extraction part 38 Classification Department 40 Processing Department 42 Allocation Section 44 Output section 60 Display 200 Decision Section [Prior art documents] [Patent Documents]

[0103] [Patent Document 1] Japanese Patent Publication No. 2021-163167

Claims

1. An extraction method for extracting log information of messages written by multiple users for communication purposes, A classification means that performs a first classification of the messages contained in the log information using predetermined conditions, classifying the messages into at least question messages and answer messages, and a second classification of the fields of each message. Assignment means for assigning the answer message to the question message based on the classification result, A determination means for determining information on the likelihood of the answer message assigned to the question message, based on the field of the answer message and the field of the user who provided the answer. An output means that outputs the question message and the answer message as a pair of messages, along with information on the likelihood of the answer message. An assignment result update means updates the pair of messages according to an operation received from an administrator terminal displaying information on the certainty of the response message and the pair of messages, An information processing device having

2. A user information storage unit that stores information in the field in which the user has expertise, It further possesses, The determination means determines information on the likelihood of the answer message assigned to the question message, based on the field in which the user has expertise, which is stored in the user information storage unit, and the field of the answer message. The information processing apparatus according to claim 1.

3. A user information storage unit that stores information on the likelihood of the user in each of the aforementioned fields, It further possesses, The determination means determines the likelihood information of the answer message assigned to the question message, based on the likelihood information of the user for each of the fields stored in the user information storage unit and the field of the answer message. The information processing apparatus according to claim 1.

4. The output means outputs information on the likelihood of the response message and the pair of messages as training data for the application program. The information processing apparatus according to any one of claims 1 to 3.

5. The aforementioned predetermined condition is a trained classification model. The information processing apparatus according to any one of claims 1 to 3.

6. In an information processing device, A procedure for extracting log information of messages written by multiple users for communication purposes. A classification procedure that performs a first classification of the messages and a second classification of the messages using predetermined conditions, wherein the messages included in the log information are classified into at least question messages and answer messages, and the messages are classified into a field for each message. Assignment procedure for assigning the answer message to the question message based on the classification results, A determination means for determining information on the likelihood of the answer message assigned to the question message, based on the field of the answer message and the field of the user who provided the answer. An output procedure that outputs the question message and the answer message as a pair of messages, along with information on the likelihood of the answer message. An assignment result update procedure that updates the pair of messages according to an operation received from an administrator terminal displaying information on the certainty of the response message and the pair of messages, A program to execute.

7. An information processing system in which an information processing device and an administrator terminal are connected in a manner that enables communication, The aforementioned information processing device is An extraction method for extracting log information of messages written by multiple users for communication purposes, A classification means that performs a first classification of the messages contained in the log information using predetermined conditions, classifying the messages into at least question messages and answer messages, and a second classification of the fields of each message. Assignment means for assigning the answer message to the question message based on the classification result, A determination means for determining information on the likelihood of the answer message assigned to the question message, based on the field of the answer message and the field of the user who provided the answer. An output means that outputs the question message and the answer message as a pair of messages, along with information on the likelihood of the answer message. An assignment result update means updates the pair of messages according to an operation received from an administrator terminal displaying information on the certainty of the response message and the pair of messages, It has, The aforementioned administrator terminal is Display means for displaying the pair of messages along with information on the likelihood of the response message. An information processing system having

8. An information processing method performed by an information processing device, An extraction step to extract log information of messages written by multiple users for communication, A classification step in which the messages included in the log information are classified into at least question messages and answer messages, and the messages are classified into a field, using predetermined conditions, An assignment step of assigning the answer message to the question message based on the classification results, A decision step to determine information on the likelihood of the answer message assigned to the question message, based on the field of the answer message and the field of the user who provided the answer; An output step that outputs the question message and the answer message as a pair of messages, along with information on the likelihood of the answer message. An assignment result update step in which the pair of messages are updated according to an operation received from an administrator terminal displaying information on the certainty of the response message and the pair of messages, An information processing method having

9. A user information storage step involves storing information in the user information storage unit that the user has expertise in. It further possesses, The aforementioned decision step determines information on the likelihood of the answer message assigned to the question message, based on the field in which the user has expertise, which is stored in the user information storage unit, and the field of the answer message. The information processing method according to claim 8.

10. A user information storage step involves storing information about the likelihood of the user in each of the aforementioned fields in a user information storage unit. It further possesses, The aforementioned decision step determines the likelihood information of the answer message assigned to the question message, based on the likelihood information of the user for each of the fields stored in the user information storage unit and the field of the answer message. The information processing method according to claim 8.

11. The output step outputs information on the likelihood of the response message and the paired messages as training data for the application program. The information processing method according to any one of claims 8 to 10.

12. The aforementioned predetermined condition is a trained classification model. The information processing method according to any one of claims 8 to 10.

Images