Information processing apparatus, information processing system, learned model, and storage medium

Abstract

To provide an information processing apparatus, an information processing system, a learned model, and a storage medium which can assist cancellation of mismatching in recognition between parties communicating with each other.SOLUTION: The present invention is directed to an information processing apparatus having a processor. The processor executes processing for accepting transmission expression and processing for inputting the accepted transmission expression to a first learned model obtained by learning the transmission expression and meaning of the transmission expression as a data set to estimate meaning of the input transmission expression.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing system, a trained model, and a storage medium that help to eliminate a misunderstanding between parties in communication. [Background technology]

[0002] In system development, system engineers (hereafter referred to as "SE") listen to the customer's requests and design and plan the system. The contents of the system design and plan are compiled into documents related to system development (hereafter referred to as "system development documents") and communicated to programmers, etc.

[0003] The system development documents to be presented include system configuration diagrams, development schedules, use case diagrams, site maps, database design documents, test cases / test scenarios, release notes, project plans, user guides / manuals, interface specifications, etc.

[0004] While presenting system development documents, the SE explains the system development documents and details of the development project to the programmer by speech or by writing (including written documents as well as electronic text. The same applies below.) Explanations are given in person or by using online tools such as web conferencing systems.

[0005] Programmers and other personnel perform tasks such as programming according to the system development documents provided by the SE and the information communicated verbally and in writing by the SE. Therefore, in order to carry out system development smoothly and accurately, it is necessary that the information communicated by the SE is accurate and that the programmers and other personnel have an accurate understanding of it.

[0006] However, when transmitting information from person to person, a gap in perception often occurs between the parties (hereafter referred to as "perception gap"). As shown in Figure 14, this type of perception gap can occur due to a gap between the intention and expression of the communicator, or a gap between the expression of the communicator and the understanding of the receiver.

[0007] In recent years, there has been a trend toward so-called "offshore development," in which system development is outsourced to companies in countries with low labor costs (e.g., Patent Document 1), and there are increasing cases of misunderstandings due to differences in language and culture. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] JP 2008-287642 A Summary of the Invention [Problem to be solved by the invention]

[0009] However, to date, no technology has been proposed that can help resolve the misunderstanding between the parties in a communication, and the problem has yet to be solved.

[0010] The present invention has been made in consideration of the above circumstances, and the problem to be solved by the present invention is to provide an information processing device, an information processing system, a trained model, and a storage medium that can help to eliminate the misunderstanding between the parties in communication. [Means for solving the problem]

[0011] [Information processing device] The information processing device of the present invention is an information processing device equipped with a processor, which executes a process of accepting a communication expression and a process of estimating the meaning of the input communication expression by inputting the accepted communication expression into a first trained model in which the communication expression and the meaning of the communication expression have been trained as a dataset.

[0012] The information processing device of the present invention can also be configured so that the processor executes a process of generating questions that can be deduced from the meaning of the input communication expression and / or questions that cannot be deduced from the meaning of the communication expression, and correct answers to the questions, by inputting the meaning of the communication expression estimated by the first trained model into a second trained model in which the meaning of the communication expression, questions that can be deduced from the meaning of the communication expression and / or questions that cannot be deduced from the meaning of the communication expression, and correct answers to the questions, as a dataset.

[0013] The information processing device of the present invention may also be configured so that the processor executes a process of calculating a score based on the answer to the generated question and the correct answer generated by the second trained model.

[0014] The information processing device of the present invention can also be configured so that the processor calculates a score by inputting answers to the generated questions into a third trained model that has been trained with question patterns, one or more answer patterns for each question pattern, and similarities as a data set.

[0015] The information processing device of the present invention can also be configured to execute a process of estimating the cause of the recognition error by inputting the calculated score into a fourth trained model in which the score and the cause of the recognition error have been trained as a data set.

[0016] In the information processing device of the present invention, the processor can also be configured to execute a process of generating a modified communication expression that reflects the intention of the communicator by inputting the communication expression into a fifth trained model that has been trained by using the meaning of the communication expression and the communication expression as a data set.

[0017] [Information Processing System] The information processing system of the present invention comprises at least a first terminal device and an information processing device connected to the first terminal device via a network line, the information processing device comprising a processor, and the processor is configured to execute a process of accepting a communication expression transmitted from the first terminal device and a process of estimating the meaning of the input communication expression by inputting the accepted communication expression into a first trained model in which the communication expression and the meaning of the communication expression have been trained as a dataset.

[0018] In the information processing system of the present invention, the processor can be configured to execute a process of generating questions that can be deduced from the meaning of the input communication expression and / or questions that cannot be deduced from the meaning of the communication expression, and correct answers to the questions, by inputting the meaning of the communication expression estimated by the first trained model into a second trained model in which the meaning of the communication expression, questions that can be deduced from the meaning of the communication expression and / or questions that cannot be deduced from the meaning of the communication expression, and correct answers to the questions, as a dataset.

[0019] The information processing system of the present invention may also be configured to include a second terminal device connected to the information processing device via a network line, and the processor may be configured to execute a process of accepting answers to questions sent from the first terminal device and / or the second terminal device, and a process of calculating a score based on the accepted answers to the questions and a correct answer generated by the second trained model.

[0020] The information processing system of the present invention can also be configured such that the processor executes a process of calculating a score by inputting the received answer into a third trained model in which a question pattern, one or more answer patterns for each question pattern, and similarity have been trained as a data set.

[0021] In the information processing system of the present invention, the processor can also be configured to execute a process of estimating the factors of the recognition error by inputting the calculated score into a fourth trained model in which the score and the factors of the recognition error have been trained as a data set.

[0022] In the information processing system of the present invention, the processor can also be configured to perform a process of generating a modified communication expression that reflects the intention of the communicator by inputting the communication expression into a fifth trained model that has been trained using the meaning of the communication expression and the communication expression as a data set.

[0023] [Trained model] The trained model of the present invention (first trained model) is a model in which a communication expression and the meaning of the communication expression are learned as a dataset, and when a communication expression is input as input data, the computer is caused to function in such a way as to output the meaning of the communication expression.

[0024] The trained model of the present invention (second trained model) is trained on the meaning of a communication expression, questions that can be derived from the meaning of the communication expression and / or questions that cannot be derived from the meaning of the communication expression, and correct answers to the questions as a dataset, and when the meaning of a communication expression estimated by the first trained model is input as input data, it causes a computer to function in such a way as to generate questions that can be derived from the meaning of the communication expression and / or questions that cannot be derived from the meaning of the communication expression, and correct answers to the questions.

[0025] The trained model of the present invention (third trained model) is trained on question patterns, one or more answer patterns for each question pattern, and similarities as a dataset, and causes a computer to function in such a way that when a received answer is input as input data, a score representing the similarity between the answer and the correct answer to the question related to the answer is calculated.

[0026] The trained model of the present invention (fourth trained model) is trained using scores and factors of recognition error as a data set, and causes a computer to function in such a way that when an answer to a question generated by the second trained model and a score calculated based on the correct answer generated by the second trained model are input as input data, the computer outputs factors of recognition error.

[0027] The trained model of the present invention (fifth trained model) is trained using the meaning of a communicative expression and the communicative expression as a data set, and causes a computer to function in such a way that when a communicative expression is input as input data, a modified communicative expression is output.

[0028] [Storage medium] The storage medium of the present invention is computer-readable and has the trained model of the present invention recorded therein. Effect of the Invention

[0029] According to the present invention, it is possible to help resolve the misunderstanding between the parties in communication and to eliminate inconveniences caused by the misunderstanding. [Brief description of the drawings]

[0030] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of an information processing system. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of a first terminal device and a second terminal device. [Diagram 3] FIG. 1 is a block diagram showing an example of a hardware configuration of an information processing device. [Figure 4] FIG. 2 is a schematic diagram showing an example of a processing flow of the information processing system. [Diagram 5] An explanatory diagram of the input and output of the first trained model and the second trained model. [Figure 6] 1A is an explanatory diagram showing an example of a generation process of a first trained model, and FIG. 1B is an explanatory diagram showing an example of a generation process of a second trained model. [Figure 7](a) is an explanatory diagram of an example of the output of the meaning of an estimated communicative expression, and (b) is an explanatory diagram of an example of the generated question and correct answer. [Figure 8] FIG. 13 is an explanatory diagram showing an example of higher-level factors, intermediate-level factors, and lower-level factors. [Figure 9] 1A is an explanatory diagram of the input and output of the third trained model, and FIG. 1B is an explanatory diagram showing an example of the generation process of the third trained model. [Figure 10] 1A is an explanatory diagram of the input and output of the fourth trained model, and FIG. 1B is an explanatory diagram showing an example of the generation process of the fourth trained model. [Figure 11] An illustration of the process of generating a modified communicative expression. [Figure 12] An illustration of the process of generating a modified communicative expression. [Figure 13] 1A is an explanatory diagram of the input and output of the fifth trained model, and FIG. 1B is an explanatory diagram showing an example of the generation process of the fifth trained model. [Figure 14] FIG. 13 is an explanatory diagram of the concept of recognition deviation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] (Embodiment) An example of an embodiment of the present invention will be described with reference to the drawings. Here, a case where two people (hereinafter referred to as a "first person" and a "second person") communicate online using information processing devices (hereinafter referred to as a "first terminal device 10" and a "second terminal device 20") will be described as an example.

[0032] <Overall system configuration> 1, the information processing system of this embodiment mainly comprises a first terminal device 10 used by a first person, a second terminal device 20 used by a second person, and an information processing device 30. The first terminal device 10, the second terminal device 20, and the information processing device 30 are connected via a network line N.

[0033] <First terminal device> The first terminal device 10 is a device used by a first person. As shown in Fig. 2, the first terminal device 10 mainly comprises a processor 11, a memory 12, a storage 13, a communication module 14, an input / output I / F 15, and an external device I / F 16. Each element is electrically connected via a bus 17. The first terminal device 10 is composed of various calculation processing modules, a PC, etc.

[0034] The processor 11 controls each device connected via a bus 17, and is configured, for example, by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an MPU (Micro Processor Unit), and the like.

[0035] The processor 11 loads programs required for executing processes from the storage 13 into the memory 12 and executes each process. Note that, if some of the programs are stored in the memory 12, the processor 11 also executes the processes of those programs.

[0036] The memory 12 is a main storage device that stores data and instructions, and is configured, for example, by a ROM (Read Only Memory) or a RAM (Random Access Memory).

[0037] The storage 13 is an auxiliary storage device that stores programs, data, and various trained models described later, and is composed of, for example, a hard disk drive (HDD), a solid state drive (SSD), flash memory, etc.

[0038] The communication module 14 is an interface for wireless communication (data communication) with external devices, and communication is performed according to various communication standards such as TCP / IP, Bluetooth Low Energy (registered trademark), Bluetooth (registered trademark), 3G, 4G (LTE), and 5G.

[0039] The input / output I / F 15 is an interface for connecting input devices and output devices to the first terminal device 10, and is connected to various input devices such as a network camera, keyboard, mouse, touch panel, microphone, scanner, etc., and various output devices such as a display, projector, printer, speaker, etc.

[0040] The external device I / F 16 is an interface for connecting various external devices other than an input / output device, and includes various serial interfaces and parallel interfaces.

[0041] <Second Terminal Device> The second terminal device 20 is a device used by a second person, and is composed of various calculation processing modules, a PC, etc. As shown in Fig. 2, the second terminal device 20 also has a processor 21, a memory 22, a storage 23, a communication module 24, an input / output I / F 25, and an external device I / F 26 as main components. Each element is electrically connected via a bus 27. Each element of the second terminal device 20 is similar to that of the first terminal device 10, so a description of the second terminal device 20 will be omitted.

[0042] <Information processing device> The information processing device 30 is a so-called server, and is composed of various calculation processing modules, a PC, etc. As shown in Fig. 3, the information processing device 30 also includes a processor 31, a memory 32, a storage 33, a communication module 34, an input / output I / F 35, and an external device I / F 36 as main components. Each element is electrically connected via a bus 37. Each component included in the information processing device 30 is similar to that of the first terminal device 10, and therefore description thereof will be omitted.

[0043] In order to provide a stable service to a plurality of terminal devices, the information processing device 30 is preferably connected to an uninterruptible power supply 38, a backup device 39, etc. The uninterruptible power supply 38 and the backup device 39 can be existing devices.

[0044] In the information processing device 30, the processor 11 executes processes such as (1) estimating the meaning of a communication expression, (2) generating a question and a correct answer to the question, (3) calculating a score, (4) estimating factors, and (5) identifying a support measure. Details of each process executed by the information processing device 30 will be described later.

[0045] In this application, "communicative expression" refers to computer-readable digital information among information transmitted in the form of documents, speech, or text. In addition, in this application, "system development documents" refers to documents related to system development among communicative expressions, including system configuration diagrams, development schedules, use case diagrams, site maps, database design documents, test cases / test scenarios, release notes, project plans, user guides / manuals, interface specifications, etc.

[0046] In addition, the "communicative expressions" are not limited to those written in Japanese or English, but include those written in various languages ​​such as Spanish, French, German, etc. The "communicative expressions" also include various diagrams other than those exemplified above.

[0047] In addition, in this application, the "meaning of a communication expression" means matters that are objectively recognized from the images and character strings contained in each communication expression, such as the purpose of development and the content explicitly stated in the communication expression.

[0048] <Processing flow of information processing system> Next, an overview of the process flow of the information processing system of this embodiment will be described below. Fig. 4 shows an example of the process flow of the information processing system of this embodiment.

[0049] As shown in FIG. 4, a first person uses a first terminal device 10 that he or she uses to request the information processing device 30 to create a space (S101).

[0050] The information processing device 30 accepts a request from the first terminal device 10, generates a space (S102), and notifies the first terminal device 10 and the second terminal device 20 of information about the generated space (hereinafter referred to as "space information"), such as the URL of the space.

[0051] When the first person receives the space information from the information processing device 30, the first person accesses the space on the first terminal device 10 and uploads a communication expression (S103). When the second person receives the space information from the information processing device 30, the second person accesses the space on the second terminal device 20.

[0052] The information processing device 30 receives the communication expression uploaded from the first terminal device 10, and estimates the meaning of the communication expression (S104). The method of estimating the meaning of the communication expression will be described later.

[0053] After estimating the meaning of the communication expression, the information processing device 30 generates a question and a correct answer to the question based on the meaning of the communication expression (S105), and transmits the question to the first terminal device 10 and the second terminal device 20. A method of generating the question and the correct answer will be described later.

[0054] The first and second persons who received the question upload answers to the question to the information processing device 30 from the first terminal device 10 and the second terminal device 20 that they use (S106).

[0055] The information processing device 30 receives the uploaded answers and calculates a score based on each answer (S107). The method of calculating the score will be described later.

[0056] The information processing device 30 estimates the cause of the recognition error based on the calculated score (S108). The method of estimating the cause will be described later.

[0057] The information processing device 30 identifies an assistance measure associated with the estimated cause (S109), and transmits the assistance measure to the first terminal apparatus 10 and / or the second terminal apparatus 20. A method for identifying an assistance measure will be described later.

[0058] Uploading of communication expressions can be done in various ways, such as direct input on a browser, voice input, automatic pick-up of speech during a meeting, e-mail, chat input, drag-and-drop of files, etc.

[0059] <Processing executed by the information processing device> Next, each process executed by the information processing device 30 will be described, specifically, (1) estimating the meaning of a communicative expression, (2) generating a question and a correct answer to the question, (3) calculating a score, (4) estimating factors, and (5) identifying support measures.

[0060] (1) Estimation of the meaning of communicative expressions When the information processing device 30 receives the communication expression transmitted from the first terminal device 10, the processor 31 of the information processing device 30 executes a process of estimating the meaning of the communication expression.

[0061] The meaning of a communicative expression can be estimated using a trained model (hereinafter, for convenience of explanation, referred to as a "first trained model"). FIG. 5 shows an example of the first trained model.

[0062] As shown in Figure 5, the first trained model takes a communicative expression as input and outputs the meaning of the communicative expression.

[0063] As shown in Fig. 6(a), the first trained model can be generated by inputting a dataset into a model generation device and subjecting the dataset to machine learning. As the dataset, a communication expression (data) and the meaning (label) of the communication expression can be used.

[0064] In other words, labeled data in which the meaning of a communication expression is assigned as a label to the communication expression can be used as a data set.

[0065] Typically, a communication expression includes one or more images and character strings. The term "labeled data" used herein refers to data in which the meaning of the communication expression of each image and each character string included in the communication expression is assigned as a label to each image and each character string.

[0066] There is no limitation on the method for performing machine learning, and various algorithms such as neural network, random forest, SVM, etc. Other methods for performing machine learning may be used.

[0067] When estimating the meaning of a communicative expression using the first trained model, the processor 11 executes the following processing.

[0068] First, the processor 11 accepts a communication expression uploaded from the first terminal device 10 and stores the communication expression in the storage 13 .

[0069] Next, the processor 11 reads the transfer representation stored in the storage 13 and inputs it into the first trained model.

[0070] When a communication expression is input to the first trained model, the first trained model performs processes such as converting the communication expression into text, analyzing the text, and outputting the meaning of the communication expression.

[0071] The communication expression can be converted into text using existing character conversion technology such as optical character recognition / reader (OCR).

[0072] When OCR is used to convert a communication expression into text, the communication expression is converted into text through separation of images and character strings contained in the communication expression, analysis of character strings, character analysis, and the like.

[0073] When the transmitted expression is speech (audio), the speech can be converted into text by existing or new speech recognition technology. In speech recognition, the speech is converted into text through processes such as acoustic analysis, phoneme identification using an acoustic model, word conversion using a language model and a pronunciation dictionary, and sentence output.

[0074] The text can be analyzed using natural language processing (NLP) technology. Specifically, the text can be analyzed by performing morphological analysis, syntactic analysis, semantic analysis, contextual analysis, etc. on the text data.

[0075] In morphological analysis, text data is divided into the smallest units (words) and information such as parts of speech and conjugations is assigned to each word. In syntactic analysis, the relationships between words, such as dependencies between words separated by morphological analysis, are analyzed.

[0076] Semantic analysis is the interpretation of the meaning of the parsed sentence. Contextual analysis is the analysis of the connections between sentences. Morphological analysis, syntactic analysis, semantic analysis, and contextual analysis can be performed using existing technologies as well as new technologies.

[0077] In this embodiment, through the above analysis, the meaning of the communication expression is inferred for each piece of text contained in the communication expression, and the inferred meaning of the communication expression is output as a list of syntactically aligned text.

[0078] FIG. 7(a) is an example of the output of the meaning of a communication expression. In the example shown in FIG. 7(a), the meaning of a communication expression is output by category. In the example of FIG. 7(a), the categories set are "purpose of development," "document content," "development management," and "quality control," but these are just examples and other categories may be used. It is preferable to set the categories in a way that leads to identifying the cause of the misunderstanding.

[0079] (2) Generate questions and answers to those questions Once the meaning of a communicative expression has been estimated in the previous step, a process is executed to generate a question and a correct answer to the question based on the meaning of the communicative expression.

[0080] The generation of questions and correct answers can be performed using a trained model. Figure 5 shows an example of a trained model (hereinafter, for convenience of explanation, referred to as a "second trained model") that generates questions and correct answers.

[0081] As shown in Figure 5, the second trained model takes as input the meaning of a communicative expression estimated in the previous step (step (1) above), and outputs questions that can be derived from the meaning of the communicative expression (hereinafter referred to as "derivable questions") and / or questions that cannot be derived from the meaning of the communicative expression (hereinafter referred to as "non-derivable questions"), along with the correct answers to those questions.

[0082] As shown in Fig. 6(b), the second trained model can be generated by inputting a dataset into a model generating device and performing machine learning on the dataset. The dataset can include the meanings (data) of communicative expressions, derivable questions and / or non-derivable questions, and correct answers (labels) to the questions.

[0083] In other words, labeled data in which the meaning of the communicative expression, which is the data, is labeled with derivable questions and / or non-derivable questions and the correct answers to each of the questions (derivable questions and / or non-derivable questions) can be used as a dataset.

[0084] As in the case of the first trained model, the method of performing machine learning is not limited, and various algorithms such as neural network, random forest, SVM, etc. Other methods of performing machine learning may be used.

[0085] When a question and a correct answer to the question are generated using the second trained model, the processor 11 performs the following processing.

[0086] First, the processor 31 reads the meaning of the communicative expression stored in the storage 33 and inputs it into the second trained model.

[0087] When the meaning of a communicative expression is input to the second trained model, the second trained model receives the meaning of the communicative expression and performs processing such as text analysis and question and answer generation. The input here is text data.

[0088] The text analysis is performed by executing morphological analysis, syntactic analysis, semantic analysis, contextual analysis, etc., on the input text data. The morphological analysis, syntactic analysis, semantic analysis, contextual analysis, etc., are performed for each input text data.

[0089] In the second trained model, feature words are extracted by text analysis. The feature words can be extracted using existing or new techniques. When feature words are extracted by text analysis, internal processing is performed based on the feature words to generate questions and correct answers.

[0090] Fig. 7(b) is an example of output of questions and correct answers. In the example shown in Fig. 7(b), questions and correct answers are output by category. The categories correspond to the semantic categories of the communicative expressions that are the basis for generating the questions and correct answers.

[0091] Therefore, in the example of FIG. 7(b), questions and correct answers are generated for each category, namely, "purpose of development," "document content," "development management," and "quality control," and are output for each category.

[0092] For example, when generating questions and correct answers in the category "purpose of development," the meaning of the communicative expression "○○ solves △△ by □□" (input) is generated as the question "What does ○○ solve by □□?" and the correct answer (output) for that question is "△△."

[0093] In addition, when generating questions and correct answers in the category "document content," for the meaning (input) of the communicative expression "○○ enters △△ with □□," the question "Who will enter △△?" and the correct answer (output) for that question, "〇〇," are generated.

[0094] In addition, when generating questions and correct answers in the category of "development management," the meaning of the communicative expression "XX is in charge of △△" (input) is used to generate the question "Who is in charge of △△?" and the correct answer (output) for that question is "XX."

[0095] In addition, when generating questions and correct answers in the category "quality control," the meaning of the communicative expression "○○ must be □□ when △△" (input) is generated as the question "When must ○○ be □□?" and the correct answer (output) for that question is "△△."

[0096] The second trained model can generate non-determinable questions in addition to determinable questions like the example above.

[0097] For example, when generating questions and correct answers in the category of "development management," in response to the meaning of the communicative expression "○○ is in charge of △△," an underived question "Who is in charge of ☆☆?" and the correct answer "Unknown" for the underived question are generated.

[0098] It is desirable to classify the questions and the correct answers in association with the cause of the misunderstanding between the parties in communication (in this embodiment, the first person and the second person).

[0099] By linking questions to factors that cause the recognition discrepancy and classifying them, the answers to the questions, the accuracy rate, the degree of accuracy, etc. can be used as data when estimating the factors described below.

[0100] The factors of the misunderstanding (in other words, the classification of questions) can be set in stages. For example, "physical factors" and "non-physical factors" can be positioned as top factors, and intermediate and lower factors can be set for each top factor.

[0101] "Physical factors" refer to factors resulting from defects in the terminal device, microphone, speaker, network, etc. Specifically, issues such as distorted audio, audio cutting out, and inaudible audio are expected. "Non-physical factors" refer to factors other than "physical factors."

[0102] In addition, the intermediate factors referred to here are factors that are positioned as subordinate concepts to the respective upper factors, and the lower factors refer to factors that are positioned as subordinate concepts to the respective intermediate factors.

[0103] For example, as shown in Figure 8, "insufficient communication information" and "differences in interpretation" can be set as intermediate factors of "non-physical factors." In this case, "expression" and "background knowledge" can be set as sub-factors of "insufficient communication information," and "language," "culture," "bias," etc. can be set as sub-factors of "differences in interpretation."

[0104] Examples of situations in which "expression" is a factor include cases where the communication expression contains a lack of information, errors, or contradictions, while examples of situations in which "background knowledge" is a factor include cases in which there is a lack of necessary information or knowledge that is not included in the communication expression.

[0105] An example of a situation where "language" is a factor would be when the first languages ​​of the communicating parties are different; an example of a situation where "culture" is a factor would be when people perceive and act differently when they receive the same words, or when they have different ideas about time, such as delivery dates; and an example of a situation where "bias" is a factor would be when people have preconceived notions or expectations about the other person they are communicating with (such as their title or career).

[0106] In this way, by hierarchizing (subdividing) the causes of the misunderstanding (classification of questions and correct answers), it is possible to identify in more detail the causes of the misunderstanding between the parties in the communication (in this embodiment, the first person and the second person), making it possible to identify specific support measures.

[0107] In order to identify the causes, it is desirable to classify the questions and correct answers by linking them to the causes of the recognition discrepancy. However, the questions and correct answers may include those that are not linked to the causes of the recognition discrepancy.

[0108] (3) Calculation of score When the answers uploaded from the first terminal device 10 and the second terminal device 20 are accepted, a process of calculating a score based on the answers is executed. The score is calculated for each of the first person and the second person.

[0109] The score can be calculated, for example, based on the answers to the generated questions and the correct answers generated by the second trained model. Specifically, the number of correct answers is calculated from the answers and the correct answers, and the ratio of the number of correct answers to the number of questions (i.e., the correct answer rate) can be used as the score (correct answer rate score).

[0110] When the correct answer rate is used as the score, the score may be calculated from the overall correct answer rate, or, if the questions and correct answers are classified in association with the causes of the misrecognition, the score may be calculated from the correct answer rate for each classification. In the following, the score calculated from the overall correct answer rate may be referred to as the "overall score," and the score calculated from the correct answer rate for each classification may be referred to as the "classification score."

[0111] For example, if factors of recognition discrepancy are set as "expression," "background knowledge," "language," "culture," and "bias," and questions and correct answers are linked and classified to each of these factors, a classification score can be calculated from the correct answer rate for questions linked to "expression," the correct answer rate for questions linked to "background knowledge," the correct answer rate for questions linked to "language," the correct answer rate for questions linked to "culture," and the correct answer rate for questions linked to "bias."

[0112] Furthermore, the calculated score of a first person makes it possible to grasp the gap between the first person's intention and expression, and the calculated score of a second person makes it possible to grasp the gap between the first person's expression and the second person's understanding.

[0113] In other words, if the first person's score is low (if the rate of correct answers is low), it can be understood that there is a large discrepancy between the first person's intention and expression, and if the first person's score is high (if the rate of correct answers is high), it can be understood that there is a small discrepancy between the first person's intention and expression.

[0114] Similarly, if the second person's score is low (the rate of correct answers is low), it can be understood that there is a large discrepancy between the first person's expression and the second person's understanding, and if the second person's score is high (the rate of correct answers is high), it can be understood that there is a small discrepancy between the first person's expression and the second person's understanding.

[0115] The method of calculating the score described here is an example, and the score can be calculated by other methods. For example, instead of calculating the score based on the correct answer rate, the score can be calculated based on the similarity (similarity score) between the correct answer and the answer.

[0116] In this case, the score can be calculated using a trained model other than the previously trained model (for convenience of explanation, hereinafter, referred to as a "third trained model"). Figure 9(a) shows an example of the third trained model.

[0117] As shown in FIG. 9(a), the third trained model takes an answer uploaded from the first terminal device 10 or the second terminal device 20 as input, and outputs the similarity (score) between the answer and the correct answer generated in the previous step. The similarity is represented by a vector and a distance. The similarity is calculated for each answer.

[0118] As shown in Fig. 9(b), the third trained model can be generated by inputting a dataset into a model generation device and performing machine learning on the dataset. The dataset can include question patterns, one or more answer patterns (data) for each question pattern, and the similarity (label) of each answer pattern for each question pattern.

[0119] In other words, labeled data in which similarity is assigned as a label to question patterns, which are data, and one or more answer patterns to each question pattern, can be used as a data set.

[0120] There is no limitation on the method for performing machine learning, and various algorithms such as neural network, random forest, SVM, etc. Other methods for performing machine learning may be used.

[0121] For the third trained model, a newly trained trained model can be used, or a model that has been trained using transfer learning or fine-tuning based on a pre-trained model (e.g., BERT) can be used.

[0122] When calculating the similarity (similarity score) between an answer and a correct answer using the third trained model, the processor 11 performs the following processing.

[0123] First, the processor 11 reads out the answer stored in the storage and inputs it to the third trained model. The answer input here is text data.

[0124] When an answer is input to the third trained model, the third trained model The answers are classified as "correct answer," "another answer," or "no answer." The classification (determination) of "correct answer," "another answer," or "no answer" can be performed using natural language processing technology or other existing technologies.

[0125] Then, in the third trained model, the answers classified as “different answers” ​​are preprocessed and vectorized, and the resulting vector is then used to calculate the similarity (score) between the answer and the correct answer.

[0126] The pre-processing includes, for example, sentence cleaning, morphological analysis, word normalization, and removal of stop words.

[0127] In the vectorization, each word obtained by the preprocessing is converted into a vector. The vectorization can be performed using existing or new methods such as a method using the number of occurrences of a word or a method using a distributed representation of a word.

[0128] Methods that use the number of times a word occurs include Bag of Words (BoW), Term Frequency - Inverse Document Frequency (TF-IDF), and Latent Semantic Indexing (LSI), while methods that use distributed representations of words include Word 2 Vec.

[0129] Once the vectorization is complete, the similarity between the answer and the correct answer is calculated using the obtained vector. The similarity can be calculated using existing or new methods, such as cosine similarity or Euclidean distance.

[0130] (4) Estimation of factors Once the accuracy score, similarity score, and other scores are calculated, a process is performed to estimate the cause of the recognition discrepancy using the scores. The cause here means the cause (background information) of the discrepancy (gap) between the first person's intention and the second person's understanding.

[0131] The factors can be estimated using a trained model. Figure 10(a) shows an example of a trained model that estimates factors (for convenience of explanation, hereinafter, referred to as a "fourth trained model").

[0132] As shown in Figure 10(a), the fourth trained model takes the first person's score or the second person's score as input, and outputs the factors behind the discrepancy between the first person's intention and the second person's understanding.

[0133] The score may be the score calculated in (3) above, specifically, the accuracy rate score, the similarity score, or the like. When the accuracy rate score is used, the score may be either the overall score or the category score, but the category score provides higher accuracy of factor estimation. When the similarity score is used, the accuracy of factor estimation is higher than when the accuracy rate score is used.

[0134] As shown in Fig. 10(b), the fourth trained model can be generated by inputting a data set to a model generation device and performing machine learning on the data set. The data set can include scores (data) and factors of recognition error (labels).

[0135] In other words, labeled data in which factors of recognition error (background information) are added as labels to the scores can be used as a dataset.

[0136] The labels to be assigned can be the upper factors, as well as the mid-level factors related to each upper factor and the lower-level factors related to each mid-level factor. The meanings of the mid-level factors and the lower-level factors are as described above.

[0137] There is no limitation on the method for performing machine learning, and various algorithms such as neural network, random forest, SVM, etc. Other methods for performing machine learning may be used.

[0138] (5) Identification of support measures Identification of a support measure can be performed, for example, by setting a support measure for each factor in advance and presenting the support measure associated with the identified factor in accordance with the factor.

[0139] It is preferable to set support measures, for example, for each higher-level factor, as well as for each intermediate factor related to each higher-level factor and for each lower-level factor related to each intermediate factor.

[0140] One example of a support measure would be to generate a communication expression (hereinafter referred to as a "corrected communication expression") that correctly reflects the intention of the first person (the person communicating).

[0141] As shown in Fig. 11, the communication expression sent by the first person does not necessarily correctly reflect the first person's intention. If there is a gap between the first person's expression and intention, the communication expression can be modified (a modified communication expression can be generated) to match the first person's intention and presented.

[0142] Specifically, as shown in Fig. 12, the communication expression can be corrected and a corrected communication expression can be generated based on the expression of the first person, the meaning of the communication expression (meaning that can be objectively grasped from the communication expression), and the difference in meaning between the first person's answer and the communication expression. The difference in meaning between the first person's answer and the communication expression can be identified, for example, from a similarity score calculated by the third trained model.

[0143] The modified communication expression can be generated using a trained model (for convenience of explanation, hereinafter referred to as the "fifth trained model"). FIG. 13(a) shows an example of the fifth trained model.

[0144] As shown in FIG. 13(a), the fifth trained model takes a transfer representation as input and a modified transfer representation as output.

[0145] As shown in Fig. 13(b), the fifth trained model can be generated by inputting a dataset into a model generating device and subjecting the dataset to machine learning. As the dataset, the meaning (data) of a communicative expression and the communicative expression (label) can be used.

[0146] In other words, labeled data in which the meaning of a communication expression, which is data, is assigned as a label can be used as a dataset. The meaning of the communication expression can be estimated by the first trained model.

[0147] There is no limitation on the method for performing machine learning, and various algorithms such as neural network, random forest, SVM, etc. Other methods for performing machine learning may be used.

[0148] When a communication expression of the fifth trained model is input, the fifth trained model executes a process in which the meaning of the communication expression is corrected to the correct meaning of the communication expression based on the answer of the first person and the meaning of the communication expression, and a corrected communication expression is generated based on the correct meaning of the communication expression.

[0149] The generated correction-transmission expression is transmitted to the first terminal device 10. At this time, it is possible to transmit only the generated correction-transmission expression to the first terminal device 10, but it is also possible to transmit the correction-transmission expression together with countermeasure information that takes into account the cause of the recognition error identified from the answer of the second person (the cause estimated by the fourth trained model).

[0150] For example, if the factor inferred by the fourth trained model is "language," the recipient can be prompted to provide a translation in their native language; if the factor inferred is "lack of background knowledge," the recipient can be prompted to convey that background knowledge to the second person.

[0151] In addition, if the inferred factor is "culture," the user can be prompted to explain how the terminology used in the communicative expression is used in the country of the person communicating it. At this time, the user can also be prompted to provide information on related sites, etc.

[0152] In addition, if the cause is a difference in "culture," such as the difference in the meaning of colors or symbols, students can be encouraged to add an explanation that will promote understanding of cultural differences, such as an explanation of the way the communicator thinks about colors or symbols.

[0153] In addition, if the cause is "background knowledge," such as not knowing technical terms, the user can be prompted to provide supplementary explanations about the lack of background knowledge, such as explaining technical terms. In addition, if the cause is some kind of "bias," the user can be prompted to provide an explanation that will eliminate the bias.

[0154] The above countermeasure information is an example, and the content of the countermeasure information is not particularly limited as long as it is information that reduces the gap between the communication expression and the second person's understanding. Note that the countermeasure information may be added as necessary. In addition, the corrected communication expression and the countermeasure information may be transmitted simultaneously or separately.

[0155] (Other embodiments) The configuration of the above embodiment is merely an example, and the present invention is not limited to the configuration of the above embodiment. The present invention can be modified and changed in various ways without departing from the gist of the invention.

[0156] In the above embodiment, two people (a first person and a second person) communicate online, but the number of communication parties may be three or more. In this case, the number of information terminals connected via the network line N may also be three or more.

[0157] In the above embodiment, as shown in FIG. 4, an example is taken of a case in which the first terminal device 10 and the second terminal device 20 are connected to an information processing terminal, but the process of estimating the meaning of a communication expression can be executed as long as at least the first terminal device 10 and the information processing terminal are connected via a network line N.

[0158] In the above embodiment, an example is given of the information processing system of the present invention being configured as a web application accessible from a web browser, but the information processing system of the present invention can also be configured as a native application installed on each information processing device terminal.

[0159] Although not described in the above embodiment, the configuration of the information processing device 30 may be designed to match the information processing system to be constructed, and the functions implemented in the information processing device 30 may be distributed across multiple server devices or consolidated into a single server device.

[0160] In the above embodiment, although not particularly limited, the sending of questions to the first terminal device 10 and the second terminal device 20 and the responses thereto from the first terminal device 10 and the second terminal device 20 can be done one question at a time or multiple questions at a time.

[0161] In either case, a model generation device that learns answers to questions as learning data (e.g., a model generation device that generates a second trained model) can be connected to the external device I / F 36 of the information processing device 30, so that questions based on the answers can be generated as subsequent questions (not necessarily immediately after).

[0162] Although not particularly limited in the above embodiment, in the information processing system of the present invention, processing for the first terminal device 10 and processing for the second terminal device 20 are executed in parallel, and it is also possible to execute processing for one of the terminal devices first and, after completion of said processing, execute processing for the other terminal device.

[0163] In the former case (when processes are executed in parallel), if there is a discrepancy between the first person's intention and the expression, the discrepancy can be resolved as it occurs. Similarly, if there is a discrepancy between the first person's expression (communicative expression) and the second person's understanding, the discrepancy can be resolved as it occurs.

[0164] In the latter case (when processing for one terminal device is executed first), for example, processing for the first terminal device 10 which uploads the communication expression can be executed first, and after eliminating the discrepancy between the first person's intention and the expression (communication expression), processing for the second terminal device 20 can be executed.

[0165] In this case, if there is a discrepancy between the first person's intention and expression, the discrepancy can be resolved before it is transmitted to the second person, thereby reducing the opportunity for the second person to come into contact with a communication expression that does not reflect the first person's intention, and reducing the probability of a discrepancy occurring between the first person's expression (communication expression) and the second person's understanding.

[0166] Additionally, it is sufficient that the information processing system functions as a whole, and which functions are implemented in which devices can be designed according to the information processing system to be constructed.

[0167] The present invention can also be used in communication situations other than system development, such as online interviews, online conferences, etc. In this case, the elements constituting the information processing system can have a configuration different from that of the above embodiment. [Industrial Applicability]

[0168] The present invention can be used when communicating using various documents, and is particularly suitable for use when communicating online using communicative expressions, as in the example described in the above embodiment. [Explanation of symbols]

[0169] 10 First terminal device 11 Processors 12. Memory 13. Storage 14 Communication Module 15 Input / Output Interface 16 External device I / F 17 Bus 20 Second terminal device 21 Processors 22 Memory 23. Storage 24 Communication Module 25 Input / Output Interface 26 External device I / F 27 Bus 30 Information processing device 31 Processors 32 Memory 33 Storage 34 Communication Module 35 Input / Output Interface 36 External device I / F 37 Bus 38 Uninterruptible power supply 39 Backup Device N Network Line

Claims

1. In an information processing device having a processor, The processor, A process for receiving a communicative expression; Executing a process of estimating the meaning of the input communication expression by inputting the received communication expression into a first trained model in which communication expressions and the meanings of the communication expressions have been trained as a data set.

23. An information processing apparatus comprising:

2. 2. The information processing device according to claim 1, The processor inputting the meaning of a communication expression estimated by the first trained model into a second trained model in which the meaning of the communication expression, questions that can be derived from the meaning of the communication expression and / or questions that cannot be derived from the meaning of the communication expression, and correct answers to the questions are trained as a dataset, thereby executing a process of generating questions that can be derived from the meaning of the input communication expression and / or questions that cannot be derived from the meaning of the communication expression, and correct answers to the questions; 23. An information processing apparatus comprising:

3. 3. The information processing device according to claim 2, The processor Executing a process of calculating a score based on the answers to the generated questions and the correct answers generated by the second trained model; 23. An information processing apparatus comprising:

4. 4. The information processing device according to claim 3, The processor Calculating a score by inputting the answers to the generated questions into a third trained model trained with question patterns, one or more answer patterns for each question pattern, and similarities as a data set; 23. An information processing apparatus comprising:

5. 5. The information processing apparatus according to claim 3, The processor The calculated score is input to a fourth trained model in which the score and the factor of the recognition error are trained as a data set, thereby performing a process of estimating the factor of the recognition error.

23. An information processing apparatus comprising:

6. 5. The information processing apparatus according to claim 4, The processor A process of generating a modified communication expression reflecting the intention of the communicator by inputting the communication expression into a fifth trained model that has been trained using the meaning of the communication expression and the communication expression as a data set.

23. An information processing apparatus comprising:

7. In an information processing system, The system includes at least a first terminal device and an information processing device connected to the first terminal device via a network line, The information processing device includes a processor, The processor, A process of receiving a communication expression transmitted from the first terminal device; Executing a process of estimating the meaning of the input communication expression by inputting the received communication expression into a first trained model in which communication expressions and the meanings of the communication expressions have been trained as a data set. An information processing system comprising:

8. 8. The information processing system according to claim 7, The processor inputting the meaning of a communication expression estimated by the first trained model into a second trained model in which the meaning of the communication expression, questions that can be derived from the meaning of the communication expression and / or questions that cannot be derived from the meaning of the communication expression, and correct answers to the questions are trained as a dataset, thereby executing a process of generating questions that can be derived from the meaning of the input communication expression and / or questions that cannot be derived from the meaning of the communication expression, and correct answers to the questions; An information processing system comprising:

9. 9. The information processing system according to claim 8, a second terminal device connected to the information processing device via a network line; The processor A process of receiving an answer to a question transmitted from the first terminal device and / or the second terminal device; Executing a process of calculating a score based on the answer to the received question and a correct answer generated by the second trained model. An information processing system comprising:

10. 10. The information processing system according to claim 9, The processor Executing a process of calculating a score by inputting the received answer into a third trained model trained with a question pattern, one or more answer patterns for each question pattern, and a similarity as a data set; An information processing system comprising:

11. 11. The information processing system according to claim 9, The processor The calculated score is input to a fourth trained model in which the score and the factor of the recognition error are trained as a data set, thereby performing a process of estimating the factor of the recognition error. An information processing system comprising:

12. 11. The information processing system according to claim 10, The processor A process of generating a modified communication expression reflecting the intention of the communicator by inputting the communication expression into a fifth trained model that has been trained using the meaning of the communication expression and the communication expression as a data set. An information processing system comprising:

13. A trained model, A data set of communicative expressions and their meanings is learned; A computer is caused to function so that, when a communicative expression is input as input data, the meaning of the communicative expression is output. A trained model characterized by:

14. A trained model, The meaning of a communicative expression, questions that can be derived from the meaning of the communicative expression and / or questions that cannot be derived from the meaning of the communicative expression, and correct answers to the questions are learned as a dataset; When the meaning of the communicative expression output in claim 13 is input as input data, the computer is caused to function so as to generate questions that can be derived from the meaning of the communicative expression and / or questions that cannot be derived from the meaning of the communicative expression, and correct answers to the questions. A trained model characterized by:

15. A trained model, A question pattern, one or more answer patterns for each question pattern, and a similarity are learned as a data set; causing a computer to function in such a way that, when the received answer is input as input data, a score representing a similarity between the answer and a correct answer to a question related to the answer is calculated; A trained model characterized by:

16. A trained model, The scores and factors of recognition discrepancies are used as a data set for learning. The computer is caused to function so that, when an answer to the question generated in claim 14 and a score calculated based on a correct answer generated by the trained model in claim 14 are input as input data, a cause of recognition deviation is output. A trained model characterized by:

17. A trained model, The meaning of communicative expressions and communicative expressions are used as a dataset for learning, causing a computer to function in response to a communicative expression being input as input data, so as to output a modified communicative expression; A trained model characterized by:

18. A computer-readable storage medium having recorded thereon the trained model according to any one of claims 13 to 17.

Images