Information Processing Systems

Abstract

An information processing system is provided that can analyze and evaluate keywords used in a dialogue with higher accuracy than before, leading to improvements in utterances in the dialogue. [Solution] The information processing system 2 comprises a voice input unit 10 that acquires the voice of the person being analyzed speaking to a person at a specified event, a reaction input unit 20 that emits an operation signal when operated by an operator, a keyword input unit 250 into which keywords are input before the start of the event, and an analysis processing unit 400 that performs analysis and evaluation of the keywords, the operator operates the reaction input unit 20 at the timing when it is determined that the person at hand has shown a reaction during the event, and the analysis processing unit 400 uses a natural language processing model to analyze and evaluate the keywords contained in the speech text of each utterance unit generated from the voice acquired by the voice input unit 10, while referring to the speech text before and after the timing when the operation signal was emitted.

Description

[Technical Field]

[0001] The present invention relates to an information processing system that inputs speech data in a dialogue into a natural language processing model such as a large language model (LLM) and analyzes the data. [Background technology]

[0002] In recent years, technological developments have been underway for information processing that analyzes the reactions of a conversation partner based on voice data acquired during a conversation, particularly for analysis using natural language processing models such as large-scale language models (LLMs). One such technology is an information processing device that uses a natural language processing model to efficiently improve the negotiation skills of sales representatives (see, for example, Cited Document 1). The information processing device described in Patent Document 1 can select products to encourage a new customer to purchase based on information about a conversation with the new customer. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7724937 Summary of the Invention [Problem to be solved by the invention]

[0004] However, although the information processing device described in the cited document 1 can predict the most suitable product for a new customer and contribute to the success of a sales negotiation, it does not analyze the salesperson's comments during the sales negotiation, especially the other party's reaction to keywords that may lead to the success of the sales negotiation, and therefore cannot lead to the improvement of the salesperson's comments.

[0005] The present invention aims to solve the above-mentioned problems and to provide an information processing system that can analyze and evaluate keywords used in dialogue with higher accuracy than conventional analysis using natural language processing models, leading to improvements in utterances in dialogue.

[0006] In order to solve the above problem, an information processing system according to one aspect of the present invention comprises: a voice input unit for acquiring the voice of a person to be analyzed speaking to a face-to-face person at a predetermined event; a reaction input unit that transmits an operation signal in response to an operation by an operator; a keyword input unit into which a keyword is input before the start of the event; an analysis processing unit that analyzes and evaluates the keywords; Equipped with the operator operates the reaction input unit at a timing when it is determined that the face-to-face person has shown a reaction during the event; The analysis processing unit is characterized in that it uses a natural language processing model to analyze and evaluate the keywords contained in the speech text of each utterance unit generated from the speech acquired by the speech input unit, while referring to the speech text before and after the timing when the operation signal is transmitted. [Effects of the Invention]

[0007] As described above, the present invention provides an information processing system that can analyze and evaluate keywords used in dialogues with greater accuracy than conventional analysis using natural language processing models, leading to improvements in utterances in dialogues. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating a system configuration of an information processing system according to an embodiment of the present invention; [Figure 2]1 is a perspective view showing a tablet computer connected to a microphone and a reaction input unit that constitute a voice input unit used in a dialogue situation, in an information processing system according to one embodiment of the present invention; [Figure 3] 10 is a perspective view schematically illustrating another example of a reaction input unit in an information processing system according to an embodiment of the present invention. FIG. [Figure 4] 10 is a perspective view schematically illustrating yet another example of a reaction input unit in an information processing system according to an embodiment of the present invention. FIG. [Figure 5] 1 is a flowchart illustrating an example of a control process for each event executed by an information processing system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Note that the embodiments described below are intended to embody the technical idea of ​​the present invention, and unless otherwise specified, the present invention is not limited to the following.

[0010] In each drawing, components having the same function may be assigned the same symbol. For convenience, the embodiments and examples may be shown separately to facilitate explanation or understanding of the main points, but partial substitution or combination of the configurations shown in different embodiments and examples is possible. In the following embodiments and examples, descriptions of matters common to the above will be omitted, and only the differences will be described. In particular, similar effects due to similar configurations will not be mentioned sequentially in each embodiment or example. The size and positional relationship of components shown in each drawing may be exaggerated to clarify the explanation.

[0011] (Information Processing System According to One Embodiment) First, the system configuration of an information processing system according to one embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram schematically showing the system configuration of an information processing system according to one embodiment of the present invention.

[0012] The information processing system 2 according to this embodiment is configured with a system configuration that generates analysis data to be used at the site of the dialogue, and a system configuration that performs analysis and proposals based on the analysis data.

[0013] The system configuration for generating analysis data includes a voice input unit 10 that acquires the voice of the person being analyzed speaking to the person in front of them at a predetermined event, a reaction input unit 20 that transmits an operation signal in response to an operator's operation, and a tablet computer 700 to which a microphone 10B constituting the voice input unit 10 and the reaction input unit 20 are electrically connected. The predetermined event includes an interview, a lecture, a speech, a play, etc.

[0014] The system configuration for performing analysis and making proposals based on the analysis data transmitted from the tablet computer 700 also includes a server 800 having a natural language processing model.

[0015] First, with reference to Figures 2 to 4, an overview of a tablet computer 700 to which a microphone 10B constituting the voice input unit 10 and a reaction input unit 20 are electrically connected will be described. Figure 2 is a perspective view schematically showing a tablet computer to which a microphone constituting the voice input unit 10 used in a dialogue site and a reaction input unit are connected, in an information processing system according to one embodiment of the present invention. Figure 3 is a perspective view schematically showing another example of a reaction input unit in an information processing system according to one embodiment of the present invention. Figure 4 is a perspective view schematically showing yet another example of a reaction input unit in an information processing system according to one embodiment of the present invention.

[0016] In the following, the speaker using this information system among the participants in the conversation will be referred to as the "analyzed person," and the person they are talking to will be referred to as the "interperson." The person operating the reaction input unit 20 will be referred to as the "operator." Basically, the case is assumed in which the analyzed person operates the reaction input unit 20, that is, the analyzed person and the operator are the same person, but there may also be cases in which the analyzed person and the operator are different people.

[0017] In addition, although the person being analyzed is generally one person, there may be cases where there are multiple people being analyzed. In this case, even when analyzing the voices of multiple people as a group, just as in the case of one person, it is possible to identify and analyze the people being analyzed by frequency, etc. (such as an analysis that **'s follow-up comment was effective).

[0018] In principle, there is only one operator, but there may be cases where there are multiple operators. Cases where there are multiple operators will be described in detail later. Note that the person you are facing will never be the person being analyzed or the operator.

[0019] <Audio input section> The voice input unit 10 is composed of a processing unit 10A and a microphone 10B that are electrically connected. In this embodiment, the processing unit 10A is provided in a tablet computer 700, and the microphone 10B, which is disposed outside the tablet computer 700, is connected to the processing unit 10A via a cable. Note that the connection of the microphone 10B is not limited to a wired connection, and it can also be connected using a short-range wireless communication technology such as Bluetooth (registered trademark).

[0020] The voice input unit 10 is configured to acquire the voice of the person being analyzed speaking to the other person from the voices acquired by the microphone 10B. For example, it is possible to acquire only the voice of the person being analyzed by using the frequency characteristics of the voice. Since the voice of the other person is not acquired, it is possible to avoid alarming the other person, the privacy of the other person is protected, and there is no risk of receiving unnecessary warnings from the other person. Since the voice of the other person is not acquired, there may be cases where the other person is not informed that they will be recorded.

[0021] 2 is a small microphone such as a pin microphone, and can be placed in a manner that makes it difficult for the person being analyzed to recognize. Also, if the environment allows clear voice capture, a microphone built into the tablet computer 700 can also be used. Furthermore, if the person being analyzed fully understands that only the voice of the person being analyzed will be captured, a medium- to large-sized microphone placed on a desk may be used.

[0022] <Reaction input section> When the operator recognizes that the person he / she is facing has shown some kind of reaction to the utterance of the person being analyzed, the operator operates the reaction input unit 20 to send an operation signal. When the operator determines that the utterance of the person being analyzed requires analysis, the operator operates the reaction input unit 20. For example, it is conceivable that the operator operates the reaction input unit 20 when the person he / she is facing shows a reaction, such as nodding or laughing, to the utterance of the person being analyzed.

[0023] As shown in FIG. 2, the reaction input unit 20 can also be connected to the tablet computer 700 by wire, or can be connected using short-range wireless communication technology such as Bluetooth (registered trademark).

[0024] 2 is a small on / off switch that can be hidden in the palm of one's hand. By inserting the operator's index finger, middle finger, etc. into the space of the finger insertion section 20B of the reaction input unit 20 and enveloping the reaction input unit 20 with the bent fingers in the palm of one's hand, the reaction input unit 20 can be held without the person facing the reaction input unit 20 being aware of its presence.

[0025] The operator can then press the push button 20A with a thumb or the like to transmit an operation signal without the other person noticing. The timing of transmitting this operation signal is associated with the voice data acquired by the voice input unit 10 and stored in a storage unit (memory) of the tablet computer 700. Note that the storage unit (memory) used in this system is not shown in the drawings. Furthermore, as will be described later, by pressing and holding the push button 20A, more detailed reaction information including the other person's emotional "intensity" information can also be transmitted.

[0026] The example of the reaction input unit 20 shown in Fig. 3 includes multiple push buttons. Therefore, although the external shape is larger than that shown in Fig. 2, using multiple push buttons makes it easier to transmit various pieces of reaction information. It is also possible to use the touch panel 30 of the tablet computer 700 as the reaction input unit 20.

[0027] 4, a push button can be displayed on the touch panel 30 of the tablet computer 700 to function as the reaction input unit 20. Furthermore, the display of the touch panel 30 shown in FIG. 4 also has a function for inputting evaluation information and a function for inputting keywords.

[0028] As described above, the reaction input unit 20 has an operation switch that the operator holds in his / her hand and operates, so that an operation signal can be transmitted at an appropriate timing without alarming the person on the other side.

[0029] <Speech text formation section> In the tablet computer 700, the voice data acquired by the voice input unit 10 and the timing of transmitting the operation signal sent from the reaction input unit 20 are stored in association with each other, and further the following control processing is performed.

[0030] 1, the tablet computer 700 includes an utterance text formation unit 100. The utterance text formation unit 100 can convert the speech of the person being analyzed, acquired by the speech input unit 10, into text and form an utterance text for each utterance. However, there may be cases where the tablet computer 700 does not have the function of the utterance text formation unit, and the function is performed by a remote server via two-way communication.

[0031] It is particularly effective to use the timing of the operation signal transmission as a reference and use a predetermined time span of the utterance text before and after the timing of the operation signal transmission for analysis within the utterance text of one event. In particular, to analyze which keyword the dialogue partner reacted to, it is effective to search for keywords in the dialogue text before the timing of the operation signal transmission. The setting for the predetermined time span before and after the timing of the operation signal transmission can be changed as desired. It is also possible to change the time span before and after the transmission timing (for example, to lengthen the time span before).

[0032] <Reaction Identification Unit> In this embodiment, as described above, reaction information can be input by operating the reaction input unit 20. The tablet computer 700 includes a reaction identification unit 200 electrically connected to the reaction input unit 20. The reaction identification unit 200 can identify the strength of the reaction of the person being met and reaction information including emotional information including at least affirmation, denial, confusion, sympathy, indifference, and attentive listening, based on the length of time the reaction input unit 20 is operated (in the case of FIG. 2) or the operation of a reaction input unit 20 having a different type of input unit (in the case of FIG. 3). Furthermore, as shown in FIG. 4, push buttons displayed on the touch panel 30 of the tablet computer 700 can be operated to function as the reaction input unit 20.

[0033] <Keyword input section> In this embodiment, the person being analyzed or the operator can input keywords before the start of an event using the touch panel 30 of the tablet computer 700 as an input unit. The tablet computer 700 has a keyword input unit 250 electrically connected to the input unit (touch panel) 30, and the keyword input unit 250 stores the input keywords as data for analysis. Note that multiple keywords can be set for one event.

[0034] <Evaluation information input section> Furthermore, in this embodiment, the subject or the operator can input evaluation information using the touch panel 30 of the tablet computer 700 as an input unit. The tablet computer 700 includes an evaluation information input unit 300 electrically connected to the input unit (touch panel) 30. By operating the input unit (touch panel) 30, the evaluation information input unit 300 can input evaluation information including information about the person in front of the person, information about the event, and keywords used for analysis. The display of the touch panel 30 of the tablet computer 700 shown in FIG. 4 functions not only as a reaction input unit 20 but also as an evaluation information input unit (touch panel) 30. Note that evaluation information is generally input after the event has ended, but it can also be input during the event. Although the items that can be input are limited, it is also possible to input the information in advance before the event.

[0035] The evaluation information is information for evaluating the event as a whole, such as the occupation of the person you are meeting and the overall score of the event, and can be called a "large evaluation function." In contrast, the reaction information obtained during the progress of the event can be called a "small evaluation function."

[0036] Next, we will explain the configuration of a system that performs analysis and proposals based on the analysis data transmitted from the tablet computer 700. In this embodiment, the system that performs analysis and proposals is provided in a server 800 that is connected to the tablet computer 700 via two-way communication.

[0037] <Analysis processing unit> The server 800 includes an analysis processing unit 400 that analyzes and processes the analysis data transmitted from the tablet computer 700. The analysis processing unit 400 can perform the following using a natural language processing model: (1) Analyze and evaluate keywords while referring to the text of the utterances before and after the timing of the operation signal being sent. (2) Analyze and evaluate keywords while referring to reaction information. (3) Analyze and evaluate keywords while referring to evaluation information. (4) Count the number of reactions of the face-to-face counterparts for each keyword in one event (the number of reactions can include positive or negative identifying information); (5) Accumulating data on the number of reactions for each keyword in each event and having the statistical analysis unit 600 learn the data. (6) Phrases contained in the spoken text other than the keywords to which the person in front of you reacted are extracted as candidate keywords for subsequent events. (7) Extract speech factors, which are words or expressions that affect the reaction of the person you are speaking to.

[0038] <Statistical analysis department> The server 800 includes a statistical analysis unit 600 that exchanges signals with the analysis processing unit 400. As described above, data on the number of reactions for each keyword in each event is accumulated as big data, and the analysis processing unit 400 learns the accumulated big data and statistically analyzes and evaluates the keywords. The statistical analysis unit 600 can perform cross-sectional analysis of keywords from multiple events that have already been held.

[0039] <Feedback formation section> The server 800 includes a feedback forming unit 500 to which a signal is sent from the analysis processing unit 400. The feedback forming unit 500 forms a suggestion for improving the next utterance based on the utterance factors extracted by the analysis processing unit 400.

[0040] Both the analysis processing unit 400 and the feedback forming unit 500 are capable of exchanging signals with the statistical analysis unit 600, and can perform the above control processing by referring to the cross-sectional analysis results of the statistical analysis unit 600 regarding keywords of multiple events.

[0041] <Notification Department> The speech factors extracted by the analysis processing unit 400 and the speech improvement suggestions formed by the feedback formation unit 500 are transmitted to the tablet computer 700 via two-way communication. The speech factors and speech improvement suggestions can be displayed by a notification unit (touch panel) 40 provided in the tablet computer 700 to notify the subject or the operator. In this embodiment, the display function of the touch panel of the tablet computer 700 can be used as the notification unit 40. However, this is not limited to this, and the information can also be displayed on a display device separate from the tablet computer 700, or can be notified by voice.

[0042] (Explanation of information processing) Next, a specific control process in the information processing system 2 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of a control process for each event executed by an information processing system according to an embodiment of the present invention.

[0043] 5, first, before the start of an event, the person to be analyzed or the operator inputs keywords for the event using the input unit (touch panel) 30 (step 2). By this input, the keyword input unit 250 stores the input keywords as data for analysis.

[0044] Next, it is determined whether or not recording has started (step S4). If it is determined that recording has not yet started (NO), the system is in a standby state. Then, for example, when the subject or the operator touches "Start Recording" displayed on the touch panel 30, it is determined in step S4 that recording has started (YES), and the voice input unit 10 starts acquiring voice data.

[0045] Next, it is determined whether an operation signal was transmitted during the event (step S6). When the operator finds that the conversation partner has shown some kind of reaction to the utterance of the person being analyzed during the event, the operator transmits an operation signal using the reaction input unit 20. If it is determined in this determination that an operation signal was not transmitted (NO), the system enters a standby state. If it is determined in step S6 that an operation signal was transmitted (YES), the timing at which the operation signal was transmitted is stored as analysis data (step S8). In other words, the timing at which the operation signal was transmitted is stored in association with the voice data acquired by the voice input unit 10. As will be described later, the analysis processing unit 400 analyzes and evaluates keywords contained in the utterance text of each utterance generated from the recorded data, while referring to the utterance text before and after the timing at which the operation signal was transmitted.

[0046] Next, it is determined whether or not reaction information has been input (step S10). As described above, the operator can input reaction information including the strength of the other person's reaction and emotional information including at least affirmation, denial, confusion, sympathy, indifference, and attentive listening by varying the length of time the reaction input unit 20 is operated (see FIG. 2) or by operating a reaction input unit 20 having a different type of input unit (see FIG. 3). Based on this operation, the reaction identification unit 200 identifies the reaction information.

[0047] If it is determined in step S10 that reaction information has been input by operating the reaction input unit 20 (YES), the reaction information identified by the reaction identification unit 200 is stored as analysis data (step S12). This allows reaction information that is a "small evaluation function" to be input. As will be described later, the analysis processing unit 400 analyzes and evaluates keywords while referring to the reaction information. If it is determined in step S10 that reaction information has not been input (NO), the control process of step S12 is not performed, and the process proceeds directly to step S14.

[0048] Then, it is determined whether or not the recording has ended (step S14). If it is determined that the recording has not ended (NO), the process returns to step S6, and while the recording continues, the control process from step S6 to step S14 is repeated every time an operation signal is received, and the analysis data is stored. The recording ends when the subject or the operator touches "End Recording" displayed on the touch panel 30. As a result, the determination in step S14 determines that the recording has ended (YES).

[0049] Next, the person being analyzed or the operator uses the input unit (touch panel) 30 to input evaluation information (step S16). Here, an example of the evaluation information input screen shown in FIG. 4 will be described. The evaluation information input screen is displayed on the touch panel 30 of the tablet computer 700. First, the person being analyzed or the operator inputs the attributes of the person being analyzed. Specifically, the person being analyzed or the operator selects the occupation of the person being analyzed according to the display of "Select Occupation" on the right side of the touch panel 30, and then selects whether the person being analyzed has decision-making authority according to the display in the center left of the touch panel 30.

[0050] Furthermore, the person being analyzed or the operator inputs an overall score for this event (here, 0 to 3) according to the display in the center of the touch panel 30. In the example shown, a score of 0 indicates that the person in front of them was not interested or was engaging in counter-sales. A score of 1 indicates that the event was merely a casual conversation. A score of 2 indicates that the person was able to exchange business cards. A score of 3 indicates that the event led to a lead in sales activities.

[0051] Touching "Save" displayed in the lower right corner of the touch panel 30 completes the input of the evaluation information shown in step S16. The evaluation information includes at least information about the attributes and decision-making authority of the dialogue partner, and a score that is the evaluation of the completed event by the person being analyzed or the operator. In this way, evaluation information that is a "large evaluation function" can be input. As will be described later, the analysis processing unit 400 analyzes and evaluates keywords while referring to the evaluation information.

[0052] Returning to the explanation of the flowchart, the speech text generation unit 100 then generates speech text for each utterance from the speech acquired by the speech input unit 10 (step S18). Next, the analysis processing unit 400 analyzes the use of keywords contained in all the speech texts and the reactions to those keywords (step S20). In other words, the reactions of the conversation partner to all utterances made by the person being analyzed in one event are analyzed.

[0053] Specifically, a natural language processing model is used to analyze the context of keywords contained in the utterance text before and after the timing at which the operation signal was emitted, and an analysis is performed to determine which keywords caused a reaction. It is particularly effective to analyze which keywords caused a reaction for a predetermined period before the timing at which the operation signal was emitted. Furthermore, if there is reaction information with a small evaluation function, the reaction information is referenced, and if there is evaluation information with a large evaluation function, the analysis is performed while referring to the evaluation information. Furthermore, as will be described later, the analysis can also be performed by referring to statistical analysis using big data accumulated from past events.

[0054] Here, the analysis processing unit 400 can determine whether the user is indifferent or listening attentively based on the speech text acquired before and after a certain period of time during which no operation signal is transmitted. Even if there is no reaction information, the analysis processing unit 400 can analyze and evaluate suitable keywords by referring to the transmission interval of the operation signal and the content of the corresponding speech text.

[0055] Next, the analysis processing unit 400 extracts speech factors, which are words or expressions that cause the reaction of the person being analyzed, based on the analysis of the keyword use and actions in step S20, and displays the speech factors on the notification unit (touch panel) 40 to notify the person being analyzed or the operator (step S22). That is, when reaction information with a small evaluation function is available, the analysis processing unit 400 refers to the reaction information, and when evaluation information with a large evaluation function is available, the analysis processing unit 400 extracts the speech factors while referring to the evaluation information. The speech factors may or may not include a preset keyword. When a keyword is not included, a new word or phrase included in the speech factors can be used as a new keyword candidate shown in step S28.

[0056] Furthermore, the feedback formation unit 500 forms speech improvement suggestions for subsequent events based on the speech factors extracted by the analysis processing unit 400, and displays the speech improvement suggestions on the notification unit (touch panel) 40 to notify the analyzed person and the operator (step S24). The analysis processing unit 400 and the feedback formation unit 500 can appropriately extract speech factors and appropriately form speech improvement suggestions using a natural language processing model.

[0057] An example of the causal factors and speech improvement proposals formed by the above control flow and displayed on the notification unit (touch panel) 40 is shown below. <Examples of causative factors> The reason this conversation resonated with the sales and marketing client is because the proposed support system has the potential to significantly improve their work efficiency. In particular, the immediate feedback provided by LLM and analysis methods based on customer attributes are directly useful for improving sales activities, which likely led to the exchange of business cards (score 2). Also, judging by the timing of the flag, we can infer that the client was interested in the specific features of the proposed system and its business benefits.

[0058] <Example of a speech improvement proposal> As a suggestion for improvement for the next sales conversation, you may be able to attract the customer's attention by mentioning more specific success stories and post-implementation effects in the conversation. It is also important to emphasize the aspects that directly relate to the customer's business characteristics and needs. This will increase the chances of generating further leads and converting them into sales (score 3).

[0059] Returning to the explanation of the flowchart, the analysis processing unit 400 then counts the number of reactions of the person being analyzed to the keywords in one event, and displays the number of reactions for each keyword on the notification unit (touch panel) 40 to notify the person being analyzed and the operator (step S26).

[0060] At this time, if there is reaction information, the analysis processing unit 400 can refer to the reaction information, distinguish the reaction of the person facing the keyword from the other person into a positive reaction or a negative reaction, count the number, and notify the person being analyzed or the operator. The counting of reactions to the keyword can be performed using a predetermined algorithm. By notifying the number of reactions of the other person facing the keyword in this way, the person being analyzed or the operator can refine the keywords in subsequent events by removing keywords with few reactions or few positive reactions.

[0061] Next, the analysis processing unit 400 extracts words and phrases other than the keywords to which the person in front of you reacted from among the words and phrases included in the spoken text. Then, for example, words and phrases that have received multiple positive reactions are extracted as keyword candidates for the subsequent event, and the candidate words and phrases are displayed on the notification unit (touch panel) 40 to notify the person being analyzed and the operator (step S28).

[0062] As described above, keywords that have received few reactions or few positive reactions for a completed event can be removed, and new effective keywords can be set for subsequent events based on the notified new keyword candidate information. This allows for the refinement of keywords, enabling the analysis and evaluation of more suitable keywords.

[0063] Then, the number of reactions counted for each keyword in the completed event is accumulated as big data (step S30), and the control process for one event is terminated. The accumulated number of reactions can be counted by distinguishing between positive and negative reactions.

[0064] The statistical analysis unit 600 can learn the big data accumulated in step 30 for each event and perform statistical analysis and evaluation of keywords. In subsequent events, the analysis processing unit 400 can perform the control processing of steps S20, S22, and S24 while referring to the statistical analysis and evaluation by the statistical analysis unit 600.

[0065] As described above, the information processing system 2 according to the present invention comprises a voice input unit 10 that acquires the voice of the person being analyzed speaking to the person being spoken to at a specified event, a reaction input unit 20 that transmits an operation signal in response to operation by an operator, a keyword input unit (e.g., a touch panel) 250 into which keywords are input before the start of the event, and an analysis processing unit 400 that performs analysis and evaluation of the keywords, wherein the operator operates the reaction input unit 20 at the timing when it is determined that the person being spoken to has shown a reaction during the event, and the analysis processing unit 400 uses a natural language processing model to analyze and evaluate the keywords contained in the speech text of each utterance unit generated from the voice acquired by the voice input unit 10, while referring to the speech text before and after the timing when the operation signal is transmitted.

[0066] In this invention, not only do we use a natural language processing model to analyze keywords in the spoken text, but also, when a person listening to the dialogue determines that the other person has reacted, they send an operation signal, and the analysis and evaluation are performed by referring to the spoken text before and after the timing of sending the operation signal, thereby achieving more accurate keyword analysis and evaluation. This makes it possible to provide an information processing system 2 that can analyze and evaluate keywords used in dialogue with higher accuracy than conventional analysis using a natural language processing model, leading to improvements in utterances in the dialogue. Furthermore, highly accurate keyword analysis and evaluation can be achieved with simple equipment without using image capture devices or various sensors.

[0067] In particular, the information processing system 2 according to the present invention is provided with a reaction identification unit 200 that identifies reaction information including the strength of the reaction of the person being met and emotional information including at least affirmation, denial, confusion, sympathy, indifference, and attentive listening, based on the length of time spent operating the reaction input unit 20 (see FIG. 2) or the operation of a reaction input unit 20 (see FIG. 3) having a different type of input unit, as shown in step S12 above, and the analysis processing unit 400 performs analysis and evaluation of keywords while referring to the reaction information.

[0068] The reaction identification unit 200 allows the input of reaction information that forms a "small analysis function" using the reaction input unit 20 with a simple configuration without alarming the interlocutor. Furthermore, since multiple complex reactions, such as not only affirmation and negation but also confusion, sympathy, indifference, and attentive listening, can be input, the accuracy of analyzing the interlocutor's reaction to the utterance of the person being analyzed can be improved. For example, since subtle human sensibilities such as atmosphere and subtleties can be used in the evaluation function, it is possible to appropriately analyze matters such as sarcasm, flattery, and social niceties that are difficult to reflect even when using a natural language processing model.

[0069] Furthermore, the information processing system 2 according to the present invention includes an evaluation information input unit 300 for inputting evaluation information including information about the person in question, information about the event, and an evaluation of the completed event, as shown in step S16 above, and the analysis processing unit 400 analyzes and evaluates the keywords while referring to the evaluation information.

[0070] The evaluation information input unit 300 can reliably obtain evaluation information that results in a "large evaluation function," so that the reaction of the person being analyzed to the statements made by the person being analyzed throughout the entire event can be analyzed accurately.

[0071] Furthermore, in the information processing system 2 according to the present invention, as shown in steps S20 and S22 above, the analysis processing unit 400 can extract speech factors, which are words or expressions that influence the reaction of the person being met, and can notify the extracted speech factors by the notification unit (e.g., a touch panel) 40.

[0072] The analysis processing unit 400 interprets the context of the extracted utterance text and extracts utterance factors, which are words or expressions that cause the reaction of the person being spoken to, so that utterance factors can be extracted accurately. In particular, when reaction information or evaluation information has been input, the utterance factors can be extracted more accurately by referring to such information. Then, the utterance factors can be notified to the person being analyzed or the operator by the notification unit (e.g., a touch panel) 40.

[0073] Furthermore, the information processing system 2 according to the present invention is provided with a feedback formation unit 500 that forms a speech improvement suggestion for the next time based on the extracted speech factors, as shown in step S24 above, and can notify the speech improvement suggestion by a notification unit (e.g., a touch panel) 40.

[0074] The feedback formation unit 500 can form utterance improvement suggestions with structural targets in response to the dialogue partner's reactions, so that the utterances of the person being analyzed can be expected to improve in subsequent events.

[0075] Furthermore, in the information processing system 2 according to the present invention, as shown in step S26 above, the analysis processing unit 400 counts the number of reactions of the face-to-face counterpart for each keyword in an event, and the number of reactions can include positive or negative identification information.

[0076] The person being analyzed or the operator can accurately grasp the evaluation of each keyword at a glance based on the number of reactions from the people they interact with for each keyword at an event. In particular, by distinguishing and counting positive or negative reactions, it is possible to grasp a more accurate evaluation of the reactions for each keyword at each event. Using this information on the number of reactions to keywords, the person being analyzed or the operator can refine the keywords in subsequent events by removing keywords with few reactions or few positive reactions.

[0077] Furthermore, in the information processing system 2 of the present invention, as shown in step S30 above, data on the number of reactions for each keyword in each event is accumulated, and a statistical analysis unit 600 is provided that learns the accumulated data and statistically analyzes and evaluates the keywords, and the number of reactions can include positive or negative identification information.

[0078] By accumulating data on the number of reactions to each keyword in each event to form big data, the statistical analysis unit 600 can perform a cross-sectional analysis of multiple events that have already been held. This allows for accurate analysis of which keywords or keyword combinations have received the most reactions. If positive or negative identification information is included, the relationship between the keyword or keyword combination and the reaction of the conversation partner can be analyzed more accurately. By obtaining such keyword analysis and evaluation, the analysis processing unit 400 and feedback generation unit 500 can use the cross-sectional analysis results to more accurately extract utterance factors and generate utterance improvement suggestions.

[0079] Furthermore, in the information processing system 2 according to the present invention, as shown in step S28 above, the analysis processing unit 400 can extract phrases contained in the spoken text other than the keyword to which the person in front of the person reacted as keyword candidates for subsequent events, and can notify the extracted keyword candidates by the notification unit (e.g., a touch panel) 40.

[0080] In step S26, the person being analyzed or the operator can remove keywords that have few reactions or few positive reactions for the next event, and set new effective keywords based on the notified new keyword candidate information. This allows the keywords to be refined, and analysis and evaluation of more appropriate keywords can be realized.

[0081] (When there are multiple operators) In an example where there are one or more subjects to be analyzed and multiple operators, multiple operators in different locations in the venue may operate individual reaction input units 20 for one or more subjects to be analyzed at a podium.

[0082] In this case, the information processing system 2 of this embodiment is equipped with multiple reaction input units 20, and multiple operators can operate their respective reaction input units 20 at any timing. For example, each operator can sense the audience's reactions from different locations in a lecture hall and input reaction information, enabling more effective analysis. In other words, this can lead to cross-sectional analysis of various evaluation scales by various operators.

[0083] (Other embodiments) The configuration for associating voice data with the timing of transmitting an operation signal, storing the voice, converting the voice into text, and analyzing the voice is not limited to the configuration using the tablet computer 700 and server 800 described above. For example, only a microphone, an operation switch, and a transmitting device for transmitting acquired data may be located on-site, with subsequent processing performed on a server on the cloud, or all functions up to analysis may be performed by a device on-site. In other words, each control component shown in FIG. 2 can be placed in any device, system, etc. at any location.

[0084] Furthermore, the reaction input unit 20 is not limited to the above-described operation switch that is held and operated by hand, but can also be a switch that is attached to the inside of the mouth, such as teeth, the sole of a shoe, clothing, glasses, behind the ear, etc.

[0085] (General Description) term <1> a voice input unit for acquiring the voice of a person to be analyzed speaking to a face-to-face person at a predetermined event; a reaction input unit that transmits an operation signal in response to an operation by an operator; a keyword input unit into which a keyword is input before the start of the event; an analysis processing unit that analyzes and evaluates the keywords; Equipped with the operator operates the reaction input unit at a timing when it is determined that the face-to-face person has shown a reaction during the event; The information processing system is characterized in that the analysis processing unit uses a natural language processing model to analyze and evaluate the keywords contained in the speech text of each utterance unit generated from the speech acquired by the speech input unit, while referring to the speech text before and after the timing when the operation signal is transmitted. term <2> a reaction identification unit that identifies reaction information including the strength of the reaction of the person being met and emotional information including at least affirmation, denial, confusion, sympathy, indifference, and attentive listening, based on the length of operation time of the reaction input unit or the operation of the reaction input unit having different types of input units; The analysis processing unit performs analysis and evaluation of the keyword while referring to the reaction information. <1> An information processing system according to claim 1. term <3> an evaluation information input unit for inputting information about the person to be met, information about the event, and evaluation information including an evaluation of the event that has ended; The analysis processing unit performs analysis and evaluation of the keyword while referring to the evaluation information. <1> or <2> An information processing system according to claim 1. term <4> the analysis processing unit counts the number of reactions of the face-to-face partner for each of the keywords in one of the events; The reaction number may include positive or negative identification information. <1> from <3> 10. An information processing system according to any one of claims 1 to 9. term <5> Data on the number of reactions for each keyword in each of the events is accumulated, a statistical analysis unit that learns accumulated data and statistically analyzes and evaluates the keywords; The number of reactions can include positive or negative identification information. <4> An information processing system according to claim 1. term <6> The analysis processing unit extracts, as keyword candidates for the subsequent event, phrases other than the keyword to which the face-to-face person has reacted, which are included in the spoken text, and The keyword candidates extracted by a notification unit are notified. <1> from <5> 10. An information processing system according to any one of claims 1 to 9. term <7> The analysis processing unit extracts speech factors that are words or expressions that are factors that affect the reaction of the person being met, a notification unit that notifies the extracted utterance factors; <1> from <6> 10. An information processing system according to any one of claims 1 to 9. term <8> a feedback forming unit that forms a proposal for improving the next utterance based on the extracted utterance factors; The notification unit notifies the user of the utterance improvement proposal. <7> An information processing system according to claim 1. term <9> The analysis processing unit determines whether the user is indifferent or listening attentively based on the speech text acquired before and after the portion where the operation signal has not been transmitted for a certain period of time. <1> from <8> 10. An information processing system according to any one of claims 1 to 9. term <10> A feature of the present invention is that a plurality of reaction input units are provided, and a plurality of operators operate the respective reaction input units at any timing. <1> from <9> 10. An information processing system according to any one of claims 1 to 9. term <11> The reaction input unit has an operation switch that the operator operates by hand. <1> from <10> 10. An information processing system according to any one of claims 1 to 9.

[0086] Although the embodiments and modes of implementation of the present invention have been described, the disclosed contents may be changed in the details of the configuration, and changes in the combination and order of elements in the embodiments and modes of implementation may be realized without departing from the scope and spirit of the claimed invention. [Explanation of symbols]

[0087] 2. Information Processing Systems 10 Audio input section 10A Processing section 10B Microphone 20 Reaction input section 20A push button 20B Finger insertion part 30 Input section (touch panel) 40 Information Department 100 Speech text formation unit 200 Reaction Identification Unit 250 Keyword input section 300 Evaluation information input section 400 Analysis processing section 500 Feedback formation section 600 Statistical Analysis Department 700 tablet computers 800 Server

Claims

1. a voice input unit for acquiring the voice of a person to be analyzed speaking to a face-to-face person at a predetermined event; a reaction input unit that transmits an operation signal in response to an operation by an operator; a keyword input unit into which a keyword is input before the start of the event; an analysis processing unit that analyzes and evaluates the keywords; Equipped with the operator operates the reaction input unit at a timing when it is determined that the face-to-face person has shown a reaction during the event; The information processing system is characterized in that the analysis processing unit uses a natural language processing model to analyze and evaluate the keywords contained in the speech text of each utterance unit generated from the speech acquired by the speech input unit, while referring to the speech text before and after the timing when the operation signal is transmitted.

2. a reaction identification unit that identifies reaction information including the strength of the reaction of the person being met and emotional information including at least affirmation, denial, confusion, sympathy, indifference, and attentive listening, based on the length of operation time of the reaction input unit or the operation of the reaction input unit having different types of input units; 2. The information processing system according to claim 1, wherein the analysis processing unit performs analysis and evaluation of the keyword while referring to the reaction information.

3. an evaluation information input unit for inputting information about the person to be met, information about the event, and evaluation information including an evaluation of the event that has ended; 2. The information processing system according to claim 1, wherein the analysis processing unit performs analysis and evaluation of the keyword while referring to the evaluation information.

4. the analysis processing unit counts the number of reactions of the face-to-face partner for each of the keywords in one of the events; 4. The information processing system according to claim 1, wherein the number of reactions can include positive or negative identification information.

5. Data on the number of reactions for each keyword in each of the events is accumulated, a statistical analysis unit that learns accumulated data and statistically analyzes and evaluates the keywords; The information processing system according to claim 4 , wherein the number of reactions can include positive or negative identification information.

6. the analysis processing unit extracts, as keyword candidates for the subsequent event, phrases other than the keyword to which the face-to-face person has reacted, which are included in the spoken text; 4. The information processing system according to claim 1, wherein a notification unit notifies the extracted keyword candidates.

7. The analysis processing unit extracts speech factors that are words or expressions that are factors that affect the reaction of the person being met, 4. The information processing system according to claim 1, wherein a notification unit notifies the extracted utterance factors.

8. a feedback forming unit that forms a proposal for improving the next utterance based on the extracted utterance factors; The information processing system according to claim 7 , wherein the notification unit notifies the user of the speech improvement suggestion.

9. The information processing system according to any one of claims 1 to 3, characterized in that the analysis processing unit determines whether the user is indifferent or listening attentively based on the speech text acquired before and after a part in which the operation signal has not been transmitted for a certain period of time.

10. 4. The information processing system according to claim 1, further comprising a plurality of the reaction input units, wherein the plurality of operators operate the reaction input units at arbitrary timings.

11. 4. The information processing system according to claim 1, wherein the reaction input unit has an operation switch that is operated by the operator's hand.

Images