Information processing method, information processing program, and information processing device

Abstract

To provide an information processing method, an information processing program, and an information processing device for asking a question considering a level of an evaluator and improving accuracy of level determination of the evaluator.SOLUTION: An information processing device 1 includes: utterance control means 100 for performing utterance control on a question in a level among a plurality of levels determined in advance for a user 4 who is an interlocutor; voice recognition means 103 for performing voice recognition of an answer of the user 4 to the question; breakdown detection means 106 for detecting breakdown of the user 4 in the answer; and capability determination means 105 for determining the level of the user 4 at least based on the level in which breakdown has been detected.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing method, an information processing program, and an information processing device. [Background technology]

[0002] As a conventional technique, an information processing method has been proposed in which an interview is scored by evaluating responses of an interviewee to questions (see, for example, Patent Document 1).

[0003] The information processing method disclosed in Patent Document 1 selects questions from prepared questions, outputs them to the interviewee, analyzes the responses to the questions using video and audio, extracts content features and communication features of the responses, and evaluates the responses. The information processing program also uses the evaluation of the responses to select questions, and tally up the evaluations of all the responses to evaluate the interview. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Pat. No. 1,067,188 Summary of the Invention [Problem to be solved by the invention]

[0005] According to the above information processing method, questions are selected according to the evaluation of each response, but because the questions are predetermined according to the development of the conversation, it is not possible to adaptively select questions of an appropriate level based on the successive evaluations of the interviewees. Also, there is no provision for a means of verifying the evaluation results, and therefore it is not always possible to make an accurate judgment.

[0006] An object of the present invention is to provide an information processing method, an information processing program, and an information processing device that ask questions taking into account the level of the evaluator and improve the accuracy of the level determination of the evaluator. [Means for solving the problem]

[0007] In order to achieve the above object, one aspect of the present invention provides the following information processing method, information processing program, and information processing device.

[0008] [1] On the computer, a speech control step of speaking a question of one of a plurality of predetermined levels to a conversation partner; a detection step for detecting a breakdown of the interlocutor in the response; and determining a level of the interlocutor based on at least the level at which the breakdown is detected. [2] The information processing method according to [1], wherein the determining step determines the level of the interlocutor for each predetermined unit of the answer and utters a question at a level higher than the determined level. [3] The information processing method according to [1] or [2], wherein, when a breakdown is detected, a question of a level other than the level at which the breakdown was detected is spoken out of the plurality of levels of questions. [4] The determining step determines the level of the interlocutor for each predetermined unit of the answer regardless of the detection of the breakdown, The information processing method according to any one of [1] to [3], wherein the speech control step utters a question of a level determined in the determination step. [5] The information processing method according to any one of [1] to [4], further comprising a display control step of controlling the display of an avatar that operates in conjunction with the speech in the speech control step. [6] The information processing method according to [5], further comprising executing an action imparting step of imparting to the avatar an action of listening attentively to the response. [7] The information processing method according to [5], further comprising executing an action imparting step of imparting an action to the avatar in response to the answer. [8] Computers, a speech control means for controlling the speech of a question of one of a plurality of predetermined levels to a conversation partner; detection means for detecting a breakdown of the interlocutor in the reply; an information processing program that further functions as a determination means for determining a level of the interlocutor based on at least the level at which the breakdown is detected; [9] A speech control means for controlling the speech of a question of one of a plurality of predetermined levels to a conversation partner; detection means for detecting a breakdown of the interlocutor in the reply; and determining means for determining a level of the interlocutor based on at least the level at which the breakdown is detected.

[0009] According to the present invention, questions can be asked taking into consideration the level of the evaluator, and the accuracy of the level determination of the evaluator can be improved. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a configuration of an information processing device according to an embodiment. [Diagram 3] FIG. 3 is a block diagram illustrating a configuration example of a terminal according to an embodiment. [Figure 4] FIG. 4 is a schematic diagram showing an example of a screen displayed on the display of the terminal. [Diagram 5] FIG. 5 is a schematic diagram showing questions posed by the speech control means of the information processing device and the contents of the user's answers. [Figure 6] FIG. 6 is a graph showing the relationship between the level of the capability judgment of an information processing device and the passage of time. [Figure 7] FIG. 7 is a flowchart showing an operation example of the information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] [Embodiment Mode] (Configuration of Information Processing System) FIG. 1 is a schematic diagram illustrating an example of a configuration of an information processing system according to an embodiment.

[0012] As an example, this information processing system is configured by connecting an information processing device 1, which judges English conversation ability as a foreign language by interacting with a user 4 as an interlocutor, and a terminal 2, which plays back information generated by the information processing device 1 and receives a response from the user 4, so that they can communicate with each other via a network 3. Note that the information processing device 1 and the terminal 2 may be configured as an integrated device, in which case the network 3 can be omitted.

[0013] The information processing device 1 is a server-type information processing device that operates in response to requests from a user 4 via a terminal 2, and is equipped with electronic components such as a CPU (Central Processing Unit) having the function of processing information within the main body, and a flash memory having the function of storing information.

[0014] The terminal 2 is a terminal device such as a PC (Personal Computer), tablet, or smartphone, and includes a CPU and flash memory having functions for processing information within the main body, as well as other electronic components such as a speaker, microphone, and camera.

[0015] The network 3 is a communication network capable of high-speed communication, and is, for example, a wired or wireless communication network such as the Internet, an intranet, or a LAN (Local Area Network).

[0016] In the above configuration, as an example, in order to determine English conversation ability, the information processing device 1 displays an avatar on the display unit of the terminal 2 via the network 3, causes the avatar to speak a question, and gives the avatar an action. The information processing device 1 collects the answer to the question of the user 4 via the microphone of the terminal 2, and performs voice recognition on the content to determine the conversation ability for the question. The ability is determined for each predetermined group of conversation, such as for each topic or question, and the level of the determination result is appropriately fed back to the selection of the next topic or question. In addition, when the level is determined, a question higher than the determined level is deliberately asked in order to make the determined level certain, and if the user 4 answers the question with a lack of understanding or non-fluency, the previously determined level is determined to be correct, and the determination result is output. The configuration will be described in more detail below.

[0017] (Configuration of information processing device) FIG. 2 is a block diagram showing an example of a configuration of the information processing device 1 according to the embodiment.

[0018] The information processing device 1 includes a control unit 10 that is composed of a CPU and the like and controls each part and executes various programs, a memory unit 11 that is composed of a storage medium such as a flash memory and stores information, and a communication unit 12 that communicates with the outside via a network 3.

[0019] The control unit 10 functions as a speech control means 100, a display control means 101, an action granting means 102, a voice recognition means 103, a video recognition means 104, an ability determination means 105, a breakdown detection means 106, etc., by executing an ability determination program 110 as an information processing program described later.

[0020] The speech control means 100 controls voice utterances in the terminal 2. The speech control means 100 selects and utters a question according to the level of the user 4 from question information 111 including questions at multiple levels prepared in advance. Note that the voice utterances include not only questions based on the question information 111, but also greetings to the user 4 in response to the question, utterances by the user 4, and interjections in response to the answer.

[0021] The display control means 101 displays the avatar on the terminal 2 while providing the avatar with a motion, using avatar information 112 that defines the image of the avatar and motion information 113 that defines the motion of the avatar.

[0022] The action imparting means 102 imparts an action to the avatar displayed on the terminal 2 by the display control means 101, by referring to the action information 113 in accordance with the actions of the speech control means 100, the voice recognition means 103, and the video recognition means 104. For example, the action of speaking is imparted to the avatar in accordance with the action of the speech control means 100. All the actions in the action information 113 are linked to, for example, a certain speech sentence, and the action having the closest inter-sentence distance to the speech sentence of the action to be generated is selected. In addition, the action of listening or the like is imparted to the avatar in accordance with the action of the voice recognition means 103, and the action of reacting to the action of the user 4 is imparted to the avatar in accordance with the action of the video recognition means 104. The action imparting means 102 may impart an action in accordance with the actions of the ability determination means 105 and the breakdown detection means 106.

[0023] The voice recognition means 103 recognizes the voice accompanying the answer of the user 4 to the question of the speech control means 100 received via the terminal 2, and stores it in the storage unit 11 as answer information 114. The voice recognition means 103 may further process the recognized voice for language understanding, or the language understanding may be performed by the ability determination means 105. For example, means such as GMM-HMM, DNN-HMM, End-to-End DNN, etc. can be used for voice recognition, and means such as keyword extraction, decision tree, neural network, etc. can be used for language understanding.

[0024] The video recognition means 104 recognizes video including the behavior, gaze, gestures, etc. accompanying the answer of the user 4 received via the terminal 2, and stores it in the storage unit 11 as answer information 114. For example, a neural network or other means can be used for the video recognition.

[0025] The ability assessment means 105 assesses the ability of the user 4 based on the contents of the answer information 114, and stores the result of assessment in the storage unit 11 as assessment result information 115. For example, the Common European Framework of Reference for Languages ​​(CEFR) is used as the assessment standard. Specific levels are A1, A2, B1, B2, C1, and C2, and the levels increase in this order. Ability assessment can be performed using means such as linear regression, decision trees, and neural networks.

[0026] Furthermore, when the breakdown detection means 106 detects a breakdown, the ability determination means 105 feeds back the provisionally determined level to the speech control means 100, and causes the speech control means 100 to select and speak a question according to the level. The feedback operation will be described later in detail. Furthermore, the ability determination means 105 comprehensively determines the level of the user 4 after answers to a plurality of questions are obtained.

[0027] When a lack of understanding, disfluency, or a decrease in grammatical accuracy is detected in the answer of the user 4 recognized by the voice recognition means 103 and the video recognition means 104, the breakdown detection means 106 detects this as a breakdown. Note that in this embodiment, the breakdown is defined as above on the premise of a case of English conversation ability assessment, but in the case of a different assessment content or a different application, the definition of the breakdown may be changed according to the assessment content or application. If it is defined in a higher-level concept, a breakdown is when the content of the response of the user 4 to the content transmitted by the information processing device 1 deviates from the distribution of normal responses, resulting in an interactive breakdown.

[0028] The breakdown detection means 106 detects non-comprehension from a state of requesting repetition of an utterance, or a state of confusion or silence while pondering to understand, etc. Regarding the latter, when the voice recognition means 103 or the video recognition means 104 detects eight characteristic actions that occur when a user does not understand, such as looking away, bringing the face closer, blinking frequently, turning to the side, moving the eyes vigorously, moving the head vigorously, becoming silent, and reducing the volume of speech, it is determined that the user is confused or silent while pondering to understand.

[0029] In addition, the breakdown detection means 106 detects disfluency from a state where speech production stagnates due to poor recall of language knowledge such as vocabulary, grammar, and pronunciation. In particular, it determines that speech production stagnates when silence occurs in the middle of a sentence or clause. This is because silence at the beginning or end of a sentence or clause is considered to be caused by content recall, such as a lack of background knowledge or a failure to plan an effective discourse structure.

[0030] The storage unit 11 stores an ability judgment program 110 for causing the control unit 10 to operate as each of the above-mentioned means 100-106, question information 111, avatar information 112, action information 113, answer information 114, judgment result information 115, and the like.

[0031] (Terminal 2 Configuration) FIG. 3 is a block diagram showing an example of a configuration of the terminal 2 according to the embodiment.

[0032] The terminal 2 includes a control unit 20 configured with a CPU and the like to control each unit and execute various programs, a storage unit 21 configured with a storage medium such as a flash memory and storing information, a communication unit 22 that communicates with the outside via the network 3, a microphone 23 that converts the input voice of the user 4 into an electrical signal, a speaker 24 that converts the signal input from the control unit 20 into voice and outputs it, a camera 25 that captures an image of the user 4 and outputs a video signal, and a display 26 such as an LCD that displays images, videos, characters, etc. In addition, the terminal 2 includes an operation unit (keyboard, mouse, track pad, touch panel) (not shown) that accepts operations from the user 4.

[0033] Although the information processing device 1 and the terminal 2 are described as separate devices, some or all of the means of the information processing device 1 may be provided in the terminal 2, and the design may be changed as appropriate without departing from the spirit of the invention.

[0034] (Operation of information processing device) Next, the operation of this embodiment will be explained by dividing it into (1) basic operation, (2) introduction operation, (3) level check operation, and (4) thrusting operation.

[0035] (1) Basic operation First, the user 4 operates the terminal 2 to request a judgment of, for example, English conversation ability. The terminal 2 communicates with the information processing device 1 via the network 3, and requests the information processing device 1 to judge the English conversation ability.

[0036] When the information processing device 1 receives a request for determining English conversation ability from the terminal 2, it issues instructions to the speech control means 100, the display control means 101, and the action granting means 102 to display an avatar on the display 26 of the terminal 2, as shown in FIG.

[0037] FIG. 4 is a schematic diagram showing an example of a screen displayed on the display 26 of the terminal 2. As shown in FIG.

[0038] The screen 101a includes an area 101a for displaying an image of the user 4 captured by the camera 25 of the terminal 2. 1and an area 101a for displaying an avatar. 2 Region 101a 1 is displayed mainly for the reference of the user 4, but may not be displayed.

[0039] area 101a 2 The avatar displayed on the screen is given a motion by the display control means 101 and the motion giving means 102, and speech is output from the speaker 24 by the speech control means 100.

[0040] In addition, the user 4's reaction (voice, facial expression, movement, etc.) to the screen 101a displayed on the display 26 and the sound output from the speaker 24 is input to the information processing device 1 via the microphone 23 and camera 25 of the terminal 2, and recognized by the voice recognition means 103 and video recognition means 104 of the information processing device 1, respectively.

[0041] Before describing the following "(2) Introduction Operation", "(3) Level Check Operation", and "(4) Push-up Operation", the basic operation of each means of the information processing device 1 will be described.

[0042] The speech control means 100 of the information processing device 1 controls the voice utterance of an avatar in the terminal 2. The speech control means 100 mainly selects a question from question information 111 including questions at multiple levels prepared in advance according to the level of the user 4 and utters it.

[0043] Furthermore, the display control means 101 of the information processing device 1 displays an avatar on the terminal 2 using avatar information 112 that defines an image of the avatar and action information 113 that defines the action of the avatar, and the action imparting means 102 of the information processing device 1 imparts an action to the avatar displayed on the terminal 2 by the display control means 101 by referring to the action information 113 in accordance with the operations of the speech control means 100, the voice recognition means 103, and the video recognition means 104. In accordance with the operation of the voice recognition means 103, an action such as a listening gesture (listening action) is imparted to the avatar, and in accordance with the operation of the video recognition means 104, an action (reaction) that reacts to the action of the user 4 is imparted to the avatar. These actions encourage self-disclosure by the user 4.

[0044] Furthermore, the voice recognition means 103 of the information processing device 1 recognizes the voice accompanying the answer of the user 4 to the question or the like of the speech control means 100 accepted via the terminal 2, and stores it in the storage unit 11 as answer information 114. The voice recognition means 103 further performs language understanding processing on the recognized voice.

[0045] Furthermore, the image recognition means 104 of the information processing device 1 recognizes an image including the behavior, gaze, gestures, etc. accompanying the answer of the user 4 received via the terminal 2 , and stores it in the storage unit 11 as answer information 114 .

[0046] Furthermore, the ability judging means 105 of the information processing device 1 judges the ability of the user 4 based on the contents of the answer information 114 and stores the judged ability in the storage unit 11 as judgment result information 115 .

[0047] Furthermore, when a lack of understanding or disfluency is detected in the answer of the user 4 recognized by the voice recognition means 103 and the video recognition means 104, the breakdown detection means 106 of the information processing device 1 detects this as a breakdown.

[0048] Hereinafter, a specific operation of the information processing device 1 for determining the English conversation ability of the user 4 while displaying the avatar will be described below.

[0049] (2) Introduction operation Fig. 5 is a schematic diagram showing an example of questions (S1-S10) by the speech control means 100 of the information processing device 1 and answers (U1-U9) from the user 4. Phase 1 shows the questions and answers during the introduction operation, Phase 2 shows the questions and answers during the level check operation, and Phase 3 shows the questions and answers during the push-up operation. Fig. 6 is a graph showing the relationship between the level of ability judgment of the information processing device 1 and the passage of time. Fig. 7 is a flowchart showing an example of the operation of the information processing device 1.

[0050] First, the information processing device 1 has a relatively simple conversation with the user 4, such as greetings and small talk, to ease the user's tension and perform introductory actions to grasp the user's general level. The information processing device 1 assigns actions to the avatar using the speech control means 100, display control means 101, and action assignment means 102, and has the avatar speak a question of level A1, for example (S100). The information processing device 1 then recognizes the user's 4's answer, facial expressions, and actions using the voice recognition means 103 and video recognition means 104, and judges the ability of the user 4 using the ability judgment means 105 (S101) (Phase 1 in Figure 6).

[0051] Specifically, the speech control means 100, the display control means 101, and the action providing means 102 cause the avatar to ask a topic introduction question such as "What is your favorite season?" (S1), as shown in phase 1 of FIG.

[0052] In response to this, for example, if the user 4 replies, "My favorite season is winter" (U1), the voice recognition means 103 and the video recognition means 104 recognize the user 4's reply, as well as his / her facial expressions and actions. At the same time, the ability determination means 105 determines the ability of the user 4. For example, it is assumed that the ability is determined to be at the CEFR A2 level.

[0053] Next, in order to confirm that user 4 can definitely hold an A2 level conversation, the speech control means 100, the display control means 101, and the action providing means 102 ask an additional question such as "Are there any activities you like to do in winter?" (S2).

[0054] In response to this, for example, if user 4 responds with "Uh ... ski and make snowmen" (U2), the user will then make a further request for continuance with the content "That sounds like a lot of fun. Could you tell me more about it?" (S3). Here, it is assumed that user 4 responded with "I like skiing with family. I go every year." (U3).

[0055] The number of times the additional questions are asked may be zero, one, or multiple, as long as the ability determination means 105 can confirm the determination result.

[0056] (3) Level check operation Next, as shown in phase 1 of FIG. 5, the speech control means 100, the display control means 101, and the action granting means 102 cause the avatar to ask a topic-introducing question (S102) according to the A2 level of ability determined in step S101, for example, "Alright. What did you eat for breakfast this morning?" (S4).

[0057] In response to this, for example, if the user 4 responds with "I ate uh... Sandwich it is chicken and salad it is very delicious." (U4), the voice recognition means 103 and the video recognition means 104 recognize the user 4's response, facial expressions, and actions. At the same time, the ability determination means 105 determines the ability of the user 4 (S103) (phase 2 in FIG. 6). For example, it is assumed that the ability is determined to be at the CEFR A2 level. Although steps S102 and S103 may be performed once, in order to ensure that the user 4 can reliably hold a conversation at the A2 level, in this embodiment, additional questions are asked multiple times to determine the ability. Note that the ability determination may be performed for each question, or may be performed for each predetermined unit such as multiple questions or each topic.

[0058] For example, the speech control means 100, the display control means 101, and the action providing means 102 ask an additional question such as "Do you usually eat breakfast?" (S5).

[0059] In response to this, if user 4 answers, for example, "Uh yes I always eat breakfast" (U5), the system asks a follow-up question such as "I see what time do you usually eat breakfast" (S6). Here, it is assumed that user 4 answers, "Uh seven AM I wake up and I go to kitchen and I eat breakfast" (U6).

[0060] In the above conversation, the voice recognition means 103 and the video recognition means 104 recognize the answers, facial expressions and actions of the user 4, and as a result the answers are given without any problems, the ability determination means 105 provisionally determines the ability of the user 4 to be at the CEFR A2 level (S103). Strictly speaking, since the level has not been finalized, it is determined to be at or above the CEFR A2 level (i.e., the lower limit is A2).

[0061] (4) Push-up movement Next, in order to verify that the result of the level check is correct, the speech control means 100, the display control means 101, and the action providing means 102 raise the level from the level of the result of the determination in step S103 and ask a topic introduction question at the CEFR B1 level (S104). Specifically, as shown in phase 3 of FIG. 5, the avatar asks a question such as "Have you ever been to a foreign country?" (S7). Note that the level may be raised by 2 or more instead of 1, the degree of the level increase may be changed depending on the answer, and the level may be lowered depending on the purpose.

[0062] In response to this, if user 4 responds, for example, with "Uh no. I never go to foreign country" (U7), the speech recognition means 103 and video recognition means 104 recognize the response, as well as the facial expressions and actions of user 4. Since the breakdown detection means 106 does not detect any lack of understanding or non-fluency in the recognized response of user 4 (S105; No), it would normally ask questions at a higher level (S106), but in this embodiment, additional questions are asked multiple times at the B1 level to ensure that user 4 can reliably carry out a B1 level conversation.

[0063] Next, in order to confirm that user 4 can certainly hold a B1 level conversation, the speech control means 100, the display control means 101 and the action granting means 102 ask an additional question such as "Ok. Which country would you like to visit in the future?" (S8).

[0064] If user 4 responds, for example, with "I would like to visit ... Singapore." (U8), the breakdown detection means 106 detects disfluency, but makes a follow-up request such as "Why is that?" (S8) to verify whether this is correct.

[0065] In response to this, if user 4 responds, for example, with "Because I want visit... I like go to nice... ah nice..." (U9), the breakdown detection means 106 detects disfluency in the recognized response of user 4 and therefore determines that a breakdown has been detected (S105; Yes), and responds with an utterance such as "That's ok. Let's move on." (S10).

[0066] When the breakdown detection means 106 detects a breakdown (S105; Yes), the ability determination means 105 determines that the ability of the user 4 is CEFR level A2, not level B1 (S107) (phase 3 in FIG. 6). In addition, the ability determination means 105 outputs the determination result to the user 4, or to an arbitrary administrator or the like other than the user 4 (S108).

[0067] On the other hand, if the breakdown detection means 106 does not detect a breakdown in response to multiple questions at level B1 (S105; No), the judgment result in step S103 is corrected and the level is raised to B2 (S106), and steps S102 to S106 are repeated to finally judge the ability (S107).

[0068] Furthermore, the ability determination means 105 may feed back the level determined in step S107 to the speech control means 100 as a provisional determination result, and cause the speech control means 100 to select and speak a question corresponding to the level again (S102). In this case, the ability determination means 105 may repeat steps S102 to S107 multiple times, and then comprehensively determine the level of the user 4. That is, the ability determination means 105 may determine the level as A2 after repeating "(3) Level check action" and "(4) Push-up action" multiple times (phase 2 and phase 3 for the second time in FIG. 6). Furthermore, a cool-down phase may be further provided after the ability determination is completed (Cool down after phase 3 in FIG. 6).

[0069] (Effects of the embodiment) According to the above-described embodiment, the ability of user 4 is judged by the ability judgment means 105, and questions of a higher level are asked to verify the judged ability, and if the breakdown detection means 106 detects a breakdown in the answers to the questions, it is judged that the previously judged ability is likely to be correct, rather than the raised level, and if no breakdown is detected, the level is further raised and the operation is continued to judge the ability of user 4. Therefore, questions can be asked that take into account the level of user 4 (person being assessed) during the ability judgment, and by trying other levels, the level can be judged accurately and with confidence.

[0070] In addition, an introductory action is adopted in which relatively simple conversation such as greetings and small talk is conducted before the ability assessment is performed, so that the user 4 can be warmed up and smoothly transitioned to the subsequent ability assessment action.

[0071] In addition, the avatar is equipped with gestures (listening, reactions) when speaking and when receiving a response, allowing for more natural questioning and answering, which in turn encourages self-disclosure from the user and elicits information.

[0072] In addition, the level is provisionally determined for each conversation, topic, etc., and questions are selected according to the level. This eliminates the need to ask all questions at random, as would be the case if the level were not provisionally determined, thereby improving efficiency in terms of the time required for questions.

[0073] [Other embodiments] The present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the spirit of the present invention.

[0074] It may be used for judging abilities other than English conversation, such as for job interviews, mental state assessment, presentation ability assessment, etc. Furthermore, it can be applied to uses other than ability assessment, such as menu selection in restaurants and tourist spot selection in tourist guides. In this case, specifically, the ability assessment means 105 is replaced with preference assessment means, and the user's preferences are determined from the user's remarks, gestures, facial expressions, etc., and the level is replaced with menus and tourist spots.

[0075] In the above embodiment, the functions of the means 100 to 106 of the control unit 10 are realized by a program, but all or part of the means may be realized by hardware such as ASIC. Also, the programs used in the above embodiment may be stored in a recording medium such as a CD-ROM and provided. Also, the steps described in the above embodiment may be replaced, deleted, added, etc., without changing the gist of the present invention.

[0076] Moreover, the functions of the means 100-106 of the control unit 10 of the information processing device 1 do not necessarily have to be realized on the information processing device 1, and may be realized on the terminal 2, as long as the gist of the present invention is not changed. Similarly, the functions of the terminal 2 do not necessarily have to be realized on the terminal 2, and may be realized on the information processing device 1, as long as the gist of the present invention is not changed. [Explanation of symbols]

[0077] 1: Information processing equipment 2: Terminal 3: Network 4: User 10: Control section 11: Storage section 12: Communications Department 20: Control section 21: Storage section 22: Communications Department 23:Mike 24: Speaker 25: Camera 26: Display 100: Speech control means 101: Display control means 102: Operation providing means 103: Voice recognition means 104: Image recognition means 105: Ability judgment means 106: Breakdown detection means 110: Ability Assessment Program 111: Question information 112: Avatar information 113: Operation information 114:Answer information 115: Judgment result information

Claims

1. a speech control step of selecting and speaking to the interlocutor a question of one level from a plurality of predetermined levels; a detection step of detecting a breakdown in the interlocutor's response to the question, such as a lack of understanding, disfluency, or a decrease in grammatical accuracy; and determining the level of the interlocutor based on at least the level at which the breakdown is detected, The detection step detects the non-understanding from a state in which the interlocutor requests repetition of an utterance or is silent while thinking deeply about how to understand.

2. A second determination step of determining a level of the interlocutor for each unit of the answer to the question; 2. The information processing method according to claim 1, further comprising a second speech control step of uttering a question of a level higher than the breakdown level when the breakdown is not detected in the detection step.

3. If a breakdown is detected in the detection step, The determining step feeds back the provisionally determined level to the speech control step, 3. The information processing method according to claim 1, wherein the speech control step selects and speaks a question corresponding to the provisional judgment level.

4. 4. The information processing method according to claim 1, wherein the speech control step selects and speaks a greeting or small talk as the first level question in the introduction operation.

5. Computer, a speech control means for selecting and speaking to a conversation partner a question of one of a plurality of predetermined levels; a detection means for detecting a breakdown in the interlocutor's response to the question, such as a lack of understanding, disfluency, or a decrease in grammatical accuracy; functioning as a determination means for determining the level of the interlocutor based on at least the level at which the breakdown is detected; The detection means is an information processing program that detects the non-understanding from a state in which the interlocutor requests repetition of an utterance or is silent while thinking hard about how to understand.

6. The determining means determines the level of the interlocutor for each unit of an answer to the question, 6. The information processing program according to claim 5, wherein the speech control means, when the detection means does not detect the breakdown, speaks a question of a level higher than the level.

7. a speech control means for selecting and speaking to a conversation partner a question of one of a plurality of predetermined levels; a detection means for detecting a breakdown in the interlocutor's response to the question, such as a lack of understanding, disfluency, or a decrease in grammatical accuracy; a determination means for determining the level of the interlocutor based on at least the level at which the breakdown is detected, The detection means detects the non-understanding from a state in which the interlocutor requests repetition of an utterance or remains silent while thinking deeply about how to understand.

8. The determining means determines the level of the interlocutor for each unit of an answer to the question, 8. The information processing apparatus according to claim 7, wherein the speech control means, when the detection means does not detect the breakdown, speaks a question at a level higher than the level.

Images