Interview support device, interview support method, and computer program

Abstract

To provide an interview support device, an interview support method, and a computer program which improve an interview skill of an interviewer.SOLUTION: An interview support system 1e comprises an interviewee device, an interviewer device, an interview support device, and a communication line. An interview support device 4e includes: an analysis unit 424e which analyzes a speech amount of an interviewer and a speech amount of an interviewee; and a feedback processing unit 426e which gives at least one of advice and warning in interview to the interviewer according to an analysis result of the speech amount of the interviewer and the speech amount of the interviewee. The analysis unit analyzes a cumulative result of speech time length of the interviewer in a predetermined period of time as the speech amount of the interviewer and analyzes a cumulative result of speech time length of the interviewee in the predetermined period of time as the speech amount of the interviewee.SELECTED DRAWING: Figure 29

Description

[Technical field]

[0001] The present invention relates to an interview support device, an interview support method, and a computer program. [Background technology]

[0002] An information processing device that supports an interview so that the purpose of the interview can be easily achieved is disclosed in Patent Document 1. In a company, a superior (interviewer) may interview a subordinate (interviewee) to encourage the subordinate to change his / her behavior. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-026657 Summary of the Invention [Problem to be solved by the invention]

[0004] In order for an interviewer to encourage the interviewee to change their behavior, the interviewer needs to have interview skills. Interview skills are, for example, communication skills during an interview. However, even though it is required that an interviewer has interview skills, there are cases where the interviewer's interview skills cannot be improved.

[0005] In view of the above circumstances, an object of the present invention is to provide an interview support device, an interview support method, and a computer program capable of improving the interview skills of an interviewee. [Means for solving the problem]

[0006] One aspect of the present invention is a method for analyzing an amount of speech of an interviewer and an amount of speech of an interviewee, The interview support device is equipped with a feedback processing unit that provides the interviewer with at least one of advice and a warning during the interview depending on the analysis results of the amount of speech of the interviewer and the amount of speech of the interviewee.

[0007] One aspect of the present invention is the above-mentioned interview support device, wherein the analysis unit analyzes the cumulative result of the speaking length of the interviewer over a specified period of time as the speaking amount of the interviewer, and analyzes the cumulative result of the speaking length of the interviewee over the specified period of time as the speaking amount of the interviewee.

[0008] One aspect of the present invention is the above-mentioned interview support device, wherein the analysis unit derives the speaking duration of the interviewer by subtracting a predetermined first margin duration from the voice duration of the interviewer in the specified period, and derives the speaking duration of the interviewee by subtracting a predetermined second margin duration from the voice duration of the interviewee in the specified period.

[0009] One aspect of the present invention is the above-mentioned interview support device, wherein the feedback processing unit derives a ratio of the amount of speech of the interviewer to the sum of the amount of speech of the interviewee and the amount of speech of the interviewee, and if the ratio is greater than or equal to a predetermined threshold, provides at least one of the advice and the warning to the interviewer.

[0010] One aspect of the present invention is an interview support method executed by the above-mentioned interview support device, which includes an analysis step of analyzing the amount of speech of the interviewer and the amount of speech of the interviewee, and a feedback processing step of providing at least one of advice and a warning to the interviewer during the interview depending on the analysis results of the amount of speech of the interviewer and the amount of speech of the interviewee.

[0011] One aspect of the present invention is a computer program for causing a computer to function as the above-mentioned interview support device. Effect of the Invention

[0012] The present invention makes it possible to improve the interview skills of an interviewer. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an interview support system in a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of the operation of the interview support system in the first embodiment. [Diagram 3] FIG. 11 is a diagram showing an example of image analysis regarding a superior's interview skills in the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of an image for selecting a personality type in the first embodiment. [Diagram 5] FIG. 2 is a diagram showing an example of an image for an interview in the first embodiment. [Figure 6] FIG. 4 is a diagram showing an example of an image for setting a target item in the first embodiment. [Figure 7] A figure showing an example of an image for feedback of interview skill scores and advice in the first embodiment. [Figure 8] FIG. 11 is a diagram showing a first example of an input image for a pre-questionnaire in the first embodiment. [Figure 9] FIG. 11 is a diagram showing a second example of an input image for a type diagnosis questionnaire in the first embodiment. [Figure 10] FIG. 11 is a diagram showing an example of an image for notifying a result of a personality type diagnosis in the first embodiment. [Figure 11] FIG. 11 is a diagram showing an example of an image for managing the progress of a target item in the first embodiment. [Figure 12] FIG. 11 is a diagram showing an example of an input image for a post-event questionnaire in the first embodiment. [Figure 13] FIG. 2 is a diagram showing a first example of the operation of the interview support device in the first embodiment. [Figure 14] FIG. 11 is a diagram showing a second example of the operation of the interview support device in the first embodiment. [Figure 15] FIG. 11 is a diagram illustrating an example of the configuration of an interview support system in a second embodiment. [Figure 16]FIG. 11 is a diagram showing an example of a voice analysis regarding a superior's interview skills in the second embodiment. [Figure 17] A figure showing an example of an image for feedback of interview skill scores and advice in the second embodiment. [Figure 18] FIG. 11 is a diagram illustrating a first example of the operation of the interview support device in the second embodiment. [Figure 19] FIG. 11 is a diagram showing a second example of the operation of the interview support device in the second embodiment. [Figure 20] FIG. 13 is a diagram illustrating an example of the configuration of an interview support system in a third embodiment. [Figure 21] FIG. 13 is a diagram illustrating a first example of the operation of the interview support device in the third embodiment. [Figure 22] FIG. 13 is a diagram showing a second example of the operation of the interview support device in the third embodiment. [Diagram 23] FIG. 13 is a diagram illustrating an example of the configuration of an interview support system in a fourth embodiment. [Figure 24] FIG. 13 is a diagram showing an example of rule-based conditional branching of a scenario in the fourth embodiment. [Diagram 25] A figure showing an example of the operation of the interview support system for a training interview associated with a real interview in the fourth embodiment. [Figure 26] FIG. 13 is a diagram showing an example of an image for an interview in the fourth embodiment. [Figure 27] FIG. 13 is a diagram illustrating an example of machine learning of a learning model in a modified example of the fourth embodiment. [Figure 28] FIG. 13 is a diagram showing an example of model-based scene transition of a scenario in a modified example of the fourth embodiment. [Figure 29] A diagram showing an example of the configuration of an interview support system in a fifth embodiment. [Diagram 30] FIG. 13 is a diagram showing an example of the duration of an audio in the fifth embodiment. [Diagram 31] FIG. 13 is a diagram showing an example of an image for an interview in the fifth embodiment. [Diagram 32] FIG. 13 is a diagram illustrating an example of the operation of the interview support device in the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings. (First embodiment) FIG. 1 is a diagram showing an example of the configuration of an interview support system 1a in the first embodiment. The interview support system 1a is a system that supports interviews. Interviews may be conducted one-to-one, one-to-many, or many-to-many. By supporting interviews with the interview support system 1a, it is possible to improve the interview skills of the person conducting the interview with the interviewee (interviewer).

[0015] The interview support system 1a is used, for example, for a one-on-one meeting between a superior (interviewer) and a subordinate (interviewee). In this case, the interview support system 1a supports the interview from before the interview to after the interview, thereby making it possible to improve the interview skills of the superior.

[0016] The interview support system 1a may be used, for example, for an interview (customer service) between a store clerk (interviewer) and a customer (interviewee). In this case, the interview support system 1a supports the interview from during the interview to after the interview, thereby making it possible to improve the interview skill (customer service skill) of the store clerk.

[0017] In the following, the interview support system is used, as an example, for a one-on-one meeting between a superior and a subordinate. The superior is the person who interviews the subordinate (interviewer) and gives advice, etc. to the subordinate (mentor). The subordinate is the person who is interviewed by the superior (interviewee) and receives advice, etc. from the superior (mentor). In addition, one or more work-related goals (promises) may be agreed upon between the superior and the subordinate during the interview.

[0018] The interview support system 1a includes an interviewee device 2, an interviewer device 3, an interview support device 4a, and a communication line 5. The communication line 5 is, for example, a communication line such as the Internet or an intranet.

[0019] The interview itself may be conducted online or offline (face-to-face). When the interview itself is conducted online, the interviewee device 2 and the interviewer device 3 may be placed at a distance greater than a predetermined distance. For example, the interviewee device 2 may be placed in a first building where a subordinate is located, and the interviewer device 3 may be placed in a second building where a superior is located.

[0020] When the interview itself is conducted offline (face-to-face), the interviewee device 2 and the interviewer device 3 may be placed close to each other within a predetermined distance. For example, the interviewee device 2 and the interviewer device 3 may be placed on a desk shared by a subordinate and a superior who are interviewed face-to-face for a one-on-one meeting. When the interview itself is conducted offline, the interview support system 1a only needs to include the interviewer device 3 used by the superior, and does not need to include the interviewee device 2 used by the subordinate.

[0021] Next, the interviewee device 2, the interviewer device 3, and the interview support device 4a will be described in detail. [Interviewee device] The interviewee device 2 is an information processing device such as a smartphone terminal, a tablet terminal, or a notebook computer. The interviewee device 2 is used by a subordinate (interviewee). The interviewee device 2 includes a camera 20, a microphone 21, an operation unit 22, a display unit 23, a speaker 24, a memory unit 25, a control unit 26, and a communication unit 27.

[0022] The camera 20 generates moving image data by capturing an image of the surroundings of the interviewee device 2. The moving image includes an image of a subordinate (interviewee). The camera 20 outputs the moving image data to the control unit .

[0023] The microphone 21 generates voice data by collecting sounds from around the interviewee device 2. The voice data includes the voices of the subordinates. The microphone 21 outputs the voice data to the control unit 26.

[0024] The operation unit 22 is an operation device such as a keyboard, a mouse, and a touch panel. The operation unit 22 is operated by a subordinate. The operation unit 22 outputs a signal corresponding to the operation to the control unit 26. The signal corresponding to the operation is, for example, a signal corresponding to an operation of entering a target item (promise) in an input field displayed on the display unit 23.

[0025] The display unit 23 is a display device such as a liquid crystal display and an organic EL (Electro Luminescence) display. When the operation unit 22 is a touch panel, the operation unit 22 and the display unit 23 may be integrated. The display unit 23 displays an image received from the communication unit 40 (relay unit) by the communication unit 27. For example, the display unit 23 displays an image (hereinafter referred to as an "interview image") used for an interview such as a one-on-one meeting. In the interview image, for example, a moving image of a superior and a moving image of a subordinate are displayed side by side. In the interview image, for example, one or more goals (promises) related to work may be displayed in response to input operations by the superior and the subordinate.

[0026] The speaker 24 outputs the voice received from the communication unit 40 (relay unit). For example, the speaker 24 outputs the voice of a superior in a one-on-one meeting.

[0027] The storage unit 25 stores identification information of the interviewee device 2. The identification information is, for example, an IP address (Internet Protocol Address). The storage unit 25 may store a computer program executed by the control unit .

[0028] The control unit 26 controls the operation of each functional unit of the interviewee device 2. For example, the control unit 26 uses the communication unit 27 to perform authentication processing for communication with the communication unit 37 via the communication unit 40. The control unit 26 causes the display unit 23 to display an image received by the communication unit 27 from the communication unit 40 through image processing. The control unit 26 causes the speaker 24 to output an audio received by the communication unit 27 from the communication unit 40 through audio processing.

[0029] The communication unit 27 executes communication with the communication unit 40. The communication unit 27 executes communication with the interviewer device 3 via the communication unit 40. For example, the communication unit 27 acquires video image data in which the face of the boss is captured from the communication unit 40. For example, the communication unit 27 acquires audio data in which the voice of the boss is recorded from the communication unit 40. The communication unit 27 may transmit identification information of the interviewee device 2 to the communication unit 40 in an authentication process for communication with the communication unit 40.

[0030] [Interviewer device] The interviewer device 3 is an information processing device such as a smartphone terminal, a tablet terminal, or a notebook computer. The interviewer device 3 is used by a superior (interviewer). The interviewer device 3 includes a camera 30, a microphone 31, an operation unit 32, a display unit 33, a speaker 34, a memory unit 35, a control unit 36, and a communication unit 37.

[0031] The camera 30 generates moving image data by capturing an image of the surroundings of the interviewer device 3. The moving image includes an image of the superior (interviewer). The camera 30 outputs the moving image data to the control unit .

[0032] The microphone 31 generates voice data by collecting sounds from around the interviewer device 3. The voice data includes the voice of the superior. The microphone 21 outputs the voice data to the control unit 36.

[0033] The operation unit 32 is an operation device such as a keyboard, a mouse, and a touch panel. The operation unit 32 is operated by a superior. The operation unit 32 outputs a signal corresponding to the operation to the control unit 36. The signal corresponding to the operation is, for example, a signal corresponding to an operation of selecting a radio button displayed on the display unit 33. The signal corresponding to the operation may be, for example, an operation of entering a target item (promise) in an input field displayed on the display unit 33.

[0034] The display unit 33 is a display device such as a liquid crystal display or an organic EL display. When the operation unit 32 is a touch panel, the operation unit 32 and the display unit 33 may be integrated. The display unit 33 displays an image received by the communication unit 37. For example, the display unit 33 displays an image for an interview.

[0035] The speaker 34 outputs the voice received from the communication unit 40 (relay unit). For example, the speaker 24 outputs the voice of a subordinate in a one-on-one meeting.

[0036] The storage unit 35 stores identification information of the interviewer device 3. The storage unit 35 may store a computer program executed by the control unit 36.

[0037] The control unit 36 ​​controls the operation of each functional unit of the interviewer device 3. For example, the control unit 36 ​​uses the communication unit 37 to perform authentication processing for communication with the communication unit 27 via the communication unit 40. The control unit 36 ​​causes the display unit 33 to display an image received by the communication unit 37 from the communication unit 40 through image processing. The control unit 36 ​​causes the speaker 34 to output an audio received by the communication unit 37 from the communication unit 40 through audio processing.

[0038] The communication unit 37 communicates with the interview support device 4a. The communication unit 37 communicates with the interviewee device 2 via the interview support device 4a. For example, the communication unit 37 acquires a video image of the subordinate's face from the communication unit 40. For example, the communication unit 37 acquires audio data in which the subordinate's voice is recorded from the communication unit 40. The communication unit 37 may transmit identification information of the interviewer device 3 to the communication unit 40.

[0039] [Interview support device] The interview support device 4a is an information processing device such as a server. The interview support device 4a may be realized by cloud computing. The interview support device 4a includes a communication unit 40, a storage unit 41, and a control unit 42a. The control unit 42a includes a selection acquisition unit 420, a scenario processing unit 421, an information output unit 422, a goal processing unit 423, an analysis unit 424a, a score processing unit 425, a feedback processing unit 426, a questionnaire processing unit 427, and a diagnosis unit 428.

[0040] Some or all of the functional units of the interview support device 4a are realized as software by a processor such as a CPU (Central Processing Unit) executing a program (computer program) stored in a storage unit 41 having a non-volatile storage medium (non-transient storage medium). The program may be recorded on a computer-readable storage medium. Examples of computer-readable storage media include portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), and storage devices such as hard disks built into a computer system.

[0041] Some or all of the functional units of the interview support device 4a may be realized using hardware including electronic circuits (electronic circuits or circuitry) using, for example, an LSI (Large Scale Integrated circuit), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array).

[0042] The communication unit 40 transmits communication data such as image data and audio data generated by the control unit 42a to at least one of the communication unit 27 and the communication unit 37. The communication unit 40 transmits communication data such as image data and audio data received from the communication unit 27 to the communication unit 37. The communication unit 40 transmits communication data such as image data and audio data received from the communication unit 37 to the communication unit 27. The communication unit 40 may temporarily store the communication data in the storage unit 41.

[0043] The memory unit 41 stores identification information of the interview support device 4a. The memory unit 41 may store a plurality of scenarios related to interviews. The memory unit 41 may store a computer program executed by the control unit 42a. The memory unit 41 may store a mathematical model. The memory unit 41 may store a model trained using a machine learning technique (hereinafter referred to as a "trained model"). This machine learning technique is not limited to a specific technique, but is, for example, supervised learning. The memory unit 41 may store a data table.

[0044] For example, at least one of the mathematical model, the data table, and the trained model may be used to derive a score for an interview skill. For example, at least one of the mathematical model, the data table, and the trained model may be used to derive advice for improving an interview skill.

[0045] The storage unit 41 may temporarily store image data received from the communication unit 27. The storage unit 41 may also temporarily store image data received from the communication unit 37. Note that the image data temporarily stored in the storage unit 41 is deleted from the storage unit 41 within a predetermined time period after it is stored.

[0046] The storage unit 41 may temporarily store the voice data received from the communication unit 27. The storage unit 41 may also temporarily store the voice data received from the communication unit 37. Note that the voice data temporarily stored in the storage unit 41 is deleted from the storage unit 41 within a predetermined time period after it is stored.

[0047] The control unit 42a controls the operation of each functional unit of the interview support device 4a. For example, the control unit 42a uses the communication unit 40 to perform authentication processing for communication with the communication unit 27. For example, the control unit 42a uses the communication unit 40 to perform authentication processing for communication with the communication unit 37.

[0048] The control unit 42a analyzes in real time the analysis target (e.g., facial expression, gaze, nod) in the image of the boss based on the result of image recognition for the image received from the communication unit 37. For example, the control unit 42a analyzes in real time the facial expression of the boss in the face area based on the recognition result of the face area of ​​the boss in the received image. The control unit 42a derives the score of the interview skill of the boss based on the analysis result of the analysis target in the image of the boss. The control unit 42a derives advice to be provided to the boss according to the analysis result of the analysis target in the image of the boss.

[0049] The control unit 42a may analyze the analysis target in the image of the subordinate in real time based on the result of image recognition for the image received from the communication unit 27. For example, the control unit 42a may analyze the facial expression of the subordinate in the face area in real time based on the recognition result of the facial area of ​​the subordinate in the received image. The control unit 42a may derive a score for the condition of the subordinate based on the analysis result of the analysis target in the image of the subordinate. The control unit 42a may derive advice to be provided to the superior according to the analysis result of the analysis target in the image of the subordinate.

[0050] The control unit 42a generates an image for an interview. The control unit 42a uses the communication unit 40 to cause the display unit 23 and the display unit 33 to display the image for an interview.

[0051] The control unit 42a generates a feedback image to be displayed on the display unit 33. That is, the control unit 42a generates a feedback image to be provided to the superior. The control unit 42a includes in the feedback image the score of the superior's interview skill. This score may be converted into a score out of a maximum score of 100, for example. The control unit 42a includes in the feedback image a character string representing advice to be provided to the superior. The control unit 42a uses the communication unit 40 to cause the display unit 33 to display the feedback image.

[0052] The control unit 42a may generate at least one of an image of a questionnaire before the interview (hereinafter referred to as a "pre-interview questionnaire"), an image of a questionnaire after the interview (hereinafter referred to as a "post-interview questionnaire"), and a questionnaire for diagnosing a personality type (hereinafter referred to as a "type diagnostic questionnaire"). The control unit 42a may use the communication unit 40 to cause the display unit 23 and the display unit 33 to display an image of the pre-interview questionnaire before the actual interview. The control unit 42a may use the communication unit 40 to cause the display unit 23 and the display unit 33 to display an image of the type diagnostic questionnaire before the actual interview. The control unit 42a may use the communication unit 40 to cause the display unit 33 to display an image of the post-interview questionnaire after the interview.

[0053] In this way, the control unit 42a supports the actual interview during the actual interview, thereby making it possible to improve the interview skills of the superior after the actual interview.

[0054] A training interview (simulation) may be conducted before the actual interview. The control unit 42a supports the training interview before the actual interview. This makes it possible to improve the interview skills of the superior before the actual interview.

[0055] A superior who receives pre-real training selects a personality type (profile) of a hypothetical subordinate who will be interviewed for the training. For example, the superior selects the same personality type as the subordinate who is scheduled to be interviewed for the real training.

[0056] In a training interview, the control unit 42a generates an avatar by computer graphics as a virtual subordinate of the selected personality type. The control unit 42a executes image processing and audio processing so that the avatar speaks according to a scenario. That is, the control unit 42a controls the image and audio actions of the avatar so that the avatar behaves (e.g., expresses facial expressions, speaks, etc.) according to a scenario corresponding to the selected personality type. For example, the control unit 42a controls the image and audio actions of the avatar so that the speech speed of the avatar varies according to the scenario. The control unit 42a uses the communication unit 40 to display an interview image including an avatar image on the display unit 33. The control unit 42a uses the communication unit 40 to output the voice of the avatar from the speaker 34. The control unit 42a analyzes the analysis target of the superior who reacts to the image and voice of the avatar speaking according to the scenario (e.g., the superior's facial expression, the superior's line of sight, the superior's nod).

[0057] When an avatar generated by the control unit 42a speaks according to a scenario in a training interview, the interview support system 1a only needs to include an interviewer device 3 used by the superior, and does not need to include an interviewee device 2. Also, in a training interview, instead of an avatar generated by the control unit 42a speaking according to a scenario, a human (actor) playing a virtual subordinate may speak according to a scenario.

[0058] Fig. 2 is a diagram showing an example of the operation of the interview support system 1a in the first embodiment. The upper part of Fig. 2 shows an example of the operation of the interview support system 1a relating to a training interview before the actual interview. Before the training interview, the selection acquisition unit 420 acquires information on the personality type selected by the superior from the communication unit 37 using the communication unit 40 (step S101).

[0059] During the training interview, the control unit 42a supports the interview. For example, the scenario processing unit 421 controls the image and audio behavior of the avatar so that the avatar operates according to a scenario corresponding to the selected personality type. The scenario processing unit 421 uses the communication unit 40 to display an interview image including an image of the avatar on the display unit 33. The information output unit 422 uses the communication unit 40 to display a guide for the tutorial, etc. on the display unit 33. The information output unit 422 uses the communication unit 40 to display an input field corresponding to the scenario on the display unit 33.

[0060] During the training interview, the analysis unit 424a analyzes the analysis target of the superior (interviewee) in real time. For example, the analysis unit 424a (image analysis unit) analyzes the analysis target in the image of the superior in real time based on the result of image recognition on the image of the superior. The analysis targets are, for example, the facial expression of the superior, the line of sight of the superior, and the nod of the superior. For example, the analysis unit 424a determines whether the facial expression of the superior in the image of the superior is positive or negative based on a plurality of feature points defined in the area of ​​the superior's face. Note that a combination of a plurality of analysis targets may be analyzed. For example, a combination of the facial expression of the superior and the line of sight of the superior may be analyzed (step S102).

[0061] During the training interview, the analysis unit 424a may analyze the analysis target in the image of the subordinate based on the scenario. Also, the analysis unit 424a may determine the subordinate's worry or empathy state (emotion) based on the scenario.

[0062] After the training interview, the score processing unit 425 acquires, for example, a mathematical model, a trained model, or a data table from the storage unit 41. The score processing unit 425 derives a score for the interview skill of the superior based on the analysis results of the superior's facial expression, line of sight, nods, etc. For example, the score processing unit 425 derives a score for the interview skill of the superior by inputting the analysis results of the superior's facial expression, etc. into the trained model. The score for the interview skill of the superior is provided to the superior. The score processing unit 425 may derive a score for the state of the subordinate according to a scenario.

[0063] After the training interview, the feedback processing unit 426 acquires the mathematical model, the learned model, or the data table from the storage unit 41. The feedback processing unit 426 derives advice to be provided to the superior according to the analysis result of the analysis target of the superior (for example, the number of times the superior nods). For example, the feedback processing unit 426 derives advice to be provided to the superior by inputting the analysis result of the analysis target of the superior into the learned model. The feedback processing unit 426 may derive advice to be provided to the superior by inputting the score of the interview skill of the superior into the learned model. The feedback processing unit 426 causes the display unit 33 to display a feedback image including the score of the interview skill of the superior and the advice to be provided to the superior using the communication unit 40 (step S104). This allows the superior to improve his / her own interview skill before the actual interview based on the score and advice provided.

[0064] The lower part of Fig. 2 shows an example of the operation of the interview support system 1a regarding an actual interview. Before the actual interview, the questionnaire processing unit 427 uses the communication unit 40 to obtain the subordinate's pre-questionnaire and the subordinate's type diagnostic questionnaire from the communication unit 27. The subordinate's pre-questionnaire is a questionnaire regarding what the subordinate wants to talk about (what he wants to consult about) with his superior during the interview (one-on-one meeting). The questionnaire processing unit 427 uses the communication unit 40 to obtain the superior's type diagnostic questionnaire from the communication unit 37 (step S201).

[0065] Before the actual interview, the diagnosis unit 428 diagnoses the personality type of the subordinate based on the type diagnosis questionnaire of the subordinate. The diagnosis unit 428 diagnoses the personality type of the superior based on the type diagnosis questionnaire of the superior. For example, the diagnosis unit 428 diagnoses the personality type based on the result of clustering processing for multiple items in the type diagnosis questionnaire. The questionnaire processing unit 427 acquires the preliminary questionnaire of the subordinate (step S202).

[0066] Before the actual interview, the goal processing unit 423 acquires the progress of the goals (promises) updated by the superior and the subordinate after the previous actual interview from the storage unit 41. The goal processing unit 423 uses the communication unit 40 to display the progress of the goals on the display unit 33 (step S203). This allows the superior to check the items written in the subordinate's pre-interview questionnaire, the subordinate's personality type, his / her own personality type, and the progress of the goals before the actual interview.

[0067] During the actual interview, the control unit 42a supports the interview. For example, the information output unit 422 uses the communication unit 40 to cause the display unit 33 to display an interview image including the personality type diagnosis result and the result of the advance questionnaire. The information output unit 422 also uses the communication unit 40 to cause the display unit 23 to display a similar interview image. This allows the superior and subordinate to conduct the interview while checking, for example, the personality type diagnosis result and the result of the advance questionnaire.

[0068] During the actual interview, the analysis unit 424a (image analysis unit) analyzes the analysis target of the superior (interviewee) in real time based on the result of image recognition on the image of the superior (interviewee). The analysis unit 424a may also analyze the analysis target of the subordinate (interviewee) in real time based on the result of image recognition on the image of the subordinate (interviewee) (step S204).

[0069] During the actual interview, the goal processing unit 423 acquires the goals (promises) entered in the input fields of the interview image displayed on the display unit 33 from the communication unit 37 using the communication unit 40. The goal processing unit 423 acquires the goals (promises) entered in the input fields of the interview image displayed on the display unit 23 from the communication unit 27 using the communication unit 40. These goals are goals that have been agreed upon between the superior and the subordinate (step S205).

[0070] After the actual interview, the questionnaire processing unit 427 uses the communication unit 40 to obtain the subordinate's post-interview questionnaire from the communication unit 27 (step S206). The score processing unit 425 derives a score for the superior's interview skills based on the analysis results of the superior's facial expression, line of sight, nodding, etc. The score processing unit 425 may derive a score for the subordinate's state of concern or empathy (feelings) based on the subordinate's post-interview questionnaire. The score processing unit 425 may derive a score for the subordinate's state based on the analysis results of the subordinate's facial expression, line of sight, nodding, etc.

[0071] After the actual interview, the feedback processing unit 426 derives advice to be provided to the superior according to the analysis result of the superior's analysis target. The feedback processing unit 426 may derive advice to be provided to the superior according to the analysis result of the superior's analysis target and the analysis result of the subordinate's analysis target. The feedback processing unit 426 causes the display unit 33 to display a feedback image including the superior's interview skill score and the advice to be provided to the superior using the communication unit 40 (step S207). This allows the superior to improve his / her own interview skill after the actual interview based on the provided score and advice.

[0072] After the actual interview, the goal processing unit 423 obtains the progress of the goals (promises) updated by the superior from the communication unit 37 using the communication unit 40. The goal processing unit 423 obtains the progress of the goals updated by the subordinate from the communication unit 27 using the communication unit 40. The goal processing unit 423 records the progress of the updated goals in the storage unit 41 (step S208). By storing the progress (status) of these goals in the storage unit 41, the superior can refer to it before the next actual interview.

[0073] Next, the analysis process will be described in detail. FIG. 3 is a diagram showing an example of image analysis regarding a superior's interview skill in the first embodiment.

[0074] [Improving communication skills of superiors during interviews] With the objective (analysis objective) of improving the communication ability of the superior in the interview, the analysis unit 424a analyzes the facial expression of the superior based on the result of image recognition of the image of the superior (interviewee). The analysis unit 424a derives the ratio of the superior's positive facial expression to the superior's negative facial expression (for example, the ratio of the length of time that the facial expression was displayed).

[0075] The score processing unit 425 derives the score of the interview skill of the boss based on the analysis result (proportion of facial expressions) of the analysis target in the image of the boss. The score processing unit 425 derives the score of the interview skill of the boss so that the score is the highest when the analysis result of the analysis target in the image of the boss satisfies a predetermined condition. For example, the score processing unit 425 derives the highest score when the ratio satisfies the condition that the positive facial expressions are at least twice but less than three times the negative facial expressions. The score of the interview skill of the boss may be converted into a score out of a maximum score of 100 points, for example.

[0076] The feedback processor 426 provides advice to the superior according to the derived ratio. For example, if the number of positive expressions is three times or more than the number of negative expressions, the feedback processor 426 includes the advice "You seem to be able to fully express trust and the like through your facial expressions. Please continue to treat your subordinates with positive facial expressions and strive to build relationships." in the feedback image. For example, if the number of positive facial expressions is two or more times but less than three times the number of negative facial expressions, the feedback processor 426 includes the advice "You seem to be able to treat your subordinates with positive facial expressions. However, you also often have negative facial expressions such as expressionless, sad, or angry. Try to make more positive facial expressions." in the feedback image. For example, if the number of positive facial expressions is less than two times the number of negative facial expressions, the feedback processor 426 includes the advice "You often have negative facial expressions such as sadness and anger, which may be affecting your relationships with your subordinates. Please practice making positive facial expressions." in the feedback image.

[0077] The analysis unit 424a may analyze the line of sight (eye movement) of the superior based on the result of image recognition on the image of the superior (interviewee). The score processing unit 425 derives the score of the superior's interview skill based on the line of sight of the superior. The feedback processing unit 426 provides advice to the superior according to the line of sight of the superior. For example, if the superior's eyes are wandering (if the superior's eyes are not directed at the subordinate), the feedback processing unit 426 includes the advice "Your eyes may wander. Be careful, as the subordinate may take that you are not open to the subordinate or that you are lying." in the feedback image. For example, if the superior is looking down, the feedback processing unit 426 includes the advice "Your eyes tend to look downward. Be careful, as the subordinate may take that you are not in good spirits or that you are closed off." in the feedback image.

[0078] The analysis unit 424a may analyze the boss's nod (interjection) based on the result of image recognition on the image of the boss (interviewee). The score processing unit 425 derives the score of the boss's interview skill based on the number of nods of the boss. The feedback processing unit 426 provides advice to the boss according to the number of nods of the boss. For example, if the number of nods of the boss is 130 or more in a 30-minute one-on-one meeting, the feedback processing unit 426 includes the advice "You are nodding unnecessarily many times, giving the subordinate the impression that you are not listening to what the subordinate is saying. Try to nod at the appropriate time." in the feedback image. For example, if the number of nods of the boss is 100 or more and less than 130, the feedback processing unit 426 includes the advice "You are showing a sufficiently receptive attitude toward the subordinate. Please continue to do so in the future." in the feedback image. For example, if the number of times the superior nods is 70 or more and less than 100, the feedback processor 426 includes in the feedback image the advice "If you nod just a little more, you can show your subordinate an accepting attitude. Let's try." For example, if the number of times the superior nods is less than 70, the feedback processor 426 includes in the feedback image the advice "It seems that your subordinate does not get the impression that he is being listened to sufficiently. Let's nod more to show your subordinate an accepting attitude."

[0079] The analysis unit 424a analyzes the facial expression of the superior based on the result of image recognition on the image of the superior. The analysis unit 424a classifies the superior's emotion based on the facial expression of the superior. The analysis unit 424a classifies emotions into, for example, anger, joy, surprise, sadness, and neutral. The analysis unit 424a analyzes the subordinate's facial expression based on the result of image recognition on the subordinate's image. The analysis unit 424a classifies the subordinate's emotion based on the subordinate's facial expression. The analysis unit 424a derives the degree of agreement (degree of empathy) between the superior's emotion and the subordinate's emotion. The score processing unit 425 derives the score of the superior's interview skill based on the degree of agreement between the superior's emotion and the subordinate's emotion. The feedback processing unit 426 provides advice to the superior according to the degree of agreement between the superior's emotion and the subordinate's emotion.

[0080] The analysis unit 424a may analyze the facial expression (state) of the subordinate based on the result of image recognition of the image of the subordinate. The score processing unit 425 derives the score of the interview skill of the superior based on the ratio (worry level) of the subordinate's positive facial expression to the subordinate's negative facial expression. The feedback processing unit 426 provides advice to the superior according to the ratio of the subordinate's positive facial expression to the subordinate's negative facial expression.

[0081] [Improving the supervisor's receptiveness during interviews (improving receptive attitude or listening attitude)] The analysis unit 424a analyzes the boss's nodding (interjections) based on the result of image recognition of the boss' image, with the aim of improving the boss's receptiveness in the interview. The score processing unit 425 derives the score of the boss' interview skill based on the number of times the boss nods. The feedback processing unit 426 judges whether the number of times the boss nods is within a predetermined range. This predetermined range is determined in advance according to, for example, the length of time the one-on-one meeting is held. The feedback processing unit 426 provides advice to the boss according to the judgment result.

[0082] Next, each image to be displayed will be described in detail. [Before the pre-production training interview] In step S101 shown in FIG. 2, the display unit 33 displays an image for selecting a personality type based on the data transmitted from the communication unit 40.

[0083] 4 is a diagram showing an example of a selection image for personality types in the first embodiment. There are "promoter" type, "supporter" type, "analyzer" type, and "controller" type according to the strength of self-assertion and the amount of emotional expression. These are examples of personality types.

[0084] The "promoter" type is a personality type that is assertive and expresses a lot of emotion. The "supporter" type is a personality type that is not assertive and expresses a lot of emotion. The "analyzer" type is a personality type that is not assertive and expresses a little emotion. The "controller" type is a personality type that is assertive and expresses a little emotion.

[0085] The boss selects the personality type of the subordinate with whom the actual interview is scheduled, for example. The boss selects the personality type "promoter" by operating the radio button 50, for example.

[0086] [During a pre-show training interview] In step S102 shown in FIG. 2, the display unit 23 displays the interview image transmitted from the communication unit 40.

[0087] FIG. 5 is a diagram showing an example of an image for an interview in the first embodiment. The image for an interview includes, for example, a moving image of a superior and a moving image of an avatar (a virtual subordinate). The image for an interview includes, for example, a character string representing the personality type of the superior and a character string representing the personality type of the avatar. The image for an interview includes, for example, an operation key image 60. The operation key image 60 is an operation key image associated with a goal (promise) set according to a scenario. When the operation key image 60 is pressed, the display unit 33 displays an image for managing the progress of the goal transmitted from the communication unit 40 (image for managing a goal) according to the control by the control unit 36.

[0088] When the operation key image 70 is pressed in step S102 shown in FIG. 2, the display unit 33 displays an image for setting the target item transmitted from the communication unit 40 in accordance with the control by the control unit .

[0089] 6 is a diagram showing an example of an image for setting a goal item in the first embodiment. The image for setting a goal item includes an input field for the goal item. When an operation key image 80 is pressed, the goal item entered in the setting image is stored in the storage unit 41. When an operation key image 90 is pressed, the display unit 33 redisplays the interview image exemplified in FIG. 5 in accordance with the control of the control unit 36.

[0090] [After the pre-show training interview] In step S104 shown in FIG. 2, the display unit 33 displays the feedback image transmitted from the communication unit 40.

[0091] FIG. 7 is a diagram showing an example of an interview skill score and advice feedback image (image to be provided) in the first embodiment. The feedback image shown in FIG. 7 includes a score display area 100 and an advice display area 110. The display unit 33 displays the score derived by the score processing unit 425 in the score display area 100. In FIG. 7, the display unit 33 displays the score of communication skills and the score of receptive attitude (receptive ability) in the score display area 100. The display unit 33 displays the advice derived by the feedback processing unit 426 in the advice display area 110. These can improve the interview skills (communicative ability, receptive ability) of the superior.

[0092] The display unit 33 may display a feedback image including a degree of satisfaction determined by the feedback processing unit 426 using artificial intelligence (AI) based on the post-questionnaire. That is, the display unit 33 may display a feedback image including a degree of satisfaction determined using a trained model. This degree of satisfaction is, for example, the degree to which the subordinate's worries have been resolved. The display unit 33 may display a feedback image including the progress of the superior's goals (promises), the progress of the subordinate's goals (promises), and the goal achievement degree, which is the average value of these.

[0093] [Before the actual interview] Fig. 8 is a diagram showing a first example of an input image for a pre-questionnaire in the first embodiment. In step S201 shown in Fig. 2, the display unit 23 displays an input image for the pre-questionnaire (consultation matters) transmitted from the communication unit 40. The pre-questionnaire includes items related to matters that a subordinate would like to consult with his / her superior in a one-on-one meeting. Each item may be selected using a pull-down menu or may be input using a check box.

[0094] Fig. 9 is a diagram showing a second example of an input image for a type diagnosis questionnaire in the first embodiment. In step S201 shown in Fig. 2, the display unit 23 and the display unit 33 display an input image for a type diagnosis questionnaire (for diagnosing a personality type) transmitted from the communication unit 40. Each item may be selected using an operation key image that can be pressed.

[0095] Fig. 10 is a diagram showing an example of a notification image of a personality type diagnosis result in the first embodiment. In step S202 shown in Fig. 2, the display unit 23 and the display unit 33 display the notification image transmitted from the communication unit 40. This notification image includes a character string representing the personality type diagnosis result based on a type diagnosis questionnaire (for diagnosing a personality type).

[0096] 2, the display unit 33 displays the management image transmitted from the communication unit 40. That is, the display unit 33 displays the management image of the progress of the goals set in the previous interviews. This enables the supervisor to check the progress of the goals set in the previous interviews before the actual interview.

[0097] [During the actual interview] In step S204 shown in FIG. 2, the display unit 23 and the display unit 33 display the interview image (for example, the interview image exemplified in FIG. 5) transmitted from the communication unit 40. The interview image includes, for example, a moving image of the superior and a moving image of the subordinate. The interview image includes, for example, a character string representing the superior's personality type and a character string representing the subordinate's personality type. The interview image may include, for example, an operation key image 60 associated with the goals (promises) set up until the previous interview. When the operation key image 60 is pressed, for example, the display unit 23 and the display unit 33 display an image for managing the progress of the goals set up until the previous interview.

[0098] 11 is a diagram showing an example of an image for managing the progress of goals in the first embodiment. The management image includes a list of set goals (promises) and each operation key image. In FIG. 11, when the operation key image 120 "Verify" is pressed, a "progress confirmation screen" showing the progress of the goal associated with the operation key image 120 is displayed on the display unit 23 and the display unit 33.

[0099] When the operation key image 70 "Goal setting" is pressed in step S204 shown in FIG. 2, the display unit 33 displays an image for setting the goal transmitted from the communication unit 40 in response to control by the control unit 36. In step S205 shown in FIG. 2, the superior (mentor) enters a new goal (promise) and deadline in the input field "Mentor's promise". When the operation key image 80 "Save" is pressed, the goal entered in the setting image is stored in the storage unit 41 in association with, for example, a time. When the operation key image 90 "Back" is pressed, the display unit 33 redisplays the interview image exemplified in FIG. 5 in response to control by the control unit 36.

[0100] When the operation key image 70 "Goal setting" is pressed in step S204 shown in FIG. 2, the display unit 23 displays an image for setting the goal transmitted from the communication unit 40 in response to control by the control unit 26. In step S205 shown in FIG. 2, the subordinate (mentor) enters a new goal (promise) and deadline in the input field "Promise of mentor". When the operation key image 80 "Save" is pressed, the goal entered in the setting image is stored in the storage unit 41 in association with, for example, a time. When the operation key image 90 "Back" is pressed, the display unit 23 re-displays the interview image exemplified in FIG. 5 in response to control by the control unit 26.

[0101] [After the actual interview] In step S206 shown in FIG. 2, the display unit 23 displays an input image for the post-event questionnaire.

[0102] FIG. 12 is a diagram showing an example of an input image for a post-event questionnaire in the first embodiment. The post-event questionnaire includes items related to the interview skills of the superior in a one-on-one meeting. Each item may be selected using a radio button, or may be input using a slider bar. The post-event questionnaire for a subordinate may include, for example, items related to key performance indicators (KPIs) (for example, progress toward targets). The post-event questionnaire for a subordinate is not shared with the superior. This allows the subordinate to fill in the items in the post-event questionnaire honestly.

[0103] In step S207 shown in Fig. 2, the display unit 33 displays the feedback image transmitted from the communication unit 40. The feedback image exemplified in Fig. 7 includes a score display area 100 and an advice display area 110. This makes it possible to improve the interview skills of the superior after the actual interview.

[0104] 2, the display unit 33 displays an image for management of the progress of the goals set in the current interview. The superior may write a message to the subordinate after the interview in the management image for each goal (promise).

[0105] In step S208 shown in Fig. 2, the display unit 23 may display an image for setting goals in response to control by the control unit 26. After the interview, the subordinate may enter new goals and deadlines in the input field "Promises for the mentor." The display unit 23 may display an image for managing the progress of the goals in which a message to the subordinate after the interview is entered in response to control by the control unit 26.

[0106] Next, an example of the operation of the interview support device 4a will be described. 13 is a diagram showing a first example (an example of a training interview) of the operation of the interview support device 4a in the first embodiment. Before the training interview, the selection acquisition unit 420 acquires information on the personality type selected by the superior from the communication unit 37 using the communication unit 40 (step S301).

[0107] During the training interview, the scenario processing unit 421 causes the display unit 33 to display an interview image including an image of an avatar that speaks according to the scenario and the personality type using the communication unit 40 (step S302). The analysis unit 424a analyzes the analysis target in the image of the superior in real time based on the result of image recognition of the image of the superior (step S303).

[0108] After the training interview, the score processor 425 derives the score of the interview skill of the superior based on the analysis results of the superior's facial expression, line of sight, nod, etc. (step S304). The feedback processor 426 derives advice to be provided to the superior according to the analysis results of the analysis target in the image of the superior (step S305). The feedback processor 426 causes the display unit 33 to display a feedback image including the score of the interview skill of the superior and the advice to be provided to the superior using the communication unit 40 (step S306).

[0109] 14 is a diagram showing a second example (example of an actual interview) of the operation of the interview support device 4a in the first embodiment. Before the actual interview, the questionnaire processing unit 427 acquires a preliminary questionnaire and a type diagnostic questionnaire for the subordinate, and a type diagnostic questionnaire for the superior (step S401). The diagnosis unit 428 diagnoses the personality type based on the type diagnostic questionnaire (step S402). The goal processing unit 423 uses the communication unit 40 to cause the display unit 33 to display the progress of the goal items (step S403).

[0110] During the actual interview, the information output unit 422 uses the communication unit 40 to display an interview image including the personality type diagnosis result and the result of the preliminary questionnaire on the display unit 33 (step S404). The analysis unit 424a analyzes the analysis target in the image of the superior in real time based on the result of image recognition of the image of the superior (step S405). The goal processing unit 423 acquires the goals (promises) entered by the superior (step S406).

[0111] After the actual interview, the questionnaire processing unit 427 acquires the subordinate's post-interview questionnaire from the communication unit 27 using the communication unit 40 (step S407). The score processing unit 425 derives the score of the interview skill of the superior based on the analysis result of the superior's facial expression, line of sight, nod, etc. (step S408). The feedback processing unit 426 derives advice to be provided to the superior according to the analysis result of the analysis target in the image of the superior (step S409). The feedback processing unit 426 causes the display unit 33 to display the feedback image including the score of the interview skill of the superior and the advice to be provided to the superior using the communication unit 40 (step S410). The goal processing unit 423 acquires the progress of the goal items (promises) updated by the superior from the communication unit 37 using the communication unit 40. The goal processing unit 423 may acquire the progress of the goal items (promises) updated by the subordinate from the communication unit 27 using the communication unit 40 (step S411). The goal processing unit 423 records the progress of the updated goal items (promises) in the storage unit 41 (step S412).

[0112] As described above, the analysis unit 424a analyzes the analysis target in the image of the interviewer based on the result of image recognition on the image of the interviewer having an interview with the interviewee. The analysis target in the image of the interviewer is, for example, at least one of the facial expression of the interviewer, the line of sight of the interviewer, and the nod of the interviewer. The score processing unit 425 provides the interviewer with a score of the interview skill of the interviewer based on the analysis result of the analysis target in the image of the interviewer. The feedback processing unit 426 provides advice to the interviewer according to the analysis result of the analysis target in the image of the interviewer.

[0113] This makes it possible to improve the interviewing skills of the interviewer using at least one of the score and advice based on the analysis results of the image of the interviewer. By improving the interviewing skills of the interviewer, it is possible to promote behavioral change (growth) of the interviewee. The behavioral change of the interviewee (growth of the subordinate) may be treated as a key goal indicator (KGI). Since images taken during the interview are analyzed, but audio (content of speech) during the interview is not analyzed, it is possible to improve the interviewing skills of the interviewer while reducing the possibility of leaking sensitive information and personal information within the company.

[0114] Second embodiment In the first embodiment, the image of the boss is the subject of analysis. In contrast, in the second embodiment, the subject of analysis is the voice of the boss, which is a difference from the first embodiment. In the second embodiment, the difference from the first embodiment will be mainly described.

[0115] 15 is a diagram showing an example of the configuration of an interview support system 1b in the second embodiment. The interview support system 1b includes an interviewee device 2, an interviewer device 3, an interview support device 4b, and a communication line 5.

[0116] The interview support device 4b is an information processing device such as a server. The interview support device 4b may be realized by cloud computing. The interview support device 4b includes a communication unit 40, a storage unit 41, and a control unit 42b. The control unit 42b includes a selection acquisition unit 420, a scenario processing unit 421, an information output unit 422, a goal processing unit 423, an analysis unit 424b, a score processing unit 425, a feedback processing unit 426, a questionnaire processing unit 427, and a diagnosis unit 428.

[0117] The analysis unit 424b (voice analysis unit) analyzes the analysis target in the voice of the superior (interviewee) in real time. The analysis target is, for example, the speaking speed of the superior, the intonation of the superior's voice, the volume (volume) of the superior's voice, and the tone (condition) of the superior's voice. The analysis target may be, for example, the content of the superior's utterance. The analysis target may be, for example, the number of questions from the superior to the subordinate. The analysis unit 424b may analyze the analysis target in the voice of the subordinate (interviewee) in real time.

[0118] Next, the analysis process will be described in detail. FIG. 16 is a diagram showing an example of a voice analysis regarding a superior's interview skill in the second embodiment.

[0119] [Improving communication skills of superiors during interviews] With the objective (analysis objective) of improving the communication ability of the superior in the interview, the analysis unit 424a analyzes the speaking speed of the superior (interviewee) in the voice of the superior. The score processing unit 425 derives a score of the superior's interview skill based on the superior's speaking speed. The feedback processing unit 426 judges whether the superior's speaking speed is within a predetermined range. The feedback processing unit 426 provides advice to the superior according to the judgment result.

[0120] The analysis unit 424b may further analyze the speech speed of the subordinate in the voice of the subordinate. The analysis unit 424b derives the degree of agreement between the speech speed of the superior and the speech speed of the subordinate. The score processing unit 425 derives the score of the interview skill of the superior based on the degree of agreement between the speech speed of the superior and the speech speed of the subordinate. The feedback processing unit 426 provides advice to the superior according to the degree of agreement between the speech speed of the superior and the speech speed of the subordinate. For example, if the degree of agreement is 80% or more, the feedback processing unit 426 includes the advice "You are able to speak at a pace that matches the subordinate's pace. The subordinate will feel relieved. Please continue like this" in the feedback image. For example, if the degree of agreement is 50% or more and less than 80%, the feedback processing unit 426 includes the advice "If you can speak more at a pace that matches the subordinate's pace, you will be able to build a stronger relationship of trust with the subordinate. Please do your best" in the feedback image. If the degree of agreement is less than 50%, the feedback processor 426 includes the advice "You may be speaking from your own perspective. Try to speak at a pace that matches your subordinate's pace" in the feedback image.

[0121] The analysis unit 424b may analyze the intonation of the boss's voice. The analysis unit 424b classifies the boss's emotion based on the intonation of the boss's voice. The analysis unit 424b may analyze the intonation of the subordinate's voice. The analysis unit 424b classifies the subordinate's emotion based on the intonation of the subordinate's voice. The analysis unit 424b derives the degree of agreement (degree of empathy) between the boss' emotion and the subordinate's emotion. The score processing unit 425 derives the score of the boss' interview skill based on the degree of agreement between the boss' emotion and the subordinate's emotion. The feedback processing unit 426 provides advice to the boss according to the degree of agreement between the boss' emotion and the subordinate's emotion. For example, if the degree of agreement is 80% or more, the feedback processing unit 426 includes the advice "It seems that you are sufficiently empathizing with and being close to the subordinate. Please continue like this" in the feedback image. For example, if the degree of match is 50% or more and less than 80%, the feedback processor 426 includes in the feedback image the advice "If you can empathize more with your subordinate's feelings, the relationship of trust between you and your subordinate will be stronger. Do your best." For example, if the degree of match is less than 50%, the feedback processor 426 includes in the feedback image the advice "It seems that you are interacting with your subordinate without being very empathetic to them. Try to empathize with your subordinate's feelings and emotions."

[0122] The analysis unit 424b may analyze the volume of the voice of the superior. The analysis unit 424b may analyze the tone of the voice of the superior. The analysis unit 424b may analyze the volume of the voice of the subordinate. The analysis unit 424b may analyze the tone of the voice of the subordinate. The analysis unit 424b derives the degree of agreement between the tone of the voice of the superior and the tone of the voice of the subordinate. The score processing unit 425 derives the score of the interview skill of the superior based on the degree of agreement between the tone of the voice of the superior and the tone of the voice of the subordinate. The feedback processing unit 426 provides advice to the superior according to the degree of agreement between the tone of the voice of the superior and the tone of the voice of the subordinate. For example, if the degree of agreement is 80% or more, the feedback processing unit 426 includes the advice "You are able to speak in accordance with the tone of the voice of the subordinate. The subordinate will feel relieved. Please continue like this" in the feedback image. For example, if the degree of match is 50% or more and less than 80%, the feedback processor 426 includes in the feedback image the advice "If you can speak more in line with the tone of your subordinate's voice, you will be able to build a stronger relationship of trust with your subordinate. Do your best." For example, if the degree of match is less than 50%, the feedback processor 426 includes in the feedback image the advice "You may be speaking from your own perspective. Try to speak in line with the volume of your subordinate's voice."

[0123] The analysis unit 424b may analyze the contents of the boss's speech in the boss's voice. The analysis unit 424b derives a ratio between the boss's positive speech content and the boss's negative speech content. The score processing unit 425 derives a score of the boss's interview skill based on the ratio between the boss's positive speech content and the boss's negative speech content. The analysis unit 424b may analyze the contents of the subordinate's speech in the subordinate's voice. The score processing unit 425 may derive a score of the boss's interview skill based on the ratio between the subordinate's positive speech content and the subordinate's negative speech content. The score processing unit 425 provides advice to the boss according to at least one of the ratios.

[0124] The analysis unit 424b deletes the voice data after the analysis of the utterance contents from the storage unit 41. The analysis unit 424b may analyze the utterance contents in the voice data received by the communication unit 40 after preventing the voice data from being stored in the storage unit 41. This can reduce the possibility that the utterance contents in the one-on-one meeting will be leaked.

[0125] [Improving the supervisor's receptiveness during interviews (improving receptive attitude or listening attitude)] The analysis unit 424b may analyze the contents of the boss's speech in the boss's voice in order to improve the boss's receptiveness in the interview. The analysis unit 424b counts the number of times the boss utters a message expressing understanding or sympathy. The score processing unit 425 derives a score of the boss' interview skill based on the number of times the boss utters a message expressing understanding or sympathy. The feedback processing unit 426 provides advice to the boss according to the number of times the boss utters a message expressing understanding or sympathy.

[0126] [Improving supervisors' coaching skills during interviews] The analysis unit 424b may analyze the contents of the boss's speech in the boss's voice in order to improve the boss's coaching ability in the interview. The analysis unit 424b counts the number of times a positive message is uttered by the boss. The score processing unit 425 derives a score of the boss' interview skill based on the number of times a positive message is uttered by the boss. The feedback processing unit 426 provides advice to the boss according to the number of times a positive message is uttered by the boss.

[0127] FIG. 17 is a diagram showing an example of an interview skill score and advice feedback image (providing image) in the second embodiment. The feedback image shown in FIG. 17 includes a score display area 100 and an advice display area 110. The display unit 33 displays the score derived by the score processing unit 425 in the score display area 100. In FIG. 17, the display unit 33 displays the score of communication ability, the score of receptive attitude (receptive ability), and the score of coaching ability in the score display area 100. The display unit 33 displays the advice derived by the feedback processing unit 426 in the advice display area 110. This makes it possible to improve the interview skills (communication ability, receptive ability, coaching ability) of the superior.

[0128] Next, an example of the operation of the interview support device 4b will be described. 18 is a diagram showing a first example (an example of a training interview) of the operation of the interview support device 4b in the second embodiment. Before the training interview, the selection acquisition unit 420 acquires information on the personality type selected by the superior from the communication unit 37 using the communication unit 40 (step S501).

[0129] During the training interview, the scenario processing unit 421 causes the display unit 33 to display an interview image including an image of an avatar that speaks according to the scenario and the personality type using the communication unit 40 (step S502). The analysis unit 424b analyzes the analysis target in the voice of the superior in real time (step S503).

[0130] After the training interview, the score processor 425 derives a score for the interview skill of the superior based on the analysis result of at least one of the speed of speech of the superior, the intonation of the superior's voice, the tone of the superior's voice, and the volume of the superior's voice. Hereinafter, the tone and volume are referred to as "tone, etc." (step S504). The feedback processor 426 derives advice to be provided to the superior according to the analysis result of the analysis target in the superior's voice (step S504). The feedback processor 426 causes the display unit 33 to display a feedback image including the score of the interview skill of the superior and the advice to be provided to the superior using the communication unit 40 (step S506).

[0131] FIG. 19 is a diagram showing a second example (example of an actual interview) of the operation of the interview support device 4b in the second embodiment. Before the actual interview, the questionnaire processing unit 427 acquires a preliminary questionnaire and a type diagnostic questionnaire for the subordinate, and a type diagnostic questionnaire for the superior (step S601). The diagnosis unit 428 diagnoses the personality type of the subordinate based on the type diagnostic questionnaire for the subordinate. The diagnosis unit 428 diagnoses the personality type of the superior based on the type diagnostic questionnaire for the superior (step S602). The goal processing unit 423 uses the communication unit 40 to cause the display unit 33 to display the progress of the goal items (step S603).

[0132] During the actual interview, the information output unit 422 uses the communication unit 40 to display an interview image including the personality type diagnosis result and the result of the preliminary questionnaire on the display unit 33 (step S604). The analysis unit 424b analyzes the analysis target in the boss's voice in real time (step S605). The goal processing unit 423 acquires the goals (promises) entered by the boss (step S606).

[0133] After the actual interview, the questionnaire processing unit 427 acquires the subordinate's post-interview questionnaire from the communication unit 27 using the communication unit 40 (step S607). The score processing unit 425 derives the score of the interview skill of the superior based on the analysis results of the superior's speech speed, the superior's voice intonation, the superior's voice tone, etc. (step S608). The feedback processing unit 426 derives advice to be provided to the superior according to the analysis results of the analysis target in the superior's voice (step S609). The feedback processing unit 426 causes the display unit 33 to display a feedback image including the score of the interview skill of the superior and the advice to be provided to the superior using the communication unit 40 (step S610). The goal processing unit 423 acquires the progress of the goal items (promises) updated by the superior from the communication unit 37 using the communication unit 40 (step S611). The goal processing unit 423 records the progress of the updated goal items (promises) in the storage unit 41 (step S612).

[0134] As described above, the analysis unit 424b analyzes the analysis target in the voice of the interviewer who is interviewing the interviewee. The score processing unit 425 provides the interviewer with a score of the interview skill of the interviewer based on the analysis result of the analysis target in the interviewer's voice. The feedback processing unit 426 provides advice to the interviewer according to the analysis result of the analysis target in the interviewer's voice. The analysis target in the interviewer's voice is, for example, at least one of the speaking speed of the interviewer, the intonation of the interviewer's voice, and the tone of the interviewer's voice.

[0135] This makes it possible to improve the interview skill of the interviewer using at least one of the score and advice based on the analysis result of the interviewer's voice. By improving the interview skill of the interviewer, it is possible to promote behavioral change (growth) of the interviewee. Even if the content of the speech during the interview is excluded from the analysis target for the purpose of reducing the possibility of leaking sensitive information and personal information within the company, it is possible to improve the interview skill of the interviewer based on the analysis result of the speech speed, intonation, and volume during the interview.

[0136] The feedback processing unit 426 may derive the subordinate's task based on the analysis result of the subordinate's speech content, for example, by using a trained model. The feedback processing unit 426 may provide the superior with advice indicating specific behavioral content based on the subordinate's task. The analysis unit 424b may determine whether the behavioral content provided to the superior has led to behavioral change of the subordinate based on the progress of the target items. The feedback processing unit 426 may provide the superior with a feedback image including the determination result.

[0137] Third embodiment In the first embodiment, the image of the boss, who is the interviewee, is the subject of analysis, but the voice of the boss is not the subject of analysis. In contrast to this, in the second embodiment, the voice of the boss is the subject of analysis, but the image of the boss is not the subject of analysis. Image-based analysis and voice-based analysis may be combined. Thus, in the third embodiment, the difference between the first and second embodiments is that the image and voice of the boss are the subjects of analysis. In the third embodiment, the differences from the first and second embodiments will be mainly described.

[0138] 20 is a diagram showing an example of the configuration of an interview support system 1c in the third embodiment. The interview support system 1c includes an interviewee device 2, an interviewer device 3, an interview support device 4c, and a communication line 5.

[0139] The interview support device 4c is an information processing device such as a server. The interview support device 4c may be realized by cloud computing. The interview support device 4c includes a communication unit 40, a storage unit 41, and a control unit 42c. The control unit 42c includes a selection acquisition unit 420, a scenario processing unit 421, an information output unit 422, a goal processing unit 423, an analysis unit 424c, a score processing unit 425, a feedback processing unit 426, a questionnaire processing unit 427, and a diagnosis unit 428.

[0140] The analysis unit 424c (image and voice analysis unit) analyzes the analysis target in the image of the superior (interviewee) in real time based on the result of image recognition on the image of the superior. The analysis unit 424c may analyze the analysis target in the image of the subordinate (interviewee) in real time based on the result of image recognition on the image of the subordinate. The analysis unit 424c analyzes the analysis target in the voice of the superior in real time. The analysis unit 424c may analyze the analysis target in the voice of the subordinate in real time.

[0141] Next, an example of the operation of the interview support device 4c will be described. 21 is a diagram showing a first example (an example of a training interview) of the operation of the interview support device 4c in the third embodiment. Before the training interview, the selection acquisition unit 420 acquires information on the personality type selected by the superior from the communication unit 37 using the communication unit 40 (step S701).

[0142] During the training interview, the scenario processing unit 421 uses the communication unit 40 to display an interview image including an image of an avatar that speaks according to the scenario and personality type on the display unit 33 (step S702). The analysis unit 424c analyzes the analysis target in the image of the boss in real time based on the result of image recognition of the image of the boss (step S703). The analysis unit 424c analyzes the analysis target in the voice of the boss in real time (step S704).

[0143] After the training interview, the score processor 425 derives a score for the interview skill of the superior based on the analysis results of the superior's facial expression, line of sight, nods, etc., and the analysis results of the superior's speech rate, the superior's voice intonation, the superior's voice tone, etc. (step S705). The feedback processor 426 derives advice to be provided to the superior based on the analysis results of the analysis targets in the image and voice of the superior (step S706). The feedback processor 426 causes the display unit 33 to display a feedback image including the score for the interview skill of the superior and the advice to be provided to the superior using the communication unit 40 (step S707).

[0144] Fig. 22 is a diagram showing a second example (example of an actual interview) of the operation of the interview support device 4c in the third embodiment. Before the actual interview, the questionnaire processing unit 427 acquires a preliminary questionnaire and a type diagnostic questionnaire for the subordinate, and a type diagnostic questionnaire for the superior (step S801). The diagnosis unit 428 diagnoses the personality type based on the type diagnostic questionnaire (step S802). The goal processing unit 423 uses the communication unit 40 to cause the display unit 33 to display the progress of the goal items (step S803).

[0145] During the actual interview, the information output unit 422 uses the communication unit 40 to display an interview image including the personality type diagnosis result and the result of the pre-questionnaire on the display unit 33 (step S804). The analysis unit 424c analyzes the analysis target in the image of the boss in real time based on the result of image recognition of the image of the boss (step S805). The analysis unit 424c analyzes the analysis target in the voice of the boss in real time (step S806). The goal processing unit 423 acquires the goals (promises) entered by the boss (step S807).

[0146] After the actual interview, the questionnaire processing unit 427 acquires the subordinate's post-interview questionnaire from the communication unit 27 using the communication unit 40 (step S808). The score processing unit 425 derives the score of the interview skill of the superior based on the analysis results of the superior's facial expression, line of sight, nods, etc., and the analysis results of the superior's speech speed, the superior's voice intonation, and the superior's voice tone, etc. (step S809). The feedback processing unit 426 derives advice to be provided to the superior according to the analysis results of the analysis target in the image and voice of the superior (step S810). The feedback processing unit 426 causes the display unit 33 to display a feedback image including the score of the interview skill of the superior and the advice to be provided to the superior using the communication unit 40 (step S811). The goal processing unit 423 acquires the progress of the goal items (promises) updated by the superior from the communication unit 37 using the communication unit 40 (step S812). The goal processing unit 423 records the progress of the updated goal items (promises) in the storage unit 41 (step S813).

[0147] As described above, the analysis unit 424c analyzes the analysis target in the image of the interviewer based on the result of image recognition on the image of the interviewer conducting an interview with the interviewee. The analysis unit 424c analyzes the analysis target in the voice of the interviewer conducting an interview with the interviewee. The score processing unit 425 provides the interviewer with a score for the interview skill based on the analysis result of the analysis target in the image of the interviewer and the analysis result of the analysis target in the voice of the interviewer. The feedback processing unit 426 provides advice to the interviewer according to the analysis result of the analysis target in the image and voice of the interviewer.

[0148] This makes it possible to efficiently improve the interview skill of the interviewer by using at least one of the score and advice based on the analysis results of the image and voice of the interviewer. By improving the interview skill of the interviewer, it is possible to promote behavioral change (growth) of the interviewee.

[0149] (Fourth embodiment) The fourth embodiment differs from the first to third embodiments in that advice, etc. is provided to the interviewee during a training interview that corresponds to a real interview, and that a scenario of the training interview is changed during the training interview depending on an analysis result of an image, etc. of the interviewee. The fourth embodiment will be described focusing on the differences from the first to third embodiments.

[0150] 23 is a diagram showing an example of the configuration of an interview support system 1d in the fourth embodiment. The interview support system 1d (interview training system) includes an interviewee device 2, an interviewer device 3, an interview support device 4d, and a communication line 5.

[0151] The interview support device 4d is an information processing device such as a server. The interview support device 4d may be realized by cloud computing. The interview support device 4d includes a communication unit 40, a storage unit 41, and a control unit 42d. The control unit 42d includes a selection acquisition unit 420, a scenario processing unit 421d, an information output unit 422, a goal processing unit 423, an analysis unit 424d, a score processing unit 425, a feedback processing unit 426d, a questionnaire processing unit 427, and a diagnosis unit 428.

[0152] The analysis unit 424d (image and audio analysis unit) executes an analysis process of an analysis target in at least one of the image and audio of the superior (interviewee), for example, at a predetermined period. For example, the analysis unit 424d analyzes (image analysis) the analysis target (e.g., facial expression) in the image of the superior based on the result of image recognition on the image of the superior. The analysis unit 424d may analyze the analysis target in the image of the subordinate (interviewee) in real time based on the result of image recognition on the image of the subordinate. The analysis unit 424d analyzes (audio analysis) the analysis target (e.g., intonation) in the voice of the superior in real time. The analysis unit 424d may analyze the analysis target in the voice of the subordinate in real time.

[0153] In addition, the contents of speech during training interviews do not have to be analyzed. Since the contents of speech (text data) during interviews are not analyzed, it is possible to improve the interview skills of the interviewee while reducing the possibility of leaking confidential and personal information within the company.

[0154] The scenario processing unit 421d changes the scenario of the training interview according to the analysis result of the analysis target of at least one of the image and the voice of the superior during the training interview. The scenario processing unit 421d controls the image and voice behavior of the avatar displayed on the display unit 33 so that the avatar behaves according to the scenario.

[0155] FIG. 24 is a diagram showing an example (modification) of a rule-based conditional branch of a scenario in the fourth embodiment. For example, the scenario processing unit 421d uses the communication unit 40 to display on the display unit 33 an interview image including an image of an avatar (virtual subordinate) that speaks according to the scenario associated with the current result (current analysis result) of the image analysis and voice analysis of the superior and the personality type of the subordinate. As an example, when the current analysis result (current analysis result) is "joy", the scenario processing unit 421d displays on the display unit 33 an interview image including a scenario associated with the analysis result "joy" and an image of an avatar that speaks according to the personality type of the selected avatar. As an example, when the current analysis result is "anger", the scenario processing unit 421d displays on the display unit 33 an interview image including a scenario associated with the analysis result "anger" and an image of an avatar that speaks according to the personality type of the selected avatar. As an example, when the current analysis result is “surprise”, the scenario processing unit 421d causes the display unit 33 to display an interview image including a scenario associated with the analysis result “surprise” and an image of an avatar speaking according to the selected avatar personality type.

[0156] For example, the scenario processing unit 421d may use the communication unit 40 to display on the display unit 33 an interview image including a scenario associated with the previous and current results of the image and voice analysis of the superior and an image of an avatar that speaks according to the selected personality type of the avatar. As an example, when the previous analysis result (past analysis result) is "joy" and the current analysis result (current analysis result) is "surprise", the scenario processing unit 421d may display on the display unit 33 an interview image including a scenario associated with the previous analysis result "joy" and the current analysis result "surprise" and an image of an avatar that speaks according to the selected personality type of the avatar. As an example, when the previous analysis result is "anger" and the current analysis result is "surprise", the scenario processing unit 421d may display on the display unit 33 an interview image including a scenario associated with the previous analysis result "anger" and the current analysis result "surprise" and an image of an avatar that speaks according to the selected personality type of the avatar.

[0157] Fig. 25 is a diagram showing an example of the operation of the interview support system 1d for a training interview associated with a real interview in the fourth embodiment. Before the training interview, the selection acquisition unit 420 acquires information on the personality type of the avatar selected by the superior based on the selection image of the personality type exemplified in Fig. 4 from the communication unit 37 using the communication unit 40. The selected personality type of the avatar is, for example, the same as the personality type of the interviewee in the real interview associated with the training interview (step S901).

[0158] The questionnaire processing unit 427 acquires the type diagnostic questionnaire of the superior from the communication unit 37 using the communication unit 40 (step S902). The diagnosis unit 428 diagnoses the personality type of the superior based on the type diagnostic questionnaire of the superior. For example, the diagnosis unit 428 diagnoses the personality type based on the result of clustering processing for a plurality of items in the type diagnostic questionnaire (step S903).

[0159] During the training interview, the analysis unit 424d analyzes the analysis target of the superior (interviewee) in real time. For example, the analysis unit 424d analyzes the analysis target in the image of the superior in real time based on the result of image recognition on the image of the superior. The analysis unit 424d outputs the analysis result of the analysis target of the superior (for example, facial expression in the image) to the scenario processing unit 421d as the analysis result regarding the interview skill of the superior.

[0160] The scenario processing unit 421d changes the scenario of the training interview according to the analysis result of the analysis target of at least one of the image and voice of the superior (the analysis result of the interview skill of the superior). The scenario processing unit 421d controls the image and voice behavior of the avatar displayed on the display unit 33 so that the avatar behaves according to the scenario (step S904).

[0161] During the training interview, the feedback processor 426d derives advice to be provided to the superior according to the diagnosis result of the superior's personality type and the analysis result of the superior's analysis target (analysis result regarding the superior's interview skill). The feedback processor 426d derives the advice to be provided to the superior by using, for example, a data table. In this data table, for example, the superior's personality type, the analysis result of the superior's analysis target, and the advice to be provided to the superior are associated with each other. The feedback processor 426d provides the derived advice to the superior (step S905). This allows the superior to improve his / her own interview skill before the actual interview based on the advice provided in real time during the training interview.

[0162] After the training interview, the feedback processor 426d derives advice to be provided to the superior again after the training interview according to the analysis result of the analysis target of the superior. The feedback processor 426d causes the display unit 33 to display a feedback image including the superior's interview skill score and the advice to be provided to the superior using the communication unit 40 (step S906). This enables the superior to improve his / her own interview skill before the actual interview based on the advice provided to the superior again after the training interview.

[0163] Fig. 26 is a diagram showing an example of an interview image in the fourth embodiment. The interview image shown in Fig. 26 is displayed on the display unit 33 during a training interview that corresponds to a real interview. The interview image includes a moving image of the superior and a moving image of an avatar (a virtual subordinate).

[0164] The interview image may further include at least one of a real-time analysis result of an analysis target (e.g., facial expression) in at least one of the image and voice of the superior, advice based on the analysis result, and advice based on the personality type diagnosis result. For example, the interview image illustrated in FIG. 26 includes an analysis result display area 200, a first advice display area 201, and a second advice display area 202.

[0165] The analysis result display area 200 is an area where real-time analysis results regarding the analysis target are displayed. The first advice display area 201 is an area where advice based on the diagnosis result of the boss's personality type is displayed. The second advice display area 202 is an area where advice based on the analysis result of the analysis target of at least one of the boss's image and voice is displayed.

[0166] During the training interview, the feedback processor 426d displays the real-time analysis results of the superior's analysis target in the analysis result display area 200. The real-time analysis results may be displayed in a graph format.

[0167] During the training interview, the feedback processing unit 426d derives advice (points to consider) based on the diagnosis result of the superior's personality type based on the diagnosis result of the superior's personality type and predetermined conditions. The feedback processing unit 426d may derive advice (points to consider) based on the diagnosis result of the superior's personality type for each level of importance. The feedback processing unit 426d displays the advice based on the diagnosis result of the superior's personality type in the first advice display area 201. The first advice display area 201 may be an area in which advice based on the diagnosis result of the personality type of an avatar (virtual subordinate) is displayed. The feedback processing unit 426d may display the advice based on the diagnosis result of the avatar's personality type in the first advice display area 201.

[0168] During the training interview, the feedback processor 426d derives advice based on the analysis results based on the real-time analysis results of the analysis target of the superior and predetermined conditions. For example, if the ratio of the analysis result "happiness" to the overall real-time analysis results of the analysis target "facial expression" of the superior is less than a predetermined threshold, the feedback processor 426d may derive advice based on the analysis results, "brighten your facial expression." The feedback processor 426d displays the advice based on the analysis results in the second advice display area 202.

[0169] As described above, the analysis unit 424d analyzes the analysis target in at least one of the image and the voice of the interviewer who is interviewing the interviewee in the training interview associated with the actual interview. The feedback processing unit 426d provides advice to the interviewer in the training interview according to the analysis result of the analysis target.

[0170] As a result, advice is provided to the interviewer in accordance with the analysis results of the analysis target in the training interview, so that the interviewer's interview skills can be efficiently improved before the actual interview. The interviewer can train his / her interview skills by himself / herself in his / her own free time. The interview support system 1d analyzes at least one of the interviewer's image and voice in real time, and during the training interview associated with the actual interview, the interview support system 1d presents the analysis results and advice, etc. to the interviewer in real time, so that the interviewer can grasp the points to be improved and points to be noted in the interviewer's own facial expressions, etc., before the interview.

[0171] The scenario processing unit 421d changes the scenario according to the analysis result of the analysis target during the training interview. The scenario processing unit 421d controls the image and audio behavior of the interviewee displayed on the display unit 33 according to the scenario. The scenario processing unit 421d may display the analysis result of the analysis target on the display unit 33 during the training interview.

[0172] In this way, the interview support system 1d changes the scenario of the interview scene that needs improvement during the training interview according to the analysis result, and provides the interviewee with a training environment (avatar image, etc.) according to the changed scenario during the training interview. This allows the interviewee to intensively train the interview scene that needs improvement.

[0173] (Modification of the fourth embodiment) The difference between the fourth embodiment and the modified example of the fourth embodiment is that the interview support device uses a model-based scenario to train the superior's interview skills. The difference between the fourth embodiment and the modified example of the fourth embodiment will be mainly described.

[0174] 27 is a diagram showing an example of machine learning of a learning model in a modified example of the fourth embodiment. The learning model is not limited to a specific machine learning model, but may be, for example, a model using a recurrent neural network (RNN) and a long-short-term memory model (LSTM).

[0175] In the learning stage, scenario branching data 300 is used to train the learning model. The scenario branching data is learning data related to the branching (selection) of the scenario associated with the scene (state) of the interview, that is, learning data related to the transition (transition) of the interview scene. In the scenario branching data, the scene before the scenario branching is associated with at least one of the image (facial expression, etc.) and voice (intonation, etc.) of the interviewee, and the scene after the scenario branching.

[0176] In addition, the contents of the speech in the training interview may not be associated with the scenario branching data. In other words, the contents of the speech in the training interview may not be the subject of analysis. Since the contents of the speech (text data) of the voice during the interview are not analyzed, it is possible to improve the interview skills of the interviewer while reducing the possibility of leaking confidential information and personal information within the company.

[0177] In the learning stage, the scenario processing unit 421d uses the scenario branching data 300 to execute learning of the learning model. For example, the scenario processing unit 421d uses the scenario branching data 300-1 to make the learning model learn that the scene "A" before the scenario branches, the analysis results of the interviewer's facial expression "neutral" and voice "neutral", and the scene "B" after the scenario branches are associated with each other. For example, the scenario processing unit 421d uses the scenario branching data 300-4 to make the learning model learn that the scene "B" before the scenario branches, the analysis results of the interviewer's facial expression "angry" and voice "angry", and the scene "E" after the scenario branches are associated with each other. Note that these analysis results do not need to be highly accurate analysis results.

[0178] Fig. 28 is a diagram showing an example of a model-based scene transition of a scenario in a modified example of the fourth embodiment. In an inference stage (during a training interview) after the learning stage, the scenario processing unit 421d inputs at least one of an image and a voice of the superior during the training interview into a learning model (scenario branching model). The scenario processing unit 421d acquires the output of the learning model. The scenario processing unit 421d changes the scenario based on the output of the learning model.

[0179] For example, when the analysis result of the interviewer's facial expression "neutral" and voice "neutral" in the interview scene "A" is obtained, the scenario processing unit 421d determines that the interview scene will transition to "B" based on learning using the scenario branching data 300-1. The scenario processing unit 421d selects a scenario according to the interview scene "B" after the transition. For example, when the analysis result of the interviewer's facial expression "angry" and voice "angry" in the interview scene "B" is obtained, the scenario processing unit 421d determines that the interview scene will transition to "E" based on learning using the scenario branching data 300-4. The scenario processing unit 421d selects a scenario according to the interview scene "E" after the transition. In this way, in the modified example of the fourth embodiment, the transition of the interview scene is managed by the scenario processing unit 421d using the learning model. The scenario processing unit 421d selects the current scenario according to the current interview scene.

[0180] As described above, in the learning stage, the scenario processing unit 421d executes learning of the learning model using the scenario branching data 300. In the inference stage, the scenario processing unit 421d inputs at least one of an image and a voice of the superior during a training interview into the learning model. The scenario processing unit 421d acquires the output of the learning model. The scenario processing unit 421d changes the scenario based on the output of the learning model.

[0181] This allows the interview scene to be managed even if the rules for the scenario's conditional branching are not strictly defined, making it possible to efficiently improve the interviewer's interview skills before the actual interview.In addition, the branching accuracy (selection accuracy) of the scenario can be improved according to the amount of accumulated scenario branching data (the amount of learning using scenario branching data).

[0182] Fifth embodiment The fifth embodiment differs from the first to fourth embodiments in that at least one of advice and warning is provided to the interviewer depending on the amount of speech of the interviewer and interviewee during the actual interview. The fifth embodiment will be described focusing on the differences from the first to fourth embodiments.

[0183] In an interview, it is often preferable for the interviewee (subordinate) to speak more than the interviewer (superior). However, the interviewer tends to speak more than the interviewee. For this reason, the interviewer himself needs to be aware during the interview whether the interviewee is speaking more than the interviewer. In such cases, the interviewer may become distracted if he becomes aware of the amount of speech he or she makes during the interview. If the interviewer becomes distracted, the quality of the interview may decline.

[0184] Therefore, in the fifth embodiment, the interview support device analyzes the amount of speech of the interviewer and the amount of speech of the interviewee in real time during the actual interview. The interview support device displays advice and warnings, etc. for the interviewer on the display unit of the interviewer device during the actual interview according to the amount of speech of each of the interviewer and the interviewee. This allows the interviewer to easily understand the amount of speech of the interviewer while concentrating on the interview. In other words, since the interviewer can easily notice that the interviewer has a large amount of speech (the interviewer's speaking time is long), it is possible to improve the interview skill of the interviewer (improve the interviewer's listening attitude). Note that the interview support device may display advice and warnings, etc. for the interviewer on the display unit of the interviewer device during the training interview according to the amount of speech of each of the interviewer and the interviewee.

[0185] 29 is a diagram showing an example of the configuration of an interview support system 1e. The interview support system 1e includes an interviewee device 2, an interviewer device 3, an interview support device 4e, and a communication line 5.

[0186] The interview support device 4e is an information processing device such as a server. The interview support device 4e may be realized by cloud computing. The interview support device 4e includes a communication unit 40, a storage unit 41, and a control unit 42e. The control unit 42e includes a selection acquisition unit 420, a scenario processing unit 421e, an information output unit 422, a goal processing unit 423, an analysis unit 424e, a score processing unit 425, a feedback processing unit 426e, a questionnaire processing unit 427, and a diagnosis unit 428.

[0187] The analysis unit 424e (image and audio analysis unit) has a voice estimation processing model. This voice estimation processing model is a trained model that executes a predetermined estimation process (analysis process) on voice data. The analysis unit 424e analyzes the analysis target in the voice of the superior (interviewee) in real time. The analysis unit 424e analyzes, for example, the amount of speech of the superior. The analysis unit 424e analyzes the analysis target in the voice of the subordinate (interviewee) in real time. The analysis unit 424e analyzes the amount of speech of the interviewer and the amount of speech of the interviewee.

[0188] The analysis unit 424e may analyze the analysis target in the voice of the superior in real time based on the result of image recognition on the image of the superior. For example, the analysis unit 424e may derive the boss's speech volume based on the length of time the superior's mouth is moving. The analysis unit 424e may also analyze the analysis target in the voice of the subordinate in real time based on the result of image recognition on the image of the subordinate. For example, the analysis unit 424e may derive the subordinate's speech volume based on the length of time the subordinate's mouth is moving. Since the speech volume is analyzed based on the image at the time of the interview (the content of the speech at the time of the interview is not analyzed), it is possible to analyze the speech volume while reducing the possibility of leakage of sensitive information and personal information within the company.

[0189] FIG. 30 is a diagram showing an example of the duration of speech in the fifth embodiment. The horizontal axis in FIG. 30 represents time. The time (predetermined period) from time "t0" to time "t5" illustrated in FIG. 30 is, for example, a fixed cycle. The duration of this fixed cycle is, for example, determined in advance. Time "t1" represents the start time of speech 400 in the boss's voice data. Time "t2" represents the end time of speech 400 in the boss's voice data. Time "t3" represents the start time of speech 401 in the subordinate's voice data. Time "t4" represents the end time of speech 401 in the subordinate's voice data.

[0190] The analysis unit 424e derives the boss's speech duration by subtracting a predetermined first margin time from the boss's speech duration from time "t0" to time "t5". Similarly, the analysis unit 424e derives the subordinate's speech duration by subtracting a predetermined second margin time from the subordinate's speech duration. The speech duration of each voice from time "t0" to time "t5" is expressed, for example, as shown in formula (1).

[0191] Speech duration = Audio duration - Margin duration … (1)

[0192] The audio time length in equation (1) is expressed, for example, as in equation (2).

[0193] Audio duration = audio end time - audio start time … (2)

[0194] The margin time length in formula (1) is expressed as formula (3), for example. Note that the margin time length of at least one of the start end and the end end may be 0.

[0195] Margin time length = margin time length at the beginning of the speech + margin time length at the end of the speech … (3)

[0196] Analysis unit 424e derives the boss's speaking time length by subtracting the margin time length of formula (3) from the boss's speech time length from time "t0" to time "t5". That is, analysis unit 424e derives the boss's speaking time length by subtracting the margin time length at the start of the speech and the margin time length at the end of the speech from the speech time length of speech 400. If the boss speaks multiple times, analysis unit 424e accumulates (sums up) each of the boss' speaking times from time "t0" to time "t5". In this way, analysis unit 424e derives the accumulated result of each of the boss' speaking times.

[0197] The analysis unit 424e derives the subordinate's speaking time length by subtracting the margin time length of formula (3) from the subordinate's speech time length from time "t0" to time "t5". That is, the analysis unit 424e derives the subordinate's speaking time length by subtracting the margin time length at the start of the speech and the margin time length at the end of the speech from the speech time length of speech 401. If the subordinate speaks multiple times, the analysis unit 424e accumulates (sums up) each of the subordinate's speaking time lengths from time "t0" to time "t5". In this way, the analysis unit 424e derives the accumulated result of each of the subordinate's speaking time lengths.

[0198] Feedback processor 426e derives the sum of the amount of speech of the superior and the amount of speech of the subordinate from time "t0" to time "t5". That is, feedback processor 426e derives the sum of the speech duration of voice 400 and the speech duration of voice 401. Feedback processor 426e derives the ratio of the amount of speech of the superior to the sum of the amount of speech of the superior and the amount of speech of the subordinate. That is, feedback processor 426e derives the ratio of the amount of speech of the superior as shown in formula (4).

[0199] Ratio of boss's speech volume = boss's speech volume / (boss's speech volume + subordinate's speech volume) … (4)

[0200] The feedback processing unit 426e derives at least one of advice and a warning when the ratio of the amount of speech of the superior is equal to or greater than a predetermined threshold. The feedback processing unit 426e generates an interview image including at least one of the derived advice and warning when the ratio of the amount of speech of the superior is equal to or greater than a predetermined threshold. During the actual interview, the display unit 23 displays the interview image transmitted from the communication unit 40. The derived advice and warning are provided to the superior by being displayed on the display unit 33 as part of the interview image.

[0201] FIG. 31 is a diagram showing an example of an image for interview in the fifth embodiment. The image for interview displayed on the display unit 33 includes a third advice display area 203 when the ratio of the amount of speech of the superior is equal to or greater than a predetermined threshold. The third advice display area 203 is an area in which at least one of advice and warning is displayed. The advice in the third advice display area 203 is expressed by a character string of advice such as "reduce the amount of speech". The warning in the third advice display area 203 is expressed by a character string of warning such as "you speak a lot". In order to make the character string of advice and warning stand out, the font of at least one of the character strings of advice and warning may be larger than the font of other character strings in the image for interview. The warning may be expressed by a predetermined icon image such as the warning image 204. The image for interview may include a numerical value (e.g., "70%") representing the ratio of the amount of speech of the superior. The image for interview may include a numerical value (e.g., "20 minutes") representing the time of speech of the superior in a predetermined period. The output of advice and warning is not limited to output using an image. For example, the speaker 34 may output a sound corresponding to the advice and warning character strings.

[0202] Next, an example of the operation of the interview support device 4e will be described. FIG. 32 is a diagram (flowchart) showing an example of the operation of the interview support device 4e in the fifth embodiment. The analysis unit 424e analyzes the amount of speech of the superior (step S1001). The analysis unit 424e analyzes the amount of speech of the subordinate (step S1002). The analysis unit 424e or the feedback processing unit 426e derives the ratio of the amount of speech of the superior to the total amount of speech of the superior and the subordinate (step S1003). The feedback processing unit 426e derives advice and warning to be provided to the superior according to the ratio of the amount of speech of the superior to the total amount of speech of the superior and the subordinate. For example, when the ratio of the amount of speech of the superior is equal to or greater than a predetermined threshold, the feedback processing unit 426e derives advice and warning to be provided to the superior (step S1004). The display unit 33 displays an interview image including at least one of advice and warning during the actual interview (step S1005).

[0203] As described above, the analysis unit 424e analyzes the amount of speech of the interviewer and the amount of speech of the interviewee. The feedback processing unit 426e provides at least one of advice and a warning to the interviewer during the interview according to the analysis result of the amount of speech of the interviewer and the amount of speech of the interviewee.

[0204] This allows the interviewer to realize that the interviewee is talking for a long time, which makes it possible to improve the interviewer's interview skills (improvement of the interviewer's listening attitude). [Industrial Applicability]

[0205] The present invention is applicable to a system for supporting improvement of interview (for example, one-on-one meetings, customer service) skills.

[0206] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0207] Reference Signs List 1a, 1b, 1c, 1d, 1e...interview support system, 2...interviewee device, 3...interviewer device, 4a, 4b, 4c, 4d, 4e...interview support device, 5...communication line, 20...camera, 21...microphone, 22...operation unit, 23...display unit, 24...speaker, 25...storage unit, 26...control unit, 27...communication unit, 30...camera, 31...microphone, 32...operation unit, 33...display unit, 34...speaker, 35...storage unit, 36...control unit, 37...communication unit, 40...communication unit, 41...storage unit, 42a, 42b, 42c...control unit, 50...radio button, 60...operation key image, 70...operation key image, 80...operation key image, 90...operation key Image, 100... score display area, 110... advice display area, 120... operation key image, 200... analysis result display area, 201... first advice display area, 202... second advice display area, 203... third advice display area, 204... warning image, 300... scenario branch data, 400... voice, 401... voice, 420... selection acquisition unit, 421, 421d, 421e... scenario processing unit, 422... information output unit, 423... goal processing unit, 424a, 424b, 424c, 424d, 424e... analysis unit, 425... score processing unit, 426, 426d, 426e... feedback processing unit, 427... questionnaire processing unit, 428... diagnosis unit

Claims

1. an analysis unit that analyzes the amount of speech of an interviewer using the interviewer device and the amount of speech of an interviewee using the interviewee device; a feedback processing unit that provides at least one of a predetermined character string, an icon image, and a sound as at least one of an advice and a warning to a display unit of the interviewer device used by the interviewer while a predetermined interview image is being displayed according to an analysis result of the amount of speech of the interviewer and the amount of speech of the interviewee; Equipped with the analysis unit analyzes an amount of speech of the interviewer based on a result of image recognition of the interviewer, and analyzes an amount of speech of the interviewee based on a result of image recognition of the interviewee, the analysis unit analyzes a cumulative result of the speaking time length of the interviewer in a predetermined period as a speaking amount of the interviewer, and analyzes a cumulative result of the speaking time length of the interviewee in the predetermined period as a speaking amount of the interviewee, An interview support device, wherein the speaking time length of the interviewer is derived by subtracting a margin time length at the start of the voice and a margin time length at the end of the voice from the voice time length of the voice of the interviewer.

2. The interview support device of claim 1, wherein the feedback processing unit derives a ratio of the amount of speech of the interviewer to the sum of the amount of speech of the interviewee and the amount of speech of the interviewee, and if the ratio is greater than or equal to a predetermined threshold, provides at least one of the advice and the warning to the interviewer.

3. An interview support method executed by an interview support device, an analysis step of analyzing an amount of speech of an interviewer using an interviewer device and an amount of speech of an interviewee using an interviewee device; a feedback processing step of providing at least one of a predetermined character string, an icon image and a sound as at least one of an advice and a warning to a display unit of the interviewer device used by the interviewer while displaying a predetermined interview image according to an analysis result of the amount of speech of the interviewer and the amount of speech of the interviewee; Including, The analyzing step includes analyzing an amount of speech of the interviewer based on a result of image recognition of the interviewer, and analyzing an amount of speech of the interviewee based on a result of image recognition of the interviewee, The analyzing step includes analyzing a cumulative result of the speaking time length of the interviewer in a predetermined period as a speaking amount of the interviewer, and analyzing a cumulative result of the speaking time length of the interviewee in the predetermined period as a speaking amount of the interviewee, The speaking time length of the interviewer is derived by subtracting a margin time length at the start of the voice and a margin time length at the end of the voice from the voice time length of the voice of the interviewer. Interview support method.

4. A computer program for causing a computer to function as the interview support device according to claim 1 or 2.

Images