Information processing device, program, and information processing method
Patent Information
- Application Number
- JP2025519337
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Priority Date
- 2023-05-10
- Filing Date
- 2024-03-26
- Publication Date
- 2024-11-14
AI Technical Summary
In remote sales interviews, sales representatives face challenges in grasping customer behavior and reactions, making it difficult to adjust the interview process effectively due to the lack of direct visual cues compared to face-to-face interactions.
An information processing device and method that estimates the customer's satisfaction level based on sensing data such as video and audio, providing real-time notifications to sales representatives about their own satisfaction levels and potential causes of dissatisfaction, allowing for adjustments during the interview.
Enhances user convenience and interview quality by enabling sales representatives to better understand customer satisfaction and respond accordingly, improving the remote interview experience.
Abstract
Description
Information processing device, program, and information processing method
[0001] The present disclosure relates to an information processing device, a program, and an information processing method.
[0002] In recent years, remote interviews, in which sales representatives meet with customers using online video or voice calls, have become widely used in sales activities for products, services, etc. Therefore, various technologies to support remote interviews by sales representatives have been considered. For example, Patent Literature 1 discloses a technology for determining a sales representative based on registered information such as the age, gender, and department of the sales partner.
[0003] Japanese Patent Application Laid-Open No. 2020-091824
[0004] In sales activities using remote interviews as described above, it can be difficult for sales representatives to grasp the customer's state or reaction compared to face-to-face interviews.
[0005] In order to solve the above problem, according to one aspect of the present disclosure, an information processing device is provided that includes a control unit that controls notifying a caller of the caller's status related to the degree of satisfaction with the call, based on the caller's level of satisfaction with the call, which is estimated based on sensing data about the caller.
[0006] Furthermore, according to the present disclosure, a program is provided that causes a computer to function as a control unit that controls the notification of a caller's status related to the degree of satisfaction with the call, based on the caller's level of satisfaction with the call, which is estimated based on sensing data about the caller.
[0007] Furthermore, according to the present disclosure, there is provided an information processing method executed by a computer, which includes controlling to notify a caller of the caller's status related to the degree of satisfaction with the call, based on sensing data about the caller.
[0008] 1 is an explanatory diagram illustrating an example configuration of an information processing system according to an embodiment of the present disclosure. FIG. 1 is a block diagram illustrating an example configuration of a caller terminal 10 according to a first embodiment of the present disclosure. FIG. 2 is a block diagram illustrating an example functional configuration of an information processing device 20 according to a first embodiment of the present disclosure. FIG. 3 is a diagram illustrating an example video playback screen generated by a control unit 230. FIG. 4 is a diagram illustrating an example call screen generated by the control unit 230. FIG. 5 is a diagram illustrating an example call screen and a notification display generated by the control unit 230. FIG. 6 is a diagram illustrating another example call screen and a notification display of busyness level generated by the control unit 230. FIG. 7 is a diagram illustrating an example notification display showing information related to speaking time generated by the control unit 230. FIG. 8 is a diagram illustrating an example screen for accepting settings of the other party, purpose, and scheduled time of a remote interview generated by the control unit 230. FIG. 9 is a flowchart illustrating an example operation of an information processing system according to the present embodiment in a learning stage of a dissatisfaction estimation model. FIG. 10 is a second flowchart illustrating an example operation of an information processing system according to the present embodiment in a learning stage of a dissatisfaction cause state detection model. FIG. 11 is a flowchart illustrating an example operation of an information processing system according to the present embodiment in an inference stage of a dissatisfaction cause state. FIG. 12 is a block diagram illustrating an example functional configuration of an information processing device 21 according to a second embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example video playback screen generated by a control unit 231. 2 is a diagram showing another example of a video playback screen generated by the control unit 231. FIG. 3 is a diagram showing an example of a call summary screen generated by the control unit 231. FIG. 4 is a flowchart showing an example of an operation of an information processing system according to a second embodiment of the present disclosure. FIG. 5 is a block diagram showing an example of a functional configuration of an information processing device 22 according to a third embodiment of the present disclosure. FIG. 6 is a diagram showing an example of a parameter setting acceptance screen related to conversion of a face image and voice by the control unit 232. FIG. 7 is a diagram showing an example of a call screen and a notification display generated by the control unit 232. FIG. 8 is a flowchart explaining an example of an operation of an information processing system according to a third embodiment of the present disclosure. FIG. 9 is a block diagram showing an example of a functional configuration of an information processing device 23 according to a fourth embodiment of the present disclosure. FIG. 10 is a diagram showing an example of a doctor-side standby screen generated by the control unit 233. FIG. 11 is a diagram showing an example of a sales-side standby screen generated by the control unit 233. FIG. 12 is a diagram showing an example of a waiting room screen generated by the control unit 233.23 is a diagram showing an example of a sales-side standby screen generated by the control unit 233. FIG. 24 is a diagram showing an example of a sales-side standby screen generated by the control unit 233 and a notification display related to waiting time. FIG. 25 is a diagram showing an example of an information exchange requester standby screen generated by the control unit 233. FIG. 26 is a diagram showing an example of an information exchange request notification screen generated by the control unit 233. FIG. 27 is a diagram showing another example of a doctor-side standby screen generated by the control unit 233. FIG. 28 is an example of a notification priority setting screen generated by the control unit 233. FIG. 29 is a diagram showing an example of a notification screen that a doctor is available for an interview generated by the control unit 233. FIG. 29 is a diagram showing an example of a notification screen generated by the control unit 233. FIG. 29 is a diagram showing an example of a post-interview evaluation questionnaire screen generated by the control unit 233. FIG. 29 is a diagram showing another example of a post-interview evaluation questionnaire screen generated by the control unit 233. FIG. 29 is a first flowchart showing an operation example of an information processing system according to a fourth embodiment. FIG. 29 is a second flowchart showing an operation example of an information processing system according to the fourth embodiment. FIG. 29 is a flowchart describing an operation example of the information processing system relating to the sales-to-sales information exchange function in the fourth embodiment of the present disclosure. FIG. 29 is a block diagram showing a hardware configuration 90 according to an embodiment of the present disclosure.
[0009] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.
[0010] In addition, in this specification and drawings, multiple components having substantially the same functional configuration may be distinguished by adding different numbers or letters after the same reference numeral. However, when there is no particular need to distinguish between multiple components having substantially the same functional configuration, only the same reference numeral is used to distinguish each of the multiple components.
[0011] The description will be given in the following order: <1. First embodiment> <1-1. Overview> <1-2. System configuration example> <1-3. Functional configuration example> <1-4. Operation example> <2. Second embodiment> <2-1. Overview> <2-2. Functional configuration example> <2-3. Operation example> <3. Third embodiment> <3-1. Overview> <3-2. Functional configuration example> <3-3. Operation example> <4. Fourth embodiment> <4-1. Overview> <4-2. Functional configuration example> <4-3. Operation example> <5. Hardware configuration example> <6. Summary>
[0012] <1. First Embodiment> <1-1. Overview> First, an overview of the first embodiment of the present disclosure will be described.
[0013] As described above, in recent years, sales activities have been conducted using remote interviews, such as interviews using video calls or voice calls, etc. Remote interviews make it possible to conduct interviews even when the customer and sales representative are in distant locations.
[0014] In a remote interview, typically, all the salesperson can see and hear during the interview is the customer's video, including their face, and their voice. Or, if the video is off, all the salesperson can hear during the interview is the customer's voice.
[0015] Therefore, in remote interviews, it can be difficult for salespeople (hereinafter referred to as "salespersons") to grasp the customer's behavior or reactions during the interview, compared to face-to-face interviews, which can make it difficult for the salespersons to respond flexibly, such as by changing the way the interview proceeds depending on the customer's reactions.
[0016] The technical concept of one embodiment of the present disclosure was conceived with the above points in mind, and provides technology that improves user convenience in remote interviews and contributes to further improving the quality of interviews.
[0017] For this purpose, the information processing device 20 according to one embodiment of the present disclosure controls to notify the caller of the caller's status related to the degree of satisfaction with the call, based on the caller's level of satisfaction with the call, which is estimated based on sensing data about the caller.
[0018] The callers may include a first caller and a second caller who sells a product or service or both to the first caller. As an example, the first caller may be a customer, and the second caller may be a salesperson.
[0019] The information processing device 20 may control the notification of the second party of the status of the second party related to the degree of satisfaction of the first party with the call, based on the first party's satisfaction with the call estimated based on sensing data regarding the first party.
[0020] For example, the information processing device 20 may perform control to notify the second party of the status of the second party when the satisfaction level of the first party is lower than a reference value based on sensing data regarding the second party.
[0021] At this time, the information processing device 20 may perform control to notify the second caller of the state of the second caller during a predetermined time period from the time when the satisfaction level of the first caller becomes lower than the reference value.
[0022] The sensing data about the caller may include either or both of time series data of video around the caller that includes the caller's face as a subject and time series data of sound around the caller that includes the caller's voice. On the other hand, the sensing data about the caller is not limited to the above and may broadly include sensing data acquired by various sensors.
[0023] By performing the above-described processing, the information processing device 20 can notify the salesperson if the customer's satisfaction level during a remote interview is lower than a reference value. This allows the salesperson to more easily understand the customer's condition during the remote interview. Furthermore, based on the notification, the salesperson can conduct the interview according to the customer's condition, such as by changing the way the interview is conducted.
[0024] Furthermore, when the satisfaction level of the first party is lower than a reference value, the information processing device 20 may analyze the state of the second party based on sensing data related to the second party. Furthermore, the information processing device 20 may output the state of the second party obtained as a result of the analysis as a dissatisfaction cause state.
[0025] In addition, the information processing device 20 may control the second caller to be notified if it detects, based on sensing data regarding the second caller, that the second caller's state is causing dissatisfaction while the call is in progress.
[0026] The dissatisfaction causing condition may be information indicating the condition of the second caller, such as the second caller's facial expression, body movements such as gestures, clothing, speech content, or tone of speech.
[0027] By performing the above-described processing, the information processing device 20 can analyze the state of the sales representative when customer satisfaction falls below a reference value during a remote interview, and can infer the state of the sales representative that is contributing to the low customer satisfaction.
[0028] Furthermore, by performing the processing described above, the information processing device 20 can notify the sales representative if the sales representative's condition becomes a dissatisfaction-causing condition during a remote interview.
[0029] This allows sales representatives to recognize their own state during remote interviews that could be a factor in lowering customer satisfaction and respond flexibly.In addition, even after the interview, sales representatives can recognize their own state that could affect customer satisfaction, reflect on the interview, and use this to improve their interview skills.
[0030] Furthermore, the information processing device 20 may calculate, based on sensing data relating to the callers, whether or not each of the first caller and the second caller has spoken, and the speech duration indicating the duration of each speech.
[0031] The information processing device 20 may perform control to notify the second caller of information regarding the length of time each of the first caller and the second caller has been speaking, based on the speech time obtained as a result of the calculation.
[0032] By performing the above-described processing, the information processing device 20 can notify the sales representative of the proportion of the conversation that each of the customer and the sales representative is doing during the remote interview, thereby preventing the sales representative from either the customer or the sales representative from talking too much, which would result in a decrease in customer satisfaction.
[0033] In addition, while a call is being made, the information processing device 20 may determine the degree of busyness of the first caller based on the conditions of the first caller's surrounding environment estimated based on sensing data related to the first caller.
[0034] The information processing device 20 may perform control to notify the second caller of information relating to how busy the first caller is, depending on the result of the above-mentioned determination.
[0035] By performing the above-described processing, the information processing device 20 can notify the sales representative if it is estimated that the customer may be busy during a remote interview. This allows the sales representative to take into consideration the customer's situation, such as confirming with the customer that there is no problem with continuing the call or rescheduling the interview for another date.
[0036] As described above, the information processing system according to the present disclosure improves the convenience of users (customers and sales representatives) during remote interviews. Therefore, the information processing system can contribute to improving the quality of remote interviews.
[0037] An example of a system configuration that achieves the above will be described in detail below. Note that in the following description, as an example of application of the information processing system according to the present disclosure, a medical representative (hereinafter referred to as MR) who is a salesperson will mainly be described as having a video call with a doctor who is a client.
[0038] 1-2. System Configuration Example FIG. 1 is an explanatory diagram showing a configuration example of an information processing system according to an embodiment of the present disclosure.
[0039] As shown in FIG. 1 , an information processing system according to an embodiment of the present disclosure includes a plurality of caller terminals 10 and an information processing device 20 .
[0040] Each caller terminal 10 and the information processing device 20 are connected to each other via a network 30 so that they can communicate with each other.
[0041] 1 illustrates an example in which the information processing system according to the present embodiment includes two caller terminals 10, namely, caller terminal 10A and caller terminal 10B, but the number of caller terminals 10 according to the present embodiment is not particularly limited. For example, the information processing system according to the present embodiment may include three or more caller terminals 10.
[0042] (Caller Terminal 10) The caller terminal 10 according to this embodiment is an information terminal used by the caller U. The customer Ua is an example of a first caller. The customer Ua is, for example, a doctor. The salesperson Ub is an example of a second caller. The salesperson Ub is, for example, an MR.
[0043] The caller terminal 10 has a function of making calls such as video calls or voice calls by communicating with other caller terminals 10 via the information processing device 20. The customer Ua and the sales person Ub can conduct remote interviews via video calls using the caller terminal 10A or the caller terminal 10B, respectively.
[0044] Each of the caller terminals 10 includes various sensors such as a camera and a microphone, and captures video and audio of the customer Ua and the salesperson Ub. The caller terminals 10 transmit the captured video and audio to the information processing device 20.
[0045] Furthermore, the caller terminal 10 may include sensors other than a camera and a microphone. For example, the caller terminal 10 may include a sensor that acquires biometric information of the caller U, such as a heart rate sensor that is worn on the arm of the salesperson Ub and measures the heart rate (pulse rate). In this case, the caller terminal 10 may transmit the acquired various sensing data to the information processing device 20 in response to a request from the information processing device 20.
[0046] The caller terminal 10 also receives call screen and audio data, including video and audio of the other caller U, from the information processing device 20. The caller terminal 10 displays the call screen and outputs audio under the control of the information processing device 20.
[0047] 1, the caller terminal 10 is implemented as a personal computer (PC), but the caller terminal 10 may be implemented as another information processing terminal. For example, the caller terminal 10 may be implemented as a smartphone, a tablet terminal, a game console, a wearable device, or the like.
[0048] (Information processing device 20) The information processing device 20 of this embodiment controls the notification of the caller U's status related to the caller U's degree of satisfaction with the call, based on sensing data related to the caller U, such as the video and audio of the caller U received from the caller terminal 10, estimated based on the caller U's satisfaction with the call.
[0049] In the example shown in FIG. 1, the information processing device 20 estimates the degree of satisfaction with the call of the customer Ua, who is a doctor, based on the video and audio of the customer Ua received from the caller terminal 10A.
[0050] Furthermore, when the estimated degree of satisfaction of the customer Ua is lower than a reference value, the information processing device 20 analyzes the state of the sales representative Ub based on the sensing data on the sales representative Ub received from the caller terminal 10B. The information processing device 20 outputs the state of the sales representative Ub obtained as a result of the analysis as a dissatisfaction cause state.
[0051] The information processing device 20 learns the relationship between the output dissatisfaction cause state and the video of sales person Ub through machine learning, and uses a model obtained as a result of the learning to detect whether sales person Ub is in a dissatisfaction cause state from the video of sales person Ub while a call is being made between the caller terminal 10A and the caller terminal 10B.
[0052] When the information processing device 20 detects that the state of the salesperson Ub has become a dissatisfaction-causing state during a call, the information processing device 20 performs control to notify the caller terminal 10B used by the salesperson Ub.
[0053] More specifically, for example, the information processing device 20 may perform control to display a notification message or notification image indicating that sales person Ub is in a state that is causing dissatisfaction in a notification area on a call screen displayed on the caller terminal 10B, which includes an image of customer Ua.
[0054] In addition, the information processing device 20 analyzes either or both of the video data and audio data received from the caller terminal 10A and the caller terminal 10B to calculate the speech time indicating whether or not each of the customer Ua and the salesperson Ub has spoken, and the length of each speech.
[0055] The information processing device 20 performs control to notify the salesperson Ub of information regarding the length of time that each of the customer Ua and the salesperson Ub has been speaking, based on the speech time obtained as a result of the calculation.
[0056] In addition, the information processing device 20 analyzes either or both of the video and audio of the customer Ua while the call is in progress, and determines the degree of busyness of the customer Ua depending on the situation in the customer Ua's surrounding environment.
[0057] The information processing device 20 performs control to notify the sales person Ub of information relating to the busyness of the customer Ua according to the result of the determination.
[0058] (Network 5) The network 5 according to this embodiment mediates communication between the caller terminal 10 and the information processing device 20.
[0059] <1-3. Example of Functional Configuration> Next, an example of the functional configuration of the caller terminal 10 and the information processing device 20 that constitute the present information processing system that realizes the above will be described in detail.
[0060] <1-3-1. Caller Terminal 10> FIG. 2 is a block diagram showing an example configuration of the caller terminal 10 according to the first embodiment of the present disclosure.
[0061] As shown in FIG. 2, the caller terminal 10 according to this embodiment may include a communication unit 110, a sensor unit 120, a control unit 130, an output unit 140, and an input unit 150.
[0062] (Communication Unit 110 ) The communication unit 110 according to this embodiment communicates with the information processing device 20 via the network 5 .
[0063] The communication unit 110 transmits and receives various data relating to calls made with other caller terminals 10 to and from the information processing device 20 under the control of the control unit 130 .
[0064] For example, the communication unit 110 transmits video data of the caller U acquired by the sensor unit 120 to the information processing device 20. The communication unit 110 also transmits voice data of the caller U acquired by the sensor unit 120 to the information processing device 20.
[0065] Furthermore, the communication unit 110 may transmit various sensing data related to the caller U other than video or audio, acquired by the sensor unit 120, to the information processing device 20 in response to a request from the information processing device 20.
[0066] The communication unit 110 also receives, from the information processing device 20, data of a call screen including a video of the other caller U who is the call destination. The communication unit 110 also receives, from the information processing device 20, data of the voice of the other caller U.
[0067] The communication unit 110 may also receive, from the information processing device 20, operation instructions relating to display control of a call screen and output control of the voice of the other caller U.
[0068] (Sensor Unit 120) The sensor unit 120 is realized by various sensors that acquire sensing data related to the caller U who is using the caller terminal 10.
[0069] For example, the sensor unit 120 may include a camera that captures video of the caller U. In this case, the sensor unit 120 captures video of the caller U's surroundings, including the caller U's face as a subject, as sensing data related to the caller U.
[0070] Hereinafter, in this specification, the video of the vicinity of the caller U, including the video of the face of the caller U, acquired by the sensor unit 120, is also referred to as the “video of the caller U.” The video of the caller U may be time-series data including a plurality of frames associated with the capture time.
[0071] Furthermore, the sensor unit 120 may include a microphone that acquires the voice of the caller U. Such a sensor unit 120 further acquires time-series data of sounds around the caller U, including the voice of the caller U, as sensing data related to the caller U.
[0072] Hereinafter, in this specification, the sounds around the caller U, including the voice of the caller U, acquired by the sensor unit 120 will also be referred to as "the voice of the caller U."
[0073] (Control Unit 130) The control unit 130 has a function of controlling the overall operation of the caller terminal 10.
[0074] For example, the control unit 130 controls the communication unit 110 to transmit and receive various data related to calls made with other caller terminals 10 to and from the information processing device 20. As an example, the control unit 130 controls the communication unit 110 to transmit video and audio of the caller U to the information processing device 20.
[0075] (Output Unit 140) The output unit 140 is realized by an output device that outputs various types of information under the control of the control unit 130.
[0076] For example, the output unit 140 may be realized by including a display device capable of visually outputting various information. Under the control of the control unit 130, the output unit 140 displays a call screen including an image of the other caller U who is the call destination and has been received by the communication unit 110.
[0077] Furthermore, the output unit 140 is realized by including a sound output device such as a speaker that outputs sound. The output unit 140 outputs the sound of the other caller U under the control of the control unit 130.
[0078] (Input Unit 150) The input unit 150 has a function of accepting input of various information by the caller U.
[0079] For example, the input unit 150 may accept input of information about another caller U who is the other party in the call.
[0080] The input unit 150 may be realized by an input device such as a keyboard or a mouse. Alternatively, the input unit 150 may be realized as a touch panel. In this case, the input unit 150 may be configured integrally with the output unit 140.
[0081] The above has described an example of the configuration of the caller terminal 10 according to the present embodiment. Note that the above configuration described with reference to Fig. 2 is merely an example, and the configuration of the caller terminal 10 according to the present embodiment is not limited to this example.
[0082] The configuration of the caller terminal 10 according to this embodiment can be flexibly modified according to the specifications and operation.
[0083] <1-3-1. Information Processing Device 20> FIG. 3 is a block diagram showing an example of the functional configuration of the information processing device 20 according to the first embodiment of the present disclosure.
[0084] (Communication Unit 210) The communication unit 210 according to this embodiment has a function of communicating with other devices. For example, the communication unit 210 receives sensing data relating to the caller U from the caller terminal 10.
[0085] More specifically, for example, the communication unit 210 receives video data of the caller U from the caller terminal 10. The communication unit 210 also receives the voice of the caller U from the caller terminal 10.
[0086] Furthermore, under the control of the control unit 230, the communication unit 210 transmits to the caller terminal 10 a call screen displayed on the caller terminal 10, including an image of another caller U who is the call destination of the caller terminal 10. Furthermore, the communication unit 210 transmits the voice of the other caller U to the caller terminal 10.
[0087] (Storage Unit 220) The storage unit 220 stores programs and data for operating the control unit 230. The storage unit 220 can also temporarily store various data required in the course of operation of the control unit 230.
[0088] For example, the storage unit 220 stores various data obtained as a result of analysis of the video of the caller U performed by the control unit 230, which will be described later.
[0089] (Control Unit 230) The control unit 230 controls the overall operation of the information processing device 20. For example, the control unit 230 controls communication between the communication unit 210 and the caller terminal 10.
[0090] Furthermore, the control unit 230 may generate a call screen to be displayed on the caller terminal 10 based on video and audio data of the caller U received from the caller terminal 10. The control unit 230 may control the communication unit 210 to transmit the generated call screen and operation instructions related to the display of the call screen to the caller terminal 10.
[0091] In addition, the control unit 230 of this embodiment controls the notification of the caller U's status related to the degree of satisfaction with the call, based on the caller U's satisfaction with the call, estimated based on sensing data about the caller U.
[0092] (Dissatisfaction Estimation Process) In more detail, the control unit 230 according to this embodiment may estimate the customer Ua's degree of satisfaction with the call based on the video of the customer Ua.
[0093] The control unit 230 may use machine learning to learn the relationship between the video of the customer Ua and information indicating the satisfaction level of the customer Ua. The control unit 230 may use a model obtained as a result of the learning (hereinafter referred to as a dissatisfaction estimation model) to estimate the satisfaction level of the customer Ua based on the video of the customer Ua.
[0094] During the learning stage of the dissatisfaction estimation model, the control unit 230 acquires video data of the customer Ua received from the caller terminal 10. In addition to the video of the customer Ua during an actual remote interview conducted between the customer Ua and the sales person Ub, the control unit 230 may also acquire video of another caller U (hereinafter referred to as the customer Uc) who plays the role of a customer in a practice remote interview conducted by the sales person Ub with the customer Uc.
[0095] The control unit 230 performs a process of assigning a label to frames in the acquired video of the customer Ua or the video of the customer role Uc at times when the customer Ua or the customer role Uc shows discomfort or dissatisfaction.
[0096] For example, the control unit 230 may perform the label assignment process by accepting a label assignment operation by an administrator of the information processing device 20 to the video data of customer Ua or the video data of customer role Uc.
[0097] Alternatively, the control unit 230 may use a known method for analyzing human facial expressions from video to analyze the facial expressions of the customer Ua or the customer role Uc from the video of the customer Ua or the customer role Uc, thereby quantifying the satisfaction of the customer Ua or the customer role Uc.
[0098] For example, the control unit 230 may calculate the satisfaction level of the customer Ua or the customer role Uc from the video of the customer Ua or the customer role Uc using a FACS (Facial Action Coding System), which is a mechanism for encoding facial expressions appearing on a human face into a machine-discernible form by combining the presence or absence of movements of various facial muscles called Action Units (AUs).The control unit 230 may calculate the satisfaction level of the customer Ua from the video of the customer Ua using a known machine learning model (e.g., OpenFace) for recognizing FACS from the video of the customer Ua.
[0099] The control unit 230 may assign a label to a frame at a time when the calculated satisfaction level of the customer Ua or the customer Uc is lower than a reference value.
[0100] The control unit 230 stores and accumulates the video data of the customer Ua or customer role Uc to which the label has been assigned in the storage unit 220.
[0101] The control unit 230 may use the video data of the labeled customer Ua or customer role Uc stored in the storage unit 220 as learning data to train the dissatisfaction estimation model.
[0102] By performing the above-described processing, the control unit 230 can estimate the satisfaction level of the customer Ua based on the video of the customer Ua.
[0103] The control unit 230 may use the dissatisfaction estimation model constructed as described above to acquire video data of customer Ua from the caller terminal 10 when a remote interview is held between customer Ua and sales person Ub, and estimate the satisfaction level of customer Ua based on the video data of customer Ua.
[0104] Based on the estimation result, the control unit 230 may generate a video playback screen that highlights frames at times when the satisfaction level of the customer Ua is lower than a reference value.
[0105] 4 is a diagram showing an example of a video playback screen generated by the control unit 230. The control unit 230 may generate a video playback screen, such as screen D1 shown in FIG. 4, that includes a video of the customer Ua and a user interface 500 that indicates the current playback location of the video.
[0106] Furthermore, the control unit 230 may highlight, as with a highlight H1 on the screen D1, a playback portion in the video of the customer Ua that corresponds to a time when the customer Ua's satisfaction level is estimated to be lower than the reference value.
[0107] The control unit 230 may transmit the generated video playback screen and the extracted video of the salesperson Ub to the caller terminal 10B used by the salesperson Ub. This allows the salesperson Ub to efficiently review the video from the time when the customer Ua's satisfaction level was estimated to be low after the remote interview ends.
[0108] (Dissatisfaction Cause State Detection Process) Furthermore, the control unit 230 according to this embodiment may analyze the state of sales Ub based on sensing data related to sales Ub when the satisfaction level of customer Ua estimated based on sensing data related to caller U is lower than a reference value. The control unit 230 may output the state of sales Ub obtained as a result of the analysis as the dissatisfaction cause state.
[0109] In more detail, for example, when a remote interview is conducted between customer Ua and sales person Ub, the control unit 230 may acquire video data of customer Ua from the caller terminal 10, and estimate the satisfaction level of customer Ua based on the video data of customer Ua.
[0110] Furthermore, the control unit 230 may analyze the video and audio of salesperson Ub at a time when it is estimated that customer Ua's satisfaction level is lower than the reference value, based on the video data from salesperson Ub's remote interview.
[0111] Alternatively, the control unit 230 may analyze the video and audio of salesperson Ub during a time period that is within a certain range from the time when the satisfaction level of customer Ua is estimated to be lower than the reference value.
[0112] More specifically, the control unit 230 may analyze the state of business Ub during a predetermined time period from the time when the satisfaction level of the customer Ua becomes lower than the reference value.
[0113] The control unit 230 may output the state of sales Ub obtained as a result of the analysis as the dissatisfaction cause state. The state of sales Ub may include, for example, facial expressions, gestures, and other body language of sales Ub, the content of statements made by sales Ub, and the tone of voice of sales Ub.
[0114] The control unit 230 may control the storage unit 220 to store the video and audio of the remote interview with salesperson Ub and the data indicating the cause of salesperson Ub's dissatisfaction output as a result of the analysis as a sales facial expression and voice database.
[0115] The control unit 230 may use machine learning to learn the relationship between the video and audio of sales Ub stored in the sales facial expression and voice database and the state of sales Ub output as the dissatisfaction cause state. The control unit 230 may use a model obtained as a result of the learning (hereinafter, dissatisfaction cause state detection model) to detect whether the state of sales Ub is a dissatisfaction cause state.
[0116] By performing the above-described processing, the control unit 230 can estimate, based on the video and audio of sales Ub, the state of sales Ub related to the degree of customer Ua's satisfaction as the cause of dissatisfaction when customer Ua's satisfaction level is lower than a reference value.
[0117] (Notification control processing) In addition, the control unit 230 according to this embodiment may control the notification to sales Ub if it detects, based on sensing data relating to sales Ub, that the state of sales Ub is in a state that is causing dissatisfaction while a call is being made.
[0118] For example, the control unit 230 may perform control to output a display indicating that Sales Ub's state is causing dissatisfaction on the call screen displayed on the caller terminal 10B used by Sales Ub while the call is in progress.
[0119] Fig. 5 is a diagram showing an example of a call screen generated by the control unit 230. Screen D2 shown in Fig. 5 includes an image of the customer Ua. Screen D2 also includes a window W1 displaying an image of the sales representative Ub.
[0120] 6 is a diagram showing an example of a call screen and a notification display generated by the control unit 230. Compared to the screen D2 shown in FIG. 5, the screen D3 shown in FIG. 6 further includes the display of the notification display A1 and the notification display A2.
[0121] The notification display A1 is an example of a notification display indicating that the state of sales Ub is a dissatisfaction-causing state. The notification display A1 may be displayed in a color, such as yellow, that indicates a pre-set dissatisfaction-causing state.
[0122] If the control unit 230 detects that the state of sales Ub has become a state that causes dissatisfaction while a call is in progress, the control unit 230 may control a notification to sales Ub, for example by controlling a display such as notification display A1.
[0123] The notification display A2 is an example of a notification display that indicates, based on an image of the customer Ua, that the satisfaction level of the customer Ua is lower than a reference value. The notification display A2 may be a display of a preset color, such as red, that indicates that the satisfaction level of the customer Ua is lower than the reference value.
[0124] If the control unit 230 estimates that the satisfaction level of customer Ua has fallen below a reference value while the call is in progress, the control unit 230 may control a display such as notification display A1 shown in Figure 6 to notify sales person Ub that the satisfaction level of customer Ua has fallen below the reference value.
[0125] The notification display A1 and the notification display A2 shown in FIG. 6 are examples of notification displays generated by the control unit 230, and the notification displays generated by the control unit 230 are not limited to these examples. For example, the notification display A1 may be an image such as a mark, illustration, photograph, or avatar indicating that the customer Ua's satisfaction level is lower than a reference value. The image may be a still image or a moving image. The image may also be an image generated using AR (Augmented Reality) technology and superimposed on the call screen. The notification display A1 may also be a message indicating that the customer Ua's satisfaction level is lower than a reference value.
[0126] Similarly, the notification display A2 may be an image such as a mark, illustration, photograph, or avatar indicating that the state of salesperson Ub is causing dissatisfaction. The notification display A1 may be a message indicating that the state of salesperson Ub is causing dissatisfaction.
[0127] (Process for detecting the degree of busyness of the customer) In addition, the control unit 230 according to the present embodiment may determine the degree of busyness of the customer Ua according to the state of the surrounding environment of the customer Ua estimated based on the sensing data related to the customer Ua while the call is being made. In accordance with the result of the determination, the control unit 230 performs control to notify the sales representative Ub of information related to the busyness of the customer Ua.
[0128] For example, the control unit 230 may perform a process of determining whether the customer Ua is speaking by analyzing either or both of the image and the audio of the customer Ua while the call is being made.
[0129] More specifically, the control unit 230 may determine whether the customer Ua is speaking by analyzing the video of the customer Ua and detecting whether the customer Ua's mouth is moving. The control unit 230 may also determine whether the customer Ua is speaking based on the volume or the like indicated by the data of the customer Ua's voice. In addition, if the sensor unit 120 of the caller terminal 10A is realized by a microphone array configured with multiple microphones, the control unit 230 may determine whether the customer Ua is speaking by performing a process of estimating the direction of a sound source based on the voice of the customer Ua acquired by each microphone.
[0130] Furthermore, when the control unit 230 determines that the customer Ua is not speaking, it may calculate the noise level in the surrounding area other than the customer Ua by analyzing the level (e.g., sound intensity) of the voice data of the customer Ua received from the caller terminal 10A used by the customer Ua.
[0131] The control unit 230 may estimate the busyness level of the customer Ua based on the calculated noise level. If the calculated noise level exceeds a reference value, the control unit 230 may determine that the busyness level of the customer Ua is high and notify the sales person Ub of information indicating that the customer Ua is busy.
[0132] Alternatively, the control unit 230 may determine whether or not a person has passed around the customer Ua by analyzing the video of the customer Ua.
[0133] For example, the control unit 230 may estimate the face and body areas of the customer Ua based on the video of the customer Ua. Furthermore, the control unit 230 may detect areas of the video of the customer Ua other than the face and body of the customer Ua as the background. The control unit 230 may analyze changes in brightness and color of the background area and determine that a person has passed by if the amount of change is greater than a reference value.
[0134] If the number of times that it is determined that a person has passed behind customer Ua within a certain period of time exceeds a reference value, the control unit 230 may determine that customer Ua is very busy and notify sales representative Ub.
[0135] 7 is a diagram showing another example of a call screen and a busyness notification display generated by the control unit 230. The screen D4 shown in FIG. 7 includes an image of the customer Ua and a notification display A3. The notification display A3 is an example of a notification display indicating that the busyness level of the customer Ua is high.
[0136] The control unit 230 may control the notification to the sales representative Ub by controlling the display of a message on the call screen indicating that the customer Ua has been determined to be very busy, such as notification display A3.
[0137] Furthermore, the control unit 230 may perform control to notify the customer Ua of a notification display such as the notification display A3.
[0138] By performing the above-described process, salesperson Ub can easily understand that customer Ua is busy. Furthermore, when salesperson Ub determines that customer Ua is busy, salesperson Ub can take measures such as ending the meeting early or asking customer Ua about how busy he or she is at the moment.
[0139] In addition, the control unit 230 may accept an instruction not to notify customer Ua and sales person Ub of the busyness determination result, even if it is determined that customer Ua's busyness level is high based on customer Ua's setting operation.
[0140] (Process for calculating speech time) In addition, the control unit 230 according to this embodiment may calculate speech time indicating whether or not each of the customer Ua and the salesperson Ub has spoken and the length of each speech based on the sensing data of the customer Ua and the salesperson Ub.
[0141] The control unit 230 may perform control to notify the salesperson Ub of information regarding the length of time each of the customer Ua and salesperson Ub has been speaking, based on the calculated speaking time.
[0142] The information regarding the length of time may be information indicating the percentage of time that each of the customer Ua and the salesperson Ub spoke during the call.
[0143] Fig. 8 is a diagram showing an example of a notification display showing information related to speech time generated by the control unit 230. The speech time display 600 shown in Fig. 8 includes a notification display A4, a notification display A5, and a setting button T1.
[0144] The notification display A4 is an example of a display showing the elapsed time since the customer Ua and the salesperson Ub started talking. The notification display A5 is an example of a display showing the cumulative total speaking time of the customer Ua and the cumulative total speaking time of the salesperson Ub.
[0145] Furthermore, the setting button T1 is a button indicating each of multiple topics included in the scheduled scenario for sales Ub during the remote interview, which has been set in advance by sales Ub. By selecting the setting button T1 on the caller terminal 10B, sales Ub can notify the information processing device 20 that the topic has changed. When the setting button T1 is selected, the control unit 230 may reset the count of the cumulative speaking time of customer Ua and the cumulative speaking time of sales Ub in the notification display A4, and perform control to restart calculation of the type time from 0.
[0146] The control unit 230 may generate a call screen including a display such as the speech time display 600 shown in Figure 8 and control the display on the caller terminal 10B used by the caller U2, thereby controlling the notification of information regarding the length of the speech time to the caller U2.
[0147] The control unit 230 may also receive in advance settings for the customer Ua with whom the sales representative Ub will conduct a remote interview, the purpose of the remote interview, and the scheduled time for the remote interview. The control unit 230 may also generate a schedule scenario for the remote interview based on the information on the customer Ua, the purpose of the remote interview, and the scheduled time information for which settings have been received. The schedule scenario may include example topics for the remote interview and an example of the scheduled time allocation for each topic.
[0148] 9 is a diagram showing an example of a screen generated by the control unit 230 for accepting settings for the remote interview partner, purpose, and scheduled time. As shown in FIG. 9, the control unit 230 may accept selection operations of selection buttons B1 to B3 by sales Ub on a setting acceptance screen such as screen D5. The control unit 230 may also generate a scheduled scenario when selection button B4 is pressed.
[0149] In addition, after the remote interview begins, the control unit 230 may calculate the remaining time of the scheduled time allocated to each topic based on the previously generated scheduled scenario, the calculation results of the customer Ua's speaking time, and the calculation results of the sales person Ub's speaking time, and may control the calculation to notify sales person Ub using notification display A4, notification display A5, etc.
[0150] By performing the above-described processing, salesperson Ub can more easily grasp situations such as when either salesperson Ub or customer Ua is talking too much, and can take action such as changing the way the interview is conducted.
[0151] An example of the functional configuration of the information processing device 20 according to this embodiment has been described above with reference to Figures 3 to 9. Note that the above description using Figures 3 to 9 is merely an example, and the configuration of the information processing device 20 according to this embodiment is not limited to this example.
[0152] The configuration of the information processing device 20 according to this embodiment can be flexibly modified according to the specifications and operation.
[0153] 1-4. Operational Example Next, an operational example of the information processing system according to this embodiment will be described with reference to FIGS.
[0154] 10 is a flowchart illustrating an example of operation of the information processing system according to this embodiment in the learning stage of the dissatisfaction estimation model. The information processing device 20 learns the dissatisfaction estimation model by performing operation processing according to the flow shown in the flowchart of FIG.
[0155] The communication unit 210 acquires the video of the customer Ua from the caller terminal 10A (S101).
[0156] The control unit 230 assigns a label to frames of the acquired video of the customer Ua that show a facial expression indicating dissatisfaction (S103).
[0157] The control unit 230 repeats the processes of S101 and S103 until all frames containing a dissatisfied facial expression among the multiple frames included in the acquired video of the customer Ua have been labeled (S105 / NO).
[0158] When the control unit 230 completes assigning labels to the frames included in the acquired video of the customer Ua (S105 / YES), the control unit 230 performs training of a dissatisfaction estimation model based on the data of the video of the customer Ua (S107).
[0159] An example of the operation process in the learning stage of the dissatisfaction estimation model of the information processing system according to this embodiment has been described above with reference to FIG.
[0160] Next, an example of operation processing in the learning stage of the dissatisfaction cause state detection model of the information processing system according to this embodiment will be described.
[0161] 11 is a second flowchart illustrating an example of operation of the information processing system according to this embodiment in the learning stage of the dissatisfaction cause state detection model. The information processing system according to this embodiment may operate according to the processing flow shown in FIG. 11 after the dissatisfaction estimation model is constructed according to the operation processing described with reference to FIG.
[0162] First, the communication unit 210 of the information processing device 20 acquires, from the caller terminal 10, an image of the customer Ua relating to the remote interview conducted between the customer Ua and the sales person Ub (S201).
[0163] The control unit 230 estimates the satisfaction level of the customer Ua based on the acquired video of the customer Ua using the dissatisfaction estimation model. If the satisfaction level of the customer Ua is lower than a reference value, the control unit 230 estimates that the customer Ua is dissatisfied (S203).
[0164] The control unit 230 identifies a frame in which the customer Ua is estimated to be dissatisfied, from among a plurality of frames included in the acquired video of the customer Ua (S205).
[0165] Next, the control unit 230 acquires video of the salesperson Ub during the remote interview, which video corresponds to the video of the customer Ua acquired in S201 (S207).
[0166] The control unit 230 analyzes the state of salesperson Ub in the frames included in the acquired video of salesperson Ub that correspond to the time of the frame in which customer Ua was identified as expressing dissatisfaction in S205 (S209).
[0167] The control unit 230 outputs the state of sales Ub obtained as a result of the analysis as a dissatisfaction cause state and stores it in the storage unit 220 (S211).
[0168] The control unit 230 repeats the processes of S207 to S211 until analysis of the state of business Ub at the time corresponding to all frames identified in S205 as the customer Ua expressing dissatisfaction is completed (S213 / NO).
[0169] When the analysis of the state of Sales Ub at the time corresponding to all identified frames is completed (S213 / YES), the control unit 230 learns a dissatisfaction cause state detection model based on the video of Sales Ub acquired in S207 and the dissatisfaction cause state of Sales Ub stored in the memory unit 220 (S215).
[0170] An example of the operation process in the learning stage of the dissatisfaction cause state detection model of the information processing system according to this embodiment has been described above with reference to FIG.
[0171] 12 is a flowchart showing an example of the operation of the information processing system according to this embodiment at the stage of inferring the cause of dissatisfaction. When a remote interview via video call between a customer Ua and a salesperson Ub is started, the information processing system according to this embodiment performs operation processing according to the flow shown in FIG. 12 while the call is continuing.
[0172] In addition, the information processing device 20 according to this embodiment may operate in accordance with the processing flow described with reference to FIG. 11, and after constructing the dissatisfaction cause state detection model, may operate in accordance with the processing flow shown in FIG. 12.
[0173] First, the control unit 230 acquires the video of the customer Ua and the video of the salesperson Ub from the caller terminal 10A and the caller terminal 10B, respectively (S301).
[0174] The control unit 230 performs a process of detecting whether or not salesperson Ub is in a dissatisfaction-causing state based on the video of salesperson Ub, using the dissatisfaction-causing state detection model (S303).
[0175] If it is not detected that sales Ub is in a dissatisfaction-causing state (S305 / NO), the process proceeds to S309.
[0176] If it is detected that Sales Ub is in a state that causes dissatisfaction (S305 / YES), the control unit 230 generates a notification indicating that Sales Ub is in a state that causes dissatisfaction, to be displayed on the caller terminal 10B used by Sales Ub (S307).
[0177] Next, the control unit 230 calculates the speaking time of each of the customer Ua and the salesperson Ub based on the video of the customer Ua and the video of the salesperson Ub (S309).
[0178] The control unit 230 calculates the busyness level of the customer Ua based on the video of the customer Ua (S311).
[0179] The control unit 230 controls the communication unit 210 to send to the caller terminal 10B a notification indicating that sales person Ub is in a dissatisfied state, a notification indicating the speaking time of customer Ua and sales person Ub, and a notification indicating the degree of busyness of customer Ua (S313).
[0180] The information processing device 20 repeats the processes of S301 to S313 until an operation to end the call is detected on the caller terminal 10A or the caller terminal 10b (S315 / NO).
[0181] If an operation to end the call is detected in the caller terminal 10A or the caller terminal 10B (S315 / YES), the control unit 230 ends the series of processes.
[0182] An example of the operation of the information processing system according to this embodiment at the stage of inferring the dissatisfaction cause state has been described above with reference to FIG.
[0183] The above describes an example of operation in which the control unit 230 acquires video of the customer Ua and video of the salesperson Ub from the caller terminal 10 as sensing data related to the caller U. However, the information processing device 20 according to this embodiment may acquire the voice of the customer Ua and the voice of the salesperson Ub in addition to the video of the customer Ua and the video of the salesperson Ub.
[0184] The above describes an example of an operation in which the control unit 230 controls to notify the sales person Ub when the satisfaction level of the customer Ua is lower than the reference value. The control unit 230 may also control to notify the customer Ua in addition to the sales person Ub when the satisfaction level of the customer Ua is lower than the reference value.
[0185] The information processing system according to the first embodiment of the present disclosure has been described above. In the above, an example has been described in which the control unit 230 provides multiple functions, including estimating the caller's dissatisfaction (estimating satisfaction level), notifying the cause of dissatisfaction, notifying information related to the length of speech, and notifying the busyness level. However, the present disclosure is not limited to such an example. For example, each of the functions of estimating the caller's dissatisfaction (estimating satisfaction level), notifying the cause of dissatisfaction, notifying information related to the length of speech, and notifying the busyness level may be provided individually. Furthermore, two or more of the above-described functions may be provided in combination. Furthermore, the processing of each function does not necessarily have to be performed in the above-described order and can be flexibly changed.
[0186] Next, an information processing system according to a second embodiment of the present disclosure will be described.
[0187] 2. Second Embodiment 2-1. Overview An information processing system according to a second embodiment of the present disclosure has a function of generating information indicating an analysis result of the remote interview, such as the customer Ua's satisfaction level, after the remote interview is conducted between the customer Ua and the sales representative Ub via a video call or voice call. The information indicating the analysis result makes it easy for the sales representative Ub to review the remote interview after the remote interview.
[0188] Such an information processing system according to the second embodiment of the present disclosure can have the same system configuration as that of the first embodiment. More specifically, the information processing system according to the present embodiment is realized by an information processing device 21 configured to be able to communicate with the caller terminal 10A and the caller terminal 10B via the network 5.
[0189] An information processing device 21 that realizes the information processing system according to this embodiment has a new functional configuration in addition to the functional configuration of the information processing device 20 described in the first embodiment.
[0190] An example of the functional configuration of the information processing device 21 that realizes the information processing system according to the second embodiment will be described in detail below.
[0191] 13 is a block diagram showing an example of the functional configuration of an information processing device 21 according to the second embodiment. As shown in FIG. 13, the information processing device 21 includes a communication unit 210, a storage unit 220, and a control unit 231.
[0192] The communication unit 210 and the storage unit 220 are the same as those described with reference to FIG. 3 in the first embodiment, and therefore a duplicated description will be omitted.
[0193] (Control unit 231) The control unit 231 is a component corresponding to the control unit 230 described with reference to Fig. 3. The control unit 231 controls the overall operation of the information processing device 21. For example, the control unit 231 controls communication between the communication unit 210 and other devices.
[0194] In addition, similar to the control unit 230 of the information processing device 20 in the first embodiment, the control unit 231 has the function of learning a dissatisfaction estimation model and a dissatisfaction cause state detection model based on sensing data regarding the caller U acquired from the caller terminal 10.
[0195] The control unit 231 also has a function of using a dissatisfaction estimation model to estimate whether or not the customer Ua is dissatisfied based on sensing data related to the customer Ua.
[0196] The control unit 231 also has a function of detecting whether the state of sales Ub is in a dissatisfaction cause state or dissatisfaction based on sensing data related to sales Ub, using a dissatisfaction cause state detection model.
[0197] Furthermore, the control unit 231 according to this embodiment has a function of detecting the degree of tension of salesperson Ub and the state causing tension of salesperson Ub based on sensing data related to salesperson Ub.
[0198] (Tension Level Estimation Process) In more detail, the control unit 231 according to this embodiment can acquire the tension level of sales person Ub using the following methods based on sensing data related to sales person Ub.
[0199] For example, the control unit 231 may construct a machine learning model (hereinafter referred to as a tension level estimation model) that estimates the tension level of sales person Ub based on sensing data related to sales person Ub.
[0200] For example, the control unit 231 may construct a tension level estimation model by learning using as learning data a video of Sales Ub and labels assigned to frames in the video of Sales Ub in which Sales Ub is nervous.
[0201] Alternatively, the control unit 231 may perform a process of analyzing the facial expression of Sales Ub from a video of Sales Ub. The control unit 231 may train a machine learning model that estimates the level of tension of Sales Ub based on the type of facial expression of Sales Ub obtained as a result of the analysis. The control unit 231 may estimate the level of tension of Sales Ub using the machine learning model obtained as a result of the training as a tension level estimation model.
[0202] Alternatively, the control unit 231 may estimate the heart rate of the salesperson Ub based on a biological signal emitted by a biological phenomenon of the salesperson Ub. The control unit 231 may estimate the level of tension of the salesperson Ub by comparing the heart rate of the salesperson Ub obtained as a result of the estimation with a reference value.
[0203] For example, the control unit 231 may estimate the heart rate of salesperson Ub by electrocardiography based on a biological signal acquired by an ECG (Electrocardiogram) sensor attached to the body of salesperson Ub.
[0204] Alternatively, the control unit 231 may estimate the heart rate (pulse rate) of salesperson Ub using photoplethysmography (PPG) based on biological signals acquired by a PPG (Photoplethysmography) sensor attached to the body of salesperson Ub.
[0205] Alternatively, the control unit 231 may estimate the heart rate of the salesperson Ub based on a biological signal acquired by an LDF (Laser Doppler Flowmetry) sensor.
[0206] Alternatively, the control unit 231 may estimate the heart rate by interpreting changes in the skin color of salesperson Ub in the video as a PPG signal.
[0207] Alternatively, the control unit 231 may estimate the level of tension of the salesperson Ub based on the blood pressure value of the salesperson Ub obtained by a continuous blood pressure monitor worn by the salesperson Ub.
[0208] Furthermore, the control unit 231 may estimate the level of tension of the sales person Ub based on other biological signals of the sales person Ub, such as electroencephalogram (EEG) or electromyography (EMG).
[0209] The control unit 231 may identify a time during the remote interview call when salesperson Ub's level of tension is higher than a reference value, and generate a video playback screen that highlights that point.
[0210] Fig. 14 is a diagram showing an example of a video playback screen generated by the control unit 231. Screen D5 shown in Fig. 14 includes a video of salesperson Ub. Screen D5 also includes a window W2 displaying a video of customer Ua.
[0211] The control unit 231 may highlight a playback portion in the video of sales person Ub that corresponds to a time when the level of tension of sales person Ub is estimated to be higher than the reference value, as in the highlight H2 shown in FIG.
[0212] The control unit 231 may transmit the generated video playback screen to the caller terminal 10B used by the salesperson Ub. This allows the salesperson Ub to efficiently review the video of the parts that made him or her feel nervous after the remote interview. Therefore, the salesperson Ub can improve his or her interview skills by reviewing situations, topics, or causes of nervousness that he or she finds difficult.
[0213] (Stress Causing State Detection Process) The control unit 231 according to this embodiment also has a function of analyzing the state of the customer Ua when the level of tension of the salesperson Ub is higher than a reference value.
[0214] The control unit 231 may analyze the state of the customer Ua in a predetermined time period from the time when the level of tension of the salesperson Ub is higher than a reference value, among frames included in the video of the salesperson Ub during the remote interview.
[0215] The control unit 231 outputs the state of the customer Ua obtained as a result of the analysis as a tension-causing state.
[0216] (Processing for calculating topics and speaking time for each topic) In addition, the control unit 231 according to this embodiment may calculate the time that customer Ua and salesperson Ub spend speaking about predetermined keywords (topics) by analyzing the video and audio of customer Ua and the video and audio of salesperson Ub.
[0217] The control unit 231 may determine the most frequently mentioned keywords in order of the number of times they were mentioned during the remote interview based on the calculated speaking time for each keyword.
[0218] The control unit 231 may also convert the voice data of the determined top keywords into text information, and output the determined top keywords.
[0219] Furthermore, the control unit 231 may generate a video playback screen that highlights the time when utterances for each keyword were detected during the remote interview call.
[0220] Fig. 15 is a diagram showing another example of a video playback screen generated by the control unit 231. Screen D6 shown in Fig. 15 includes videos of customer Ua and salesperson Ub. Screen D6 may also include the display of speech time display 600 described with reference to Fig. 8 in the first embodiment.
[0221] 15, the screen D6 includes a user interface 510. The control unit 231 may generate a video playback screen including a display indicating keywords to be detected, such as a keyword list display 511 shown in FIG.
[0222] Furthermore, the control unit 231 may generate a video playback screen including a display showing a color example indicating the utterance time of each keyword, such as the keyword color display 512 shown in FIG.
[0223] Furthermore, the control unit 231 may generate a video playback screen including a highlight display indicating the time when speech for each keyword was detected, for each color of each keyword indicated by the keyword color display 512, such as the speech time by keyword 513 shown in Figure 15.
[0224] The control unit 231 may transmit the generated video playback screen to the caller terminal 10B used by the sales representative Ub. This allows the sales representative Ub to easily review which topics were mentioned most frequently after the remote interview. Therefore, the sales representative Ub can, for example, analyze topics that interest the customer Ua and use the analysis results in future interviews with the customer Ua.
[0225] (Generation of call summary screen) In addition, the control unit 231 in this embodiment has the function of generating a call summary screen that includes information indicating the customer Ua's satisfaction level and the cause of dissatisfaction of the sales person Ub from the start to the end of the remote interview call.
[0226] The call summary screen may include information on the results of analyzing the sensing data regarding the other customers Ua and the salesperson Ub, such as the level of tension of the salesperson Ub or the most frequently mentioned topics, as described above.
[0227] Fig. 16 is a diagram showing an example of a call summary screen generated by the control unit 231. In the example shown in Fig. 16, the call summary screen SD includes eight analysis result items C, namely, analysis result items C1 to C8.
[0228] The control unit 231 may generate a call summary screen including an analysis result indicating an overall evaluation of the degree of satisfaction of the customer Ua from the start to the end of the remote interview call, such as the analysis result item C1 shown in Fig. 16. The control unit 231 may calculate the overall evaluation based on the change in the value of the satisfaction of the customer Ua estimated using the dissatisfaction estimation model.
[0229] The control unit 231 may also generate a call summary screen that includes information indicating the topics that were mentioned frequently during the call and the proportion of speaking time for each topic, such as analysis result item C2.
[0230] The control unit 231 may also generate a call summary screen that includes information on the state of tension causing the time when it was detected that salesperson Ub's tension level was higher than the reference value during the call, such as analysis result item C3.
[0231] The control unit 231 may also generate a summary screen including information on the analysis results of the state causing tension at the time when it was detected that salesperson Ub's tension level was higher than the reference value during the call, such as analysis result item C4.
[0232] The control unit 231 may also generate a graph showing the time when a situation that is likely to cause mental stress to salesperson Ub was detected, as in analysis result item C5. The control unit 231 may also generate a call summary screen including the graph. For example, the control unit 231 may analyze video of customer Ua to calculate the time when customer Ua showed an angry expression or the time when customer Ua glared at salesperson Ub, and generate a graph like analysis result item C5.
[0233] Furthermore, the control unit 231 may generate a call summary screen including information indicating the analysis result of Sales Ub's state at the time when Customer Ua's satisfaction level was lower than the reference value, such as analysis result item C6. The analysis result of Sales Ub's state may be a cropped image of Sales Ub's video showing Sales Ub's state. Alternatively, the analysis result of Sales Ub's state may be text information indicating the content of Sales Ub's remarks. Furthermore, the analysis result of Sales Ub's state may be information indicating Sales Ub's gestures or body language.
[0234] The control unit 231 may also generate a call summary screen including information indicating the transition of the customer Ua's facial expression, such as the analysis result item C7, which shows the analysis results of the customer Ua's facial expression at regular intervals.
[0235] The control unit 231 may also generate a call summary screen that includes facial expression information indicating the average level of satisfaction of the customer Ua from the start to the end of the call, such as the analysis result item C8.
[0236] The control unit 231 may perform control to transmit the generated call summary screen to the caller terminal 10B used by sales Ub.
[0237] By performing the above-described processing, salesperson Ub can more easily analyze customer Ua's satisfaction level and salesperson Ub's words and actions related to customer Ua's satisfaction after the remote interview is completed.
[0238] An example of the functional configuration of the information processing device 21 according to the second embodiment of the present disclosure has been described above with reference to Fig. 13 to Fig. 16. Next, an example of the operation of the information processing system according to the second embodiment of the present disclosure will be described with reference to Fig. 17. <2-3. Example of Operation> Fig. 17 is a flowchart showing an example of the operation of the information processing system according to the second embodiment of the present disclosure.
[0239] First, the control unit 231 acquires sensing data of the customer Ua and the salesperson Ub (S401).
[0240] The control unit 231 estimates the satisfaction level of the customer Ua based on the acquired video of the customer Ua (S403).
[0241] The control unit 231 analyzes the state of the sales Ub at the time when the satisfaction level of the customer Ua is lower than the reference value, and outputs the dissatisfaction cause state (S405).
[0242] The control unit 231 detects the degree of tension of the salesperson Ub based on the video of the salesperson Ub or sensing data such as biological signals (S407).
[0243] The control unit 231 analyzes the state of the customer Ua at the time when the tension level of the salesperson Ub is higher than the reference value, and outputs the tension-causing state (S409).
[0244] The control unit 231 analyzes the video and audio of the customer Ua and the video and audio of the salesperson Ub, and calculates the speaking time (S411).
[0245] The control unit 231 generates a call summary screen based on the analysis results obtained as a result of the processes of S403 to S411 (S413).
[0246] The control unit 231 transmits the call summary screen from the communication unit 210 to the caller terminal 10 and controls the display thereof (S415).
[0247] The information processing system according to the second embodiment of the present disclosure has been described above. In the above description, an example has been described in which the control unit 231 provides multiple functions, including a process for estimating a caller's tension level, a process for detecting a state that causes tension, a calculation of a speaking time, and a process for generating a call summary screen. However, the present disclosure is not limited to such an example. For example, each of the functions, including a process for estimating a caller's tension level, a process for detecting a state that causes tension, a calculation of a speaking time, and a process for generating a call summary screen, may be provided individually. Furthermore, two or more of the above-described functions may be provided in combination. Furthermore, the processing of each function does not necessarily have to be performed in the above-described order, and can be flexibly changed.
[0248] Next, an information processing system according to a third embodiment of the present disclosure will be described.
[0249] <3. Third embodiment> <3-1. Overview> An information processing system according to a third embodiment of the present disclosure provides functions that can be applied mainly to cases where a remote interview is conducted as practice between a sales person Ub and a customer person Uc who plays the role of a customer.
[0250] More specifically, the information processing system according to the third embodiment of the present disclosure processes the video and audio of the customer Uc, thereby changing the appearance and voice of the customer Uc in the video to make him appear as if he were a different person. This allows the salesperson Ub to feel as if the customer Uc is unfamiliar to him, even when a remote interview is conducted as part of a practice session. This increases the realism of the remote interview practice session for the salesperson Ub. This is also expected to improve the effectiveness of the remote interview practice session.
[0251] The information processing system according to the third embodiment of the present disclosure can have the same system configuration as the first and second embodiments. More specifically, the information processing system according to the present embodiment is realized by the caller terminal 10C used by the customer Uc, the caller terminal 10B, and the information processing device 22.
[0252] The information processing device 22 that realizes the information processing system according to this embodiment further has the following new functional configuration in addition to the functional configuration of the information processing device 20 described in the first embodiment.
[0253] An example of the functional configuration of the information processing device 22 that realizes the information processing system according to the third embodiment will be described in detail below.
[0254] 18 is a block diagram showing an example of the functional configuration of the information processing device 22 according to the third embodiment. As shown in FIG. 18, the information processing device 22 includes a communication unit 210, a storage unit 220, and a control unit 232.
[0255] The communication unit 210 and the storage unit 220 are the same as those described with reference to FIG. 3 in the first embodiment, and therefore a duplicated description will be omitted.
[0256] (Control unit 232) The control unit 232 is a component corresponding to the control unit 230 described with reference to Fig. 3. The control unit 232 controls the overall operation of the information processing device 22. For example, the control unit 232 controls communication between the communication unit 210 and other devices.
[0257] Similar to the control unit 230 of the information processing device 20 in the first embodiment, the control unit 232 has the function of learning a dissatisfaction estimation model and a dissatisfaction cause state detection model based on sensing data relating to the caller U acquired from the caller terminal 10.
[0258] Furthermore, the control unit 232 acquires the video of the customer Uc from the caller terminal 10C, and estimates the time when the satisfaction level of the customer Uc will be lower than the reference value using the dissatisfaction estimation model.
[0259] (Facial Image and Voice Conversion Processing) Furthermore, the control unit 232 according to this embodiment analyzes the facial image of the customer Uc, and performs processing to synthesize and process the facial image of the customer Uc with other facial images prepared in advance.
[0260] At this time, the control unit 232 performs calibration based on the facial image of the customer Uc capturing the facial movements of the customer Uc so that the movements of each part of the converted facial image coincide with the movements of each part of the actual face of the customer Uc.
[0261] Alternatively, the control unit 232 may accept a setting operation by the customer Uc to set the value of a parameter that adjusts the degree of transformation of the face image and voice.
[0262] FIG. 19 is a diagram showing an example of a parameter setting acceptance screen relating to the conversion of a facial image and a voice by the control unit 232. As shown in FIG.
[0263] As shown in screen D7 in FIG. 19, the control unit 232 may generate a parameter setting acceptance screen that includes a display of the pre-conversion image of the customer role Uc and the post-conversion caller image GU showing the post-conversion image.
[0264] Furthermore, the control unit 232 may change the value of a parameter that determines the degree of conversion of the facial image and voice of the customer Uc in response to the pressing operation of the selection button B5 and the selection button B6 by the customer Uc.
[0265] Furthermore, when the customer Uc presses the selection button B7, the control unit 232 may perform image processing to change the facial expression of the converted caller image GU to one showing dissatisfaction.
[0266] (Notification Processing According to a Plan Scenario) Furthermore, the control unit 232 according to this embodiment may receive a setting of a plan scenario for a practice remote interview to be conducted between the sales person Ub and the customer Uc. The plan scenario may include topics to be discussed, time allocation for each topic, and planned points in which the customer Uc expresses dissatisfaction.
[0267] The control unit 232 may estimate the satisfaction level of the customer Uc while the remote interview call is being held between the sales person Ub and the customer Uc. If the satisfaction level of the customer Uc at the time set as a point of dissatisfaction in the scheduled scenario is higher than a reference value, the control unit 232 may perform control to notify the customer Uc.
[0268] 20 is a diagram showing an example of a call screen and a notification display generated by the control unit 232. The screen D8 shown in FIG. 20 includes a converted caller image GU of the customer Uc. The screen D8 also includes a window W1 displaying an image of the salesperson Ub.
[0269] The control unit 232 may generate a notification display, such as notification display A6 shown in Figure 20, indicating that the dissatisfaction level planned for the customer Uc in the scheduled scenario does not match the dissatisfaction level of the customer Uc's current facial expression.
[0270] An example of the functional configuration of the information processing device 22 according to the third embodiment of the present disclosure has been described above with reference to FIGS.
[0271] Next, an example of the operation of the information processing system according to this embodiment will be described with reference to FIG.
[0272] 3-3. Operation Example FIG. 21 is a flowchart illustrating an operation example of the information processing system according to the third embodiment of the present disclosure.
[0273] First, the control unit 232 acquires video and audio of the customer Uc (S501).
[0274] The control unit 232 accepts the setting operation of the parameters of the face image, the voice, and the dissatisfaction level (S503).
[0275] The control unit 232 accepts a setting operation for the scheduled scenario, which includes information on the scheduled parts where the customer Uc expresses dissatisfaction (S505).
[0276] If an operation to start a call has not been performed on the caller terminal 10B or the caller terminal 10C (S507 / NO), the process proceeds to S517.
[0277] When an operation to start a call is performed on the caller terminal 10B or the caller terminal 10C (S507 / YES), the control unit 232 converts the facial image and voice of the customer Uc based on the setting values of the parameters received in S503 (S509).
[0278] The control unit 232 estimates the satisfaction level of the customer Uc using the dissatisfaction estimation model (S511).
[0279] If the estimated dissatisfaction level of the customer Uc is in line with the scheduled scenario (S513 / YES), the process proceeds to S517.
[0280] If the estimated dissatisfaction level of the customer Uc is not the same as the planned scenario (S513 / NO), the control unit 232 performs control to notify the customer Uc (S515).
[0281] The control unit 230 repeats the processes of S509 to S515 until an operation to end the call is detected in the caller terminal 10B or the caller terminal 10C (S517 / NO).
[0282] When an operation to end the call is detected in the caller terminal 10B or the caller terminal 10C (S517 / UES), the information processing device 22 ends the series of processes.
[0283] The information processing system according to the third embodiment of the present disclosure has been described above. The above describes an example in which the control unit 232 provides multiple functions, including a process for converting a caller's facial image, a process for converting a caller's voice, and a notification process according to a scheduled scenario. However, the present disclosure is not limited to such an example. For example, the functions of the process for converting a caller's facial image, the process for converting a caller's voice, and a notification process according to a scheduled scenario may be provided individually. Furthermore, two or more of the above-described functions may be provided in combination. Furthermore, the processing of each function does not necessarily have to be performed in the order described above, and can be flexibly changed.
[0284] Next, an information processing system according to a fourth embodiment of the present disclosure will be described.
[0285] <4. Fourth embodiment> <4-1. Overview> An information processing system according to a fourth embodiment of the present disclosure provides an application function that enables a plurality of salespeople who communicate with a customer Ua to conduct remote interviews via video or voice calls.
[0286] The information processing system according to the fourth embodiment of the present disclosure can have the same system configuration as the first, second, and third embodiments. More specifically, the information processing system according to the present embodiment is realized by a caller terminal 10A, a caller terminal 10B used by each of a plurality of sales representatives, and an information processing device 23.
[0287] An example of the functional configuration of the information processing device 23 that realizes the information processing system according to the fourth embodiment will be described in detail below.
[0288] 22 is a block diagram showing an example of the functional configuration of an information processing device 23 according to the fourth embodiment. As shown in FIG. 22, the information processing device 23 includes a communication unit 210, a storage unit 220, and a control unit 233.
[0289] The communication unit 210 and the storage unit 220 are the same as those described with reference to FIG. 3 in the first embodiment, and therefore a duplicated description will be omitted.
[0290] (Control unit 233) The control unit 233 is a component corresponding to the control unit 230 described with reference to Fig. 3. The control unit 233 controls the overall operation of the information processing device 23. For example, the control unit 233 controls communication between the communication unit 210 and other devices.
[0291] (Online Waiting Room Function) The control unit 233 according to this embodiment provides, for each customer Ua, an online waiting room function that enables salespeople who wish to have a remote interview with the customer Ua to wait their turn.
[0292] More specifically, the control unit 233 has a function of generating various user interface screens related to the online waiting room function in response to requests received by the communication unit 210 from the caller terminal 10. The control unit 233 may cause the communication unit 210 to transmit data related to the generated user interface screens to the caller terminal 10.
[0293] (Doctor's call availability notification function) For example, the control unit 233 according to this embodiment has a function of generating a doctor-side standby screen that accepts an operation indicating that the customer Ua is ready to start a call.
[0294] Fig. 23 is a diagram showing an example of a doctor-side standby screen generated by the control unit 233. As shown in Fig. 23, the screen D9 includes a selection button E1.
[0295] When the control unit 233 receives a notification from the caller terminal 10 that the pressing of the selection button E1 has been detected on the caller terminal 10A, the control unit 233 may perform control to notify the caller terminal 10 used by the waiting sales representative that the customer Ua is now able to start a call.
[0296] 24 is a diagram showing an example of a sales-side standby screen generated by the control unit 233. As shown in FIG. 24, screen D10 includes a notification display A7. The control unit 233 may control the transmission and display of a screen such as screen D10 to the caller terminal 10, thereby notifying the waiting sales person that the customer Ua is ready to start a call.
[0297] Furthermore, the control unit 233 may control the start of a call between the customer Ua and the sales representative when, after the customer Ua has been notified that the customer Ua is now able to start a call, both the customer Ua and the sales representative to whom the notification was sent perform a call start operation.
[0298] This allows the customer Ua to start a meeting with the salesperson when they have time to do so during their busy schedules. Furthermore, the customer Ua can attend the meeting without feeling rushed, compared to when the salesperson directly visits the hospital where the customer Ua works.
[0299] (Sales Information Display and Priority Setting Function) The control unit 233 according to this embodiment may also generate a doctor-side standby screen including information on sales personnel who are waiting to meet with the customer Ua.
[0300] Fig. 25 is a diagram showing an example of a waiting room screen generated by the control unit 233. As shown in Fig. 25, the screen D11 includes information 700 about sales with appointments and information 710 about sales without appointments.
[0301] The control unit 233 may divide sales into those with prior appointments and those without appointments who wish to have a walk-in interview, and generate a waiting room screen that includes a display showing information about each sales person who is waiting.
[0302] When the control unit 233 receives a request from the caller terminal 10 used by the salesperson to enter the waiting room for a remote interview with the customer Ua, the control unit 233 may control the display of information about the salesperson on the doctor's waiting screen.
[0303] The control unit 233 may set the waiting order for each business in the order in which requests to enter the waiting room are received.
[0304] Sales representative information 701 and sales representative information 711 shown in Figure 25 are examples of displays showing information about salespeople who have entered the waiting room and are waiting. In the example shown in Figure 25, sales representative information 701 and sales representative information 711 are generated as displays imitating business cards, including photographs of the faces of each salesperson. Note that the display format of the information showing the salesperson information is not limited to the example shown in Figure 25. For example, the display showing the salesperson information may be text information only, or may be image information only.
[0305] As shown in FIG. 25, the sales representative information 701 and the sales representative information 711 may include information on the elapsed time or remaining time from the time of each sales appointment or visit.
[0306] Furthermore, the sales representative information 701 and the sales representative information 711 may include information relating to the organization and position of each salesperson.
[0307] In addition, sales representative information 701 and sales representative information 711 may include the purpose for which each salesperson wishes to meet with customer Ua, information on the pharmaceuticals handled by each salesperson, and information on their area of expertise.
[0308] Furthermore, the control unit 233 may accept an operation to set the sales priority on a doctor-side standby screen such as screen D11, which will be notified when the customer Ua becomes available to start an interview.
[0309] More specifically, for example, when the communication unit 210 receives a notification from the caller terminal 10A that an operation has been performed to select one of the displays of information about each sales person displayed on the screen D11, the control unit 233 may set priorities in the order of the sales person for whom the selection operation was performed, and change the waiting order.
[0310] When the control unit 233 receives a notification from the caller terminal 10A that the selection button E2 has been pressed on the screen D11, it may control the caller terminal 10 used by each sales department to notify them that the customer Ua is available to start an interview, in accordance with the waiting order changed based on the set priority.
[0311] This allows customer Ua to view information about each sales representative on a doctor's standby screen such as screen D11, and to prioritize any sales representative for a remote interview depending on customer Ua's level of interest, etc.
[0312] Alternatively, the above-described operation of setting the priority may be performed by a nurse or the like who is capable of operating the caller terminal 10A, in addition to the customer Ua.
[0313] In addition, if the customer Ua does not perform a priority setting operation, the control unit 233 may automatically set the priority based on information such as whether or not each salesperson has an appointment, the appointed time of the appointment, and the elapsed time since each salesperson entered the waiting room.
[0314] (Waiting Time and Related Information Notification Function) The control unit 233 according to this embodiment may perform control to notify a salesperson who is waiting and has requested a remote interview with a customer Ua of the current number of people waiting and an estimate of the waiting time.
[0315] Fig. 26 is a diagram showing an example of a sales side standby screen generated by the control unit 233. As shown in Fig. 26, the screen D12 includes a notification display A8, a selection button E3, and a selection button E4.
[0316] The control unit 233 may generate a sales-side standby screen including notification text such as notification display A8 for each salesperson on standby. The control unit 233 may notify each salesperson of information such as the current waiting time by controlling the communication unit 210 to transmit and display the sales-side standby screen to the caller terminal 10 used by each salesperson.
[0317] This allows each salesperson to make effective use of the time they spend waiting for a remote interview, such as by deciding to perform other tasks while the waiting time is up.
[0318] The control unit 233 may also perform control to notify the waiting sales staff of various information related to the customer Ua, such as the customer Ua's areas of interest or information about events at the customer Ua's hospital.
[0319] For example, when the control unit 233 receives a notification from the caller terminal 10 that a press of the selection button E3 on the screen D12 shown in Figure 26 has been detected, the control unit 233 may control the sending of a screen displaying various information about the customer Ua to the caller terminal 10.
[0320] This allows the salesperson on standby to view information about customer Ua that is likely to become a topic of conversation during the remote interview until the waiting time has passed.
[0321] In addition, the control unit 233 may perform control to notify each salesperson of the change in waiting time when a change in the waiting time of each salesperson occurs, such as when an operation is performed to set the priority of the waiting salesperson on the doctor's waiting screen.
[0322] FIG. 27 is a diagram showing an example of a sales side standby screen generated by the control unit 233 and a notification display regarding waiting time.
[0323] The control unit 233 may perform control to notify each sales representative of the change in waiting time by displaying a notification such as the notification display A9 on the screen D13, for example.
[0324] (Information Exchange Function Between Salespeople) Furthermore, the control unit 233 according to this embodiment may provide a function that allows waiting salespeople to exchange information with each other if they so desire.
[0325] More specifically, for example, when the control unit 233 receives a notification from the caller terminal 10 that the selection button E4 has been pressed on the sales side standby screen described above with reference to Figure 26, it may determine that the sales person using the caller terminal 10 wishes to exchange information with other sales people.
[0326] If there is a salesperson who is determined to wish to exchange information, the control unit 233 may generate an information exchange requester waiting screen that displays information about the salesperson.
[0327] Fig. 28 is a diagram showing an example of an information exchange requester waiting screen generated by the control unit 233. As shown in Fig. 28, the screen D13 includes a notification display A9, a notification display A10, and a selection button E5.
[0328] The control unit 233 may generate an information exchange requester waiting screen including a notification display indicating the number of salespeople who have performed an operation and wish to exchange information, such as notification display A9.
[0329] In addition, the control unit 233 may generate an information exchange requester standby screen that includes a notification display indicating information about the salesperson who performed an operation to request information exchange, such as notification display A10.
[0330] The control unit 233 may control the communication unit 210 to transmit and display data of an information exchange requester waiting screen to the caller terminal 10 used by the salesperson who has been determined to desire information exchange.
[0331] In addition, when the control unit 233 receives from the caller terminal 10 an operation to select one of the sales representatives displayed as those wishing to exchange information on an information exchange waiting screen such as screen D13, and a notification that the selection operation of selection button E5 has been performed, the control unit 233 may perform control to notify the selected sales representative.
[0332] Fig. 29 is a diagram showing an example of an information exchange request notification screen generated by the control unit 233. As shown in Fig. 29, the screen D14 includes a notification display A11, a notification display A12, and a selection button E6.
[0333] The control unit 233 may generate an information exchange request notification screen that includes a display indicating a time limit for deciding whether or not to accept the information exchange request, as in the notification display A11.
[0334] The control unit 233 may also generate an information exchange request notification screen, such as notification display A12, that includes a display indicating that a request for information exchange has been notified by another salesperson.
[0335] If an operation to start a call is performed using the selection button E6 before the time limit indicated by the notification display A11 has elapsed, the control unit 233 may control the start of a call between the sales person who requested information exchange and the selected sales person.
[0336] If no operation to start a call is performed using the selection button E6 before the time limit indicated by the notification display A11 has elapsed, the control unit 233 may not start a call between the sales person who requested information exchange and the selected sales person.
[0337] By performing the above-described process, the salesperson waiting for the start of the remote interview with the customer Ua can exchange information with other salespersons, thereby enabling each salesperson to make more effective use of their waiting time.
[0338] The information exchange seeker standby screen described above with reference to FIG. 28 is an example, and the display format of the information exchange seeker standby screen generated by the control unit 233 is not limited to this example.
[0339] For example, the control unit 233 may generate an image of a virtual space simulating a waiting room, including the display of avatars representing each business, as a waiting screen for those wishing to exchange information.
[0340] (Notification control according to the doctor's free time) Furthermore, the control unit 233 in this embodiment may control the notification to the caller terminal 10 used for sales, which has been set as the notification destination in advance, when the customer Ua accesses the doctor's waiting screen by performing an operation on the caller terminal 10A.
[0341] Alternatively, the control unit 233 may control the caller terminal 10A to notify the sales representative who has been set as the notification destination when the customer Ua accesses the doctor's waiting screen on the caller terminal 10A and performs an operation to notify the sales representative who has been set as the notification destination that an interview is available.
[0342] Fig. 30 is a diagram showing another example of the doctor-side standby screen generated by the control unit 233. As shown in Fig. 30, the screen D15 includes information 720 about sales with appointments and information 730 about sales without appointments.
[0343] 30, it is assumed that there are no salespeople currently on standby. In this case, the control unit 233 may display displays such as notification display A13 and notification display A14 on the doctor-side standby screen.
[0344] The control unit 233 may also display information about salespeople with upcoming appointments starting from the current time on the doctor-side standby screen, such as the notification display A15 and salesperson information 721.
[0345] When the selection button E7 is pressed, the control unit 233 may perform control to notify the caller terminal 10 used for sales, which has been set as the notification destination in advance, that the customer Ua is currently available for an interview.
[0346] In addition, when there are multiple sales representatives who have been set as notification destinations in advance, the control unit 233 may accept the notification order set by the customer Ua.
[0347] FIG. 31 is a diagram showing an example of a notification priority setting screen generated by the control unit 233. As shown in FIG.
[0348] The control unit 233 may calculate an estimate of the free time of the customer Ua starting from the current time based on information such as the doctor's consultation schedule, etc. Furthermore, the control unit 233 may notify the customer Ua of the calculated estimate of the free time by displaying a notification display A16 or the like.
[0349] The control unit 233 may also display information on groups of sales that have been set as notification destinations in advance, such as notification display A17, on the notification priority setting screen.
[0350] The control unit 233 may determine the order of notification destinations by prioritizing sales classified into the group selected by the customer Ua using the selection button E8.
[0351] When the selection button E9 is pressed, the control unit 233 may control the notification to the sales representatives to be notified in accordance with the determined order.
[0352] Fig. 32 is a diagram showing an example of a notification screen that indicates that a doctor is available for an interview, generated by the control unit 233. As shown in Fig. 32, the screen D17 includes a notification display A18, a notification display A19, and a selection button E10.
[0353] The control unit 233 may generate a display, such as a notification display A18, indicating a time limit for determining whether or not to start a meeting with the customer Ua.
[0354] The control unit 233 may also notify the sales department that the customer Ua is currently available for an interview, for example, by using a message such as a notification display A19.
[0355] When a call start operation with customer Ua is performed on a caller terminal 10 that has received a notification screen such as screen D17 using selection button E17, the control unit 233 may notify customer Ua that a sales person using the caller terminal 10 has received a request to meet with customer Ua.
[0356] The control unit 233 may also display information about the salesperson who initiated the call with customer Ua in addition to the display of salespersons with appointments on the doctor's standby screen as described above with reference to Figure 30.
[0357] The control unit 233 may set an upper limit on the number of salespeople with whom the customer Ua can have an interview, depending on the estimated free time of the customer Ua. After notifying the salespeople that the customer Ua is currently available for an interview, if the number of salespeople who have performed a call start operation with the customer Ua reaches the upper limit, the control unit 233 may perform control to notify the salespeople who have not performed a call start operation that the specified number has been reached.
[0358] Fig. 33 is a diagram showing an example of a notification screen generated by the control unit 233. The control unit 233 may perform control to notify the customer Ua that the number of salespeople who have requested a meeting with the customer Ua has reached the upper limit, using wording such as the notification display A20 shown in Fig. 33.
[0359] (Processing for collecting post-interview evaluation) Furthermore, the control unit 233 according to this embodiment has a function of collecting information regarding the customer Ua's satisfaction with the remote interview after the end of the remote interview between the customer Ua and the sales person.
[0360] More specifically, for example, the control unit 233 may estimate the change in the satisfaction level of the customer Ua during the remote interview using the dissatisfaction estimation model described above in the first embodiment. Furthermore, the control unit 233 may estimate the overall evaluation of the customer Ua regarding the remote interview based on the average satisfaction level of the customer Ua.
[0361] 34 is a diagram showing an example of a post-interview evaluation questionnaire screen generated by the control unit 233. The control unit 233 may generate a screen that displays the estimated results of the customer Ua's overall evaluation of the remote interview on a five-point scale, for example, as shown in the selection button E11 in FIG.
[0362] The control unit 233 may cause the communication unit 210 to transmit a post-interview evaluation questionnaire screen such as screen D19 to the caller terminal 10A used by the customer Ua and display it.
[0363] In addition, the control unit 233 may accept a modification of the customer Ua's overall evaluation of the remote interview by accepting a selection operation of the selection button E11 on the screen D19 displayed on the caller terminal 10A. When the selection button E12 is pressed, the control unit 233 may receive from the caller terminal 10A the overall evaluation of the remote interview selected with the selection button E11.
[0364] 35 is a diagram showing another example of a post-interview evaluation questionnaire screen generated by the control unit 233. The control unit 233 may generate a questionnaire screen for dissatisfaction items regarding the remote interview, such as screen D20 shown in FIG.
[0365] The control unit 233 may receive information on the complaint item selected using the selection button E13 on the screen D20 from the caller terminal 10A.
[0366] The control unit 233 may also control the notification of the overall evaluation of the remote interview and information on dissatisfaction items received from the caller terminal 10A to the caller terminal 10 used by the sales person who is the other party in the remote interview.
[0367] By performing this process, the customer Ua can more easily convey their evaluation of the salesperson and their complaints compared to when they meet face-to-face with the salesperson in a remote interview.
[0368] In addition, by knowing the customer Ua's overall evaluation of the remote interview and information on the items of dissatisfaction, the sales department can reflect on the interview and use this information to improve the quality of the interview.
[0369] An example of the functional configuration of the information processing device 23 according to the fourth embodiment of the present disclosure has been described above with reference to Fig. 22 to Fig. 35. Next, an example of the operation of the information processing system according to the fourth embodiment of the present disclosure will be described with reference to Fig. 36 to Fig. 38.
[0370] FIG. 36 is a first flowchart illustrating an example of the operation of the information processing system according to the fourth embodiment.
[0371] First, the control unit 233 acquires the waiting status of a salesperson who has entered the waiting room and wishes to have a remote interview with the customer Ua (S601).
[0372] Next, the control unit 233 acquires appointment information between the customer Ua and the sales department (S603).
[0373] The control unit 233 controls the caller terminal 10A to display the doctor side waiting screen generated based on the business waiting status and appointment information (S605).
[0374] The control unit 233 accepts an operation to set the priority of the waiting order for business hours on the doctor-side waiting screen, and changes the waiting order (S607).
[0375] If there is a salesperson on standby (S609 / YES), the process proceeds to S611.
[0376] If there is no salesperson on standby (S609 / NO), the process proceeds to S615.
[0377] In S611, the control unit 233 determines whether or not the customer Ua has performed an operation to start a call for a remote consultation with a sales representative (S611).
[0378] The control unit 233 waits until a call start operation is performed (S611 / NO).
[0379] If a call start operation has been performed (S611 / YES), the control unit 233 performs control to notify the sales department in accordance with the standby order (S613). Next, the process proceeds to S617 in FIG.
[0380] In S615, the control unit 233 determines whether or not the customer Ua has performed an operation to notify the customer that an appointment is available (S615).
[0381] The control unit 233 waits until an operation to notify the availability of an interview is performed (S615 / NO).
[0382] If an operation to notify the availability of an interview has been performed (S615 / YES), the process proceeds to S613.
[0383] FIG. 37 is a second flowchart illustrating an example of the operation of the information processing system according to the fourth embodiment.
[0384] After the process of S613 described with reference to FIG. 36, the control unit 233 determines whether or not the notified salesperson has performed an operation to start a call with the customer Ua (S617).
[0385] If the call start operation is not performed (S617 / NO), the control unit 233 waits until a certain time has elapsed (S619 / NO).
[0386] When a certain period of time has elapsed (YES in S619), the control unit 233 performs control to notify the next salesperson on standby (S621). Next, the control unit 233 proceeds to the process of S617 again.
[0387] In S617, if a call start operation has been performed (S617 / YES), the control unit 233 performs control to start a call between the customer Ua and the sales person who performed the call start operation (S623).
[0388] The control unit 233 waits until an operation to end the call is performed (S625 / NO).
[0389] When an operation to end the call is performed (S625 / YES), the control unit 233 controls the caller terminal 10A to display a questionnaire regarding the satisfaction level of the remote interview (S627).
[0390] Next, the information processing device 23 repeats the processes of S601 to S609 again until an operation to end the series of processes is performed (S629 / NO).
[0391] If an operation to end the series of processes is performed, the information processing device 23 ends the process (S629 / YES).
[0392] Next, referring to FIG. 38, an example of operation of the information processing system relating to the function of exchanging information between sales representatives, which is performed in parallel with the example of operation described above with reference to FIGS. 36 and 37, will be described.
[0393] 38 is a flowchart illustrating an example of the operation of the information processing system relating to the information exchange function between sales representatives according to the fourth embodiment of the present disclosure. When there is a sales representative waiting to have a remote interview with the customer Ua, the control unit 233 may operate according to the flow shown in FIG.
[0394] The control unit 233 determines whether an operation requesting information exchange has been performed by a salesperson on standby on the doctor-side standby screen (S701).
[0395] If an operation for requesting information exchange has not been performed (S701 / NO), the control unit 233 ends the series of processes.
[0396] When an operation is performed to request information exchange (S701 / YES), the control unit 233 controls the display of information about other people wishing to exchange information on the caller terminal 10 used by the salesperson who requested the information exchange (S703).
[0397] Next, if the sales person who wishes to exchange information performs an operation to request information exchange with another sales person who wishes to exchange information (S705 / YES), the process proceeds to S711.
[0398] If no operation to request information exchange with the sales representative of another person desiring information exchange has been performed (S705 / NO), the process proceeds to S707.
[0399] In S707, if a request for information exchange has not been received from another person wishing to exchange information (S707 / NO), the process proceeds to S709.
[0400] In S709, if the salesperson who performed the operation to request information exchange performs an operation to end information exchange (S709 / YES), the control unit 233 ends the series of processes.
[0401] If the operation to end the information exchange has not been performed (S709 / NO), the control unit 233 repeats the processing from S701 onwards.
[0402] In S711, if a call start operation is performed by both the sales person who requested the information exchange and the sales person who received the information exchange request (S711 / YES), the control unit 233 controls to start the connection for the information exchange call (S713).
[0403] Next, the control unit 233 waits until an operation to end the information exchange call is performed (S715 / NO).
[0404] If an operation to end the information exchange call has been performed (S715 / YES), the process proceeds to S709.
[0405] In S711, if the call start operation has not been performed by either or both of the sales person who requested the information exchange and the sales person who received the information exchange request (S711 / NO), the control unit 233 determines whether a certain amount of time has elapsed since the request was made (S717).
[0406] If the certain time has not elapsed (S717 / NO), the control unit 233 repeats the processing from S711 onwards.
[0407] If the predetermined time has elapsed (S717 / YES), the process proceeds to S709.
[0408] Above, with reference to FIG. 38 , an example of the operation of the information processing system relating to the sales-to-sales information exchange function in the fourth embodiment of the present disclosure has been described. Note that the above describes an example in which the control unit 233 provides multiple functions, including a doctor's call availability notification function, a sales information display and priority order setting function, a wait time and related information notification function, a sales-to-sales information exchange function, notification control according to the doctor's free time, and post-interview evaluation collection processing. However, the present disclosure is not limited to such an example. For example, each of the above-described functions may be provided individually. Furthermore, two or more of the above-described functions may be provided in combination. Furthermore, the processing of each function does not necessarily have to be performed in the above-described order and can be flexibly changed.
[0409] 5. Hardware Configuration Example> The embodiments of the present disclosure have been described above. Next, a hardware configuration example common to the caller terminal 10, the information processing device 20, the information processing device 21, the information processing device 22, and the information processing device 23 according to the embodiments of the present disclosure will be described.
[0410] FIG. 39 is a block diagram illustrating a hardware configuration 90 according to an embodiment of the present disclosure.
[0411] The hardware configuration 90 can be applied to the caller terminal 10, the information processing device 20, the information processing device 21, the information processing device 22, and the information processing device 23.
[0412] 39 , the hardware configuration 90 includes, for example, a processor 901, a read-only memory (ROM) 903, a random access memory (RAM) 905, a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 923, and a communication device 925. Note that the hardware configuration shown here is an example, and some of the components may be omitted. Furthermore, the hardware configuration may include components other than those shown here.
[0413] (Processor 901) The processor 901 functions, for example, as an arithmetic processing device and a control device, and controls the overall operation of each component or part of it based on various programs recorded in the ROM 903, the RAM 905, the storage device 919, or the removable recording medium 927.
[0414] (ROM 903, RAM 905) The ROM 903 is a means for storing programs to be read into the processor 901 and / or data used in calculations. The RAM 905 temporarily or permanently stores, for example, programs to be read into the processor 901 and / or various parameters that change as appropriate when the programs are executed.
[0415] (Host bus 907, bridge 909, external bus 911, interface 913) The processor 901, ROM 903, and RAM 905 are connected to one another via, for example, a host bus 907 that is capable of high-speed data transmission. On the other hand, the host bus 907 is connected to, for example, an external bus 911 that has a relatively low data transmission speed via a bridge 909. In addition, the external bus 911 is connected to various components via an interface 913.
[0416] (Input Device 915) Examples of the input device 915 include a mouse, keyboard, touch panel, button, switch, and lever. Furthermore, a remote controller (hereinafter referred to as a remote control) capable of transmitting control signals using infrared rays or other radio waves may also be used as the input device 915. The input device 915 also includes an audio input device such as a microphone.
[0417] The input device 915 may also include an imaging device and a sensor. The imaging device is a device that captures real space and generates a captured image using an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and various components such as a lens for controlling the formation of a subject image on the imaging element. The imaging device may capture still images or moving images.
[0418] The sensors include various types of sensors, such as a distance measurement sensor, an acceleration sensor, a gyro sensor, a geomagnetic sensor, a vibration sensor, a light sensor, and a sound sensor. The sensors acquire information about the state of the hardware configuration 90 itself, such as the posture of the housing of the hardware configuration 90, or information about the surrounding environment of the hardware configuration 90, such as the brightness or noise around the hardware configuration 90. The sensors may also include a GNSS sensor that receives GNSS signals and measures the latitude, longitude, and altitude of the device. The sensing data acquired by the various sensors may be used as the sensing data related to the caller described above.
[0419] (Output device 917) The output device 917 includes various vibration devices that can visually or audibly notify the user of acquired information, such as display devices such as CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), or organic EL (Electro-Luminescence), audio output devices such as speakers and headphones, printers, mobile phones, or facsimiles.
[0420] (Storage Device 919) The storage device 919 is a device for storing various types of data. As the storage device 919, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device is used.
[0421] (Drive 921) The drive 921 is a device that reads information recorded on a removable recording medium 927 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, or writes information to the removable recording medium 927.
[0422] (Removable recording medium 927) The removable recording medium 927 is, for example, a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, various semiconductor storage media, etc. Of course, the removable recording medium 927 may also be, for example, an IC card equipped with a contactless IC chip, an electronic device, etc.
[0423] (Connection Port 923) The connection port 923 is a port for connecting an external device 929, such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface) port, an RS-232C port, or an optical audio terminal.
[0424] (Externally Connected Device 929) The externally connected device 929 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder.
[0425] (Communication device 925) The communication device 925 is a communication device for connecting to a network, such as a communication card for wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), or WUSB (Wireless USB), a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.
[0426] 6. Summary Although preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.
[0427] For example, in the above embodiment, an example in which the present disclosure is applied to a remote interview conducted between one customer Ua and one sales person Ub has been described. However, the application of the present disclosure is not limited to this example. The information processing system according to the present disclosure may also be applied to a remote interview conducted between one or more customers and one or more sales people, for example.
[0428] In the above embodiment, an example has been described in which the information processing devices 20, 21, 22, and 23 control notifications to callers by generating and transmitting notification displays output on the display unit of the caller terminal 10. However, the present disclosure is not limited to such an example. For example, the information processing devices 20, 21, 22, and 23 may control notifications to callers by generating audio data related to the content of the notification to the callers and transmitting the audio data to the caller terminal 10 for output.
[0429] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.
[0430] Note that the following configurations also fall within the technical scope of the present disclosure. (1) An information processing device comprising: a control unit that controls notifying a caller of a state of the caller related to the degree of satisfaction with the call, in accordance with the caller's level of satisfaction with the call estimated based on sensing data related to the caller. (2) The callers include a first caller and a second caller who engages in sales activities for either a product or a service or both with the first caller, and the control unit controls notifying the second caller of the state of the second caller related to the degree of satisfaction of the first caller, in accordance with the first caller's level of satisfaction with the call estimated based on sensing data related to the first caller. (3) The information processing device described in (2), in which the control unit controls notifying the second caller of the state of the second caller when the first caller's level of satisfaction with the call is lower than a reference value, based on sensing data related to the second caller. (4) The information processing device according to (3), wherein the control unit, when the satisfaction level of the first caller is lower than a reference value, analyzes the state of the second caller based on the sensing data related to the second caller, and outputs the state of the second caller obtained as a result of the analysis as a dissatisfaction causing state. (5) The information processing device according to (3), wherein the control unit controls to notify the second caller of the state of the second caller for a predetermined time period from the time when the satisfaction level of the first caller is lower than the reference value. (6) The information processing device according to (4), wherein the control unit, when detecting, based on the sensing data related to the second caller, that the state of the second caller has become the dissatisfaction causing state, while the call is being conducted, controls to notify the second caller. (7) The information processing device according to (6), wherein the control unit, when detecting, based on the sensing data related to the first caller, that the satisfaction level of the first caller is lower than the reference value, controls to notify the second caller.(8) The information processing device according to (7), wherein the control unit learns, by machine learning, a relationship between sensing data related to the first caller and information indicating the satisfaction level of the first caller, and estimates the satisfaction level of the first caller using a model obtained as a result of the learning. (9) The information processing device according to (8), wherein the control unit learns, by machine learning, a relationship between sensing data related to the second caller and a state of the second caller output as the dissatisfaction cause state, and detects whether the state of the second caller has become the dissatisfaction cause state using the model obtained as a result of the learning. (10) The information processing device according to (6), wherein the control unit acquires a degree of tension of the second caller estimated based on the sensing data related to the second caller, and if the degree of tension is higher than a reference value, analyzes the state of the first caller, and outputs the state of the first caller obtained as a result of the analysis as a tension cause state. (11) The information processing device according to (10), wherein the control unit analyzes the state of the first caller for a predetermined time period from a time when the tension level of the second caller is higher than a reference value, and outputs the state of the first caller obtained as a result of the analysis as the tension-causing state. (12) The information processing device according to (2), wherein the control unit performs control to notify the first caller when it is estimated that the satisfaction level of the first caller is below a reference value during the call. (13) The information processing device according to (11), wherein the control unit calculates, based on the sensing data, a speech time indicating whether or not each of the first caller and the second caller has spoken and the length of each speech, and performs control to notify the second caller of information regarding the length of time each of the first caller and the second caller has spoken, based on the speech time obtained as a result of the calculation. (14) The information processing device according to (13), wherein the control unit performs control to notify the second caller of information indicating a proportion of time during which the first caller and the second caller each spoke based on the calculated speaking time.(15) The information processing device according to (14), wherein the control unit determines a level of busyness of the first caller according to a state of the first caller's surrounding environment estimated based on sensing data related to the first caller while the call is being made, and controls to notify the second caller of information related to the busyness of the first caller according to a result of the determination. (16) The information processing device according to (15), wherein the control unit generates a call summary screen including information indicating the satisfaction level of the first caller and the dissatisfaction cause state of the second caller from the start to the end of the call. (17) The information processing device according to (1), wherein the sensing data related to the callers includes either or both of time series data of video around the caller that includes the caller's face as a subject and time series data of sound around the caller that includes the caller's voice. (18) The information processing device according to (8), wherein the sensing data related to the callers includes a biosignal emitted by a biophenomenon of the caller. (19) A program that causes a computer to function as a control unit that controls notifying a caller of a state of a caller that is related to a degree of satisfaction with the call, in accordance with the caller's satisfaction with the call that is estimated based on sensing data about the caller. (20) An information processing method executed by a computer, including: controlling notifying the caller of a state of a caller that is related to a degree of satisfaction with the call, in accordance with the caller's satisfaction with the call that is estimated based on sensing data about the caller.
[0431] REFERENCE SIGNS LIST 10 caller terminal 110 communication unit 120 sensor unit 130 control unit 140 output unit 150 input unit 20 information processing device 210 communication unit 220 storage unit 230 control unit 21 information processing device 22 information processing device 23 information processing device 5 network
Claims
1. An information processing device comprising: a control unit that performs control to notify a caller of the state of the caller related to the degree of satisfaction with the call, in accordance with the caller's degree of satisfaction with the call, which is estimated based on sensing data regarding the caller.
2. The information processing device of claim 1, wherein the parties include a first party and a second party who engages in sales activities for either a product or a service or both to the first party, and the control unit controls to notify the second party of the status of the second party related to the degree of satisfaction of the first party in accordance with the first party's degree of satisfaction with the call estimated based on sensing data regarding the first party.
3. The information processing device according to claim 2, wherein the control unit performs control to notify the second party of the status of the second party when the satisfaction level of the first party is lower than a reference value based on sensing data regarding the second party.
4. The information processing device of claim 3, wherein when the satisfaction level of the first caller is lower than a reference value, the control unit analyzes the state of the second caller based on the sensing data regarding the second caller, and outputs the state of the second caller obtained as a result of the analysis as a dissatisfaction cause state.
5. The information processing device according to claim 3, wherein the control unit controls to notify the second caller of the state of the second caller during a predetermined time period from the time when the satisfaction level of the first caller is lower than a reference value.
6. The information processing device according to claim 4, wherein the control unit performs control to notify the second caller when it detects, based on sensing data regarding the second caller, that the state of the second caller is in the dissatisfaction causing state while the call is being made.
7. The information processing device according to claim 6, wherein the control unit performs control to notify the second caller when it detects, based on sensing data regarding the first caller, that the satisfaction level of the first caller is lower than the reference value while the call is being made.
8. The information processing device according to claim 7, wherein the control unit learns the relationship between the sensing data regarding the first caller and information indicating the satisfaction level of the first caller through machine learning, and estimates the satisfaction level of the first caller using a model obtained as a result of the learning.
9. The information processing device according to claim 8, wherein the control unit learns through machine learning the relationship between the sensing data regarding the second caller and the state of the second caller output as the dissatisfaction causing state, and detects whether the state of the second caller is the dissatisfaction causing state using a model obtained as a result of the learning.
10. The information processing device of claim 6, wherein the control unit acquires a degree of tension of the second caller estimated based on sensing data regarding the second caller, analyzes a state of the first caller when the degree of tension is higher than a reference value, and outputs the state of the first caller obtained as a result of the analysis as a state causing tension.
11. The information processing device according to claim 10, wherein the control unit analyzes the state of the first caller during a predetermined time period from the time when the level of tension of the second caller is higher than a reference value, and outputs the state of the first caller obtained as a result of the analysis as the tension-causing state.
12. The information processing device according to claim 2, wherein the control unit performs control to notify the first caller when it is estimated that the satisfaction level of the first caller is below a reference value while the call is being made.
13. The information processing device according to claim 11, wherein the control unit calculates, based on the sensing data, a speech time indicating whether or not each of the first and second callers spoke and the length of each speech, and controls to notify the second caller of information regarding the length of time each of the first and second callers spoke based on the speech time obtained as a result of the calculation.
14. The information processing device according to claim 13, wherein the control unit controls to notify the second caller of information indicating the proportion of time that each of the first caller and the second caller spoke during the call based on the calculated speaking time.
15. The information processing device according to claim 14, wherein the control unit determines the degree of busyness of the first caller in accordance with the state of the first caller's surrounding environment estimated based on sensing data related to the first caller while the call is being made, and controls to notify the second caller of information regarding the busyness of the first caller in accordance with the result of the determination.
16. The information processing device according to claim 15, wherein the control unit generates a call summary screen including information indicating the satisfaction level of the first caller and the dissatisfaction cause state of the second caller from the start to the end of the call.
17. The information processing device of claim 1, wherein the sensing data relating to the caller includes either or both of time series data of video around the caller including the caller's face as a subject, and time series data of sounds around the caller including the caller's voice.
18. The information processing device according to claim 8, wherein the sensing data relating to the caller includes a biosignal emitted by a biological phenomenon of the caller.
19. A program that causes a computer to function as a control unit that controls the notification of a caller's status related to the degree of satisfaction with a call, based on sensing data about the caller's level of satisfaction with the call, in accordance with the caller's level of satisfaction with the call, which is estimated based on sensing data about the caller.
20. An information processing method executed by a computer, comprising: controlling to notify a caller of a status of the caller related to a degree of satisfaction with the call, the status being estimated based on sensing data regarding the caller.
