Information processing system, information processing device, information processing program, and information processing method

Abstract

An information processing system, an information processing device, an information processing program, and an information processing method are provided that allow an operator to respond efficiently. [Solution] An information processing system 10 includes a central control device 12, an operator terminal 16, a user terminal 18, and an automatic response device 20. When a response request is received from a user terminal, the central control device assigns the user of that user terminal to an available operator. At this time, the automatic response device automatically interviews the user for a first predetermined time, and then the user using the user terminal that made the response request is assigned to the available operator, with the operator responding for a second predetermined time that is shorter than the first predetermined time. An available operator is an operator to whom no user is assigned to respond, or an operator whose first predetermined time for interviewing the user using the user terminal that made the response request overlaps with the entire second predetermined time for serving another previously assigned user.

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing device, an information processing program, and an information processing method, and more particularly to an information processing system, an information processing device, an information processing program, and an information processing method for assigning an operator to respond to a user, for example. [Background technology]

[0002] An example of an information processing system of the background art is disclosed in Patent Document 1. The automated transaction device operation support system of Patent Document 1 switches to an operator response mode after an automatic response mode using chat, and in the operator response mode, the support terminal device receives history data and displays it on a support operation screen. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-215729 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, since the burden on the operator would be heavy if the operator had to respond to all inquiries from users, it is conceivable to reduce the burden on the operator by automatically responding via chat, as in the background art described above. Furthermore, in the background art described above, responses can be automatically made via chat during the time it takes for an operator to become available, thereby reducing the waiting time for users compared to responses by an operator alone.

[0005] However, in the background art described above, when a response cannot be made in chat-based automatic response mode, the response mode is switched to an operator response mode, and therefore the length of time spent responding in automatic response mode varies from user to user. Therefore, if the response time in automatic response mode is long, the operator may have to wait, and there is room for improvement in allowing the operators to respond to users efficiently.

[0006] Therefore, a primary object of the present invention is to provide a novel information processing system, an information processing device, an information processing program, and an information processing method.

[0007] Another object of the present invention is to provide an information processing system, an information processing device, an information processing program, and an information processing method that allow an operator to efficiently respond to each of a plurality of users. [Means for solving the problem]

[0008] A first invention is an information processing system comprising a central control device, a plurality of user terminals used by each of a plurality of users, an operator terminal used by an operator who responds to the plurality of users, and an automatic response device that can automatically respond to each of the plurality of users, wherein the central control device comprises an allocation means that, when a response request is received from a user terminal, assigns the response of the user using the user terminal to an available operator, and a response instruction means that instructs the operator terminal used by the operator to respond to the user to which the user has been assigned by the allocation means, and the allocation means assigns the user using the user terminal that has made the response request to an available operator, such that after the automatic response device has responded for at least a first predetermined time, the operator will respond for a second predetermined time that is shorter than the first predetermined time.

[0009] A second invention is dependent on the first invention, and the central control device further comprises a first determination means for determining, when a response request is received from a user terminal, an operator to whom no responding user is assigned, or an operator whose time to automatically respond to a user using the user terminal by an automatic response device can overlap with all of the time to respond to other users who have already been assigned, as an available operator.

[0010] A third invention is dependent on the first invention and includes a plurality of operator terminals used by a plurality of operators, and the central control device further includes a switching means for having another operator other than the operator assigned to handle the user handle the user if the automatic response device finishes handling the user earlier than a first predetermined time.

[0011] A fourth invention is according to the third invention, and the central control device further comprises a second determination means for determining the operator assigned to the other user as the other operator when the time to start serving the assigned other user is later than the time when the operator finishes serving the user assigned to the operator earlier than a first predetermined time, and the time for automatically serving the other user can be extended.

[0012] A fifth invention is dependent on the fourth invention, and the automatic response device acquires answers from each of a plurality of users to a plurality of questions, and the second determination means determines that if the progress of the other users in answering the plurality of questions is slow, the time for automatically responding to the other users can be extended.

[0013] A sixth invention is an information processing device that assigns a user of a user terminal to an operator who is available to respond to a response request from the user terminal, and includes an allocation means that, when a response request is received from the user terminal, assigns the response of the user using the user terminal to an operator who is available to respond, and a response instruction means that instructs the operator terminal used by the operator to which the user has been assigned by the allocation means to respond, and the allocation means assigns the user using the user terminal that has received the response request to an operator who is available to respond, assuming that after the automatic response device has responded for at least a first predetermined time, the operator will respond for a second predetermined time that is shorter than the first predetermined time.

[0014] A seventh invention is an information processing program for an information processing device that assigns a user of a user terminal to an operator who is available to respond to a response request from the user terminal, the information processing program causing a processor of the information processing device to, when a response request is received from the user terminal, assign an available operator to respond to the user using the user terminal that has made the response request, assuming that after an automatic response device has responded for at least a first predetermined time, the operator will respond for a second predetermined time that is shorter than the first predetermined time, and to instruct the operator terminal used by the assigned operator to respond to the user who has made the response request.

[0015] An eighth invention is an information processing method of an information processing device that assigns a user of a user terminal to an operator who is available to respond to a response request from the user terminal, wherein when a response request is received from the user terminal, the processor of the information processing device assigns the user who uses the user terminal that has made the response request to an operator who is available to respond, assuming that after the automatic response device has responded for at least a first predetermined time, the operator will respond for a second predetermined time that is shorter than the first predetermined time, and instructs the operator terminal used by the assigned operator to respond to the user who has made the response request. [Effects of the Invention]

[0016] According to this invention, the time for which an automatic response device automatically responds to a call is fixed to be longer than the time for which an operator responds, and if the entire time for responding to one user can overlap with the time for automatically responding to another user, the other user is assigned to this operator, so that the operator can efficiently respond to each of multiple users.

[0017] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the preferred embodiments, which proceeds with reference to the accompanying drawings. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram showing an information processing system according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of the electrical configuration of the central control device. [Figure 3] FIG. 3 is a block diagram showing an example of the electrical configuration of the operator terminal. [Figure 4] FIG. 4 is a block diagram showing an example of the electrical configuration of a user terminal. [Figure 5] FIG. 5 is a block diagram showing an example of the electrical configuration of the automatic answering device. [Figure 6] FIG. 6 is a diagram showing an example of a response screen displayed on the display device of the user terminal. [Figure 7] FIG. 7 is a diagram showing an example of a hearing screen displayed on the display device of the user terminal. [Figure 8] FIG. 8 is a diagram showing an example of a case where users who respond to a certain operator are sequentially assigned. [Figure 9] FIG. 9 is a diagram showing an example of a memory map of the RAM of the central control unit. [Figure 10] FIG. 10 is a diagram showing an example of a memory map of the RAM of the operator terminal. [Figure 11] FIG. 11 is a diagram showing an example of a memory map of the RAM of the user terminal. [Figure 12]FIG. 12 is a diagram showing an example of a memory map of the RAM of the automatic answering device. [Figure 13] FIG. 13 is a flowchart showing a part of an example of operator management processing by the CPU of the central control device. [Figure 14] FIG. 14 is a flowchart showing another example of the operator management process of the CPU of the central control device, which follows FIG. [Figure 15] FIG. 15 is a flowchart showing a part of an example of response processing by the CPU of the operator terminal. [Figure 16] FIG. 16 is a flowchart showing another part of an example of the response processing by the CPU of the operator terminal, and follows FIG. [Figure 17] FIG. 17 is a flowchart showing another part of an example of the response processing by the CPU of the operator terminal, and follows FIG. [Figure 18] FIG. 18 is a flowchart showing an example of the hearing process of the CPU of the automatic answering device. [Figure 19] FIG. 19 is a diagram showing an example of the case where users are assigned in turn to respond to two operators in the second embodiment. [Figure 20] FIG. 20 is a diagram showing an example of a case where two operators shown in FIG. 19 alternately handle users assigned to each of them. [Figure 21] FIG. 21 is a flowchart showing a part of an example of operator management processing by the CPU of the central control device in the second embodiment. [Figure 22] FIG. 22 is a flowchart showing another example of the operator management process of the CPU of the central control device in the second embodiment, and follows FIG. [Figure 23] FIG. 23 is a flowchart showing another example of the operator management process of the CPU of the central control device in the second embodiment, which follows FIG. [Figure 24] FIG. 24 is a flowchart showing a part of an example of response processing by the CPU of the operator terminal in the second embodiment. [Figure 25]FIG. 25 is a flowchart showing another part of an example of the response process of the CPU of the operator terminal in the second embodiment, and follows FIG. DETAILED DESCRIPTION OF THE INVENTION

[0019] <First Example> Referring to FIG. 1, an information processing system 10 (hereinafter simply referred to as "system 10") according to a first embodiment of the present invention includes a central control device 12. The central control device 12 is communicably connected to an operator terminal 16, a user terminal 18, an automated response device 20, and the like via a network 14. The system 10 is applied to a store or facility (hereinafter referred to as a "store, etc."). As will be described in detail later, the system provides online remote response (customer service) to customers (i.e., users) of the store, etc., who use the user terminal 18.

[0020] The central control device 12 is an information processing device such as a server, and controls the entire system 10. The operator terminal 16 is an information processing device such as a personal computer, and is used by a human operator (operator) who is a staff member of a store or the like. The operator terminal 16 is installed in the store's office, a contact center, the operator's home, or the like. The user terminal 18 is an information processing device such as a device for making inquiries, automatically entering and exiting, and performing payments, and is used by users who use the store or the like. The user terminal 18 is installed in an appropriate location such as the entrance or exit of the store, an information desk, or a payment counter. The automated response device 20 is an information processing device such as a personal computer or a server, and asks questions to users, conducts surveys, and obtains responses from users. The automated response device 20 may also have a chatbot function that can automatically respond to users, that is, an automated response function that utilizes artificial intelligence (i.e., AI).

[0021] The stores and the like include hotels, stations, bus terminals, airports, hospitals, government offices, libraries, banks, theme parks, shopping malls, etc. The stores and the like also include predetermined websites on the Internet. In this case, the user terminal 18 is an information processing device such as a personal computer, smartphone, or tablet terminal used by the user.

[0022] 1 shows one each of the operator terminal 16, the user terminal 18, and the automatic response device 20, but in reality, one or more operator terminals 16, one or more user terminals 18, and one or more automatic response devices 20 are connected to the network 14. Basically, the number of installed operator terminals 16 is smaller than the number of installed user terminals 18. Furthermore, the operator terminal 16 and the user terminal 18, and the user terminal 18 and the automatic response device 20, can perform two-way communication via the network 14 using a P2P (Peer to Peer) method.

[0023] The network 14 is composed of an IP network (or IP network) including the Internet, and an access network (or access network) for accessing this IP network. The access network may be a public telephone network, a mobile phone network, a wired LAN, a wireless LAN, or CATV (Cable Television). Furthermore, WebRTC (Web Real-Time Communication) is used to realize two-way communication using the P2P method. For this purpose, appropriate elements such as a signaling server, STUN server, and TURN server (not shown) are provided, but as these are well known, detailed description will be omitted.

[0024] Fig. 2 is a block diagram showing an example of the electrical configuration of central control device 12. As shown in Fig. 2, central control device 12 includes a CPU 30. CPU 30 is connected to RAM 32, a communication interface (hereinafter referred to as "communication I / F") 34, and an input / output interface (hereinafter referred to as "input / output I / F") 36 via an internal bus.

[0025] The CPU 30 is a processor that controls the central control device 12 and, ultimately, the overall control of the system 10. However, instead of the CPU 30, a SoC (System-on-a-Chip) including multiple functions such as a CPU function and a GPU (Graphics Processing Unit) function may be provided.

[0026] RAM 32 is the main storage device of central control device 12 and is used as a work area or buffer area for CPU 30. Although not shown, central control device 12 is provided with a HDD and ROM as auxiliary storage devices. However, non-volatile memory such as an SSD may be used instead of or in addition to the HDD. The auxiliary storage device stores various software such as an operating system, middleware, and various application programs.

[0027] The communication I / F 34 is a wired interface for transmitting and receiving control signals and data to and from external computers such as the operator terminal 16, the user terminal 18, and the automatic answering device 20 via the network 14 under the control of the CPU 30. However, the communication I / F 34 may also be a wireless interface for connecting to a wireless LAN.

[0028] Input devices such as a keyboard and a computer mouse, and a display device such as a liquid crystal display device are connected to the input / output I / F 36 as appropriate. The input / output I / F 36 outputs operation data (or operation information) input from the input device to the CPU 30. It also outputs image data generated by the CPU 30 to the display device, causing the display device to display a screen or image corresponding to the image data. Note that the configuration of the central control device 12 shown in FIG. 2 is an example, and is not limited to this.

[0029] Fig. 3 is a block diagram showing an example of the electrical configuration of operator terminal 16. As shown in Fig. 3, operator terminal 16 includes a CPU 40. CPU 40 is connected to RAM 42, a communication I / F 44, and an input / output I / F 46 via an internal bus.

[0030] The CPU 40 is a processor that controls the overall operation of the operator terminal 16. However, instead of the CPU 40, an SoC including multiple functions such as a CPU function and a GPU function may be provided.

[0031] The RAM 42 is the main storage device of the operator terminal 16 and is used as a work area or buffer area for the CPU 40. Although not shown, the operator terminal 16 is also provided with a HDD and ROM as auxiliary storage devices. However, a non-volatile memory such as an SSD may be used instead of or in addition to the HDD. The auxiliary storage device stores various software such as an operating system, middleware, and various application programs.

[0032] The communication I / F 44 is a wired interface for transmitting and receiving control signals and data to and from external computers such as the central control device 12, user terminal 18, and automatic response device 20 via the network 14 under the control of the CPU 40. However, the communication I / F 44 may also be a wireless interface for connecting to a wireless LAN.

[0033] An input device 48, a display device 50, a microphone 52, a speaker 54, a camera 56, and the like are connected to the input / output I / F 46. The input device 48 is a keyboard, a computer mouse, a touch panel, and the like. The display device 50 is, for example, a liquid crystal display device. The microphone 52 is an operator voice detection means for detecting voices emitted by an operator using the operator terminal 16, and is provided so as to be able to detect the operator voices. The speaker 54 is a user voice output means for outputting the user's voice, and is provided so that the operator can hear the output sound from the speaker 54. Note that a headset in which the microphone 52 and the speaker 54 are integrated may also be employed. The camera 56 is an operator imaging means for imaging the operator using the operator terminal 16, and is provided so as to be able to image the operator.

[0034] The input / output I / F 46 outputs operation data (or operation information) input from the input device 48 to the CPU 40. The input / output I / F 46 also outputs image data generated by the CPU 40 to the display device 50, causing a screen corresponding to the image data to be displayed on the display device 50. However, there are also cases where image data received from an external computer (for example, the user terminal 18) is output by the CPU 40.

[0035] Furthermore, the input / output I / F 46 converts the operator's voice detected by the microphone 52 into digital voice data and outputs it to the CPU 40, and converts voice data output by the CPU 40 into an analog voice signal and outputs it from the speaker 54. Furthermore, the input / output I / F 46 outputs data of an image (operator image) including the operator captured (detected) by the camera 56 to the CPU 40. Note that the voice data output from the CPU 40 includes voice data of the user received from the user terminal 18. Note that the image of the operator captured by the camera 56 may be a still image or a moving image. Note that the configuration of the operator terminal 16 shown in FIG. 3 is an example and is not limited to this.

[0036] Fig. 4 is a block diagram showing an example of the electrical configuration of the user terminal 18 shown in Fig. 1. As shown in Fig. 4, the user terminal 18 includes a CPU 60. The CPU 60 is connected to a RAM 62, a communication I / F 64, and an input / output I / F 66 via an internal bus.

[0037] The CPU 60 is a processor that is responsible for overall control of the user terminal 18. However, instead of the CPU 60, an SoC including multiple functions such as a CPU function and a GPU function may be provided.

[0038] The RAM 62 is the main storage device of the user terminal 18 and is used as a work area or buffer area for the CPU 60. Although not shown, the user terminal 18 is provided with a HDD and ROM as auxiliary storage devices. However, a non-volatile memory such as an SSD may be used instead of or in addition to the HDD. Various software such as an operating system, middleware, and various application programs are stored in this auxiliary storage device.

[0039] The communication I / F 64 is a wired interface for transmitting and receiving control signals and data to and from external computers such as the central control device 12, the operator terminal 16, and the automatic response device 20 via the network 14 under the control of the CPU 60. However, the communication I / F 64 may also be a wireless interface for connecting to a wireless LAN.

[0040] The input / output I / F 66 is connected to the above-mentioned input device 68, display device 70, microphone 72, speaker 74, camera 76, etc. This input / output I / F 66 outputs operation data (or operation information) input from the input device 68 to the CPU 60. The input / output I / F 66 also outputs image data generated by the CPU 60 to the display device 70, causing the display device 70 to display a screen corresponding to the image data. However, there are also cases where image data received from an external computer (for example, the operator terminal 16 and the automatic response device 20) is output by the CPU 60.

[0041] In addition, the input / output I / F 66 converts the user's voice detected by the microphone 72 into digital voice data and outputs it to the CPU 60, and converts the voice data output by the CPU 60 into an analog voice signal and outputs it from the speaker 74.

[0042] Furthermore, the input / output I / F 66 outputs data of an image (user image) including the user captured by the camera 76 to the CPU 60. The voice data output from the CPU 60 includes voice data of the operator received from the operator terminal 16 and voice data of the AI ​​received from the automatic response device 20. The image of the user captured by the camera 76 may be a still image or a moving image. The configuration of the user terminal 18 shown in FIG. 4 is an example and need not be limited to this.

[0043] Fig. 5 is a block diagram showing an example of the electrical configuration of the automated answering device 20. As shown in Fig. 5, the automated answering device 20 includes a CPU 80. The CPU 80 is connected to a RAM 82, a communication I / F 84, and an input / output I / F 86 via an internal bus.

[0044] The CPU 80 is a processor that controls the overall operation of the automatic answering device 20. However, instead of the CPU 80, an SoC including multiple functions such as a CPU function and a GPU function may be provided.

[0045] The RAM 82 is the main storage device of the automated response device 20 and is used as a work area or buffer area for the CPU 80. Although not shown in the figure, the automated response device 20 is provided with a HDD and a ROM as auxiliary storage devices. However, a non-volatile memory such as an SSD may be used instead of or in addition to the HDD. Various software such as an operating system, middleware, and various application programs are stored in this auxiliary storage device.

[0046] The communication I / F 84 is a wired interface for transmitting and receiving control signals and data to and from external computers such as the central control device 12 and the user terminal 18 via the network 14 under the control of the CPU 80. However, the communication I / F 84 may also be a wireless interface for connecting to a wireless LAN.

[0047] An input device 88 and a display device 90 are connected to the input / output I / F 86. The input device 88 is a keyboard, a computer mouse, a touch panel, or the like. The display device 90 is, for example, a liquid crystal display device. The input / output I / F 86 outputs operation data (or operation information) input from the input device 88 to the CPU 80, and also outputs image data generated by the CPU 80 to the display device 90, causing a screen or image corresponding to the image data to be displayed on the display device 90. Note that the configuration of the automatic answering device 20 shown in FIG. 6 is an example, and is not limited to this.

[0048] In such a system 10, an operator using the operator terminal 16 can remotely respond to (or serve) a user using the user terminal 18. In this case, a user who desires remote response performs a call operation to call the operator using any user terminal 18.

[0049] The central control device 12 is notified that this call operation has been performed. Upon receiving this notification (i.e., an answering request), the central control device 12 determines an available operator, that is, assigns the available operator to answer the user who made the answering request, and instructs the operator terminal 16 of the determined operator to answer the user. In this first embodiment, the central control device 12 connects the operator terminal 16 of the determined operator to the user terminal 18 that made the call operation, that is, makes a state in which two-way communication by P2P is possible. In other words, a connection state between the operator terminal 16 and the user terminal 18 is established. Therefore, the operator at the determined operator terminal 16 starts answering the user remotely.

[0050] In this remote response, operation data input via the input device 68 of the user terminal 18 or an operation result according to the operation data is transmitted to the operator terminal 16. Then, an image of the operation result according to the operation data received by the operator terminal 16 or an image of the operation result received by the operator terminal 16 is displayed on the display device 50 of the operator terminal 16. In addition, data of the user voice detected by the microphone 72 of the user terminal 18 (hereinafter referred to as "user voice data") is also transmitted to the operator terminal 16. The user voice corresponding to the user voice data received by the operator terminal 16 is output from the speaker 54 of the operator terminal 16. Furthermore, data of a user image captured by the camera 76 of the user terminal 18 (hereinafter referred to as "user image data") can also be transmitted to the operator terminal 16. In this case, a user image corresponding to the user image data received by the operator terminal 16 is displayed on the display device 50 of the operator terminal 16.

[0051] Furthermore, in remote call handling, a call handling screen 100 is displayed on display device 70 of user terminal 18. FIG. 6 is a diagram showing an example of call handling screen 100. As shown in FIG. 6, call handling screen 100 displays an avatar image (hereinafter referred to as "avatar image") 102, which is a substitute character for the operator handling the call remotely. Avatar image 102 moves in accordance with the operator image captured by camera 56 of operator terminal 16. To this end, operator terminal 16 performs a well-known motion detection process on the operator image to generate avatar image data for displaying avatar image 102 moving in accordance with the operator image. This avatar image data is transmitted to user terminal 18. Upon receiving the avatar image data, user terminal 18 generates display screen data for displaying call handling screen 100 based on the avatar image data, and call handling screen 100 is displayed on display device 70 based on the display screen data.

[0052] However, an image of the operator himself / herself may be displayed on response screen 100 instead of avatar image 102. Also, operator terminal 16 may generate motion data for motions corresponding to the operator image and transmit the data to user terminal 18, and avatar image data may be generated on the user terminal 18 side.

[0053] Data of the operator's voice detected by the microphone 52 of the operator terminal 16 (hereinafter referred to as "operator voice data") is also transmitted to the user terminal 18. The operator's voice corresponding to the operator voice data received by the user terminal 18 is output from the speaker 74 of the user terminal 18. Note that the sound output from the speaker 74 may be a voice that has been processed by a voice changer.

[0054] Although not shown in the drawings, the user terminal 18 is provided with a call button (or an operator that functions as a call button) for accepting the above-mentioned call operation on the input device 68 or the display device 70. In addition, the display device 70 of the user terminal 18 in the standby state displays a message screen informing the user that remote response is possible and guiding the user to operate the call button if remote response is desired.

[0055] As described above, in this system 10, when a response request is made from the user terminal 18, an operator determined by the central control device 12 basically responds using the operator terminal 16. However, questions or questionnaires (hereinafter simply referred to as "questions") about content that the operator wants to obtain from the user in advance before responding are carried out by the automatic response device 20 on behalf of the operator.

[0056] Therefore, in this first embodiment, when an operator responds to a user, the operator first greets the user and asks the user to answer a question posed by the automatic response device 20 (hereinafter, this may be referred to as the "introduction to the response"), and then the operator requests the central control device 12 to ask the user a question (hereinafter, referred to as "interview").

[0057] Then, the central control device 12 connects the automatic response device 20 to the user terminal 18 used by the user whom the operator responds to, that is, makes it possible for two-way communication by the P2P method. In other words, a connection state between the automatic response device 20 and the user terminal 18 is established. Therefore, the automatic response device 20 conducts a hearing with the user.

[0058] Fig. 7 shows an example of a hearing screen 150 displayed on the display device 70 of the user terminal 18. In the example shown in Fig. 7, a question is written on the hearing screen 150, and multiple selectable answers to the question are written. In addition, button images 152 are displayed below the multiple answers.

[0059] The questions are prepared in advance and are to be asked of the user regarding the services provided by the system 10. The questions are displayed in order on the hearing screen 150.

[0060] The multiple answers are each an answer to a question, and two or more answers are set. However, the answers may also be entered by the user as text.

[0061] Button image 152 is an icon for proceeding to the next question. Although not shown, for the second question and subsequent questions, a button image is also displayed as an icon for returning to the previous question. By returning to the previous question, it is possible to confirm and correct the answer.

[0062] After the user interview is completed, the answer result is transmitted from the automatic response device 20 to the operator terminal 16 via the central control device 12. The central control device 12 also instructs the operator terminal 16 to start (resume) the user interview. Therefore, the operator of the operator terminal 16 resumes the user interview. However, before resuming the user interview, the operator refers to the answer received from the central control device 12 to learn the answer. In the interview-response part (hereinafter sometimes referred to as the "second half of the response"), the operator responds by answering the user's questions, asking the user questions, explaining the services provided by the system 10, or, in the case of sales, making an offer and closing the deal. In the first embodiment, the duration of the second half of the response is set to the average time (for example, 10 minutes) when multiple operators actually respond to a user.

[0063] As described above, since the automatic response device 20 is configured to conduct the hearing, while the automatic response device 20 is conducting the hearing for a certain user, the operator can respond to another user. Therefore, it is possible to respond to users efficiently.

[0064] However, the time it takes to answer all questions during a hearing varies from user to user. Therefore, if an operator is listening to a user while assisting another user and then resuming the listening session for a user who has finished listening, the operator may have to wait until the user has finished answering all of the questions. Therefore, there is room for improvement in order to allow operators to respond to users more efficiently.

[0065] Therefore, in this first embodiment, by setting the hearing time to a first predetermined time (for example, 15 minutes) regardless of whether the user has answered all questions, the operator can determine the time to start the second half of the user's conversation as the time to end the conversation for that user. However, the first predetermined time for the conversation is set longer than the time for the second half of the conversation (hereinafter referred to as the "second predetermined time"). Therefore, while the operator is conducting a conversation for a previously assigned user, the operator can execute the introductory part of the conversation for another user who is subsequently assigned, and can then start a conversation for this other user. Furthermore, the operator can execute the second half of the conversation for a previously assigned user while conducting a conversation for another user. This allows the operator to more efficiently interact with users.

[0066] In the first embodiment, when a user is assigned to an operator, the duration of the introductory portion of the response is set to a third predetermined time (for example, 3 minutes). When the operator actually executes the introductory portion of the response to the user, the duration may deviate slightly from the third predetermined time, but it will not deviate significantly, so it is considered that the responses of the user and other users will not be affected.

[0067] FIG. 8 is a diagram for explaining an example of time change when multiple users are sequentially assigned to one operator. In FIG. 8, one rectangle (or one band) corresponds to the response time for one user. Also, "OP" means response by the operator. Furthermore, as described above, hearing means hearing (i.e., response) by the automatic response device 20. Also, as described above, the introductory part of the response is executed before the hearing, and the latter part of the response is executed after the hearing.

[0068] As shown in Fig. 8, when a user requests assistance, the central control device 12 determines an available operator. Whether an operator is available is determined based on whether a user is assigned to the operator and, if a user is assigned to assist the operator, whether the operator is currently being interviewed. However, a user who has already been attended to by the operator is not included in the users assigned to the operator.

[0069] That is, if no user is assigned to the operator, the user who requested the response is assigned to the operator. Even if a user is assigned to the operator, if the latter half of the user's response overlaps with the time of the response of the user who requested the response, another user is assigned.

[0070] However, in the first embodiment, the introductory part of the response is executed for each user, so it is necessary to ensure that the time at which the introductory part of the response for the newly assigned user starts does not overlap with the latter part of the response for the previously assigned user.

[0071] Therefore, as shown in FIG. 8, when a first user is assigned to a certain operator (herein referred to as "the operator") and the first user is being interviewed, a second user can be assigned. Also, when the second user is being interviewed, a third user can be assigned. In other words, users can be assigned to the operator in sequence. Note that the three users assigned to the operator are different users.

[0072] However, as described above, the second user is assigned to the operator so that the introductory part of the second user's response does not overlap with the latter part of the first user's response and the second user's listening time overlaps with the entire latter part of the first user's response. Furthermore, the third user is assigned to the operator so that the introductory part of the third user's response does not overlap with the latter part of the second user's response and the third user's listening time overlaps with the entire latter part of the second user's response.

[0073] Furthermore, when a user is assigned to the operator, the user is assigned to the operator under the constraints described above, and therefore, it may not be possible to start responding immediately. For this reason, when a user is assigned to an operator, the time at which each user's response will begin (hereinafter referred to as the "start time") is also set. The start time is the time at which the introductory part of each user's response begins. Note that, when a user is assigned to an operator, the times at which the introductory part of the response, the interview, and the latter part of the response are executed are fixedly set, so when the time at which the introductory part of the response begins is set, the time at which the interview will begin and the time at which the latter part of the response begins are also set.

[0074] Specifically, if no user is assigned to the operator, when a response request is received from a user, the start time is set to the current time or a time from a few seconds to a few minutes in the future, and the operator will respond to the user. When a second or subsequent user is assigned, the start time is set taking into consideration the first predetermined time for hearing from the user assigned immediately before and the second predetermined time for the latter half of the response.

[0075] Therefore, while the operator is hearing the first user, the operator executes the introductory part of the second user's response, starts the second user's response, and when the first user's response ends while the second user's response is being executed, the operator executes the second half of the first user's response. Also, while the operator is hearing the second user, the operator executes the introductory part of the third user's response after finishing the second half of the first user's response, starts the third user's response, and when the second user's response ends while the third user's response is being executed, the operator executes the second half of the second user's response. Similarly, the operator responds to the third and subsequent users.

[0076] Although the explanation is omitted, users are similarly assigned to other operators, and the assigned users respond in turn.

[0077] By fixing the hearing time in this way, it is possible to know in advance when each user's hearing will end, and therefore it is possible to assign multiple users to a single operator in sequence with almost no waiting time. Therefore, it is possible to assign users more efficiently than when the hearing time is not fixed.

[0078] Information about user allocation according to the time series as shown in FIG. 8 (hereinafter referred to as "allocation information") is managed for each operator by the central control device 12, and the central control device 12 determines the operator to which the user who has requested a response should be assigned based on this allocation information.

[0079] However, if the hearing time is fixed at the first predetermined time, there will be users who finish answering all questions before the first predetermined time. For such users, in the first embodiment, a video explaining the service or product provided by the system 10 is shown to the users until the first predetermined time has elapsed, and the users are made to wait for the operator to resume the conversation, i.e., for the start of the second half of the conversation.

[0080] When a user is assigned to an operator, the times for the introductory and latter portions of the operator's interaction are fixed, but the actual interaction may be longer or shorter than the fixed times. As described above, when the operator performs the introductory portion of the interaction with a user, the time may deviate slightly from the third predetermined time. However, since the introductory portion is simply the operator greeting the user, it does not deviate significantly. Therefore, in this first embodiment, once the introductory portion of the interaction is completed, the hearing begins in accordance with the operator's instructions. On the other hand, if no restrictions are imposed when the operator performs the latter portion of the interaction with a user, the second predetermined time may be exceeded. In this case, a delay occurs in the interaction of a user assigned later in time. For this reason, in this first embodiment, the latter portion of the interaction is forcibly terminated when the second predetermined time has elapsed.

[0081] 9 shows an example of a memory map 200 of the RAM 32 built into the central control unit 12. As shown in FIG. 9, the RAM 32 includes a program storage area 202 and a data storage area 204. The program storage area 202 stores information processing programs executed by the central control unit 12 of this first embodiment. The information processing programs of the central control unit 12 include a main processing program 202a, a communication program 202b, an operator management program 202c, and a standby program 202d.

[0082] The main processing program 202a is a program for executing the main routine of information processing of the central control unit 12 of this first embodiment.

[0083] The communication program 202b is a program for communicating with external devices such as the operator terminal 16, the user terminal 18, and the automatic answering device 20 (sending and receiving data, etc.).

[0084] The operator management program 202c is a program that manages allocation information for each operator (or each operator terminal 16), and based on this allocation information, assigns a user (or user terminal 18) to respond to each operator (or operator terminal 16), and has each operator execute the response of the user assigned to that operator.

[0085] The standby program 202d is a program for making the automatic response device 20 play a video until the first predetermined time has elapsed when the automatic response device 20 finishes hearing the user earlier than the first predetermined time, and for the automatic response device 20 to wait for the operator to resume answering the call.

[0086] Although not shown, the program storage area 202 also stores other programs for executing the information processing program of the central control unit 12.

[0087] The data storage area 204 stores operator management data 204a, a first predetermined time counter 204b, a second predetermined time counter 204c, a moving image playback flag 204d, and the like.

[0088] The operator management data 204a is management information data related to each operator, and includes allocation information for each operator as well as data on the status (hereinafter referred to as "response status") of each user assigned to each operator, such as whether the user is currently performing the introductory part of the response, the interview, or the latter part of the response. Note that the management information for a user whose response has been completed by the operator is deleted.

[0089] The first predetermined time counters 204b are provided for each operator, one for each user currently being served, and are counters or timers for counting the first predetermined time for hearing each user.

[0090] The second predetermined time counter 204c is provided for each operator and is a counter or timer for counting the second predetermined time in the latter half of the user's response.

[0091] The video playback flag 204d is provided for each operator, one for each user currently being served, and is a flag indicating whether or not a video is being played. When a video is being played in the user terminal 18, the video playback flag 204d for the user of that user terminal 18 is turned on, and the corresponding bit is set to 1. On the other hand, when a video is not being played in the user terminal 18, the video playback flag 204d for the user of that user terminal 18 is turned off, and the corresponding bit is set to 0.

[0092] Although not shown in the figure, the data memory area 204 stores other data necessary for the central control unit 12 to execute information processing, and is provided with other timers (counters) and flags necessary for executing information processing.

[0093] Fig. 10 shows an example of a memory map 300 of the RAM 42 built into the operator terminal 16. As shown in Fig. 10, the RAM 42 includes a program storage area 302 and a data storage area 304. The program storage area 302 stores information processing programs executed by the operator terminal 16 of this first embodiment. The information processing programs for the operator terminal 16 include a main processing program 302a, an operation detection program 302b, a communication program 302c, a display program 302d, a voice detection program 302e, a voice output program 302f, a photography program 302g, and a remote response program 302h.

[0094] The main processing program 302a is a program for executing the main routine of information processing of the operator terminal 16 of this first embodiment.

[0095] The operation detection program 302b is a program for detecting operation data 304a input from the input device 48 in accordance with an operation by the operator.

[0096] The communication program 302c is a program for communicating with external devices such as the central control unit 12 and the user terminal 18.

[0097] The display program 302d is a program for generating and outputting display screen data required for displaying various screens on the display device 50.

[0098] The voice detection program 302e is a program for detecting voice by the microphone 52.

[0099] The audio output program 302f is a program for outputting audio from the speaker .

[0100] The photography program 302g is a program for taking an image (a still image or a moving image) by the camera 56.

[0101] Remote response program 302h is a program that allows an operator to remotely respond to users using operator terminal 16. Remote response program 302h includes various programs for realizing remote response, including an avatar image data generation program for generating avatar image data.

[0102] Although not shown in the figure, the program storage area 302 also stores other programs required for executing the information processing program of the operator terminal 16.

[0103] Various types of data are stored in data storage area 304. The various types of data include operation data 304a, image generation data 304b, operator voice data 304c, operator image data 304d, avatar image data 304e, user terminal data 304f, and response data 304g.

[0104] The operation data 304a is data that indicates the operation state of the operator on the input device 48 detected by the operation detection program 302b.

[0105] Image generation data 304b is data such as polygon data and texture data used to generate display screen data by display program 302d.

[0106] The operator voice data 304c is voice data of the operator detected by the microphone 52.

[0107] The operator image data 304d is image data of the operator captured by the camera 56.

[0108] Avatar image data 304e is image data of an avatar generated based on image data of the captured operator.

[0109] The user terminal data 304f is data transmitted and received between the user terminal 18. The user terminal data 304f includes user voice data and user image data received from the user terminal 18, and also includes operator voice data and avatar image data to be transmitted to the user terminal 18.

[0110] The response data 304g is data on the content of responses given by each user in the hearing, and is received from the central control device 12 and stored so as to be identifiable for each user.

[0111] Although not shown in the figure, the data storage area 304 stores other data necessary for the operator terminal 16 to execute information processing, and is provided with timers (counters) and flags necessary for executing information processing.

[0112] 11 shows an example of a memory map 400 of the RAM 62 built into the user terminal 18. As shown in FIG. 11, the RAM 62 includes a program storage area 402 and a data storage area 404. The program storage area 402 stores information processing programs executed by the user terminal 18 of this first embodiment. The information processing programs for this user terminal 18 include a main processing program 402a, an operation detection program 402b, a communication program 402c, a display program 402d, a voice detection program 402e, a voice output program 402f, an imaging program 402g, an operator response program 402h, and an automatic response program 402i.

[0113] The main processing program 402a is a program for executing the main routine of information processing of the user terminal 18 of this first embodiment.

[0114] The operation detection program 402b is a program for detecting operation data 404a input from the input device 68 in accordance with a user's operation.

[0115] The communication program 402c is a program for communicating with external devices such as the central control device 12, the operator terminal 16, and the automatic response device 20.

[0116] The display program 402d is a program for generating display screen data required for displaying various screens on the display device .

[0117] The voice detection program 402 e is a program for detecting voice by the microphone 72 .

[0118] The audio output program 402f is a program for outputting audio from the speaker 74.

[0119] The photography program 402g is a program for taking an image (a still image or a moving image) by the camera 76.

[0120] The operator response program 402h is a program for remotely responding in cooperation with the operator terminal 16, and includes various programs for realizing remote response.

[0121] The automatic response program 402i is a program for performing automatic response using artificial intelligence in cooperation with the automatic response device 20, and includes various programs for realizing automatic response. In this first embodiment, the automatic response executes a hearing with the automatic response device 20, and executes video playback processing as necessary.

[0122] Although not shown, the program storage area 402 also stores other programs for executing the information processing program of the user terminal 18.

[0123] Various types of data are stored in the data storage area 404. The various types of data include operation data 404a, image generation data 404b, user voice data 404c, user image data 404d, operator terminal data 404e, and automatic answering device data 404f.

[0124] The operation data 404a is data that indicates the state of the user's operation on the input device 68 detected by the operation detection program 402b.

[0125] Image generation data 404b is data such as polygon data and texture data used to generate display screen data by display program 402d.

[0126] The user voice data 404c is voice data of the user detected by the microphone 72.

[0127] The user image data 404d is image data of the user captured by the camera 76.

[0128] The operator terminal data 404e is data transmitted and received between the operator terminal 16. The operator terminal data 404e includes the operator's voice data and avatar image data received from the operator terminal 16, and also includes the user's voice data and user's image data to be transmitted to the operator terminal 16.

[0129] The automatic response device data 404f is data transmitted and received between the automatic response device 20. The automatic response device data 404f includes question data and video data received from the automatic response device 20, and response data to be transmitted to the automatic response device 20.

[0130] Although not shown in the figure, the data storage area 404 stores other data necessary for the user terminal 18 to execute information processing, and is provided with timers (counters) and flags necessary for executing information processing.

[0131] 12 shows an example of a memory map 500 of the RAM 82 built into the automated answering device 20. As shown in FIG. 12, the RAM 82 includes a program storage area 502 and a data storage area 504. The program storage area 502 stores information processing programs executed by the automated answering device 20 of the first embodiment. The information processing programs for the automated answering device 20 include a main processing program 502a, a communication program 502b, a hearing program 502c, and a video playback program 502d.

[0132] The main processing program 502a is a program for executing the main routine of information processing of the automated answering device 20 of the first embodiment.

[0133] The communication program 502b is a program for communicating with external devices such as the central control device 12 and the user terminal 18.

[0134] The hearing program 502c is a program for transmitting question data 504a for each question to the user terminal 18 and receiving answer data to the questions from the user terminal 18. However, when the hearing program 502c is executed, the communication program 502b is also executed.

[0135] The video playback program 502d is a program for transmitting video data to the user terminal 18 of the user and playing the video data when the user's hearing is completed earlier than the first predetermined time. However, when the video playback program 502d is executed, the communication program 502b is also executed.

[0136] Although not shown, the program storage area 502 also stores other programs for executing the information processing program of the automated answering device 20.

[0137] The data storage area 504 stores question data 504a, video data 504, answer data 504c, and the like.

[0138] The question data 504a is data on a plurality of questions to be asked to the user in the hearing process.

[0139] The video data 504b is data about a video that explains the services provided by the system 10, etc.

[0140] The response data 504c is user response data received from the user terminal 18, and is stored so as to be identifiable for each user.

[0141] Although not shown in the figure, the data storage area 504 stores other data necessary for the automated answering device 20 to execute information processing, and is provided with timers (counters) and flags necessary for executing information processing.

[0142] 13 is a flow diagram showing an example of operator management processing, which is an example of information processing by the CPU 30 of the central control device 12. The operator management processing is executed for each operator terminal 16 or its operator, and each user assigned to that operator. Below, the operator management processing will be described for a certain operator terminal 16 or its operator, and a certain user assigned to that operator, but the same applies to this operator terminal 16 or other users assigned to that operator. The same also applies to other operator terminals 16 or their operators, users assigned to that operator, and other users. In this operator management processing, a certain operator terminal 16 will be referred to as the "operator terminal 16," its operator will be referred to as the "operator," and a certain user assigned to that operator will be referred to as the "user."

[0143] Although not shown in the drawings, when a response request is received from a certain user terminal 18, the CPU 30 of the central control device 12 determines an operator who can respond to the user of this user terminal 18. The method for determining the available operator is as described above. Once the CPU 30 determines an available operator, that is, once the CPU 30 assigns the user of the user terminal 18 that made the response request to the operator, the CPU 30 executes operator management processing for the user of the operator when the response start time arrives.

[0144] 13, when the CPU 30 starts the operator management process, in step S1, it instructs the operator terminal 16 to handle the user. In step S1, the CPU 30 updates the operator management data 204a. Specifically, the CPU 30 adds management information, which includes assignment information assigning the user to the operator of the operator terminal 16 and indicates that the introductory part of the response is being executed for the user, to the operator management data 204a, linking it to the operator.

[0145] In the next step S3, it is determined whether or not there is a request to start a hearing about the user from the operator terminal 16. If the answer is "NO" in step S3, that is, if there is no request to start a hearing about the user from the operator terminal 16, the process returns to step S3.

[0146] On the other hand, if the result in step S3 is "YES," that is, if there is a request from the operator terminal 16 to start an interview about the user, in step S5 the automatic response device 20 is instructed to interview the user, and in step S7 counting of the first predetermined time is started, and the process proceeds to step S9 shown in Fig. 14. Also, in step S7, the CPU 30 updates the operator management data 204a by updating the management information to include allocation information that assigns the user to the operator of the operator terminal 16 and indicates that an interview about the user is being conducted.

[0147] 14, in step S9, it is determined whether a first predetermined time (for example, 15 minutes) has elapsed since the start of hearing. If the determination in step S9 is "NO," that is, if the first predetermined time has not elapsed since the start of hearing, it is determined in step S11 whether a video is being played. Here, the CPU 30 determines whether the corresponding bit of the video playback flag 204e is set to 1.

[0148] If the answer is "YES" in step S11, that is, if a video is being played, the process returns to step S9. On the other hand, if the answer is "NO" in step S11, that is, if a video is not being played, the process determines in step S13 whether or not there is a notification from the automated answering device 20 that the hearing has ended.

[0149] If step S13 is "NO", that is, if there is no notice of the end of the hearing from the automatic response device 20, the process returns to step S9. On the other hand, if step S13 is "YES", that is, if there is a notice of the end of the hearing from the automatic response device 20, the process instructs the automatic response device 20 to play a video in step S15, and turns on the video playback flag 204e for the user terminal 18 of the user in question in step S17, and the process returns to step S9.

[0150] If step S9 returns "YES," that is, if a first predetermined time has elapsed since the start of the hearing, step S19 instructs the automated answering device 20 to end the hearing for the user. Also, in step S19, the CPU 30 updates the operator management data 204a by updating the management information to include allocation information that assigns the user to the operator of the operator terminal 16 and indicates that the latter half of the response for the user is being performed.

[0151] In the next step S21, it is determined whether or not the answer data of the user has been received from the automatic response device 20. If the answer data of the user has not been received from the automatic response device 20, the process returns to step S21.

[0152] On the other hand, if the answer is "YES" in step S21, that is, when the user's response data is received from the automatic response device 20, the operator terminal 16 is called in step S23, and the user's response data is sent to the operator terminal 16 in step S25, and the process proceeds to step S27.

[0153] In step S27, the operator terminal 16 is instructed to attend to the user, in step S29, counting of a second predetermined time period is started, and in step S31, it is determined whether or not the attendance of the user has ended. Here, the CPU 30 determines whether or not a notification of the end of attendance has been received from the operator terminal 16, or whether or not the second predetermined time period has elapsed.

[0154] If step S31 returns "NO," that is, if the user has not finished handling the call, the process returns to step S31. On the other hand, if step S31 returns "YES," that is, if the user has finished handling the call, the operator management process for the operator for that user is terminated. However, if the second predetermined time has elapsed, the CPU 30 instructs the operator terminal 16 to end handling. Furthermore, when the CPU 30 terminates the operator management process for the operator for that user, the CPU 30 updates the operator management data 204a by updating the management information to include assignment information that assigns the user to the operator of the operator terminal 16 and indicates that handling the user has ended. However, the management information for the user for which handling has ended may be erased.

[0155] 15 to 17 are flow charts showing an example of the response processing included in the information processing of the CPU 40 of the operator terminal 16 shown in FIG. 3. The CPU 40 of the operator terminal 16 starts the response processing when it receives a response instruction from the central control device 12. However, the response processing is executed for each user assigned to respond. Therefore, when multiple users are assigned to the operator, the response processing is executed for each user in parallel. Furthermore, although not shown in the figures, processing for receiving control signals or data from other devices and photographing processing are also executed in parallel with the response processing.

[0156] 15, when the CPU 40 starts the answering process, in step S101, it establishes a connection with the user terminal 18 (hereinafter referred to as "the user terminal 18" in this answering process) used by the user who has been instructed to answer (hereinafter referred to as "the user" in this answering process). However, connection information for the user terminal 18 is transmitted from the central control device 12 to the operator terminal 16 together with the answering instruction.

[0157] In the next step S103, it is determined whether or not the operator's voice has been detected. If "NO" in step S103, that is, if the operator's voice has not been detected, the process proceeds to step S107.

[0158] On the other hand, if "YES" in step S103, that is, if the operator's voice has been detected, then in step S105, the operator's voice data is transmitted to the user terminal 18, and the process proceeds to step S107.

[0159] In step S107, avatar image data 304e is generated based on operator image data 304d, and in step S109, avatar image data 304e is sent to user terminal 18, and the process proceeds to step S111.

[0160] In step S111, it is determined whether data has been received from the user terminal 18. Here, the CPU 40 determines whether user voice data and / or user image data of the user has been received. If the determination in step S111 is "NO," that is, if data has not been received from the user terminal 18, the process proceeds to step S115.

[0161] On the other hand, if "YES" in step S111, that is, if data is received from the user terminal 18, the voice and / or image of the user is output in step S13, and the process proceeds to step S115.

[0162] In step S115, it is determined whether or not the operator has issued an instruction to start hearing. If "NO" in step S115, that is, if the operator has not issued an instruction to start hearing, the process returns to step S103.

[0163] On the other hand, if the answer is "YES" in step S115, that is, if the operator has issued an instruction to start a hearing, a request to start a hearing is sent to the central control device 12 in step S117, and the process proceeds to step S119 shown in Fig. 16. Steps S103 to S115 correspond to the introduction part of the dialogue.

[0164] 16, in step S119, it is determined whether or not there is a call from the central control device 12. If "NO" in step S119, that is, if there is no call from the central control device 12, the process returns to step S119. On the other hand, if "YES" in step S119, that is, if there is a call from the central control device 12, the process proceeds to step S121.

[0165] In step S121, it is determined whether or not response data 304g has been received from the central control device 12. If "NO" in step S121, that is, if response data 304g has not been received from the central control device 12, the process returns to step S121. On the other hand, if "YES" in step S121, that is, if response data 304g has been received from the central control device 12, the process proceeds to step S123.

[0166] In step S123, the answer is displayed. Therefore, the operator can know the answer of the user and whether there are any unanswered questions. Therefore, the operator refers to the answer and performs the second half of the response to the user.

[0167] 17, in the next step S125, it is determined whether or not the operator's voice has been detected. If "NO" in step S125, the process proceeds to step S129. On the other hand, if "YES" in step S125, in step S127, the operator's voice data is transmitted to the user terminal 18, and the process proceeds to step S129.

[0168] In step S129, avatar image data 304e is generated based on operator image data 304d, and in step S131, avatar image data 304e is sent to user terminal 18, and the process proceeds to step S133.

[0169] In step S133, it is determined whether data has been received from the user terminal 18. If "NO" in step S133, the process proceeds to step S137. On the other hand, if "YES" in step S133, the voice and / or image of the user is output in step S135, and the process proceeds to step S137.

[0170] In step S137, it is determined whether an instruction to end the service has been given. Here, the CPU 40 determines whether an instruction to end the service has been input by the operator or whether an instruction to end the service has been received from the central control device 12.

[0171] If step S137 is "NO," that is, if there is no instruction to end the service, the process returns to step S125. On the other hand, if step S137 is "YES," that is, if the operator has input an instruction to end the service, the service processing for the user is terminated. Steps S125 to S137 correspond to the latter half of the dialogue. However, if the operator has input an instruction to end the service, the CPU 40 notifies the central control device 12 that the service has been terminated.

[0172] Fig. 18 is a flow chart showing an example of hearing processing included in the information processing of the CPU 80 of the automatic response device 20 shown in Fig. 5. The CPU 80 of the automatic response device 20 starts the hearing processing when a hearing instruction is received from the central control device 12. However, the hearing processing is executed for each user who is answered by one or more operators. Therefore, when hearings are executed for multiple users, the hearing processing is executed for each user in parallel.

[0173] 17, when the CPU 80 starts the hearing process, in step S201, it establishes a connection with the user terminal 18 of the user who will be hearing. However, the automatic response device 20 receives from the central control device 12 a hearing instruction as well as connection information for the user terminal 18 of the user who will be hearing.

[0174] In the next step S203, initial question data is sent to the user terminal 18, and in step S205, it is determined whether answer data to the question data has been received. If the answer is "NO" in step S205, that is, if answer data to the question data has not been received, the process returns to step S205.

[0175] On the other hand, if "YES" in step S205, that is, if answer data to the question data has been received, it is determined in step S207 whether or not there is next question data.

[0176] If step S207 is "YES", that is, if there is next question data, then in step S209 the next question data is sent to the user terminal 18, and the process proceeds to step S213. On the other hand, if step S207 is "NO", that is, if there is no next question data, then in step S211 the central control device 12 is notified of the end of the hearing, and the process proceeds to step S213.

[0177] In step S213, it is determined whether or not an instruction to end the hearing has been received from the central control device 12. If the answer is "YES" in step S213, that is, if an instruction to end the hearing has been received from the central control device 12, then in step S215, answer data is transmitted to the central control device 12, and the hearing process is terminated. However, the answer data transmitted in step S213 is one or more answers to one or more questions, and it is not necessarily the case that all answers have been obtained for all questions.

[0178] On the other hand, if step S213 is "NO", that is, if there is no instruction to end the hearing, it is determined in step S217 whether or not a video is being played. If step S217 is "YES", that is, if a video is being played, the process proceeds to step S221. On the other hand, if step S217 is "NO", that is, if a video is not being played, it is determined in step S219 whether or not there is an instruction to play a video from the central control device 12.

[0179] If step S219 is "NO", that is, if there is no instruction to play the video from the central control device 12, the process returns to step S205. On the other hand, if step S219 is "YES", that is, if there is an instruction to play the video from the central control device 12, the process executes video playback processing in step S221 and returns to step S213. In step S221, video data and an instruction to play the video data are transmitted to the video playback device of the user terminal 18 of the user who ended the hearing before the instruction to end the hearing was received.

[0180] As described above, according to the first embodiment, the time for the automated answering device to ask the user is fixed to the first predetermined time, so that it is possible to predict the time when the operator will attend to the user after that, and therefore, when there is an answering request from another user, it is possible to efficiently assign the operator to attend to the other user. Therefore, it is possible to have the operator efficiently attend to each of the multiple users.

[0181] In the first embodiment described above, if all questions are answered within the first predetermined time during the hearing, a video about the services provided by the system is played, but this does not need to be limited to this.

[0182] Alternatively, the device may simply wait until the first predetermined time has elapsed. In this case, the user may be notified by outputting audio and / or images to the user terminal that the device is waiting until the operator starts (restarts) responding. In this case, the user may be notified by outputting audio and / or images to the user terminal that the operator will resume responding when the first predetermined time has elapsed.

[0183] <Second Example> The system 10 of the second embodiment is the same as the system 10 of the first embodiment, except that if an operator answers all questions in a hearing of a user assigned to that operator before a first predetermined time, the latter half of the response for that user is performed by another operator, and the latter half of the response for another user assigned to that other operator is performed by that operator. Therefore, in the second embodiment, only the details that are different from the first embodiment will be described, and the description of the same details as those described in the first embodiment will be omitted or will be briefly described.

[0184] In the second embodiment, the handling of a user assigned to a certain operator is performed by another operator, so that users who finish their interview early can be handled efficiently without having to wait.

[0185] FIG. 19 shows an example of a state in which multiple users are assigned to each of two operators. In the example shown in FIG. 19, the first operator and the second operator are different operators, and "OP1" indicates that the first operator is in charge, and "OP2" indicates that the second operator is in charge. In the example shown in FIG. 19, two users are assigned to the first operator, and three users are assigned to the second operator. The assignments of the first operator are shown above the dashed-dotted line, and the assignments of the second operator are shown below the dashed-dotted line. For convenience of explanation, the two users assigned to the first operator are referred to as the first user and the second user, and the three users assigned to the second operator are referred to as the first user, the second user, and the third user, but this is merely an example. Note that the two users assigned to the first operator and the three users assigned to the second operator are all different users.

[0186] 19, at time t1, the first operator is serving the first user and is conducting a hearing with the second user. At time t1, the third and subsequent users have not been assigned.

[0187] On the other hand, at time t1, the second operator has finished serving the first user, and is currently conducting interviews with the second and third users.

[0188] At time t1, if the second user assigned to the first operator finishes the hearing early, the first operator will not be able to assist the second user because he or she is already assisting the first user.

[0189] On the other hand, at time t1, the second operator is not serving any users, but it has been decided that the second and third users will be served after time t1, and the time to serve the second user (perform the latter half of the service) is approaching.

[0190] Therefore, in this second embodiment, when a hearing of a user (here, referred to as the "first user") assigned to a certain operator (here, the first operator) ends early, if another operator (here, the second operator) who starts responding to a user (here, referred to as the "second user") assigned to him / her after the end of the hearing ends and the hearing of the second user can be extended, the time of the hearing is extended and instead of performing (the latter half of) the response of the second user, the response of (the latter half of) the first user who ended the hearing early is performed. Conversely, a certain operator, i.e., the first operator, performs (the latter half of) the response of the second user whose hearing time has been extended. In other words, the first operator and the second operator take turns responding to users.

[0191] Whether the hearing can be extended is determined based on the progress of the hearing. If the progress of the hearing is behind schedule, it is determined that the hearing can be extended. The progress of the hearing can be determined by the ratio of the number of answered questions to the total number of questions. Furthermore, whether the progress of the hearing is behind schedule is determined by comparing the ratio of the hearing execution time to a first predetermined time set for the hearing (for convenience of explanation, referred to as the "first ratio") with the ratio of the number of answered questions to the total number of questions to be heard (for convenience of explanation, referred to as the "second ratio"). If the second ratio is smaller than the first ratio, it can be said that the progress of the hearing is behind schedule.

[0192] However, if the first and second rates are approximately the same, extending the first predetermined time for the hearing may cause the user who is currently receiving the hearing during the extended first predetermined time to wait for a long time until the second half of the conversation begins, which would not be considered an efficient allocation of operators. For this reason, in the second embodiment, if the second rate is smaller than the first rate by a certain value or more, it is determined that the hearing can be extended. For example, if the first and second rates are each expressed as a number between 0 and 1, the certain value is set to 0.3.

[0193] In the second embodiment, the time for extending the hearing is determined so that there is no delay in responding to the user assigned to the operator after the operator takes over and performs the latter half of the response of the user assigned to the other operator. Specifically, the extension time is the time remaining until the first predetermined time when the user who has taken over the response from the other operator finishes the hearing early.

[0194] Therefore, in the example shown in Fig. 19, if it is determined that the hearing of the second user assigned to the second operator can be extended, the hearing time of this second user is extended by the time remaining until the first predetermined time if the second user assigned to the first operator finishes the hearing early, as shown in Fig. 20. Also, the second operator handles (the latter half of) the second user assigned to the first operator, and the first operator handles (the latter half of) the second user assigned to the second operator after the hearing of that user.

[0195] 21-23 are flow charts showing a portion of the operator management processing of the CPU 30 of the central control device 12 in the second embodiment. The processing from the start of the operator management processing to step S11 shown in FIG. 13 is the same in the second embodiment, so illustrations and explanations are omitted. Also, as shown in FIGS. 21-23, step S11 shown in FIG. 14 is deleted, and steps S51, S53, S55, S57, S59, S61, and S63 are provided instead of steps S15 and S17. Furthermore, if step S13 is "NO," steps S65, S67, S69, S71, S73, S75, S77, S79, S81, and S83 are executed.

[0196] As shown in Fig. 21, if the answer is "NO" in step S9, it is determined in step S13 whether or not there is a notification of the end of the hearing, and if the answer is "NO" in step S13, the process proceeds to step S65 shown in Fig. 22, and if the answer is "YES" in step S13, it is determined in step S51 whether or not there is an operator who can take over. Here, the CPU 30 determines whether or not the second half of the response of the user assigned to the other operator starts at a later time and the hearing of the user can be extended.

[0197] If step S51 is "NO", that is, if there is no operator who can take over, the process returns to step S9. On the other hand, if step S51 is "YES", that is, if there is an operator who can take over, the process goes to step S53, where it is determined whether or not response data has been received.

[0198] If step S53 is "NO", that is, if the answer data has not been received, the process returns to step S53. On the other hand, if step S53 is "YES", that is, if the answer data has been received, the process calls the replacement operator (hereinafter referred to as "the other operator" in this operator management process) in step S55, and transmits the answer data to the operator terminal 16 of the other operator (hereinafter referred to as "the other operator terminal 16" in this operator management process) in step S57, and then proceeds to step S59.

[0199] In step S59, the other operator is instructed to attend to the user (meaning the user assigned to the operator, as explained in the first embodiment), and in step S61, counting of a second predetermined time is started. In step S63, it is determined whether the other operator has finished attending to the user. In this step S63, the CPU 30 determines whether there is a notification of the end of attending to the user from the operator terminal 16 of the other operator who has taken over, and whether the second predetermined time has elapsed.

[0200] If "NO" in step S63, that is, if the user has not finished answering the call, the process returns to step S63. On the other hand, if "YES" in step S63, that is, if the user has finished answering the call, the operator management process for the operator ends.

[0201] If step S13 is "NO", then in step S65 shown in Fig. 22 it is determined whether the call is from a user assigned to the other operator (hereinafter referred to as "the other user" in this operator management process). If step S65 is "YES", that is, if the call is from the other user, then in step S67 the time for hearing the other user is extended, and the process proceeds to step S69.

[0202] In step S69, it is determined whether the extended hearing time has elapsed. If the answer is "NO" in step S69, that is, if the extended hearing time has not elapsed, the process returns to step S69. On the other hand, if the answer is "YES" in step S69, that is, if the extended hearing time has elapsed, the process instructs the automated answering device 20 to end the hearing for the other user in step S71.

[0203] In the next step S73, it is determined whether or not response data has been received. If "NO" in step S73, the process returns to step S73. On the other hand, if "YES" in step S73, the process proceeds to step S75.

[0204] In step S75, the operator terminal 16 is called, in step S77, response data is sent to the operator terminal 16, in step S79, the operator terminal 16 is instructed to respond to the other user, and the process proceeds to step S81.

[0205] In step S81, counting of the second predetermined time is started, and in step S83, it is determined whether the response has ended. In this step S83, it is determined whether there is a notification of the response end from the operator terminal 16, or whether the second predetermined time has elapsed. If "NO" in step S83, the process returns to step S83, and if "YES" in step S83, the operator management process for the operator ends.

[0206] Also, as shown in Fig. 21, if the answer is "YES" in step S9, that is, if the call is not from the other user, the process proceeds to step S19 as shown in Fig. 23. The process from step S19 onwards shown in Fig. 23 is the same as that explained in the first embodiment, and therefore a duplicate explanation will be omitted.

[0207] 24 and 25 show a portion of an example of the response processing of the CPU 40 of the operator terminal 16 of the second embodiment. The processing from the start of the response processing to step S123 shown in FIGS. 15-17 is the same in the second embodiment, and therefore will not be illustrated or described again. Also, as shown in FIGS. 24 and 25, step S151 is provided between step S123 and step S125, and further, if step S151 returns "YES," steps S153, S155, S157, S159, S161, S163, S165, and S167 are executed.

[0208] As shown in Fig. 24, in step S151, it is determined whether the call is from a user assigned to another operator. If "NO" in step S151, that is, if the call is not from a user assigned to another operator, the process proceeds to step S125. The processes in steps S125 to S137 are the same as those described in the first embodiment, so redundant explanations will be omitted. On the other hand, if "YES" in step S151, if the call is from a user assigned to another operator, the process proceeds to step S153 shown in Fig. 25.

[0209] As shown in FIG. 25, in step S153, a connection state is established with the user terminal 18 (hereinafter referred to as "the other user terminal 18" in this response process) of the user assigned to the other operator (hereinafter referred to as "the other user" in this response process).

[0210] In the next step S155, it is determined whether the operator's voice has been detected. If "NO" in step S155, the process proceeds to step S159. On the other hand, if "YES" in step S155, the operator's voice data is transmitted to the other user terminal 18 in step S157, and the process proceeds to step S159.

[0211] In step S159, avatar image data 304e is generated based on operator image data 304d, and in step S161, avatar image data 304e is transmitted to the other user terminal 18, and the process proceeds to step S163.

[0212] In step S163, it is determined whether data has been received from the other user terminal 18. If "NO" in step S163, the process proceeds to step S167. On the other hand, if "YES" in step S163, the voice and / or image of the other user is output in step S165, and the process proceeds to step S167.

[0213] In step S167, it is determined whether an instruction to end the service has been given. If the answer is "NO" in step S167, the process returns to step S155. Steps S155 to S167 correspond to the latter half of the dialogue with the other user. On the other hand, if the answer is "YES" in step S167, the service processing ends. However, if the operator has input an instruction to end the service, the CPU 40 notifies the central control device 12 that the service with the other user has ended.

[0214] According to the second embodiment, a user who finishes the hearing early is handled by another operator other than the operator who assigned the user, and the hearing time of the user assigned to the other operator is extended so that the other operator can handle the user, i.e., the other operator and the user are alternately handled, so the user who finishes the hearing early does not have to wait. In other words, the operators can handle the user efficiently.

[0215] In the second embodiment, the user who extends the hearing time is a user who is experiencing delays in the progress of the hearing, so extending the hearing time does not reduce the efficiency of the operator's response.

[0216] Furthermore, in each of the above-described embodiments, the central control unit 12 and the automatic response unit 20 are provided separately, but the central control unit 12 and the automatic response unit 20 may be combined into one unit.

[0217] Furthermore, in each of the above-described embodiments, an introductory portion of the response is provided, but the introductory portion of the response does not have to be provided.

[0218] Furthermore, in the above-described embodiments, the operator responds by voice using an avatar, but the operator may respond by text chat. When responding only by voice, a system may be configured in which the central control device, operator terminal, user terminal, and automatic response device each make a call or communicate via a telephone network.

[0219] Furthermore, in the above-described embodiments, a CG avatar, which is a representative character of the operator, is used to respond to calls. However, a robot avatar can also be used. In such cases, a robot is provided instead of the user terminal, and the robot's movements and speech are controlled by the operator or automatically. The robot also acquires the user's image and voice and transmits them to the operator terminal or an automatic response device. In other words, the term "avatar" should be understood to encompass remotely controlled robots and CG agents, and also to include robots and CG agents that interact with the user partially or completely autonomously. In addition, if the user terminal is a robot, it has movable parts such as arms, and the movements of these movable parts express the movements of the operator.

[0220] In addition, in each of the above-mentioned embodiments, the automatic response device conducts a hearing by sending a question in a predetermined format to the user terminal, but the hearing may also be conducted through a dialogue using an AI avatar, a chat using a chatbot, or a call using voice guidance.

[0221] Furthermore, in the above-described embodiments, when a user is assigned to an operator, the second half of the response is set to a second predetermined time, and when the operator responds to the user, the second predetermined time is not exceeded. However, when responding to a user, the second predetermined time may be allowed to exceed. In such a case, the start time of the second half of the response for the subsequently assigned user is delayed. When the start time of the second half of the response is delayed, a video explaining the services provided by the system may be played, as in the case where the interview is ended early.

[0222] It should be noted that the order of processing steps in the flow charts shown in the above embodiments can be changed if the same results are obtained. Also, the various screens and specific numerical values ​​shown in this specification and drawings are merely examples and can be changed as needed. [Explanation of symbols]

[0223] 10. Information Processing Systems 12...Central control unit 14...Network 16...Operator terminal 18...User terminal 20...Automatic answering device

Claims

1. An information processing system including a central control device, a plurality of user terminals used by a plurality of users, an operator terminal used by an operator who responds to the plurality of users, and an automatic response device that can automatically respond to each of the plurality of users, The central control unit an allocation means for allocating, when a response request is received from the user terminal, the response of the user using the user terminal to the operator who is available to respond; a response instruction means for instructing a response of the user to be assigned to the operator terminal used by the operator to which the user is assigned by the assignment means; The allocation means assigns the user using the user terminal that made the response request to the available operator, assuming that after the automatic response device has responded for at least a first predetermined time, the operator will respond for a second predetermined time that is shorter than the first predetermined time.

2. The information processing system of claim 1, further comprising a first determination means for determining, when a response request is received from the user terminal, as the available operator, an operator to whom no user is assigned to respond, or an operator whose time for automatically responding to the user using the user terminal by the automatic response device can overlap with all of the time for responding to other users who have already been assigned.

3. a plurality of the operator terminals used by the plurality of operators, The information processing system of claim 1, wherein the central control device further includes a switching means for having another operator other than the operator assigned to handle the user handle the user if the automatic response device finishes handling the user earlier than the first predetermined time.

4. The information processing system of claim 3, wherein the central control device further comprises a second determination means for determining the operator assigned to the other user as the other operator when the time to start serving the assigned other user is later than the time when the operator finishes serving the user assigned to the operator earlier than the first predetermined time and the time for automatically serving the other user can be extended.

5. the automated answering device acquires answers from the plurality of users to a plurality of questions, The information processing system according to claim 4 , wherein the second determination means determines that the time for automatically responding to the other user can be extended if the other user's progress in answering the plurality of questions is slow.

6. An information processing device that assigns a user of a user terminal to an operator who can respond to a response request from the user terminal, an allocation means for allocating, when a response request is received from the user terminal, the response of the user using the user terminal to the operator who is available to respond; a response instruction means for instructing a response of the user to be assigned to the operator terminal used by the operator to which the user is assigned by the assignment means; The allocation means is an information processing device that assigns the user using the user terminal that made the response request to the available operator, assuming that after an automatic response device has responded for at least a first predetermined time, the operator will respond for a second predetermined time that is shorter than the first predetermined time.

7. An information processing program for an information processing device that assigns a user of a user terminal to an operator who is available to respond to a response request from the user terminal, A processor of the information processing device when a response request is received from a user terminal, an automatic response device responds for at least a first predetermined time, and then the operator responds for a second predetermined time that is shorter than the first predetermined time, and the operator is assigned to respond to the user who uses the user terminal that has made the response request; an information processing program that instructs the operator terminal used by the assigned operator to respond to the user who has made the response request;

8. 1. An information processing method of an information processing device that assigns a user of a user terminal to an operator who is available to respond to a response request from the user terminal, comprising: The processor of the information processing device when a response request is received from a user terminal, after an automatic response device has responded for at least a first predetermined time, the operator is assumed to respond for a second predetermined time that is shorter than the first predetermined time, and the operator is assigned to respond to the user who uses the user terminal that has made the response request; an information processing method for instructing the operator terminal used by the assigned operator to respond to the user who has made the response request;

Images