Customer service system, server, control method and program

Abstract

To provide a customer service system, a server, a control method, and a program that provide an appropriate customer service according to a reaction of a visitor.SOLUTION: In a customer service system for providing a customer service using a robot for customer service, a program causes a computer to function as first acquisition means S2 that acquires first information relating to a subject of conversation provided by a responsible person in a business negotiation of a visitor and the responsible person, second acquisition means S3 that acquires a detection result relating to words and actions of the visitor by the robot, estimation means S4 that estimates a reaction of the visitor based on the detection result acquired by the second acquisition means, and output means S5 that outputs second information relating to the reaction estimated by the estimation means.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a customer service system, a server, a control method, and a program. [Background technology]

[0002] Recently, various technologies related to customer service systems using customer service robots have been proposed as a means of resolving labor shortages. For example, in Patent Document 1, a robot explains exhibits to visitors and guides them around the exhibition hall. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6142306 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when a visitor to a facility and a person in charge of handling the visitor conduct business negotiations, the person in charge may miss what the visitor says or may not be able to read the visitor's emotions, which may result in the person being unable to make an appropriate proposal to the visitor.

[0005] The present invention provides a technique that can provide appropriate customer service in response to a visitor's reaction. [Means for solving the problem]

[0006] According to one aspect of the present invention, A customer service system for serving customers using a customer service robot, a recognition means for recognizing the actions of the responder; An instruction means for moving the robot to a location where a business negotiation is to be held; With visitors The above During business negotiations with the person in charge The above A first acquisition means for acquiring first information related to a topic provided by a responder; A second acquisition means for acquiring a detection result regarding the behavior of the visitor by the robot; an estimation means for estimating a reaction of the visitor based on the detection result acquired by the second acquisition means; and an output means for outputting second information regarding the reaction estimated by the estimation means. A customer service system is provided. Effect of the Invention

[0007] According to the present invention, it is possible to provide appropriate customer service in accordance with the reaction of visitors. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an outline of a customer service system. [Diagram 2] FIG. 1 is a schematic diagram showing a configuration of a robot. [Diagram 3] FIG. 1 is a block diagram showing a hardware configuration of a customer service system. [Figure 4] FIG. 4 is a block diagram showing functions realized by a management server. [Diagram 5] 10 is a flowchart showing an example of processing by a processing unit of a management server. [Figure 6] 1A is a diagram illustrating an example of information acquired by a business negotiation content acquisition unit, FIG. 1B is a diagram illustrating an example of information acquired by a reaction estimation unit, and FIG. 1C is a diagram illustrating an example of information output by a reaction information output unit to a visitor DB. [Figure 7] 10 is a flowchart showing an example of processing by a processing unit of a management server and a processing unit of a terminal device. [Figure 8] 5A and 5B are diagrams showing examples of screens displayed on a display unit serving as an output unit of the terminal device. [Figure 9] 6A and 6B are flowcharts showing an example of processing by a processing unit of a management server. [Figure 10] 10 is a flowchart showing an example of processing by a processing unit of a management server and a processing unit of a robot. [Figure 11] 13(a) to 13(c) are diagrams showing examples of display of a robot's face. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more features among the multiple features described in the embodiments may be arbitrarily combined. In addition, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] First Embodiment <Overview of the customer service system SY> Fig. 1 is a diagram showing an outline of the customer service system SY. The customer service system SY is a system for providing customer service using a robot 2. Fig. 1 shows an example of the customer service system SY being used in a showroom 5 of a car dealership. The customer service system SY includes a management server 1, a robot 2, a terminal device 3, and a camera unit 4.

[0011] The management server 1 is a server that manages information about the store in which the showroom 5 is located or information about visitors to the store, and controls the robot 2, the terminal device 3, and the camera unit 4. The management server 1 is installed, for example, in the office of each store. The management server 1 installed in each store may be capable of communicating with a higher-level server that accepts reservations from visitors and manages personal information via a network such as the Internet.

[0012] The robot 2 is a robot for serving customers in the showroom 5. In Fig. 1, a robot 2a that serves as a receptionist near the entrance 51 of the showroom 5 and a robot 2b that is present at a business negotiation between a sales representative 8 and a visitor 9 in a business negotiation space 52a are shown. In the following description, when there is no need to distinguish between these, they will be referred to as robot 2.

[0013] The terminal device 3 is a terminal operated by the person in charge 8. The person in charge 8 uses the terminal device 3 to present various materials to the visitor 9 and to confirm information about the visitor 9. The camera unit 4 is a unit that photographs the showroom 5. In FIG. 1, camera unit 4a and camera unit 4b that photograph the negotiation spaces 52a and 52b, respectively, and camera unit 4c that photographs the entire showroom 5 are provided. In the following description, when the negotiation spaces 52a and 52b are not differentiated from each other, they are referred to as the negotiation space 52, and when the camera units 4a to 4c are not differentiated from each other, they are referred to as the camera unit 4.

[0014] <Robot 2 Configuration> 2 is a schematic diagram showing the configuration of the robot 2. The robot 2 is formed in a roughly upright gourd shape, and is roughly composed of a head 201, a constricted portion 202, and a torso 203. In this embodiment, the robot 2 is about 110 cm tall. The robot 2 has no arms or legs, and is configured to be able to move freely in any direction within 360 degrees, including forward, backward, left, right, and diagonal directions, by means of a running device 204 provided at the bottom end of the torso 203. Note that a description of the specific configuration of the running device 204 will be omitted.

[0015] In this way, the head 201 of the robot 2 is slightly larger than the torso 303, and the robot 2 has a shape without arms or legs, which makes it easy for a child to hug and facilitates communication with the child. In addition, the running device 204 of the robot 2 enables the robot 2 to move while swinging forward / backward and left / right, which makes it easy for the visitor 9 to notice the visitor 9 approaching, and enables the robot 2 to move in a way that facilitates communication with the visitor 9.

[0016] The head 201 of the robot 2 is provided with a face 205 having a substantially elliptical shape elongated in the horizontal direction, and the face 205 is configured to be capable of displaying the facial expressions of the robot 2, simple text images, and the like. In this embodiment, a pair of pseudo eyes 206 representing eyes is displayed on the face 205 of the robot 2, and various facial expressions can be expressed by the pair of pseudo eyes 206 (see Figs. 11(a) to (c)). For example, by changing the shape of the pair of pseudo eyes 206, facial expressions such as joy, anger, sadness, and happiness can be expressed. In this embodiment, a pseudo mouth 207 representing a mouth is also displayed, and by changing the shape of the pseudo mouth 207 in addition to the pair of pseudo eyes 206, it is possible to make the change in facial expression easier to understand.

[0017] The robot 2 is also configured to be able to move the positions of the pair of artificial eyes 206 within the face 205. The robot 2 expresses the movement of the line of sight by changing the positions of the pair of artificial eyes 206 within the face 205. The robot 2 guides the line of sight of the visitor 9 by changing the positions of the pair of artificial eyes 206 in front of the visitor 9 and expressing the movement of the line of sight. At this time, by having the robot 2 perform a rotational movement or the like in conjunction with the movement of the line of sight, it becomes easier to guide the line of sight of the visitor 9.

[0018] The embodiment of the robot 2 is merely an example, and other embodiments of the robot may be adopted. For example, the shape of the robot 2 may be changed as appropriate. The robot may be a stationary robot, and for example, stationary robots may be placed near the entrance 51 of the showroom 5 and in each of the negotiation spaces 52a and 52b.

[0019] <Hardware configuration> 3 is a block diagram showing a hardware configuration of the customer service system SY. The management server 1, the robot 2, the terminal device 3, and the camera unit 4 are configured to be able to communicate with each other via a network NW such as a wireless local area network (wireless LAN).

[0020] The server 1 includes a processing unit 11, a storage unit 12, and a communication unit 13, which are connected by a bus (not shown). The processing unit 11 is a processor represented by a CPU, and realizes various functions related to the management server 1 by executing a program stored in the storage unit 12. That is, information processing by software stored in the storage unit 12 can be specifically realized by the control unit 13, which is an example of hardware, and executed as each functional unit included in the control unit 13. The storage unit 12 is, for example, a read only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), or a solid state drive (SSD), and stores various data in addition to the program executed by the processing unit 11. The communication unit 13 is a communication interface with an external device.

[0021] The robot 2 includes a processing unit 21, a memory unit 22, a communication unit 23, an input unit 24, an output unit 25, a traveling unit 26, a sensor group 27, and an imaging unit 28. The processing unit 21, the memory unit 22, and the communication unit 23 may have the same configuration as the processing unit 11, the memory unit 12, and the communication unit 13 of the management server 1, and therefore a description thereof will be omitted.

[0022] The input unit 24 accepts various inputs. The input unit 24 may be, for example, various switch buttons that can be operated during maintenance or the like, a microphone that accepts voice input from the visitor 9, or an operation panel that accepts input of reservation information by the visitor 9, or the like.

[0023] The output unit 25 outputs various information. The output unit 25 may be, for example, a speaker capable of outputting sound, or a display unit such as a display capable of displaying images. In this embodiment, the face unit 205 functions as the output unit 25 that outputs (displays) an image.

[0024] The traveling unit 26 is composed of the aforementioned traveling device 204 and the like that allows the robot 2 to self-propel. The traveling unit 26 includes, for example, a battery and an electric motor, and is configured so that the electric motor is driven by power supplied from the battery. The traveling unit 26 can be appropriately configured using known electric technology.

[0025] The sensor group 27 includes sensors for detecting the running and stopped states of the robot 2, such as a running speed sensor, an acceleration sensor, and a gyro sensor, as well as an obstacle sensor, a human sensor, a moving object sensor, etc. The robot 2 may include various sensors such as sensors for detecting the conditions around the robot 2.

[0026] The imaging unit 28 is a camera having an imaging element such as a CCD or a CMOS, and is configured to be able to capture images of visitors who visit the showroom 5. The imaging unit 28 is provided, for example, in the head 201 of the robot 2. This arrangement makes it easier to capture an image of the face of the visitor 9. Furthermore, the imaging unit 28 may be provided near the pair of artificial eyes 206 of the robot 2 from the viewpoint of capturing an image of the face of the visitor 9.

[0027] The terminal device 3 includes a processing unit 31, a storage unit 32, a communication unit 33, an input unit 34, and an output unit 35. The processing unit 31, the storage unit 32, and the communication unit 33 may have the same configuration as the processing unit 11, the storage unit 12, and the communication unit 13 of the management server 1, and therefore the description thereof will be omitted.

[0028] The input unit 34 accepts various inputs from the person in charge 8 or the like. The input unit 34 is, for example, a touch panel. The output unit 35 outputs various information. The output unit 35 may be, for example, a speaker capable of outputting sound, or a display unit such as a display capable of displaying images.

[0029] The camera unit 4 includes an imaging section 41 and a communication section 42. The imaging section 41 is a camera having an imaging element such as a CCD or a CMOS, and is configured to be able to capture images of visitors to the showroom 5. The communication section 42 is an interface with an external device, and transmits images captured by the imaging section 41 to the management server 1 via the network NW.

[0030] <Functional configuration of Management Server 1> 4 is a block diagram showing functions realized by the management server 1. The processing unit 11 of the management server 1 functions as each of the functional units 111 to 119 shown in FIG.

[0031] The in-facility image acquisition unit 111 acquires images of the inside of the showroom 5 captured by the multiple camera units 4 installed in the showroom 5. Specifically, the in-facility image acquisition unit 111 inputs image (including still images and videos) data of the interior of the showroom 5 captured by the multiple camera units 4 via the communication unit 13. For example, the in-facility image acquisition unit 111 outputs control signals for causing the multiple camera units 4 to capture images of the showroom 5 via the communication unit 13, and inputs image data of the inside of the showroom captured by the multiple camera units 4 via the communication unit 13.

[0032] The robot image acquisition unit 112 acquires an image including a facial image of the visitor 9 captured by the robot 2 placed in the showroom 5. Specifically, the robot image acquisition unit 112 inputs image (including still images and videos) data including the facial image of the visitor 9 captured by the robot 2 via the communication unit 13. For example, the robot image acquisition unit 112 outputs a control signal for causing the robot 2 to capture an image of the face of the visitor 9 via the communication unit 13, and inputs image data including the facial image of the visitor 9 captured by the robot 2 via the communication unit 13.

[0033] The business negotiation content acquisition unit 113 acquires the content of the business negotiation between the person in charge 8 and the visitor 9. For example, the business negotiation content acquisition unit 113 acquires data on the business negotiation content transmitted by the terminal device 3 to the management server 1 via the communication unit 13. The data on the business negotiation content includes data on the content displayed on the display unit as the output unit 35. For example, the business negotiation content acquisition unit 113 acquires the content (new car catalog, quotation, etc.) displayed on the display unit of the terminal device 3 during the business negotiation between the person in charge 8 and the visitor 9. This allows the processing unit 11 to grasp the content of the business negotiation. Alternatively, the business negotiation content acquisition unit 113 may acquire voice data of the conversation between the person in charge 8 and the visitor 9 from the robot 2. Then, the business negotiation content acquisition unit 113 may acquire the content of the business negotiation based on the content of the conversation. The business negotiation content acquisition unit 113 also recognizes that the business negotiation has started and ended.

[0034] The visitor identification unit 114 identifies the visitor 9 who has visited the showroom 5 from the image inside the showroom acquired by the facility image acquisition unit 111. For example, the visitor identification unit 114 extracts a person from the image inside the showroom, and further extracts (recognizes) a face image from the extracted person. Then, the visitor identification unit 114 searches for visitor data having a face image matching the extracted face image from the visitor data stored in the visitor DB 123, and identifies the visitor 9. If there is no visitor data having a face image matching the extracted face image, the visitor is stored in the visitor DB 123 as a new visitor 9.

[0035] The reaction estimation unit 115 estimates the reaction of the visitor 9. For example, the reaction estimation unit 115 estimates the reaction of the visitor 9 based on an image of the visitor 9 acquired by the robot image acquisition unit 112 during a business negotiation between the attendant 8 and the visitor 9. The reaction of the visitor 9 may be, for example, the feelings of the visitor 9 toward a topic provided by the attendant 8.

[0036] As a method for identifying the emotion, a known technique can be appropriately adopted. For example, the emotion of the visitor 9 can be estimated based on the gestures and behavior of the visitor 9. Specifically, the reaction estimation unit 115 may estimate the emotion of the visitor 9 from a facial image of the visitor 9 extracted from an image inside the showroom 5, the behavior of the visitor 9, and the like. For example, the reaction estimation unit 115 estimates that the visitor 9 is in a bad mood when the visitor 9 has an angry face, is looking around, is fidgeting, or is otherwise restless. On the other hand, the reaction estimation unit 115 estimates that the visitor 9 is in a good mood when the visitor 9 has a happy face, is talking to another visitor 9, or the like.

[0037] The reaction information output unit 116 outputs the reaction information. For example, the reaction information output unit 116 outputs the reaction information, which is information about the reaction estimated by the reaction estimation unit 115, to the storage unit 12 and stores it in the visitor DB 123. In addition, for example, the reaction information output unit 116 outputs the reaction information to the robot 2 or the terminal device 3. The robot 2 or the terminal device 3 that has received the reaction information outputs the reaction information (for example, displays it on a screen), allowing the person in charge 8 to grasp the reaction of the visitor 9 to the content of the business negotiation.

[0038] The behavior identifying unit 117 identifies the behavior of the facility staff (service representative 8) of the showroom 5. For example, the behavior identifying unit 117 may identify the behavior of the service representative 8 based on the position or traffic line of the service representative 8 recognized based on the image of the inside of the showroom 5 acquired by the facility image acquiring unit 111. As an example, the behavior identifying unit 117 identifies that the service representative 8 starts a business negotiation when the service representative 8 enters the negotiation space 52a together with the visitor 9. Alternatively, the behavior identifying unit 117 may receive data on the operation content accepted by the terminal device 3 from the terminal device 3, and identify the behavior of the service representative 8 based on the received data content.

[0039] The robot movement instruction unit 118 issues a movement instruction to the robot 2. For example, when the robot movement instruction unit 118 detects that a visitor 9 has entered the store through the entrance 51 based on an image acquired by the in-facility image acquisition unit 111, the robot movement instruction unit 118 moves the robot 2 to the vicinity of the entrance 51. Also, for example, when the behavior identification unit 117 identifies that a business negotiation between the person in charge 8 and the visitor 9 is to start, the robot movement instruction unit 118 moves the robot 2 to the business negotiation space 52.

[0040] At least a part of the functions realized by the processing unit 11 of the management server 1 may be realized by hardware. For example, at least a part of the functions realized by the processing unit 11 may be realized by a known semiconductor device such as a programmable logic device (PLD) or an application specific integrated circuit (ASIC). Although the processing unit 11 is shown here as a single element, the processing unit 11 may be divided into two or more elements as necessary.

[0041] The same applies to the processing unit 21 of the robot 2 and the processing unit 31 of the terminal device 3, and at least a part of the functions realized by these may be realized by hardware. Also, the processing unit 21 and the storage unit 22 may be divided into two or more elements as necessary.

[0042] Next, a description will be given of the databases constructed by the storage unit 12 of the management server 1. In this embodiment, the storage unit 12 constructs a robot database (DB) 121, a showroom database (DB) 122, a visitor database (DB) 123, and a business negotiation content database (DB) .

[0043] The robot DB 121 manages the ID of the robot 2, basic information or maintenance information related to the robot 2, etc. The showroom DB 122 stores data corresponding to the arrangement of the exhibited vehicles 53 and other items arranged in the showroom 5, data related to reservations for use of the negotiation space 52, etc.

[0044] The visitor DB 123 stores visitor information of the visitor 9 who visits the showroom 5. The visitor information includes basic information of the visitor such as the visitor's address, name, age, occupation, and gender, as well as a facial image and visit history of the visitor 9. The visit history may include not only the content of the business negotiation at the time of the visit, but also casual conversation before the business negotiation. In this embodiment, the visitor information includes a reservation to visit the showroom 5. The visit reservation includes the reservation date and time, information of the person in charge 8 at the showroom 5, etc.

[0045] <Processing example> <Reaction information output> Fig. 5 is a flowchart showing an example of processing by the processing unit 11. Fig. 5 shows processing when the processing unit 11 identifies the reaction of the visitor 9 and outputs information about the reaction during a business negotiation between the person in charge 8 and the visitor 9. This flowchart is realized by the processing unit 11 reading and executing a program stored in the storage unit 12, causing the processing unit 11 to function as each of the functional units shown in Fig. 4.

[0046] This flowchart is repeatedly executed at a predetermined cycle based on the start of a business negotiation between the person in charge 8 and the visitor 9. For example, in the business negotiation space 52a, when the person in charge 8 inputs an instruction to start a business negotiation into the terminal device 3, information to the effect that the business negotiation is about to start is transmitted from the terminal device 3 to the management server 1. The processing unit 11 starts the flowchart of FIG. 5 based on the information to the effect that the business negotiation is about to start being received by the communication unit 13.

[0047] S1 is a conditional branch as to whether the business negotiation is continuing or not. If the business negotiation is continuing, the business negotiation content acquisition unit 113 proceeds to S2, and if the business negotiation is finished, the business negotiation content acquisition unit 113 proceeds to S5. For example, when the business negotiation is finished, the person in charge 8 inputs information that the business negotiation has been finished through the input unit 34. Then, the processing unit 31 transmits information that the business negotiation has been finished to the management server 1 through the communication unit 33. When the business negotiation content acquisition unit 113 receives information that the business negotiation has been finished from the terminal device 3 through the communication unit 13, the business negotiation content acquisition unit 113 determines that the business negotiation has been finished. Alternatively, the business negotiation content acquisition unit 113 may acquire voice data on the conversation between the person in charge 8 and the visitor 9 from the robot 2b and analyze the content of the conversation to determine whether the business negotiation is continuing or finished.

[0048] In S2, the business negotiation content acquisition unit 113 acquires the topic of the business negotiation. More specifically, the business negotiation content acquisition unit 113 acquires information on the topic provided by the person in charge 8 in the business negotiation between the visitor 9 and the person in charge 8.

[0049] Specifically, the negotiation content acquisition unit 113 acquires information about the topic provided by the person in charge of handling 8 based on the operation of the terminal device 3 operated by the person in charge of handling 8. For example, assume that the person in charge of handling 8 operates the terminal device 3 to display a catalog of new car A on the display unit of the terminal device 3. In this case, the terminal device 3 transmits data to the management server 1 from which it can be recognized that the catalog of new car A has been displayed. Based on the received data, the negotiation content acquisition unit 113 recognizes that the content of the topic of the negotiation is an introduction of new car A.

[0050] More specifically, the business negotiation content acquisition unit 113 acquires information about the topic provided by the person in charge 8 based on the detection result of the voice of the person in charge 8 by the robot 2. The business negotiation content acquisition unit 113 acquires voice data of the conversation during the business negotiation from the robot 2. Then, the business negotiation content acquisition unit 113 generates text data from the acquired voice data, and recognizes the content of the business negotiation based on the content of the generated text data. For example, the business negotiation content acquisition unit 113 may extract specific terms (car names such as "New Car A", "quote", "delivery date", etc.) registered in advance from the text, and recognize the content of the business negotiation based on the extraction result.

[0051] 6(a) is a diagram illustrating an example of information acquired by the negotiation content acquisition unit 113. For example, the negotiation content acquisition unit 113 acquires information that associates the acquired topic content with the time and the name (or identification information, etc.) of the visitor 9. In this embodiment, the negotiation content acquisition unit 113 may store information about the acquired topic content in a temporary storage area, such as a RAM, included in the storage unit 12.

[0052] In S3, the robot image acquisition unit 112 acquires the detection result regarding the behavior of the visitor 9 by the robot 2. For example, the robot image acquisition unit 112 outputs a control signal for causing the robot 2 to capture an image of the face of the visitor 9 via the communication unit 13, and inputs image data including the facial image of the visitor 9 captured by the imaging unit 28 of the robot 2 via the communication unit 13. Note that, although the image data captured by the imaging unit 28 of the robot 2 is acquired as the detection result regarding the behavior of the visitor 9 here, for example, voice data of the visitor 9 detected by a microphone serving as the input unit 24 of the robot 2 may also be acquired.

[0053] In S4, the reaction estimation unit 115 estimates (acquires) the reaction of the visitor 9 based on the detection result acquired in S3. As described in the explanation of Fig. 4, the reaction estimation unit 115 estimates the reaction of the visitor 9 based on the image data as the acquired detection result. In this embodiment, the reaction estimation unit 115 determines whether the emotion of the visitor 9 corresponds to normal, positive, or negative.

[0054] 6(b) is a diagram illustrating an example of information acquired by the reaction estimation unit 115. For example, the reaction estimation unit 115 acquires information in which an estimation result of the emotion of the visitor 9 is associated with a time and the name (or identification information, etc.) of the visitor 9. In this embodiment, the reaction estimation unit 115 stores the acquired information in a temporary storage area, such as a RAM, included in the storage unit 12.

[0055] In S5, the reaction information output unit 116 outputs information regarding the reaction of the visitor 9 estimated by the reaction estimation unit 115 in S4. In this embodiment, the reaction information output unit 116 outputs the information regarding the reaction of the visitor 9 to the memory unit 12. That is, the reaction information output unit 116 stores the information regarding the reaction of the visitor 9 in the visitor DB 123 of the memory unit 12. In this embodiment, the reaction information output unit 116 associates the information regarding the topic of the business negotiation acquired in S2 with the information regarding the detection result by the robot 2 acquired in S4 and stores them in the visitor DB 123.

[0056] FIG. 6(c) is a diagram illustrating information that the reaction information output unit 116 outputs to the visitor DB 123. The reaction information output unit 116 associates the information stored in the temporary storage area in S2 and S4 and outputs the information to the visitor DB 123. Specifically, the reaction information output unit 116 stores information in which the topic of the business negotiation acquired in S2 and the reaction of the visitor 9 to the topic are associated in the visitor DB 123. The association at this time may be performed based on the time included in each piece of information. For example, the reaction information output unit 116 recognizes from the information acquired in S2 that the topic between 13:00 and 13:15 was the introduction of a new car A. Furthermore, the reaction information output unit 116 recognizes in S4 that the estimated results of the emotions of the visitor 9 between 13:00 and 13:15 were normal, negative, and positive once each. Based on the three estimation results in total, the reaction information output unit 116 may associate the reaction (emotion) of the visitor 9 to the topic "Introduction of New Car A" as "normal" in total.

[0057] In this processing example, after the business negotiation is completed, information regarding the reaction of visitor 9 is output by the reaction information output unit 116, but information acquired during the processing may be output to visitor DB 123 at any time and registered.

[0058] <Display instructions to terminal device 3> Fig. 7 is a flowchart showing a processing example of the processing unit 11 of the management server 1 and the processing unit 31 of the terminal device 3. Fig. 7 shows a processing example of the processing unit 11 and the processing unit 31 when reaction information is displayed on the terminal device 3 as one mode of output of reaction information by the reaction information output unit 116.

[0059] In S11, the processing unit 31 transmits a request to display the business negotiation details to the management server 1 via the communication unit 33. This request is a request for displaying the business negotiation details on the display unit serving as the output unit 35 of the terminal device 3 after the business negotiation is completed or during the business negotiation.

[0060] In S12, the processing unit 11 receives a request to display the business negotiation details from the processing unit 31 via the communication unit 13. Thereafter, in S13, the reaction information output unit 116 of the processing unit 11 generates display data for displaying the business negotiation details on the display unit serving as the output unit 35. The reaction information output unit 116 generates this display data based on the data shown in FIG. 6(c) stored in the visitor DB 123. That is, the display data is data in which information on the topic of the business negotiation and information on the reaction of the visitor 9 to the topic are associated with each other. In S14, the reaction information output unit 116 transmits a display instruction for the business negotiation details to the terminal device 3 via the communication unit 13. The display instruction here may include the display data generated in S13 in addition to a command to display a screen regarding the business negotiation details on the display unit serving as the output unit 35.

[0061] In S15, the processing unit 31 receives the display instruction transmitted from the processing unit 11 through the communication unit 33. In S16, the processing unit 31 causes the display unit serving as the output unit 35 to display the business negotiation content based on the display data received in S15. FIG. 8 is a diagram showing a screen example 35a displayed on the display unit serving as the output unit 35 of the terminal device 3. The screen example 35a displays an outline of the meeting (date and time, customer name, and person in charge name), the customer's reaction to the topic of the business negotiation, and comments. In this way, the reaction of the visitor 9 to the topic of the business negotiation and comments encouraging a corresponding response are displayed on the terminal device 3, so that the person in charge 8 can recognize the reaction of the visitor 9 with a higher accuracy and can provide customer service according to the reaction of the visitor 9.

[0062] <Robot movement instructions> Fig. 9(a) is a flowchart showing an example of processing by the processing unit 11. Fig. 9(a) shows an example of processing by the processing unit 11 when the management server 1 issues an instruction to move the robot 2 to a negotiation site. For example, this flowchart is repeatedly executed at a predetermined cycle during business hours of a store having a showroom 5.

[0063] In S111, the in-facility image acquisition unit 111 acquires images of the inside of the showroom 5 captured by the multiple camera units 4 installed in the showroom 5. In S112, the behavior identification unit 117 performs a behavior recognition process of the person in charge 8 based on the images acquired in S111.

[0064] FIG. 9B is a flowchart showing a specific example of the behavior recognition process in S112. S1121 is a conditional branch based on the movement of the person in charge 8. If the person in charge 8 has moved to the negotiation room, the behavior identification unit 117 proceeds to S1122. If not, the flow chart of FIG. 9(b) is terminated and the flow chart of FIG. 9(a) is returned to. For example, the behavior identification unit 117 may determine that the person in charge 8 has moved to the negotiation room when the person in charge 8 can be identified by the camera unit 4 provided in the negotiation space 52. Alternatively, the behavior identification unit 117 may determine that the person in charge 8 has moved to the negotiation room when the person in charge 8 is recognized to be approaching the negotiation space 52 based on the data of a video acquired by the camera unit 4 or a plurality of still images acquired in time series.

[0065] S1122 is a conditional branch based on the reservation status of the business negotiation. If there is a reservation for the business negotiation handled by the target person in charge of the response 8, the behavior identification unit 117 proceeds to S1123 and recognizes that the person in charge of the response 8 will conduct the business negotiation, and if not, ends the flowchart of FIG. 9(b) and returns to the flowchart of FIG. 9(a). For example, the behavior identification unit 117 checks the business negotiation reservation of the person in charge of the response 8 stored in the visitor DB 123. If there is a business negotiation reservation of the person in charge of the response 8 within a predetermined time before or after the current time, the behavior identification unit 117 proceeds to S1123. In this way, the behavior identification unit 117 may recognize the behavior of the person in charge of the response 8 based on the visit reservation information of the visitor 9.

[0066] S113 is a conditional branch based on the recognition result in S 112. If the behavior identification unit 117 recognizes in S112 that the person in charge 8 is going to negotiate, the process proceeds to S114, and if not, the process ends.

[0067] In S114, the robot movement instruction unit 118 issues a movement instruction to the robot 2. The robot movement instruction unit 118 issues a movement instruction to the robot 2 to move to the negotiation space 52 where the salesperson 8 will conduct negotiations. At this time, the robot movement instruction unit 118 may issue a movement instruction to the robot 2 that is not conducting other negotiations or reception among the robots 2 arranged in the showroom 5. Specifically, the robot DB 121 may store the current status of the robot 2, such as reception, negotiation, waiting, etc. Then, the robot movement instruction unit 118 may issue a movement instruction to the robot 2 whose status is waiting, based on the status of each robot 2 stored in the robot DB 121. Alternatively, the robot movement instruction unit 118 may issue a movement instruction to the robot 2 that has conducted reception of the visitor 9 who will conduct negotiations, to the negotiation space 52, while also guiding the visitor 9 to the negotiation space 52.

[0068] In this way, the robot 2 can be present at the business negotiation by moving the robot 2 to the place of business negotiation in response to the action of the person in charge 8. This allows the robot 2 present at the business negotiation to obtain information regarding the reaction of the visitor 9.

[0069] As described above, according to this embodiment, information related to the reaction of the visitor 9 estimated by the reaction estimation unit 115 is output by the reaction information output unit 116. Therefore, the person in charge 8 can provide appropriate customer service according to the reaction of the visitor using the output information. Furthermore, in this embodiment, information that associates the topic of the business negotiation with the reaction of the visitor 9 to the topic is output. Therefore, the person in charge 8 can provide more appropriate customer service based on the reaction of the visitor 9 to the content of the business negotiation.

[0070] <Second embodiment> Next, a second embodiment will be described. In summary, the second embodiment differs from the first embodiment in the manner in which information regarding the reaction of the visitor 9 during a business negotiation is output. Hereinafter, the same components as those in the first embodiment will be denoted by the same reference numerals and will not be described.

[0071] 10 is a flowchart showing an example of processing by the processing unit 11 of the management server 1 and the processing unit 21 of the robot 2. This flowchart shows processing when the robot 2 appropriately outputs the reaction of the visitor 9 during a business negotiation between the person in charge 8 and the visitor 9. In other words, this flowchart is repeatedly executed at a predetermined cycle while the business negotiation between the person in charge 8 and the visitor 9 is taking place.

[0072] In S211, the negotiation content acquisition unit 113 receives (acquires) information transmitted from the terminal device 3. In S212, the robot image acquisition unit 112 requests an image of the visitor 9 captured by the robot 2 as information on the behavior of the visitor 9. The processing unit 11 may also request voice data of the visitor 9 as information on the behavior of the visitor 9.

[0073] In S221, the processing unit 21 accepts a request from the management server 1. In S222, the processing unit 21 transmits the detection result of the behavior information to the management server 1 via the communication unit 23. In this embodiment, the detection result of the behavior information is image data of the image of the visitor 9 captured by the imaging unit 28.

[0074] In S213, the negotiation content acquisition section 113 receives the information transmitted from the robot 2 in S222 via the communication section 13. Note that steps S212 and S213 may be the same as step S3 in FIG.

[0075] In S214, the reaction estimation unit 115 estimates (acquires) the reaction of the visitor 9 based on the information acquired in S213. This step may be the same as S4 in Fig. 5. After that, in S215, the reaction information output unit 116 issues an instruction to the robot 2 to display the reaction information via the communication unit 13 as an output of reaction information related to the reaction acquired in S214. For example, the reaction information output unit 116 transmits information regarding whether the current emotion of the visitor 9 corresponds to normal, positive, or negative, together with the display instruction, to the robot 2.

[0076] In S223, the processing unit 21 receives the display instruction transmitted from the management server 1 in S215 via the communication unit 23. In S224, the processing unit 21 performs display based on the received display instruction. Specifically, the processing unit 21 causes the face unit 205 functioning as the output unit 25 of the robot 2 to perform a display corresponding to the emotion of the visitor 9.

[0077] 11(a) to 11(c) are diagrams showing display examples of the face 205 of the robot 2. FIG. 11(a) shows a display example when the emotion of the visitor 9 estimated in S214 is positive. FIG. 11(b) shows a display example when the emotion of the visitor 9 estimated in S214 is neutral. FIG. 11(c) shows a display example when the emotion of the visitor 9 estimated in S214 is negative. In this way, by changing the facial expression of the robot 2 to match the emotion of the visitor 9, the person in charge 8 can more intuitively grasp the reaction of the visitor 9.

[0078] As described above, according to this embodiment, the person in charge 8 can check the reaction of the visitor 9 while the business negotiation is being conducted. Therefore, the person in charge 8 can provide more appropriate customer service based on the reaction of the visitor 9 to the content of the business negotiation. Note that in this embodiment, although the information on the reaction of the visitor 9 is not associated with the information on the topic of the business negotiation, the person in charge 8 can check the reaction of the visitor 9 in real time, and therefore the person in charge 8 can understand the reaction of the visitor 9 to the content of the topic. The configurations of the first and second embodiments can be combined as appropriate.

[0079] <Other embodiments> In the above embodiment, the customer service system SY is used in the showroom 5 of an automobile dealership. However, the customer service system SY can be used in, for example, stores that retail various products, art museums, museums, art galleries, and other art-related facilities, science museums, memorial museums, exhibitions, workshops, and other business entities. Stores that retail various products include, for example, department stores, supermarkets, and specialty stores.

[0080] Furthermore, in the customer service system SY, the allocation of functions among the devices may be changed as appropriate. For example, the robot 2 may execute part of the processing of the management server 1. As an example, the processing unit 21 of the robot 2 may execute the process of estimating the reaction of the visitor 9 shown in S4 of Fig. 5. In this case, in response to the processing unit 11 requesting the processing unit 21 for information on the reaction of the visitor 9, the processing unit 21 may estimate the reaction of the visitor 9 based on the captured image of the visitor 9.

[0081] Furthermore, the processing unit 21 may acquire voice data during the business negotiation detected by a microphone serving as the input unit 24 of the robot 2 from the robot 2, and generate text data based on the acquired voice data. That is, the processing unit 21 may transcribe the voice during the business negotiation. The processing unit 21 may then store the generated text data in the visitor DB 123 as a visit record. By storing this text data, for example, when the person in charge 8 wants to look back in detail on a topic to which the visitor 9 did not respond well, the specific content of the conversation can be referred to.

[0082] <Summary of the embodiment> The above embodiment discloses at least the following customer service system, server, control method, and program.

[0083] 1. The customer service system of the above embodiment is A customer service system (SY) for serving customers using a customer service robot, A first acquisition means (11, S2) for acquiring first information on a topic provided by the person in charge of handling during a business negotiation between the visitor and the person in charge of handling; A second acquisition means (11, S3) for acquiring a detection result regarding the behavior of the visitor by the robot; an estimation means (11, S4) for estimating a reaction of the visitor based on the detection result acquired by the second acquisition means; and an output means (11, S5) for outputting second information on the reaction estimated by the estimation means.

[0084] According to this embodiment, the information on the reaction of the visitor estimated by the estimation means is output by the output means, so that the attendant can provide appropriate service to the visitor according to the reaction by using the information output by the output means.

[0085] 2. According to the above embodiment, The second information is information regarding the visitor's emotions.

[0086] According to this embodiment, customer service can be provided according to the emotions of the visitor.

[0087] 3. According to the above embodiment, The first acquisition means acquires the first information based on the operation content of a terminal device operated by the person in charge of handling the call.

[0088] According to this embodiment, the contents of the business negotiation can be recognized from the operation of the terminal device of the person in charge.

[0089] 4. According to the above embodiment, The first acquisition means acquires the first information based on a detection result of the voice of the person in charge by the robot.

[0090] According to this embodiment, the person in charge can recognize the details of the negotiation without performing any specific process such as operating a terminal device.

[0091] 5. According to the above embodiment, The output means outputs the first information and the second information to a storage unit in association with each other.

[0092] According to this embodiment, since information in which the first information and the second information are associated with each other becomes available, more appropriate customer service can be provided based on the visitor's response to the content of the business negotiation.

[0093] 6. According to the above embodiment, The output means outputs the second information to the robot.

[0094] According to this embodiment, the person in charge can check the reaction of the visitor via the robot.

[0095] 7. According to the above embodiment, The output means outputs the first information and the second information in association with each other to a terminal device operated by the person in charge of handling the case.

[0096] According to this embodiment, the person in charge can check the visitor's reaction to the contents of the business negotiation using the terminal device.

[0097] 8. According to the above embodiment, A recognition means (11, S112) for recognizing the action of the responder; and an instruction means (11, S114) for moving the robot to a location where the business negotiation takes place.

[0098] According to this embodiment, the robot moves to the location where the business negotiation takes place, so that the robot can obtain information regarding the reaction of the visitor during the business negotiation.

[0099] 9. According to the above embodiment, The facility further includes a management unit (123) for managing the visit reservation information of the visitor, The recognition means recognizes the behavior based on the visit reservation information.

[0100] According to this embodiment, the recognition means can recognize the actions of the responder with higher accuracy.

[0101] 10. According to the above embodiment, The device further includes a generating means for generating text data based on voice data of the conversation between the visitor and the person in charge during the business negotiation.

[0102] According to this embodiment, the person in charge can check the contents of the conversation during the business negotiation in text form, so that the contents of the business negotiation can be checked efficiently.

[0103] 11. The server (1) of the above embodiment is Function as each of the means of the customer service system described above in 1. to 10.

[0104] According to this embodiment, a server is provided for providing more appropriate customer service based on the reaction of the visitor 9 to the content of the business negotiation.

[0105] 12. The control method of the above embodiment is A method for controlling a customer service system for serving customers using a customer service robot, comprising: A first acquisition step (S2) of acquiring first information related to a topic provided by the person in charge of handling during a business negotiation between the visitor and the person in charge of handling; A second acquisition step (S3) of acquiring a detection result regarding the behavior of the visitor by the robot; an estimation step (S4) of estimating a reaction of the visitor based on the detection result acquired in the second acquisition step; and an output step (S5) of outputting second information regarding the reaction estimated in the estimation step.

[0106] According to this embodiment, the attendant can provide appropriate service according to the visitor's reaction by using the information output in the output step.

[0107] 13. The program of the above embodiment is Computer, A first acquisition means (S2) for acquiring first information on a topic provided by the person in charge of handling during a business negotiation between the visitor and the person in charge of handling; A second acquisition means (S3) for acquiring a detection result regarding the behavior of the visitor by the customer service robot; an estimation means (S4) for estimating a reaction of the visitor based on the detection result acquired by the second acquisition means; an output means (S5) for outputting second information regarding the reaction estimated by the estimation means; It will function as a means to:

[0108] According to this embodiment, the attendant can provide appropriate service to the visitor based on the visitor's reaction using the information output by the output means.

[0109] The invention is not limited to the above-described embodiment, and various modifications and variations are possible within the scope of the gist of the invention. [Explanation of symbols]

[0110] 1: Management server, 2: Robot, 3: Terminal device, 11: Processing unit, SY: Customer service system

Claims

1. A customer service system for serving customers using a customer service robot, a recognition means for recognizing the actions of the responder; An instruction means for moving the robot to a location where a business negotiation is to be held; A first acquisition means for acquiring first information related to a topic provided by the person in charge of handling during a business negotiation between the visitor and the person in charge of handling; A second acquisition means for acquiring a detection result regarding the behavior of the visitor by the robot; an estimation means for estimating a reaction of the visitor based on the detection result acquired by the second acquisition means; and an output means for outputting second information regarding the reaction estimated by the estimation means. A customer service system characterized by the above.

2. The customer service system according to claim 1, The second information is information regarding the emotion of the visitor. A customer service system characterized by the above.

3. The customer service system according to any one of claims 1 to 2, the first acquisition means acquires the first information based on an operation of a terminal device operated by the person in charge of handling the case. A customer service system characterized by the above.

4. The customer service system according to any one of claims 1 to 3, the first acquisition means acquires the first information based on a detection result of the voice of the person in charge by the robot; A customer service system characterized by the above.

5. The customer service system according to any one of claims 1 to 4, The output means outputs the first information and the second information to a storage unit in association with each other. A customer service system characterized by the above.

6. The customer service system according to any one of claims 1 to 5, The output means outputs the second information to the robot. A customer service system characterized by the above.

7. The customer service system according to any one of claims 1 to 6, the output means outputs the first information and the second information to a terminal device operated by the person in charge of handling the case in association with each other; A customer service system characterized by the above.

8. The customer service system according to any one of claims 1 to 7, A management unit for managing the visit reservation information of the visitor is further provided, The recognition means recognizes the behavior based on the visit reservation information. A customer service system characterized by the above.

9. The customer service system according to any one of claims 1 to 8, A generating unit is further provided for generating text data based on voice data of a conversation between the visitor and the person in charge during the business negotiation. A customer service system characterized by the above.

10. A server functioning as each of the means of the customer service system according to any one of claims 1 to 9.

11. A method for controlling a customer service system for serving customers using a customer service robot, comprising: a recognition step for recognizing the actions of the responder; An instruction step of moving the robot to a location where a business negotiation is to be held; a first acquisition step of acquiring first information related to a topic provided by the person in charge of handling during a business negotiation between the visitor and the person in charge of handling; a second acquisition step of acquiring a detection result regarding the behavior of the visitor by the robot; an estimation step of estimating a reaction of the visitor based on the detection result acquired in the second acquisition step; and an output step of outputting second information regarding the reaction estimated in the estimation step. A control method comprising:

12. Computer, a recognition means for recognizing the actions of responders; A command means for moving the customer service robot to a location where a business negotiation takes place; a first acquisition means for acquiring first information related to a topic provided by the person in charge of handling during a business negotiation between the visitor and the person in charge; A second acquisition means for acquiring a detection result regarding the behavior of the visitor by the robot; an estimation means for estimating a reaction of the visitor based on the detection result acquired by the second acquisition means; an output means for outputting second information regarding the reaction estimated by the estimation means; A program that functions as a means to

Images