Information processing device, information processing method and program

Abstract

To support a person in charge of receiving visitors online or a person in charge of receiving visitors at a site online.SOLUTION: An information processing device has a processor, and the processor acquires the position of a terminal connected to an own device online and having a voice collection part, acquires first information related to the acquired position, and presents the acquired first information as second information for supporting a conversation with a person existing in front of the terminal to a user operating the own device.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Today, with the increasing awareness of measures against infectious diseases such as coronavirus and influenza, there is a growing need for salespersons who are in a location separate from the sales floor to serve customers through monitors installed on the sales floor. Hereinafter, this type of customer service is referred to as "online customer service". Online customer service is also effective in ensuring social distance between customers and salespersons.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By adopting online customer service, a salesperson can be in charge of multiple sales floors because there is no distance restriction in the online environment. For example, it becomes possible to be in charge of sales floors scattered several tens of kilometers, several hundreds of kilometers away, and in some cases, across national borders. On the other hand, when in charge of multiple sales floors, it may be difficult to grasp information about the regions where the sales floors are located. Especially in online sales, a real-time feeling is important. Therefore, providing information about the regions where the sales floors are located becomes more important than ever. In the future, it is expected that personnel of various ages and nationalities will be active as salespersons. In such a case as well, tools for realizing good communication are required.

[0005] An object of the present invention is to support, online, a person in charge of serving customers online or a person in charge of serving customers on-site.

Means for Solving the Problem

[0006] The invention according to claim 1 has a processor, and the processor acquires the position of a terminal having a sound collection unit that is connected online to the own device, acquires first information related to the acquired position, and presents the acquired first information to a user who operates the own device as second information for assisting a conversation with a person existing in front of the terminal. The invention according to claim 2 has a processor, and the processor acquires the position of a terminal having a sound collection unit that is connected online to the own device, acquires first information related to the acquired position, and presents the acquired first information to a user who operates the own device as second information for assisting a conversation between a person existing in front of the terminal and a third party. The invention according to claim 3 is the information processing device according to claim 1 or 2, wherein the processor displays a screen for selecting whether to provide the first information, and when receiving an instruction to provide the first information, provides the first information as the second information. The invention according to claim 4 is the information processing device according to claim 3, wherein when receiving an instruction not to provide the first information, the processor does not provide the first information as the second information. The invention according to claim 5 is the information processing device according to claim 3, wherein when receiving an instruction to provide the first information, the processor executes acquisition of the first information. The invention according to claim 6 is the information processing device according to claim 1, wherein the processor acquires a topic of the area where the terminal is located as the first information. The invention according to claim 7 is the information processing device according to claim 6, which prioritizes highly immediate content as the topic of the area. The invention according to claim 8 is the information processing device according to claim 2, wherein the processor acquires a topic regarding the destination visited by the person in charge of carrying the terminal as the first information. The invention according to claim 9 is the information processing apparatus according to claim 8, wherein the topic regarding the destination includes the history of visits. The invention according to claim 10 is the information processing apparatus according to claim 1 or 2, wherein the processor presents an example sentence of a topic regarding the item for which the designation has been received as the second information. The invention according to claim 11 is the information processing apparatus according to claim 10, wherein candidates for the item are provided so as to be selectable. The invention according to claim 12 is the information processing apparatus according to claim 1, wherein the processor provides information regarding the person as the second information. The invention according to claim 13 is the information processing apparatus according to claim 2, wherein the processor provides information regarding the third party as the second information. The invention according to claim 14 is the information processing apparatus according to claim 1, wherein the processor performs processing to emphasize the movement of the user in the image displayed by the terminal as compared with the actual movement of the user. The invention according to claim 15 is the information processing apparatus according to claim 14, wherein the magnitude of the user's movement is enlarged by image processing as compared with the actual magnitude of the movement. The invention according to claim 16 is the information processing apparatus according to claim 14, wherein the speed of the user's movement is increased or decreased by image processing as compared with the actual speed of the movement. The invention according to claim 17 is the information processing apparatus according to claim 14, wherein the degree of emphasis is variable according to the user's settings. The invention according to claim 18 is the information processing apparatus according to claim 1, wherein during a conversation with the person, the processor generates an image of a standing posture prepared in advance or an image representing a standing posture and transmits it to the terminal. The invention according to claim 19 is the information processing apparatus according to claim 1, wherein when the person is sitting, the processor generates an image of a sitting posture prepared in advance or an image representing a sitting posture and transmits it to the terminal, and when the person is standing, the processor generates an image of a standing posture prepared in advance or an image representing a standing posture and transmits it to the terminal. The invention according to claim 20 is the information processing apparatus according to claim 1, wherein when transmitting a live image of a user during a conversation to the terminal, the processor guides the user with respect to the person's posture. The invention according to claim 21 is an information processing method executed by an information processing apparatus that provides a service for assisting communication. In the information processing method, among a first terminal and a second terminal connected online, a process of obtaining the position of the first terminal used at the location where a person who is a conversation partner is present, a process of obtaining first information related to the obtained position, and a process of presenting the obtained first information to a user who operates the second terminal as second information for assisting the conversation with the person are included. The invention according to claim 22 is an information processing method executed by an information processing apparatus that provides a service for assisting communication. In the information processing method, among a first terminal and a second terminal connected online, a process of obtaining the position of the first terminal used at the location where a person who is having a conversation with a third party is present, a process of obtaining first information related to the obtained position, and a process of presenting the obtained first information to a user who operates the second terminal as second information for assisting the conversation between the person and the third party are included. The invention according to claim 23 is a program for causing a computer to realize a function of obtaining the position of a terminal having a sound collection unit connected online to the own device, a function of obtaining first information related to the obtained position, and a function of presenting the obtained first information to a user who operates the own device as second information for assisting a conversation with a person existing in front of the terminal. The invention according to claim 24 is a program for causing a computer to realize a function of obtaining the position of a terminal having a sound collection unit connected online to the own device, a function of obtaining first information related to the obtained position, and a function of presenting the obtained first information to a user who operates the own device as second information for assisting a conversation between a person existing in front of the terminal and a third party. The invention according to claim 25 is a program for causing a computer to implement a function of acquiring the position of a first terminal used at a location where a person who is a conversation partner is present among a first terminal and a second terminal connected online, a function of acquiring first information related to the acquired position, and a function of presenting the acquired first information to a user who operates the second terminal as second information for assisting the conversation with the person. The invention according to claim 26 is a program for causing a computer to implement a function of acquiring the position of a first terminal used at a location where a person who is having a conversation with a third party is present among a first terminal and a second terminal connected online, a function of acquiring first information related to the acquired position, and a function of presenting the acquired first information to a user who operates the second terminal as second information for assisting the conversation between the person and the third party. [[Effect of the Invention]]

[0007] According to the invention described in claim 1, it is possible to support a person in charge of online customer service. According to the invention described in claim 2, it is possible to support a person in charge of on-site customer service online. According to the invention described in claim 3, a user operating an information processing device at a remote location can select whether to provide first information. According to the invention described in claim 4, when the user does not desire it, it is possible to eliminate the need to provide first information. According to the invention described in claim 5, when the provision of first information is unnecessary, the processing load can be reduced. According to the invention described in claim 6, it is possible to provide information that serves as a conversation trigger. According to the invention described in claim 7, it is possible to promote real-time customer service. According to the invention described in claim 8, it is possible to provide information useful for supporting a person in charge of on-site customer service. According to the invention described in claim 9, it is possible to enable customer service according to the number of visits. According to the invention described in claim 10, even a user with low skills can make a specific response. According to the invention described in claim 11, even a user with low skills can take specific actions. According to the invention described in claim 12, the quality of the conversation can be improved by confirming the information of the person who is the conversation partner. According to the invention described in claim 13, the quality of the conversation can be improved by confirming the information of the third party with whom the person serving customers on-site is talking. According to the invention described in claim 14, even through the screen, the movement of the user can be reliably conveyed to the person in front of the terminal. According to the invention described in claim 15, by showing a large movement, the transmission of intention becomes easy. According to the invention described in claim 16, even through the screen, it becomes easier to convey momentum. According to the invention described in claim 17, the user can instruct the degree of emphasis. According to the invention described in claim 18, even if the user during the conversation is sitting, the user can express courtesy to the person in front of the terminal. According to the invention described in claim 19, the posture of the user visible to the person in front of the terminal can be adjusted to match the posture of the person in front of the terminal. According to the invention described in claim 20, the posture of the user visible to the person in front of the terminal device can be adjusted to match the posture of the person in front of the terminal device. According to the invention described in claim 21, a service for supporting users who serve customers online can be provided. According to the invention described in claim 22, a service for supporting the person serving customers on-site online can be provided. According to the invention described in claim 23, a user who serves customers online can be supported. According to the invention described in claim 24, the person serving customers on-site can be supported online. According to the invention described in claim 25, a user who serves customers online can be supported. According to the invention described in claim 26, the person serving customers on-site can be supported online.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. <Embodiment 1> <Configuration of the System> FIG. 1 is a diagram for explaining a configuration example of the customer service support system 1 used in Embodiment 1. The customer service support system 1 shown in FIG. 1 assumes a scenario where a staff member Q who is not in the sales floor receives customers (hereinafter referred to as "visiting customers") who come to a plurality of stores (Store A, Store B, Store C,...) online.

[0010] The terminal 10 installed in each store and the terminal 20 operated by staff Q are connected via the network 30. The terminal 10 is an example of a terminal, and the terminal 20 is also an example of an information processing device. Note that the terminal 10 is an example of a first terminal, and the terminal 20 is also an example of a second terminal. In FIG. 1, a customer visiting the store is an example of a person present in front of the terminal. Also, staff Q in charge of serving the customer is an example of a person in charge. Further, staff Q is also an example of a user operating the terminal 20.

[0011] In the case of FIG. 1, store A is in region X of country J, and the current weather is sunny. Store B is in region Y of country J, and the current weather is rainy. Store C is in region Z of country J, and the current weather is snowy. Also, the workplace of staff Q is in region T of country J, and the current weather is cloudy. Thus, the weather at each store and the workplace of staff Q depends on the location. The workplace of staff Q is not limited to a specific office and may be a private room-type office that can be used on an hourly basis or at home.

[0012] In the case of FIG. 1, store A, store B, store C, and the workplace of staff Q are all in different regions within the same country, but they may also be in the same region within the same country or in different countries. Since it is online customer service, there is no restriction on the distance between each store and the workplace of staff Q. In the example of FIG. 1, staff Q is providing online customer service to a two-person customer who visited store A. For this reason, the faces of the two-person customer are displayed on the terminal 20 operated by staff Q. Also, the face of staff Q in charge of customer service is displayed on the terminal 10 installed in store A.

[0013] In the case of FIG. 1, a live image of staff Q during customer service is displayed on the terminal 10. The live image is a so-called moving image and is an image captured in real time of staff Q during customer service. However, a photo of staff Q (i.e., a still image) may be displayed on the terminal 10, or a photo of someone other than staff Q may be displayed. Alternatively, instead of the live images or photos of Staff Q, images or animations of characters representing individual staff members in charge of customer service (hereinafter also referred to as "customer service staff") may be displayed. The characters here are not limited to humans, but may also be animals, plants, natural objects, or artificial objects, or virtual entities imitating them.

[0014] FIG. 2 is a diagram for explaining an example of a character displayed on the terminal 10 on the store side. In the case of FIG. 2, a character imitating the customer service staff is displayed on the monitor 11 constituting the terminal 10. In the case of FIG. 2, the character is a full-body standing image, but it may also be a sitting image, an image of only the upper body, or an image of only the face. In the case of FIG. 2, a camera 12 is attached to the upper left portion of the monitor 11. The image captured by the camera 12 is displayed on the terminal 20 (see FIG. 1) operated by Staff Q during customer service.

[0015] In the case of FIG. 2, the camera 12 is attached to the upper part of the monitor 11, but the camera 12 may also be integrated with the monitor 11. In the case of FIG. 2, a microphone 13 and a speaker 14 are attached to the upper right portion of the monitor 11. In the case of FIG. 2, the microphone 13 and the speaker 14 are shown as one member, but they may also be separate members or integrated with the monitor 11.

[0016] Returning to the description of FIG. 1. In the case of FIG. 1, only Staff Q is represented as the customer service staff, but there may be multiple customer service staff. For the network 30, for example, a LAN or the Internet is used. For communication on the network 30, LAN communication, Internet communication, a mobile communication system, etc. are used. The mobile communication system may be any of the fourth generation (i.e., 4G), the fifth generation (i.e., 5G), or the sixth generation (i.e., 6G).

[0017] <Configuration of the Terminal> FIG. 3 is a diagram for explaining an example of the hardware configuration of the terminal 10 installed in the store visited by customers. The terminal 10 shown in Fig. 3 has a terminal body 100, a monitor 11, a camera 12, a microphone 13, and a speaker 14. Furthermore, the terminal body 100 has a processor 101, a semiconductor memory 102, a hard disk device 103, and a communication module 104.

[0018] For the monitor 11, for example, an organic EL (=Electro Luminescent) display or a liquid crystal display is used. On the monitor 11, for example, images used for explaining products and services (hereinafter also referred to as "products, etc.") and images of staff Q for online customer service are displayed. For the camera 12, for example, a CMOS (=Complementary Metal Oxide Semiconductor) image sensor or a CCD (=Charge-Coupled Device) image sensor is used. The camera 12 may be integrated with the terminal body 100 or may be external.

[0019] The microphone 13 is, for example, a device that converts the voice of a customer who visits the store into an electrical signal. The microphone 13 may be integrated with the terminal body 100 or may be external. The microphone 13 is an example of a sound collection unit. The speaker 14 is, for example, a device that converts the explanation of a product or the voice of staff Q given as an electrical signal into sound and outputs it. The speaker 14 may be integrated with the terminal body 100 or may be external.

[0020] The processor 101 is composed of, for example, a CPU (=Central Processing Unit). The processor 101 realizes various functions through the execution of programs. The semiconductor memory 102 has a ROM (=Read Only Memory) in which the BIOS (=Basic Input Output System) etc. are stored, and a RAM (=Random Access Memory) used as a main memory device. The processor 101 and the semiconductor memory 102 constitute a so-called computer. The processor 101 uses the RAM as a work space for programs.

[0021] The hard disk device 103 is an auxiliary storage device, and stores, in addition to programs, data of images displayed on the monitor 11, data of images captured by the camera 12, etc. Note that a semiconductor memory may be used instead of the hard disk device 103. The communication module 104 is a device that realizes communication with the terminal 20 (see Fig. 1) operated by the staff Q.

[0022] Fig. 4 is a diagram for explaining an example of the hardware configuration of the terminal 20 operated by the staff Q who provides online customer service. The terminal 20 shown in Fig. 4 is a so-called computer, and may be a desktop type or a notebook type. Further, the terminal 20 may be a tablet type or a wearable type. The wearable type terminal 20 may be any of a watch, earphones, glasses, etc.

[0023] The terminal 20 has a terminal main body 200, a monitor 21, a camera 22, a microphone 23, a speaker 24, a mouse 25, and a keyboard 26. Furthermore, the terminal main body 200 has a processor 201, a semiconductor memory 202, a hard disk device 203, and a communication module 204. Note that the configuration of the terminal 20 shown in Fig. 4 is an example, and it is not necessary to have all of them when implementing.

[0024] For the monitor 21, for example, an organic EL display or a liquid crystal display is used. On the monitor 21, for example, images of the inside of a store where online customer service is being conducted, images of customers who have come to the store during online customer service, images presented to the customers who have come to the store, information for identifying the store during online customer service, information for assisting the customer service staff, etc. are displayed. The image of the inside of the store is, for example, an image captured by the camera 12 (see FIG. 2) attached to the terminal 10 (see FIG. 2) before the start or after the end of online customer service, or an image of a surveillance camera or the like installed separately from the camera 12.

[0025] The image of the customer who has come to the store during online customer service is, for example, an image captured by the camera 12 during online customer service. The image presented to the customer who has come to the store is, for example, an image of the customer service staff captured by the camera 22, or an image for explaining products or the like. The information for identifying the store during online customer service is, for example, the store name and the location. The information for assisting the customer service staff includes information related to the location of the store targeted for online customer service, the weather in the area where the store is located, and candidates for topics to be used in conversations with the customers who have come to the store. In the case of this embodiment, the information for assisting the customer service staff is displayed when the staff Q selects the display, and is not displayed when not selected.

[0026] For the camera 22, for example, a CMOS image sensor or a CCD image sensor is used. The camera 22 may be integrated with the terminal body 200 or may be external. The microphone 23 is, for example, a device that converts the voice of the staff Q into an electrical signal. The microphone 23 may be integrated with the terminal body 200 or may be external. The speaker 24 is, for example, a device that converts the voice of the customer who has come to the store during online customer service into sound and outputs it. The speaker 24 may be integrated with the terminal body 200 or may be external. Note that the microphone 23 and the speaker 24 may be a headset worn by the staff Q on the head.

[0027] The mouse 25 and the keyboard 26 are examples of input devices. They are used for inputting instructions to the terminal 20 by staff Q and inputting characters and the like. The processor 201 is composed of, for example, a CPU. The processor 201 realizes various functions through program execution. The semiconductor memory 202 has a ROM in which the BIOS and the like are stored and a RAM used as a main storage device. The processor 201 and the semiconductor memory 202 constitute a so-called computer. The processor 201 uses the RAM as a work space for programs.

[0028] The hard disk device 203 is an auxiliary storage device, and stores programs, image data received from the store-side terminal 10 (see FIG. 1), image data captured by the camera 22, image data used for descriptions of products and the like, the positions of stores connected online, weather information obtained according to the positions of each store, time situation information obtained according to the positions of each store, topics according to the positions of each store, and the like. Note that a semiconductor memory may be used instead of the hard disk device 203. The communication module 204 is a device that realizes communication with the store-side terminal 10 (see FIG. 1).

[0029] <Processing operation> FIG. 5 is a flowchart for explaining the processing operation executed by the terminal 20 (see FIG. 1) operated by the staff Q who conducts online customer service. Note that the symbol S in FIG. 5 means step. The processing operation shown in FIG. 5 is realized through the execution of a program by the processor 201 (see FIG. 4).

[0030] First, the processor 201 receives a selection of a connection destination (step 1). In the case of this embodiment, online customer service is started when the staff Q selects a connection destination. Staff Q selects the corresponding store as the connection destination when, for example, there is an operation to call from a customer visiting the store or when a customer showing interest in the exhibit is confirmed on monitor 21 (see FIG. 4). Online customer service is handled by Staff Q on standby, and staff on break do not handle calls even if there is one. However, an administrator who manages online customer service may determine the connection destination and notify the staff on standby of the connection destination. Alternatively, the system may determine the combination of the connection destination and the staff according to a predetermined rule and notify the staff of the determined connection destination information.

[0031] In the case of FIG. 5, Staff Q selects the connection destination. When Staff Q selects the connection destination, the processor 201 acquires the position of the connection destination (step 2). The position of the connection destination is an example of the position of the terminal. The processor 201 acquires, for example, "Store A in Region X of Country J" or the location from the hard disk device 203 (see FIG. 4). The position information may include, for example, the name of the building in which the store is located and the floor number of the floor on which the store is located.

[0032] Next, the processor 201 determines whether to execute a mode for assisting online customer service (hereinafter referred to as "online customer service support mode") (step 3). In the present embodiment, the execution of the online customer service support mode is queried after the selection of the connection destination, but the availability of the execution of the online customer service support mode may be queried before the selection of the connection destination.

[0033] If an affirmative result is obtained in step 3, the processor 201 determines whether to display support information (step 4). The support information includes, for example, weather information and time information. In addition, the support information may include the name of the street facing the store of the connection destination, information on landmarks existing near the store of the connection destination, and business hours. The support information in the present embodiment is an example of first information related to the position of the store that is the connection destination.

[0034] FIG. 6 is a diagram for explaining an example of screens 210 and 220 used for receiving selections in steps 3 and 4 (see FIG. 5). Screens 210 and 220 are displayed in a partial area of, for example, monitor 21 (see FIG. 4). Screens 210 and 220 are also called windows.

[0035] Screen 210 is used for receiving a selection in step 3. In the case of FIG. 6, on screen 210, there are arranged sentence 211 "Do you want to start communication by turning on the online customer service support mode?", button 212 to be operated in the case of "YES", and button 213 to be operated in the case of "NO".

[0036] Screen 220 is used for receiving a selection in step 4. Screen 220 is displayed when "YES" is selected in step 3. In the case of FIG. 6, on screen 220, there are arranged sentence 221 "Do you want to display weather information and time situation information of the connection destination?", button 222 to be operated in the case of "YES", and button 223 to be operated in the case of "NO".

[0037] Return to the description of FIG. 5. When an affirmative result is obtained in step 4, the processor 201 displays support information including information related to the position of the connection destination and position information (step 5). The support information is the first information and is also an example of the second information for assisting in the conversation with the customer. The second information also includes information not related to the position of the store during online customer service. For example, the second information includes greetings and topics according to the age of the customer.

[0038] Also, the processor 201 displays shared information on the monitor 11 (see FIG. 3) of the connection destination terminal 10 (see FIG. 3) (step 6). The shared information is the information displayed on both the store-side monitor 11 and the staff Q-side monitor 21 (see FIG. 4). The shared information is the information shared between the customers visiting the store and staff Q. The shared information includes, for example, images of products or the like that are the objects of explanation, and characters and diagrams explaining the features of products or the like.

[0039] In the case of this embodiment, the information presented on the terminal 20 (see FIG. 1) operated by staff Q is distinguished from the information presented to the customers visiting the store. The information presented to the customers visiting the store includes, for example, the information that staff Q has instructed to be displayed among the information presented on the terminal 20 operated by staff Q, information regarding products or the like being explained, and information regarding the topic being discussed.

[0040] The content of the shared information displayed may differ for each customer visiting the store. Also, the content of the shared information displayed may be determined according to the content of the conversation with the customer visiting the store. The content of the shared information may be individually instructed by staff Q, or may be determined by the processor 201 according to the content of the conversation. Note that the content of the conversation may be given to the processor 201 as a result of analysis by a program that recognizes voices, or the selection of a topic by staff Q may be given to the processor 201.

[0041] On the other hand, when a negative result is obtained in step 4, the processor 201 displays only the location information (step 7). The location information includes information for specifying the store name and location. Also, the processor 201 displays the shared information on the monitor 11 of the connected terminal (see FIG. 3) (step 8). Note that when a negative result is obtained in step 3, the processor 20 executes another operation mode (step 9). In the case of this embodiment, the other operation mode includes a mode for supporting business online, which will be described later.

[0042] <Example of the operation screen> Hereinafter, an example of the screen displayed during online customer service will be described with reference to FIGS. 7 to 11. <Example 1> FIG. 7 is a diagram for explaining an example of a screen when support information is not displayed during online customer service. (A) shows an example of a screen 230 displayed on a monitor 21 of a terminal 20 (see FIG. 1) operated by staff Q, and (B) shows an example of a screen 110 displayed on a monitor 11 of a terminal 10 (see FIG. 1) installed on the store side.

[0043] In the screen 110 of FIG. 7(B), a live image of staff Q during customer service is displayed. By displaying the live image, it becomes easier for the arriving customers to convey that staff Q is responding in real time. Although the delay is small even in Internet communication, communication with small delays is also possible in international roaming using a mobile communication system after 5G, enabling online customer service without sacrificing the sense of real time. However, instead of the live image, it is also possible to display a face photo of staff Q or an image of a character, etc. on the monitor 11 and conduct customer service only with the voice of staff Q.

[0044] In the screen 230 of FIG. 7(A), an image 231 of the arriving customer, an image 232 of the monitor 11 that the arriving customer is looking at, and information 234 on the location of the store where the arriving customer is during online customer service are displayed. In the case of FIG. 7(A), as the information 234, the local date and time on the store side and the store name, etc. are displayed.

[0045] The reason for displaying the local date and time is to avoid giving a sense of discomfort to the conversation with the arriving customer even when there are multiple standard times within the same country, when daylight saving time is provided, or when the place of work of staff Q and the store are in different countries. Also, by including the time in the conversation, it becomes possible to impress the arriving customer with real-time customer service. In many cases, the store is indoors, and it is difficult to estimate the local time from the image of the arriving customer. However, as shown in FIG. 7(A), by displaying the current date and time, staff Q can conduct online customer service while being aware of the store side time.

[0046] <Example 2> FIG. 8 is a diagram for explaining an example of a screen when support information is displayed during online customer service. (A) shows an example of a screen 230A displayed on a monitor 21 of a terminal 20 (see FIG. 1) operated by staff Q, and (B) shows an example of a screen 110 displayed on a monitor 11 of a terminal 10 (see FIG. 1) installed on the store side. In FIG. 8, reference numerals corresponding to corresponding parts in FIG. 7 are shown.

[0047] On the screen 230A of FIG. 8, support information 235 and a topic column 236 are additionally displayed. In the case of FIG. 8(A), as the support information 235, the current weather, temperature, and news in the area where the store is located are displayed. In this example, the current weather in the area where the store is located is "sunny", and the temperature is "30 °C". Also, as news regarding the area where the store is located, "election bulletin" and "festival" are displayed. News regarding the weather and the area also serves as an appeal for real-time customer service.

[0048] The "election bulletin" and "festival" shown in FIG. 8(A) are just examples. Also, in FIG. 8(A), two news items are displayed, but the number of news items to be displayed may be one or three or more. Incidentally, the news may be obtained from a news site or the like at the stage when the location of the store is acquired, or may be obtained from a news site or the like after the execution of the online customer service support mode is selected. It is desirable to prioritize news with highly immediate content as the support information 235. In other words, information with a newer publication time is prioritized. In the example of FIG. 8(A), the election bulletin is prioritized over the festival. By including a highly immediate topic during the conversation, a sense of familiarity and real-time feeling can be easily conveyed to the customers visiting the store.

[0049] In addition, the support information 235 may include information regarding meteorology and disasters such as humidity and earthquakes. Also, when a customer visiting the store is identified as an individual from an image of the customer visiting the store, the visit history of the corresponding customer visiting the store, the name of the staff in charge last time, notes during customer service, etc. may be displayed as support information. Identifying the customers who come to the store here does not require identifying the names of individuals. For example, by matching based on face or clothing, it may be determined that the customer is the same person as the one served an hour ago. In the case of matching based on face or clothing, there is no need to prepare a database storing personal information.

[0050] In the topic column 236, buttons for topics that are likely to be useful in conversations with customers are displayed. In the case of Fig. 8(A), "Greetings", "Current Affairs", "Clothing", etc. are displayed. The candidates for topics displayed in the topic column 236 may be general content regardless of the differences among customers, or may be content corresponding to the gender, age, family composition, clothing, etc. of the customers recognized from the image. The image processing and the selection of the topic content are executed by the processor 201. For example, if there are children, information related to children may be included as candidates for topics. Also, if the customer is an elderly person, information related to health may be included in the candidates for topics.

[0051] <Example 3> FIG. 9 is a diagram for explaining another example of a screen when support information is displayed during online customer service. (A) shows an example of a screen 230B displayed on the monitor 21 of the terminal 20 (see FIG. 1) operated by staff Q, and (B) shows an example of a screen 110 displayed on the monitor 11 of the terminal 10 (see FIG. 1) installed on the store side. In FIG. 9, reference numerals corresponding to the corresponding parts in FIG. 8 are shown.

[0052] In the screen 230B of FIG. 9(A), "Greetings" among the plurality of candidates displayed in the topic column 236 is selected by staff Q. In FIG. 9(A), the selected state is represented by shading. Also, in the screen 230B, example sentences related to the topic selected by staff Q are displayed in the example sentence column 237. In FIG. 9(A), three example sentences are displayed. The example sentences are not limited to stereotyped content, and example sentences reflecting the weather at the current time in the store during online customer service, the weather forecasted after the current time, and past weather information may be provided by the processor 201.

[0053] In the case of online customer service, staff Q does not only take charge of specific stores, and the place of work does not necessarily exist within the same area as the store. Therefore, it is easy to be confused about the content of topics that do not give a sense of discomfort to the customers visiting the store during online customer service. However, in the case of this embodiment, by displaying examples of topics according to the location of the store during online customer service and the customers visiting the store, an improvement in the quality of the conversation is expected. Also, even when the ages and nationalities of the staff providing online customer service are diversified, by displaying examples of sentences for each topic according to the location of the store and the customers visiting the store, it is possible to avoid giving a sense of discomfort to the customers visiting the store and improve the quality of customer service.

[0054] <Example 4> FIG. 10 is a diagram for explaining an example of a screen during online customer service. (A) shows an example of a screen 230C displayed on a monitor 21 of a terminal 20 (see FIG. 1) operated by staff Q, and (B) shows an example of a screen 110A displayed on a monitor 11 of a terminal 10 (see FIG. 1) installed on the store side. In FIG. 10, reference numerals corresponding to the corresponding parts in FIG. 9 are attached and shown.

[0055] In the case of FIG. 10(B), a live image 111 of staff Q during customer service, an image 112 of product information, and an image 113 of information regarding the topic during the conversation are displayed on the monitor 11 of the terminal 10 on the store side. Of course, the same image 232 as the screen 110A shown in FIG. 10(B) is also displayed on the screen 230C on the side of the terminal 20 operated by staff Q.

[0056] In the case of FIG. 10(B), a live image 111 of staff Q is displayed in the approximate left half of the screen 110A, an image 112 of product information is displayed in the upper half of the right area, and an image 113 of information regarding the topic during the conversation is displayed in the lower half of the right area. The image 112 of product information here is an example of shared information. The image 112 of product information is composed of a moving image, a photograph, a description text, etc. The moving image here is an image that was captured and recorded in advance, different from the live image. In addition, figures and graphs may be displayed in the image 112. By displaying the product information image 112, in-store customers can view the product information while looking at the face of staff Q.

[0057] In Fig. 10(B), the same information as the support information 235 displayed on the monitor 21 of the terminal 20 operated by staff Q is shown as the image 113 of information related to the topic during the conversation. However, the display layout and the way of presenting the information may be different. For example, on the monitor 11 that in-store customers view, a simplified image with a limited number of characters or the like may be displayed. This is because overly detailed information not only impairs the distinctiveness but also may divert the in-store customers' attention from the conversation with staff Q to understanding the information. The image 113 here is also an example of shared information.

[0058] Note that in Fig. 10(B), the live image 111 of staff Q during online customer service, the product information image 112, and the image 113 of information related to the topic during the conversation are arranged within the screen so as not to overlap. However, the product information image 112 and the image 113 of information related to the topic during the conversation may be superimposed and displayed on the live image 111 of staff Q. That is, the product information image 112 and the image 113 of information related to the topic during the conversation may be displayed in front of the live image 111.

[0059] <Example 5> Fig. 11 is a diagram for explaining another example of the screen displayed on the store-side monitor 11 during online customer service. (A) shows an example of the screen 110B where only the information related to the topic during the conversation is displayed as shared information, and (B) shows an example of the screen 110C mainly displaying product information. In Fig. 11, reference numerals corresponding to the corresponding parts in Fig. 10 are shown. In the case of Fig. 11(A), the layout of the area where the live image 111 of staff Q is displayed is the same as that in the case of Fig. 10(B). The difference is that only the image 113 is displayed in the area to the right of the live image 111. The screen 110B shown in Fig. 11(A) is used, for example, at the initial stage when online customer service starts.

[0060] In the case of Fig. 11(B), the image 112 of the product information is displayed using most of the screen 110C, and the live image 111 of Staff Q and the image 113 of the information regarding the topic during the conversation are arranged side by side in the empty area below the image 112. The screen 110C shown in Fig. 11(B) is used, for example, after the proportion of the conversation has shifted to the explanation of products or the like. This is because after the proportion has shifted to the product explanation, the proportion of the live image of Staff Q and the information regarding the topic during the conversation becomes small. However, the layout of the images displayed on the monitor 11 on the store side may be freely switched by the selection or instruction by Staff Q. The change of the layout is executed by the operation of the mouse 25 (see Fig. 4) or the keyboard 26 (see Fig. 4) by Staff Q.

[0061] <Embodiment 2> In the present embodiment, in step 3 (see Fig. 5), an optional function that is executed when the execution of the mode for supporting online customer service is selected will be described. Since online customer service is customer service through the screen, the movements of the hand and arm of Staff Q will inevitably look two-dimensional.

[0062] Therefore, as a device for Staff Q in charge of online customer service to convey his / her feelings and emotions to the customers who visit the store, there is a method of intentionally expressing large body movements. However, continuously making large body movements throughout the day places a large physical burden regardless of age. Therefore, in the present embodiment, an optional function is provided to emphasize the movement of Staff Q in the live image presented to the customers who visit the store as compared with the actual movement.

[0063] <Processing operation> Fig. 12 is a flowchart for explaining an example of the optional function executed by the terminal 20 operated by Staff Q who provides online customer service. The processing operation shown in Fig. 12 is executed when an affirmative result is obtained in step 3 (see Fig. 5), that is, when the online customer service support mode is selected. First, the processor 201 determines whether to emphasize gestures or the like (step 11).

[0064] FIG. 13 is a diagram for explaining an example of a screen 240 used for receiving the selection in step 11 (see FIG. 12). The screen 240 is displayed in a partial area of, for example, the monitor 21 (see FIG. 4). The screen 240 is used for receiving the selection in step 11. In the case of FIG. 13, on the screen 240, there are a reception field 241 used for receiving the volume of the voice output, the speed of movement, and the correction amount of the magnitude of movement, a button 242 to be operated in the case of "YES", and a button 243 to be operated in the case of "NO".

[0065] In the reception field 241, an area is prepared for selectively inputting the magnification of the correction amount. In the case of the present embodiment, the options for the magnification are natural numbers. Incidentally, in the case of FIG. 13, the magnification of the voice output volume is 3 times, the magnification of the movement speed is 5 times, and the magnification of the movement magnitude is 2 times. Note that instead of options, it may be possible to input any numerical value. For example, it may be possible to set 1.1 times or 2.5 times.

[0066] Note that it is also possible to set the magnification to 1 times or less, such as 0.5 times. For example, a staff member who originally has a loud voice may sometimes want to intentionally lower the volume. Also, a staff member who originally has overly large gestures may sometimes want to intentionally reduce the movement. Thus, in the case of the present embodiment, the degree of emphasis can be freely varied according to the setting of the staff Q. However, a fixed value may be used for the degree of emphasis. Also, the processor 201 may set the degree of emphasis according to the age, gender, and family composition of the customer estimated through image processing. In this case, the setting used for online customer service is read from a table recording the correspondence between the age etc. of the customer and the degree of emphasis.

[0067] Returning to the description of FIG. 12. If a positive result is obtained in step 11, the processor 201 corrects the video and audio of staff Q according to the settings (step 12). Specifically, the correction of the volume of the audio output is realized by amplifying the electrical signal or digital signal output from the microphone 23 (see FIG. 4) according to the settings. However, the processor 201 may specify an increase in the volume of the sound output from the speaker 14 (see FIG. 3) of the terminal 10 (see FIG. 3) on the store side.

[0068] The correction of the movement speed generates an image obtained by accelerating the playback speed of the online image at a set magnification. The difference from the real time caused by the acceleration of the playback speed may be adjusted by inserting a still image or an image explaining a product or the like. For example, when playing back a 10 - second image in 2 seconds, the remaining 8 seconds are filled with still images. The correction of the movement amplitude detects the movement of the hands and arms of staff Q by image processing, and generates an image in which the detected movement amount is amplified at a set magnification.

[0069] For example, the processor 201 detects the start position and end position of the movement in the image to obtain the distance of the movement. Next, the processor 201 multiplies the obtained distance by the set magnification, and corrects the position where the movement of the hands and arms in the image ends so as to match the multiplied distance. Thereby, even if the actual hand movement by staff Q is 10 cm, on the monitor 11 (see FIG. 3) that the customer sees, an image in which the hand movement appears to be 20 cm is generated.

[0070] Note that the movements of the hands and arms used in online customer service can be classified into several types. For example, it is possible to classify them into movements such as spreading both hands, waving the hands left and right, and moving the hands from bottom to top or diagonally upward. Therefore, a program for image processing corresponding to each movement may be prepared, and when correcting the image, the prepared program may be used to correct the image. When the correction in step 12 is completed, the processor 201 transmits the corrected video and audio to the terminal 10 (see FIG. 3) at the connection destination (step 13). In addition, when a negative result is obtained in Step 11, the processor 201 transmits the uncorrected video and audio to the destination terminal 10 (Step 14).

[0071] <Example of correction processing> <Correction of the volume of audio output> FIG. 14 is a diagram for explaining the correction of the volume of audio output. In FIG. 14, the same reference numerals as the corresponding parts in FIG. 1 are shown. In the case of FIG. 14, the staff Q during online customer service is speaking "For that product" at "Volume 2". However, since the correction magnification is 3 times, the voice of the staff Q is reproduced at "Volume 6" from the terminal 10 on the customer side. Since the magnification can be freely set by the staff Q, it is also possible to set a smaller magnification in the morning and a larger magnification in the evening when getting tired.

[0072] In addition, the processor 201 may adjust the magnification according to the loudness of the voice of the staff Q. For example, it may be adjusted so that the volume of the audio reproduced from the speaker 14 (see FIG. 3) on the store side is constant. Also, when the visitor shows a gesture of difficulty in hearing or asks for an explanation to be repeated several times, the processor 201 may increase the volume of the audio reproduced from the speaker 14.

[0073] <Correction of image> FIG. 15 is a diagram for explaining an example of displaying a corrected image emphasizing the gestures of the staff Q on the monitor 11 on the customer side. In FIG. 15, the same reference numerals as the corresponding parts in FIG. 1 are shown. In the example of FIG. 15, an example of correcting the movement of waving the hand left and right at the set magnification is shown. For example, even if the actual movement of the hand of the staff Q is 10 cm, the movement amount of the hand in the image displayed on the monitor 11 is 20 cm, and the speed of the hand waving left and right is corrected to 5 times the actual speed. By this correction, even through the screen, the gestures of the staff Q can be more easily conveyed to the visitor.

[0074] Note that the body movement correction and voice correction may be executed only when staff Q gives an explicit instruction. For example, when the correction is instructed by the mouse 25 (see FIG. 4) or the keyboard 26 (see FIG. 4), when a foot switch (not shown) is stepped on, or when a predetermined specific catchphrase or line is detected, the correction may be executed at the set magnification. In addition, a threshold may be set for the volume and body movement of the voice of staff Q, and the correction may be executed at the set magnification only when a voice or body movement with a volume exceeding the threshold is detected.

[0075] <Embodiment 3> During online customer service, staff Q works in an office or at home different from the store that is the target of the online customer service. Therefore, staff Q is often in a seated position. On the other hand, customers visiting the store are often standing. Therefore, at least the greeting to the customer visiting the store is preferably given in a standing position like the customer visiting the store. Therefore, in this embodiment, a function of displaying an image of the standing posture of staff Q on the monitor 11 (see FIG. 3) that the customer visiting the store sees will be described even when actually providing online customer service while sitting.

[0076] <Processing operation> FIG. 16 is a flowchart for explaining an example of another optional function executed by the terminal 20 operated by staff Q providing online customer service. The processing operation shown in FIG. 16 is executed when an affirmative result is obtained in step 3 (see FIG. 5), that is, when the online customer service support mode is selected. First, the processor 201 determines whether to display a replacement image (step 21). In the case of this embodiment, staff Q is providing online customer service in a seated state.

[0077] If an affirmative result is obtained in step 21, the processor 201 transmits a full-body image of the standing state to the destination terminal 10 (see FIG. 1) (step 22). The full-body images here are the moving images and photos that were taken in advance of Staff Q. In the case of the images taken in advance, since it is difficult to convey a real-time feeling, the display of the replacement images may be limited to only the scene where the online customer service starts.

[0078] On the other hand, when a negative result is obtained in Step 21, the processor 201 transmits the uncorrected video and audio to the terminal 10 at the connection destination (Step 23). Since it is uncorrected, a live image of Staff Q sitting is transmitted. However, it is also possible to make the online customer service in the sitting state less conspicuous by using an image of Staff Q's face taken up close or an image of Staff Q's upper body.

[0079] FIG. 17 is a diagram for explaining the image of online customer service assumed in Embodiment 3. (A) shows the posture of Staff Q during online customer service, and (B) shows the image of Staff Q displayed on the monitor 11 of the terminal 10 on the customer side who visits the store. As shown in FIG. 17, in the present embodiment, even when Staff Q is providing online customer service while sitting, it appears as if Staff Q is providing customer service in the same standing state as the customer who visits the store. With this function, it becomes possible to convey respect and courtesy to the customer who visits the store.

[0080] In the case of FIG. 17, an image of Staff Q actually providing online customer service is transmitted to the terminal 10, but an image of another staff may be displayed on the terminal 10. Also, instead of an image of Staff Q or the like, a character or the like imitating Staff Q or the like may be transmitted to the terminal 10. As shown in FIG. 17, in the present embodiment, even though Staff Q is actually sitting, it is possible to provide online customer service with an image in a standing posture.

[0081] <Embodiment 4> In the above-described Embodiment 3, it is assumed that the customer who visits the store is always standing, but in the present embodiment, an optional function that enables online customer service according to the posture of the customer who visits the store will be described.

[0082] <Processing Operation> FIG. 18 is a flowchart for explaining an example of another optional function executed by the terminal 20 operated by the staff Q who provides online customer service. The processing operation shown in FIG. 18 is also executed when an affirmative result is obtained in step 3 (see FIG. 5), that is, when the online customer service support mode is selected.

[0083] First, the processor 201 determines whether to switch and display an image according to the posture of the customer who has come to the store (step 31). Also in this embodiment, the staff Q provides online customer service while sitting. If an affirmative result is obtained in step 31, the processor 201 determines whether the customer who has come to the store is sitting (step 32). The posture of the customer who has come to the store is determined by image processing.

[0084] If an affirmative result is also obtained in step 32, the processor 201 transmits the sitting image to the terminal 10 (see FIG. 1) at the connection destination (step 33). In this case, a live image of the staff Q who is providing online customer service while sitting is transmitted. That is, both the staff Q and the customer who has come to the store will have a conversation while sitting. Note that in this embodiment, even when the staff Q is providing online customer service while standing, an image in a sitting state is transmitted to the terminal 10 by executing step 33.

[0085] On the other hand, if a negative result is obtained in step 32, the processor 201 transmits an image of a standing posture to the terminal 10 at the connection destination (step 34). In this case, the staff Q is providing online customer service while sitting, but an image of the standing staff Q or an image of a character is displayed to the customer who has come to the store. As described above, in customer service using a replacement image, it is difficult to convey a sense of real time, so customer service using an image of a standing posture may be used only for the initial greeting scene.

[0086] Incidentally, if a negative result is obtained in step 31, the processor 201 determines whether the customer who has come to the store is sitting (step 35). If an affirmative result is obtained in step 35, the processor 201 transmits the live image to the terminal 10 of the destination (step 36). That is, if the staff Q during online customer service is sitting, the image of the sitting state is transmitted to the terminal 10. If the staff Q during online customer service is standing, the image of the standing state is transmitted to the terminal 10 on the store side.

[0087] If a negative result is obtained in step 35, the processor 201 guides the staff Q to serve customers in a standing state (step 37). In this case, since the image is not switched, the staff Q is urged to serve customers in a standing state in the same manner as the customers who come to the store. The guidance here may be performed by displaying on the monitor 21 (see FIG. 4) that the staff Q can visually recognize, or by voice notification from the speaker 24 (see FIG. 4). After that, the processor 201 proceeds to step 36 and transmits the live image to the terminal 10 of the destination.

[0088] FIG. 19 is a diagram for explaining the image of online customer service assumed in the fourth embodiment. (A) shows the posture of the staff Q during online customer service, and (B) shows the image of the staff Q displayed on the monitor 11 and the customer who comes to the store on the store side. In the case of FIG. 19, since the customer who comes to the store is in a standing posture, even if the staff Q during online customer service is sitting, an image of the staff Q serving customers in a standing posture or an image of a character is displayed on the monitor 11 of the terminal 10.

[0089] FIG. 20 is a diagram for explaining another image of online customer service assumed in the fourth embodiment. (A) shows the posture of the staff Q during online customer service, and (B) shows the image of the staff Q displayed on the monitor 11 and the customer who comes to the store on the store side. In the case of FIG. 20, the customer who comes to the store is sitting on a chair. For this reason, a live image of the staff Q serving customers in a sitting state or an image of a character is displayed on the monitor 11 of the terminal 10. FIG. 20 exemplifies the case where an image of a character is displayed instead of the live image of the staff Q.

[0090] FIG. 21 is a diagram for explaining another image of online customer service assumed in Embodiment 4. (A) shows the posture of staff Q during online customer service, and (B) shows the image of staff Q displayed on monitor 11 and the in-store customer. In FIG. 21, it is assumed that a negative result is obtained in step 31 (see FIG. 18). Therefore, the switching display of the image by the processor 201 is not executed. That is, the live image of staff Q is directly transmitted to the terminal 10.

[0091] In FIG. 21, the in-store customer is standing. Therefore, a function that notifies staff Q of the posture of the in-store customer and prompts a change in posture operates from the terminal 20 operated by staff Q. For example, a voice such as "The customer is standing. Let's serve the customer while standing." is output from the terminal 20. This notification serves as an opportunity for staff Q to change their posture during online customer service.

[0092] <Embodiment 5> <System Configuration> In this embodiment, a system for online support of staff during business hours will be described. FIG. 22 is a diagram for explaining a configuration example of the business support system 1A used in Embodiment 5. In FIG. 22, the corresponding parts to those in FIG. 1 are denoted by the same reference numerals. The business support system 1A shown in FIG. 22 assumes a scenario where a supervisor supports subordinates who are on business at business locations (Branch E of Company D, Sales Office G of Company F, Store I of Company H,...) online.

[0093] In the case of FIG. 22, Branch E of Company D is located in Region X of Country J, and the current weather is sunny. Sales Office G of Company F is located in Region Y of Country J, and the current weather is rainy. Store I of Company H is located in Region Z of Country J, and the current weather is snowy. Also, the supervisor's workplace is in Region T of Country J, and the current weather is cloudy. The supervisor's workplace is not limited to a specific office and may be a private room-type office or home that can be used on an hourly basis.

[0094] In the case of Fig. 22, the E branch of Company D, the G sales office of Company F, the I store of Company H, and the workplace of the supervisor are all in different regions within the same country, but they may also be in the same region within the same country or in different countries. Since it is online support, there is no restriction on the distance between the business destination visited by the subordinate (hereinafter also referred to as the "person in charge") and the workplace of the supervisor. The subordinate here is an example of a person present in front of the terminal. Note that the person who is serving customers at the business destination is an example of a third party. In the example of Fig. 22, the supervisor assumes a scenario of online supporting the business conducted by the subordinate who visited the E branch of Company D.

[0095] In the case of Fig. 22, there is a terminal 10A at the business destination. A terminal 20 operated by the supervisor is connected to the terminal 10A via a network 30. The terminal 10A in Fig. 22 is composed of a smartphone, earphones, a camera, and a microphone. The terminal 10A here is an example of the first terminal. Note that in the case of Fig. 22, the earphones and the microphone are depicted as a headset. In the case of this embodiment, the camera is placed on the table and captures images of the customer and the subordinate. Therefore, in the example of Fig. 22, the supervisor is observing the business situation on the monitor of the terminal 20.

[0096] However, the camera is not limited to being placed on the table. For example, the image during business may be captured by the camera of the smartphone. In that case, the subordinate may, for example, store the smartphone in the chest pocket and capture the business situation. In addition, when the business situation is captured by the camera for video conferencing installed in the meeting room or the like at the business destination, the image may be shared on the terminal 20 on the supervisor's side.

[0097] However, there may be a case where imaging by a camera is not permitted during business operations. In that case, naturally, imaging by the camera is not performed. However, for the purpose of supporting good-quality and high-quality communication, in the present embodiment, it is required that at least acquisition of voice during business operations by a microphone (not shown) and transmission of the acquired voice to the terminal 20 can be executed in real time.

[0098] In the case of the present embodiment, the earphone is used to receive support from a supervisor. Therefore, in the present embodiment, a subordinate conducts business while wearing the earphone on the ear. However, if it is possible to check the display of the smartphone during business operations, an instruction from the supervisor may be displayed on the display of the smartphone. In that case, wearing the earphone becomes unnecessary.

[0099] In the case of FIG. 22, a smartphone is assumed as the terminal 10A connected via the network 30 to the terminal 20 operated by the supervisor, but a notebook computer may also be used. If it is possible to operate the computer during the explanation to the customer, the subordinate can check the information provided by the supervisor on the display. In the case of FIG. 22, the supervisor is an example of an assistant. Further, the terminal 20 operated by the supervisor is an example of an information processing apparatus and also an example of a second terminal.

[0100] Further, in the case of FIG. 22, it is assumed that the person receiving support is a subordinate and the person providing support is a supervisor, but the person receiving support may be a supervisor and the person providing support may be a subordinate. Also, the person receiving support and the person providing support may be colleagues. In any case, the person providing support becomes an assistant.

[0101] <Configuration of Terminal> FIG. 23 is a diagram for explaining an example of the hardware configuration of the terminal 10A used at a business site. In FIG. 23, reference numerals corresponding to the corresponding parts in FIG. 3 are attached and shown. The terminal 10A shown in FIG. 23 includes a terminal main body 100A, a display 11A, a camera 12A, a microphone 13A, and an earphone 14A. Furthermore, the terminal main body 100A includes a processor 101, a semiconductor memory 102 as an internal memory, a semiconductor memory 103A as an external memory, and a communication module 104.

[0102] For the display 11A, for example, an organic EL display or a liquid crystal display is used. In the case of this embodiment, the display 11A is provided integrally with the smartphone. Note that the display 11A in this embodiment is referred to when a subordinate during business receives support by means of characters or images. Therefore, when receiving support only by voice, the display 11A is not necessarily required.

[0103] For the camera 12A, for example, a CMOS image sensor or a CCD image sensor is used. The camera 12A may be integrated with the terminal main body 100A or may be an external attachment. The microphone 13A is, for example, a device that converts the voices of a customer and a subordinate into electrical signals. The microphone 13A may also be integrated with the terminal main body 100A or may be an external attachment. The microphone 13A is also an example of a sound collection unit. The earphone 14A is, for example, a device that converts the voice of a supervisor given as an electrical signal into sound and outputs it.

[0104] The processor 101 is constituted by, for example, a CPU. The processor 101 realizes various functions through program execution. The semiconductor memory 102 as an internal memory has a ROM in which a BIOS or the like is stored and a RAM used as a main storage device. The processor 101 and the semiconductor memory 102 constitute a so-called computer.

[0105] The processor 101 uses the RAM as a work space for programs. The semiconductor memory 103A as an external memory is an auxiliary storage device, and stores, in addition to programs, data of images to be displayed on the display 11A, data of images captured by the camera 12A, and the like. When using a notebook computer as the terminal 10A, a hard disk device 103 (see FIG. 3) may be used instead of the semiconductor memory 103A. The communication module 104 is a device that realizes communication with the terminal 20 (see FIG. 22) operated by a supervisor who supports sales.

[0106] <Processing operation> FIG. 24 is a flowchart for explaining the processing operation executed by the terminal 20 (see FIG. 22) operated by a supervisor who supports sales online. In FIG. 24, reference numerals corresponding to the corresponding parts in FIG. 5 are shown. The processing operation shown in FIG. 24 is realized through the execution of a program by the processor 201 (see FIG. 4).

[0107] First, the processor 201 accepts the selection of a connection destination (step 1). In the case of this embodiment, the online sales support is started by the supervisor who supports sales selecting a connection destination. The selection of the connection destination is made, for example, when there is a call operation or a call from a subordinate during business hours, or when the start of business is confirmed on the monitor 21 (see FIG. 4).

[0108] When the supervisor selects a connection destination, the processor 201 acquires the position of the connection destination (step 2). The processor 201 acquires, for example, information such as "Branch E of Company D in Region X of Country J" or the location information of the business destination. The location information of the connection destination is registered in advance in the hard disk device 203 (see FIG. 4) or the like. Note that the location information may include information such as the name of the building where the business destination is located and the floor number. The location here is an example of the location of the terminal.

[0109] Next, the processor 201 determines whether to execute a mode for supporting business operations online (hereinafter referred to as the "online business support mode") (step 41). In the present embodiment, after selecting the connection destination, it is inquired whether to execute the online business support mode, but it is also possible to select it in advance.

[0110] If an affirmative result is obtained in step 41, the processor 201 determines whether to display support information (step 42). The support information includes, for example, weather information and industry information of the business destination. The support information in the present embodiment is an example of first information related to the location of the store that is the connection destination. Also, the business destination is an example of the visit destination.

[0111] FIG. 25 is a diagram for explaining an example of screens 250 and 260 used for receiving the selections in steps 41 and 42 (see FIG. 24). The screens 250 and 260 are displayed in a partial area of, for example, the monitor 21 (see FIG. 4).

[0112] The screen 250 is used for receiving the selection in step 41. In the case of FIG. 25, on the screen 250, there are arranged a sentence 251 "Do you want to turn on the online business support mode and start communication?", a button 252 to be operated in the case of "YES", and a button 253 to be operated in the case of "NO".

[0113] The screen 260 is used for receiving the selection in step 42. The screen 260 is displayed when the answer in step 41 is "YES". In the case of FIG. 25, on the screen 260, there are arranged a sentence 261 "Do you want to display weather information, industry information, etc. of the business destination?", a button 262 to be operated in the case of "YES", and a button 263 to be operated in the case of "NO".

[0114] Return to the description of FIG. 24. If an affirmative result is obtained in step 42, the processor 201 displays support information including information related to the location of the business partner and location information (step 43). The information related to the location of the business partner includes, for example, information on the industry related to the business of the business partner, visit history, and minutes of meetings. The industry information has recent topics, such as information on new products, new goods, new services, exhibitions, and academic societies. The support information here is the first information and is also an example of the second information that supports conversations with customers. The second information includes, for example, information not related to the location of the business partner, such as greetings, topics according to the history and age of the customers talking at the business partner.

[0115] On the other hand, if a negative result is obtained in step 42, the processor 201 displays only the location information (step 44). The location information includes information identifying the store name and location. If a negative result is obtained in step 41, the processor 201 executes another operation mode (step 45). In the case of this embodiment, the other operation mode includes the mode for supporting the above-described online customer service.

[0116] <Example of operation screen> Hereinafter, with reference to FIGS. 26 to 29, an example of a screen displayed on the terminal 20 operated by a supervisor who supports sales will be described. <Example 1> FIG. 26 is a diagram for explaining an example of a screen 270 that does not display support information when supporting sales online. The screen 270 is displayed on the monitor 21 of the terminal 20 (see FIG. 22) operated by a supervisor who supports the sales of a subordinate.

[0117] In the case of FIG. 26, an image 271 of a customer (hereinafter referred to as "customer") and a subordinate taken by the monitor 21 and information 272 for identifying the business partner are displayed on the screen 270. In the case of FIG. 26, as the information 272, the local date and time of the business partner and the name of the business partner are displayed. The reason for displaying the local date and time, etc. is to facilitate the confirmation of the location and time of day visited by subordinates during business operations, even when there are multiple standard times within the same country, when daylight saving time is in effect, when the work location of the supervisor supporting the business and the business destination are in different countries, etc. In addition, in Fig. 26, an image 271 showing both the customer and the subordinate is displayed, but there may be cases where only the customer's image is displayed, or cases where the image 271 is not displayed.

[0118] <Example 2> Fig. 27 is a diagram for explaining an example of a screen 270A that displays support information when supporting business online. In Fig. 27, reference numerals corresponding to the corresponding parts in Fig. 26 are used for illustration. In the case of the screen 270A, support information 273 and a topic column 274 are additionally displayed. In the case of Fig. 27, as the support information 273, the current weather, temperature, industry topics, negotiation history, and previous visit date in the region where the business destination is located are displayed.

[0119] Specifically, the current weather in the region where the business destination is located is "sunny" and the temperature is "30°C". Also, as the negotiation history, it is displayed that it is the "third appointment" and the previous visit date is "2020 / 08 / 07". These pieces of information provide basic information to the assistant who supports the person in charge, that is, the supervisor. In addition, in the case of Fig. 27, the industry topic related to the business destination is blank. When displaying industry topics, the number is not limited to one and may be multiple.

[0120] Incidentally, industry topics are obtained from news sites, etc. at the stage when the location of the business destination is acquired. It is desirable to prioritize information with a high degree of immediacy for the topics displayed as the support information 273. In other words, information with a newer time when the news was published is prioritized. If new topics that the subordinate may not be aware of can be provided, the development of the conversation between the subordinate and the other party is expected.

[0121] In addition, the support information 273 may include information related to weather and disasters such as humidity and earthquakes. In addition, when a customer is identified as an individual from an image of the customer, the number and history of business negotiations between the corresponding customer and the company, the number and history of business negotiations with subordinates, the name of the subordinate who was in charge of the previous business negotiation, the memo during the business negotiation, etc. may be displayed as support information. The identification of the customer here does not necessarily require the identification of the individual's name.

[0122] In the topic column 274, buttons for topics that are likely to be useful for conversations with customers are displayed. In the case of FIG. 27, "Greetings", "Current Affairs", "Minutes of the Meeting", "History of the Other Party", "Advice", etc. are displayed. Note that the candidates for topics displayed in the topic column 274 may be general-purpose content regardless of the difference in business destinations, or may be content selected by the processor 201 according to the gender, age, clothing, etc. recognized from the image of the customer.

[0123] <Example 3> FIG. 28 is a diagram for explaining an example of another screen 270B that displays support information when supporting sales online. In FIG. 28, reference numerals corresponding to the corresponding parts in FIG. 27 are shown. In the screen 270B shown in FIG. 28, "Minutes of the Meeting" among the plurality of items displayed in the topic column 274 is selected by the supervisor. In FIG. 28, the selected state is represented by hatching.

[0124] On the screen 270B, an example sentence related to the topic selected by the supervisor is displayed in the example sentence column 275. In FIG. 28, the content "There is a provisional specification determined at the previous visit" is displayed. However, the full text of the minutes of the meeting may be read out and displayed. In addition, when "History of the Other Party" is selected, the history recorded about the customer and the company history of Company D are displayed in the example sentence column 275.

[0125] In this way, since the information and specific examples that should be known as the premise of support are displayed in the example column 275, it becomes possible to provide useful information to the subordinates who are visiting the business partner. Specifically, by conveying the advice based on the displayed information and the displayed examples to the subordinates online, it becomes possible to improve the conversation and the quality of sales at the business partner. In addition, when access rights are set for matters associated with the topic, information within the range permitted by the access rights is displayed in the example column 275.

[0126] In this embodiment, the superior is supporting the subordinate during business. However, as described above, the relationship between the person receiving support and the person providing support is diverse. Therefore, by displaying the topic column 274 and the example column 275, regardless of the combination of the person providing support and the person receiving support, the quality of support is improved, and an improvement in the conversation with the customer and the quality of sales at the business partner is realized. For example, even when one or both of the combination have little sales experience, by referring to the display of the topic column 274 and the example column 275, the variation in the quality of sales due to the difference in the combination is reduced. Also, when the combination of the nationalities of the person receiving support and the person providing support is different, a similar effect is expected.

[0127] Note that the content of the topics displayed in the topic column 274 and the examples displayed in the example column 275 may be customized according to the attributes of the person receiving support, the business partner, and the customer. For example, when the person receiving support is old and the customer is young, topics corresponding to the customer's age may be selectively displayed.

[0128] <Example 4> FIG. 29 is a diagram for explaining a specific example of online sales support. (A) is an example of the screen 270C displayed on the terminal 20 (see FIG. 22) operated by the superior who supports sales online. (B) shows an example in which support for the subordinate during business is executed in text, and (C) shows an example in which support for the subordinate during business is executed by voice. In FIG. 29, reference numerals corresponding to the corresponding parts in FIG. 28 are shown.

[0129] On the screen 270C shown in FIG. 29(A), an advice column 276 is added for the supervisor operating the terminal 20 to input advice given to the subordinate at the business customer. In FIG. 29(A), “Please check if there are any changes to the provisional specifications” is input in the input column of the advice column 276. In FIG. 29(B), a message 114 from the supervisor is displayed on the display 11A of the terminal 10A carried by the subordinate during business. On the other hand, in FIG. 29(C), the voice of the supervisor acquired by a microphone 23 (see FIG. 4) not shown is output from the earphone 14A. Thus, in the case of this embodiment, the supervisor who is providing online support does not directly participate in the conversation between the customer and the subordinate. In this regard, it is different from a so-called video conference. Moreover, it becomes possible to support the conversation between the subordinate and the customer and the improvement of the quality of sales online.

[0130] <Other Embodiments> (1) As described above, the embodiments of the present invention have been described. However, the technical scope of the present invention is not limited to the scope described in the above-described embodiments. It is clear from the description of the claims that those obtained by making various changes or improvements to the above-described embodiments are also included in the technical scope of the present invention.

[0131] (2) In the above-described Embodiment 1, the support information for the area where the store is located is acquired when the area is selected. However, the acquisition of the support information may be executed after the display of the support information is instructed in Step 4 (see FIG. 5).

[0132] (3) In the above-described Embodiment 1, the case of serving customers online has been described. However, it may also be used for online sales. In that case, a live image or the like of the staff Q conducting online sales will be displayed to the customer. Similarly, the technologies described in Embodiments 2 to 4 may also be used for online sales.

[0133] (4) In the foregoing Embodiment 5, the case of online support for sales activities of subordinates or the like visiting business partners has been described. However, the same technology may also be used to support on-site staff serving in-store customers. For example, it may be used when online supporting the sales activities of a new staff member serving customers at the storefront. In this case, the in-store customer becomes an example of a third party, and the new staff member serving customers at the storefront becomes an example of a person existing in front of the terminal.

[0134] (5) In the case of the foregoing embodiments, the information necessary for support is presented to the staff Q during online customer service or the supervisor supporting sales online through the program executed on the terminal 20 (see FIGS. 1 and 22). However, the same program may be executed on a server connected to the network 30 (see FIGS. 1 and 22). FIG. 30 is a diagram for explaining a configuration example of another customer service support system 1B. In FIG. 30, reference numerals corresponding to the corresponding parts in FIG. 1 are given.

[0135] In the case of FIG. 30, communication between the terminal 10 and the terminal 20 is realized via the server 40 connected to the network 30. The server 40 provides, for example, a support service as a cloud service. In this case, the terminal 20 operated by the staff Q is used as an input / output device for the server 40. The server 40 here is an example of an information processing device. Note that the support service shown in FIG. 30 is premised on online customer service, but may also be used for the online sales support described in Embodiment 5.

[0136] (6) In each of the above embodiments, the processor refers to a processor in a broad sense and includes a general-purpose processor (for example, CPU: Central Processing Unit, etc.) and a dedicated processor (for example, GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.). Also, the operations of the processor in each of the above embodiments may be performed not only by a single processor but also by a plurality of physically separated processors working in cooperation. Further, the order of each operation of the processor is not limited to the order described in each of the above embodiments and may be changed as appropriate.

Explanation of Signs

[0137] 1, 1B... Customer service support system, 1A... Sales support system, 10, 10A, 20... Terminals, 30... Network, 40... Server

Claims

1. It has a processor, The processor acquires the position of a terminal having a sound collection unit that is connected online to its own device, acquires first information related to the acquired position, and presents the acquired first information to a user who operates its own device as second information for assisting a conversation with a person existing in front of the terminal. An information processing device.

2. It has a processor, The processor acquires the position of a terminal having a sound collection unit that is connected online to its own device, acquires first information related to the acquired position, and presents the acquired first information to a user who operates its own device as second information for assisting a conversation between a person existing in front of the terminal and a third party. An information processing device.

3. The processor displays a screen for selecting whether to provide the first information, and provides the first information as the second information when an instruction to provide the first information is received. The information processing device according to Claim 1 or 2.

4. The processor does not provide the first information as the second information when an instruction not to provide the first information is received. The information processing device according to Claim 3.

5. The processor executes acquisition of the first information when an instruction to provide the first information is received. The information processing device according to Claim 3.

6. The processor acquires a topic of the area where the terminal is located as the first information. The information processing device according to Claim 1.

7. Prioritize highly immediate content as the topic of the area. The information processing device according to Claim 6.

8. The processor acquires a topic related to the visit destination of the person carrying the terminal as the first information. The information processing device according to Claim 2.

9. The topic related to the visit destination includes a visit history. The information processing device according to Claim 8.

10. The processor presents an example sentence of a topic related to an item for which a designation has been received as the second information. The information processing device according to Claim 1 or 2.

11. Candidates for the item are provided so as to be selectable. The information processing device according to Claim 10.

12. The processor provides information about the person as the second information. The information processing device according to Claim 1.

13. The processor provides information about the third party as the second information. The information processing apparatus according to claim 2.

14. The processor performs a process of emphasizing the movement of the user in the image displayed by the terminal as compared with the actual movement of the user. The information processing apparatus according to claim 1.

15. The magnitude of the user's movement is enlarged by image processing as compared with the actual magnitude of the movement. The information processing apparatus according to claim 14.

16. The speed of the user's movement is increased or decreased by image processing as compared with the actual speed of the movement. The information processing apparatus according to claim 14.

17. The processor varies the degree of emphasis according to the user's setting. The information processing apparatus according to claim 14.

18. During the conversation with the person, the processor generates an image of a standing posture prepared in advance or an image representing a standing posture and transmits it to the terminal. The information processing apparatus according to claim 1.

19. The processor when the person is sitting, generates an image of a sitting posture prepared in advance or an image representing a sitting posture and transmits it to the terminal, when the person is standing, generates an image of a standing posture prepared in advance or an image representing a standing posture and transmits it to the terminal. The information processing apparatus according to claim 1.

20. When transmitting the live image of the user during the conversation to the terminal, the processor guides the user with respect to the posture of the person. The information processing apparatus according to claim 1.

21. In an information processing method executed by an information processing apparatus that provides a service for assisting communication, a process of acquiring the position of the first terminal used at the location where the person who is the conversation partner is present among the first terminal and the second terminal connected online, a process of acquiring first information related to the acquired position, a process of presenting the acquired first information to the user operating the second terminal as second information for assisting the conversation with the person. An information processing method having the above.

22. In an information processing method executed by an information processing apparatus that provides a service for assisting communication, a process of acquiring the position of the first terminal used at the location where the person who is having a conversation with a third party is present among the first terminal and the second terminal connected online, a process of acquiring first information related to the acquired position, a process of presenting the acquired first information to the user operating the second terminal as second information for assisting the conversation between the person and the third party. An information processing method having

23. In a computer, a function of acquiring the position of a terminal having a sound collection unit that is connected online to the own device; a function of acquiring first information related to the acquired position; a function of presenting the acquired first information to a user who operates the own device as second information for assisting a conversation with a person existing in front of the terminal; A program for realizing the above.

24. In a computer, a function of acquiring the position of a terminal having a sound collection unit that is connected online to the own device; a function of acquiring first information related to the acquired position; a function of presenting the acquired first information to a user who operates the own device as second information for assisting a conversation between a person existing in front of the terminal and a third party; A program for realizing the above.

25. In a computer, a function of acquiring the position of a first terminal that is used at a location where a person who is a conversation partner is present, among a first terminal and a second terminal that are connected online; a function of acquiring first information related to the acquired position; a function of presenting the acquired first information to a user who operates the second terminal as second information for assisting a conversation with the person; A program for realizing the above.

26. In a computer, a function of acquiring the position of a first terminal that is used at a location where a person who converses with a third party is present, among a first terminal and a second terminal that are connected online; a function of acquiring first information related to the acquired position; a function of presenting the acquired first information to a user who operates the second terminal as second information for assisting a conversation between the person and the third party; A program for realizing the above.

Images