Communication system, communication method, and program

Abstract

To convey to a customer an atmosphere of a place where a trip or the like is planned, with an enhanced sense of realism, in order to solve the problem in which: compared to face-to-face customer services, remote customer services lack in excitement of customers when making travel plans; in a case of a conventional face-to-face customer service, a customer and an explainer can plan a trip while sharing materials of travel pamphlets and exchanging opinions; during this step, listening directly to experience of the explainer livens up the conversation and increases the customer's expectations for the trip; on the other hand, even the remote customer service involves sharing electronic data of travel pamphlets through screen sharing while planning a trip, however, because it is not the face-to-face customer service, it is difficult to raise the customer's expectations for the trip.SOLUTION: According to an instruction from an explainer terminal side, a personal image of the explainer is displayed in a main image column 140, a personal image of a customer is displayed in a sub image column 150, and images of landscapes and the like of a travel destination are displayed in each column 140 and 150 as a background image.SELECTED DRAWING: Figure 33

Description

[Technical Field]

[0001] The present disclosure relates to a communication system, a communication method, and a program. [Background technology]

[0002] In recent years, remote concierge (remote customer service) services have been introduced in order to streamline and streamline store operations. As a result, even if a customer (guest) has to wait his / her turn for a customer service because a store employee is busy serving customers, the customer (guest) can receive customer service by accessing a terminal of a concierge with specialized skills at a remote service center via the Internet or the like from a store terminal and having a video call with the concierge (see Patent Document 1).

[0003] Remote customer service has been increasingly adopted by travel agencies in recent years. This service allows customers (guests) to be served without having to wait in stores, and it is beginning to spread as a service that helps travel agencies prevent lost opportunities to acquire new customers. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-154059 Summary of the Invention [Problem to be solved by the invention]

[0005] However, several issues have become apparent. Compared to face-to-face customer service, remote customer service poses the challenge of not getting the customer (guest) excited when planning a trip. In traditional face-to-face customer service, customers and the person providing the tour can share travel brochures and exchange opinions while planning the trip. In this case, hearing the tour guide's personal experiences directly creates a lively conversation and increases the customer's expectations for the trip. On the other hand, even in remote customer service, travel plans are made while sharing electronic data from travel brochures via screen sharing, but because it is not a face-to-face format, it is difficult to get the customer excited about the trip. As a result, when using video calls, it becomes more difficult to get the conversation going compared to face-to-face meetings. [Means for solving the problem]

[0006] The invention of claim 1 is a communication management device that mediates video data of a subject at a first location sent by a first communication terminal so that a second communication terminal can receive the video data of the subject at the first location, and is characterized by having: a receiving means that receives a request to send a specific background image sent by the first communication terminal; and a transmitting means that transmits data of the specific background image to the second communication terminal based on the transmission request so that the second communication terminal displays the specific background image as a background image for the subject at the first location while the second communication terminal is displaying video data of the subject at the second location photographed by the second communication terminal. [Effects of the Invention]

[0007] As described above, according to the present invention, even in the case of a video call, each user can share the same background image with each other during the video call, which has the effect of making the conversation more lively. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating the overall configuration of a communication system according to an embodiment of the present invention. [Figure 2] FIG. 10 is an image diagram showing a situation in which a customer at a store and a presenter at a service center are remotely making a video call. [Figure 3] FIG. 2 is a hardware configuration diagram of each terminal and each device that constitutes the communication system of the present embodiment. [Figure 4] FIG. 2 is a functional block diagram of the communication system according to the embodiment. [Figure 5] FIG. 2 is a functional block diagram of the communication system according to the embodiment. [Figure 6] 1A is a conceptual diagram showing a presenter management table, and FIG. 1B is a conceptual diagram showing a skill group management table. [Figure 7] 1A is a conceptual diagram showing an authentication management table, and FIG. 1B is a conceptual diagram showing a terminal status management table. [Figure 8] 1A is a conceptual diagram showing a destination list management table, FIG. 1B is a conceptual diagram showing a session management table, and FIG. 1C is a conceptual diagram showing a communication information management table. [Figure 9] 1A is a conceptual diagram of a system update information management table, FIG. 1B is a conceptual diagram of a provider management table, and FIG. 1C is a conceptual diagram of an area management table. [Figure 10] 1A is a conceptual diagram of a terminal management table, and FIG. 1B is a conceptual diagram of a skill management table. [Figure 11] 1A is a conceptual diagram of a login information management table, FIG. 1B is a conceptual diagram of a subgroup management table, and FIG. 1C is a conceptual diagram of a user update information management table. [Figure 12] FIG. 10 is a sequence diagram illustrating a registration process of service information. [Figure 13] FIG. 10 is a diagram showing a login screen displayed on the administrator terminal. [Figure 14] FIG. 10 is a diagram showing a store list screen displayed on the administrator terminal. [Figure 15] FIG. 10 is a diagram showing a presenter list screen displayed on the administrator terminal. [Figure 16] FIG. 10 is a diagram showing a group list screen displayed on the administrator terminal. [Figure 17] FIG. 10 is a diagram showing an explainer information editing screen displayed on the manager terminal. [Figure 18] FIG. 10 is a diagram showing a group information editing screen displayed on the administrator terminal. [Figure 19] FIG. 10 is a diagram showing a store information editing screen displayed on the manager terminal. [Figure 20] FIG. 10 is a sequence diagram showing a preparation process for a video call by a store terminal, and in particular showing a process for acquiring service information. [Figure 21] FIG. 10 is a sequence diagram showing a preparation process for a video call at a store terminal, and in particular showing a process for acquiring the operating status of each terminal. [Figure 22] 10 is a flowchart showing a preparation process for a video call at a store terminal, and in particular a process for determining the operating status of a group of presenter terminals for each skill. [Figure 23] FIG. 10 is a diagram showing a screen displayed on a store terminal. [Figure 24] FIG. 10 is a sequence diagram showing a process for starting a video call. [Figure 25] FIG. 10 is a sequence diagram showing detailed processing of communication establishment control. [Figure 26] 10 is a flowchart showing various processes for starting a call. [Figure 27] 10 is an example of a call screen displayed on a presenter terminal. [Figure 28] 10 is an example of a call screen displayed on a store terminal. [Figure 29] FIG. 10 is a sequence diagram showing a process in which a presenter terminal and a store terminal start a video call from a dedicated application and create a travel plan. [Figure 30] 10 is an example of a call screen (including an input window and a map window) displayed on the presenter terminal. [Figure 31] 10 is an example of a call screen (input window, map window) displayed on the presenter terminal. [Figure 32] 10 is an example of a call screen (including an input window, a map window, and an image of a tourist spot) displayed on the presenter terminal. [Figure 33]10 is an example of a call screen (including an image of a tourist spot) displayed on a store terminal. [Figure 34] 10 is an example of a call screen (including an input window, a map window, an image of the exterior of the accommodation facility, and a price list) displayed on the presenter terminal. [Figure 35] 10 is an example of a call screen (including an image of the exterior of an accommodation facility and a price list) displayed on a store terminal. [Figure 36] 10 is an example of a call screen (including an input window, a map window, an image of a room, and a price list) displayed on the presenter terminal. [Figure 37] 10 is an example of a call screen (including an image of a room and a price list) displayed on a store terminal. DETAILED DESCRIPTION OF THE INVENTION

[0009] [Overall configuration of communication system] The overall configuration of a communication system will be described using Figure 1. Figure 1 is an overall configuration diagram of a communication system according to an embodiment of the present invention. Here, a customer, a travel agency that sells travel products to the customer, and a system provider that provides the travel agency with a system for remote concierge (remote customer service) appear.

[0010] Travel agencies operate staffed stores, unmanned stores, and service centers in multiple areas, such as Kanto and Kansai. FIG. 1 shows that a travel agency operates a staffed store MS, an unmanned store US, and a service center SC1 in area E1. A staffed store MS is a store that has employees who serve customers. An unmanned store is a store that does not have employees who serve customers. Note that while FIG. 1 shows only one staffed store MS and one unmanned store US, there may be multiple of each.

[0011] In a manned store MS, an employee usually introduces travel products to customers who visit the store, but there are cases where the employee is busy serving other customers and is unable to serve new customers. Furthermore, in an unmanned store US, there are no employees to serve customers at all. In such cases, a customer can use the store terminal 1c1 in the manned store MS or the store terminal 1c2 in the unmanned store US to access the presenter terminal of any presenter in the service center SC1 via a communication network 100 such as the Internet, and receive remote customer service regarding travel products from the presenter. The presenter is, for example, the remote concierge mentioned above. Since each presenter basically uses their own dedicated presenter terminal, a specific presenter and a specific presenter terminal are linked.

[0012] Each of the store terminals 1c1 and 1c2 is configured by a PC (personal computer). A manned store MS and an unmanned store US may have multiple store terminals installed. Hereinafter, the store terminals 1c1, 1c2, and other store terminals will be collectively referred to as "store terminal 1."

[0013] The service center SC1 includes a manned store priority response organization MG1 that receives, on a priority basis, a call start request to start remote customer service from a store terminal 1c1 in a manned store MS, and an unmanned store priority response organization UG1 that receives, on a priority basis, a call start request to start remote customer service from a store terminal 1c2 in an unmanned store US. Note that there may be three or more priority response organizations.

[0014] Furthermore, the staffed store priority response organization MG1 includes an organization SR11 responsible for skill SK1 and an organization SR12 responsible for skill SK2. For example, skill SK1 is the ability of a presenter to make suggestions regarding domestic travel, and skill SK2 is the ability of a presenter to make suggestions regarding overseas travel. Similarly, the unmanned store priority response organization UG1 includes an organization SR21 responsible for skill SK1 and an organization SR22 responsible for skill SK2. In other words, the staffed store priority response organization MG1 and the unmanned store priority response organization UG1 have organizations SR11 and SR21 (or organizations SR12 and SR22) that employ presenters with the same skills. Note that there may be three or more skills and skill-responsible organizations.

[0015] Furthermore, in the manned store priority response organization MG1, the organization SR11 in charge of skill SK1 includes presenters mg11 and mg12 who have skill SK1. The presenter mg11 uses the presenter terminal 9mg11, and the presenter mg12 uses the presenter terminal 9mg12.

[0016] Furthermore, the organization SR12 in charge of skill SK2 has presenters mg21 and mg22 who have skill SK2. The presenter mg21 uses a presenter terminal 9mg21, and the presenter mg22 uses a presenter terminal 9mg22.

[0017] Similarly, in unmanned store priority response organization UG1, organization SR21 responsible for skill SK1 has presenters ug11 and ug12 who possess skill SK1. Presenter ug11 uses presenter terminal 9ug11, and presenter ug12 uses presenter terminal 9ug12. Furthermore, organization SR22 responsible for skill SK2 has presenters ug21 and ug22 who possess skill SK2. Presenter ug21 uses presenter terminal 9ug21, and presenter ug22 uses presenter terminal 9ug22.

[0018] There may be three or more presenters in each responsible organization (SR11, SR12, SR21, SR22). In this case, a presenter terminal is assigned to each of the three or more presenters. In addition, hereinafter, when referring to multiple presenter terminals such as presenter terminals 9mg11, 9mg12, 9mg21, 9mg22, 9ug12, 9ug12, 9ug21, 9ug22, etc., they will be referred to collectively as "presenter terminal 9."

[0019] The travel agency also has an administrator ad1 who manages the terminals in each service area. The administrator ad1 uses the administrator terminal 4 to access the service information management device 7 via the communication network 100 and sets various types of service information. This service information is information related to the processing and settings of each terminal of the travel agency, The administrator terminal 4 is configured as a PC.

[0020] Customers c1 and c2 are customers who visited a staffed store MS and an unstaffed store US, respectively.

[0021] The system provider manages and operates the relay device 3, communication management device 5, and service information management device 7. The relay device 3, communication management device 5, and service information management device 7 are computers such as servers. The relay device 3 relays image data and audio data when a video call is made between any of the store terminals 1 and any of the presenter terminals 9. The communication management device 5 centrally manages login authentication from the store terminal 1 and the presenter terminal 9, management of the communication status of the store terminal 1 and the presenter terminal 9, management of the destination list, and communication status of the relay device 30. The service information management device 7 manages various set service information. The relay device 3, communication management device 5, and service information management device 7 will be explained in detail later.

[0022] The store terminal 1, the manager terminal 4, the presenter terminal 9, the relay device 3, the communication management device 5, and the service information management device 7 can be connected to a communication network 100 such as the Internet.

[0023] The relay device 3, communication management device 5, and service information management device 7 may be constructed by a single computer, or may be constructed by multiple computers in which each part (function or means) is divided and assigned arbitrarily.

[0024] Figure 2 is an illustration showing a situation in which a customer at a store and a presenter at a service center are having a remote video call. Figure 2 shows a situation in which customer c2 who has visited an unmanned store US is using store terminal 1c2, and a presenter ug11 from unmanned store priority support organization UG1 is using presenter terminal 9ug11. In this way, customer c2 can receive an explanation of a travel product through remote customer service.

[0025] [Hardware configuration of communication system] Next, using FIG. 3, the hardware configuration of each terminal (store terminal 1, manager terminal 4, presenter terminal 9) and each device (relay device 3, communication management device 5, service information management device 7) that constitute the communication system shown in FIG. 1 will be described. In this embodiment, it is assumed that each terminal and each device have the same configuration, so the hardware configuration of the store terminal 1 will be described and descriptions of the hardware configurations of the other terminals and devices will be omitted.

[0026] As shown in Figure 3, the store terminal 1 is composed of a computer and includes a CPU 101, a ROM 102, a RAM 103, a HD 104, an HDD (Hard Disk Drive) controller 105, a display 106, an external device connection I / F (Interface) 108, a network I / F 109, a data bus 110, a keyboard 111, a pointing device 112, a DVD-RW (Digital Versatile Disk Rewritable) drive 114, and a media I / F 116.

[0027] Of these, the CPU 101 controls the operation of the entire computer. The ROM 102 stores programs used to drive the CPU 101, such as the IPL. The RAM 103 is used as a work area for the CPU 101. The HD 104 stores various data, such as programs. The HDD controller 105 controls the reading and writing of various data from and to the HD 104 under the control of the CPU 101. The display 106 displays various information, such as a cursor, menus, windows, characters, or images. The external device connection I / F 108 is an interface for connecting various external devices. In this case, the external devices are, for example, USB (Universal Serial Bus) memories, printers, etc. The network I / F 109 is an interface for data communication using the communication network 100. The bus line 110 is an address bus, a data bus, etc. for electrically connecting the components, such as the CPU 101, shown in FIG. 3.

[0028] The keyboard 111 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 112 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 114 controls reading and writing of various data from a DVD-RW 113, which is an example of a removable recording medium. Note that the medium is not limited to a DVD-RW, and may be a DVD-R, a Blu-ray Disc, or the like. The media I / F 116 controls reading and writing (storing) of data from a recording medium 115, such as a flash memory.

[0029] Also, a microphone 121 as an example of a sound collection device, a speaker 122 as an example of a sound output device, and a camera 123 as an example of an imaging device are connected to the external device connection I / F. Note that, although the microphone 121, the speaker 122, and the camera 123 are shown as external devices in Fig. 3, they may also be built-in devices without going through the external device connection I / F.

[0030] [Functional configuration of the communication system] Next, each functional configuration of the communication system of this embodiment will be described with reference to Fig. 3 to Fig. 11. Fig. 4 and Fig. 5 are block diagrams of each function of the communication system according to this embodiment.

[0031] <Store terminal functional configuration> First, the functional configuration of the store terminal 1 will be described with reference to FIGS. 3 and 4. As shown in FIG. 4, the store terminal 1 includes a transmitting / receiving unit 11, a receiving unit 12, a communication control unit 13, a display control unit 14, a determination unit 15, and a storage / readout processing unit 19. Each of these units is a function or a means for performing the function, which is realized when any of the components shown in FIG. 3 operates in response to an instruction from the CPU 101 in accordance with a program loaded from the HD 104 onto the RAM 103. The store terminal 1 also includes a memory unit 1000 constructed by the RAM 103 and HD 104 shown in FIG. 3. A presenter management DB 1001 and a skill group management DB 1002 are constructed in the memory unit 1000. The information managed in the presenter management DB 1001 and the skill group management DB 1002 is included in the client information sent from the service information management device 7 in step S19, which will be described later. The store terminal 1 uses the information managed in the presenter management DB 1001 and the skill group management DB 1002 when displaying a screen shown in FIG. 23, which will be described later.

[0032] (Explainer management table) Fig. 6(A) is a conceptual diagram of a presenter management table. A presenter management DB 1001 configured by the presenter management table shown in Fig. 6(A) is constructed in the storage unit 1000. In this presenter management table, various information of destination candidates (presenter terminals 9 in this case) of the store terminal 1 (communication ID, presenter name, operating status (presence) of the destination candidate, and skill name indicating the skill of the presenter who exclusively uses the presenter terminal) are associated and managed.

[0033] The communication ID is identification information for identifying the communication destination in the communication system 1. The communication ID is not particularly limited, but may be, for example, a user ID for identifying the presenter using the presenter terminal 9, a communication ID for identifying a store terminal, etc. The presenter name in FIG. 6(A) is the presenter name shown in FIG. 23(C) described later. The operating status indicates the operating status of the terminal on the other side of the video call. There are three types of operating status: Online (available for communication), Chat (in communication), and Offline (unable to communicate). The operating status is reflected in the displays in FIGS. 23(A) and (C). As mentioned above, the skill indicates the name of the skill possessed by the presenter. As shown in FIG. 6(A), one presenter may have multiple skills (Specialty 1, Specialty 2).

[0034] (Skill group management table) (B) of Fig. 6 is a conceptual diagram showing a skill group management table. A skill group management DB 1002 is constructed in the storage unit 1000 using a skill group management table such as that shown in (B) of Fig. 6. In this skill group management table, skill names, operating status (presence) of skill groups, communication IDs of presenter terminals that can be contacted by a store terminal, and communication IDs of presenter terminals that are currently in communication are associated and managed.

[0035] The operating status of a skill group indicates the overall operating status of each presenter terminal of a group of presenters who have the same skill, such as Specialty 1. For example, if there are presenter terminals of a group of presenters who have the same skill, and if the operating status of at least one of these presenter terminals is Online, the operating status of the Specialty 1 skill group is managed as Online. Also, even if the operating status of all of the presenter terminals is not Online, if the operating status of at least one of these presenter terminals is Chat, the operating status of the Specialty 1 skill group is managed as Chat. Furthermore, if the operating status of all of the presenter terminals is neither Online nor Chat, i.e., if all of the operating statuses are Offline, the operating status of the Specialty 1 skill group is managed as Offline.

[0036] (Functional configuration of store terminal) Next, we will explain each component of the store terminal 1. The transmitting / receiving unit 11 is realized by commands from the CPU 101 shown in Fig. 3, the external device connection I / F 108, and the network I / F 109, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 100.

[0037] The reception unit 12 is realized mainly by commands from the CPU 101 shown in FIG. 3, as well as the keyboard 111 and pointing device 112, and receives various inputs from the user.

[0038] The communication control unit 13 is realized by instructions from the CPU 101 shown in Fig. 3, and inputs image data obtained by an external camera connected to the external device connection I / F 108 by capturing an image of a subject. The communication control unit 13 also inputs sound data obtained by an external microphone connected to the external device connection I / F 108 by collecting sound. The communication control unit 13 also outputs image data received from the other end of the video call and image data obtained by its own terminal to the display control unit 14. The communication control unit 13 also outputs sound data received from the other end of the video call and sound data obtained by its own terminal to an external speaker connected to the external device connection I / F 108.

[0039] The display control unit 14 is mainly realized by instructions from the CPU 101 shown in FIG. 3, and displays an image by outputting image data to the display 106 or an external display connected to the external device connection I / F.

[0040] The determination unit 15 is mainly realized by instructions from the CPU 101 shown in Fig. 3, and makes various determinations, the contents of which will be described later.

[0041] The storage / readout processing unit 19 is mainly executed by instructions from the CPU 101 shown in FIG. 3 and the HDD controller 105, and performs processes such as storing various data in the memory unit 1000 and reading out various data stored in the memory unit 1000.

[0042] Furthermore, image data and sound data received during communication with other terminals are overwritten and stored in the storage unit 1000. Of these, the image data before being overwritten displays an image on the display 106 or an external display, and the sound data before being overwritten outputs sound from an external speaker.

[0043] <Administrator terminal> Next, the functional configuration of the administrator terminal 4 will be described with reference to Fig. 3 and Fig. 4. As shown in Fig. 4, the administrator terminal 4 has a transmission / reception unit 41, a reception unit 42, a display control unit 44, and a storage / readout processing unit 49. In addition, the administrator terminal 4 has a storage unit 4000 constructed by the RAM 103 and HD 104 shown in Fig. 3.

[0044] The transmitter / receiver unit 41, reception unit 42, display control unit 44, and memory / readout processing unit 49 of the manager terminal 4 have the same functions as the transmitter / receiver unit 11, reception unit 12, display control unit 14, and memory / readout processing unit 49 of the store terminal 1, respectively, and therefore their explanations will be omitted.

[0045] <Functional configuration of the presenter terminal> Next, the functional configuration of the presenter terminal 9 will be described with reference to Figures 3 and 4. As shown in Figure 4, the presenter terminal 9 has a transmitting / receiving unit 91, a receiving unit 92, a communication control unit 93, a display control unit 94, a determination unit 95, and a storage / readout processing unit 99. The presenter terminal 9 also has a memory unit 9000 constructed by the RAM 103 and HD 104 shown in Figure 3. A presenter management DB 9001 and a skill group management DB 9002 are constructed in the memory unit 9000.

[0046] The transmitter / receiver unit 91, reception unit 92, communication control unit 93, display control unit 94, judgment unit 95, and storage / readout processing unit 99 of the presenter terminal 9 have the same functions as the transmitter / receiver unit 11, reception unit 12, communication control unit 13, display control unit 14, judgment unit 15, and storage / readout processing unit 19 of the store terminal 1, respectively, and therefore descriptions thereof will be omitted. Also, the presenter management DB 9001 and skill group management DB 9002 of the presenter terminal 9 have the same data structure as the presenter management DB 1001 and skill group management DB 1002 of the store terminal 1, and therefore descriptions thereof will be omitted.

[0047] <Functional configuration of relay device> Next, the functional configuration of the relay device 3 will be described with reference to Figures 3 and 5. As shown in Figure 5, the relay device 3 has a transmitting / receiving unit 31 that also functions as a forwarding unit, a determining unit 32, and a storage / reading processing unit 39. Each of these units is a function or a means for performing a function that is realized when any of the components shown in Figure 3 operates in response to an instruction from the CPU 101 in accordance with a program loaded from the HD 104 onto the RAM 103. The relay device 3 also has a storage unit 3000 constructed by the RAM 103 and HD 104 shown in Figure 3.

[0048] (Functional configuration of relay device) Next, a detailed description will be given of each functional configuration of the relay device 3. The transmitting / receiving unit 31 of the relay device 3 shown in Fig. 5 is realized by commands from the CPU 101 shown in Fig. 3 and the network I / F 109 shown in Fig. 3, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 100. The transmitting / receiving unit 31 also serves as a transfer unit, and transfers image data and sound data transmitted from a predetermined terminal to other terminals.

[0049] The determination unit 32 is realized by an instruction from the CPU 101 shown in FIG. 3, and determines the delay state of data, etc.

[0050] The storage / readout processing unit 39 is realized by instructions from the CPU 101 shown in FIG. 3 and the HDD controller 105 shown in FIG. 3, and performs processing such as storing various data in the storage unit 3000 and reading out various data stored in the storage unit 3000.

[0051] <Functional configuration of the communication management device> Next, the functional configuration of the communication management device 5 will be described with reference to Figures 3 and 5. As shown in Figure 5, the communication management device 5 has a transmitting / receiving unit 51, a selecting unit 53, a creating unit 54, a determining unit 55, and a storing / reading processing unit 59. Each of these units is a function or a means of functioning that is realized when any of the components shown in Figure 3 operates in response to an instruction from the CPU 101 in accordance with a program loaded from the HD 104 onto the RAM 103. The communication management device 5 also has a memory unit 5000 constructed by the RAM 103 and HD 104 shown in Figure 3.

[0052] (Authentication management table) Fig. 6(A) is a conceptual diagram showing an authentication management table. An authentication management DB 5001 configured with an authentication management table such as that shown in Fig. 6(A) is constructed in the storage unit 5000. In this authentication management table, each password is associated with each communication ID of all terminals 1, 9 managed by the communication management device 5 and managed. Note that a password is an example of authentication information, and authentication information also includes an access token.

[0053] (Terminal status management table) 7B is a conceptual diagram showing a terminal status management table. A terminal status management DB 5002 configured by the terminal status management table shown in FIG. 7B is constructed in the storage unit 5000. In this terminal status management table, for each communication ID of each terminal 1, 9, the destination name when each terminal 1, 9 is the destination, the operating status of each terminal 1, 9, the date and time when login request information (described later) was received by the communication management device 5, and the IP address of the store terminal 1, 9 are associated and managed. The communication ID, destination name, and terminal IP address are stored when each terminal 1, 9 pre-registers to receive services provided by the communication management device 5.

[0054] (Address list management table) Fig. 8(A) is a conceptual diagram showing a destination list management table. A destination list management DB 5003 configured with a destination list management table such as that shown in Fig. 8(A) is constructed in the storage unit 5000. In this destination list management table, the communication IDs of all destination terminals registered as candidate destination terminals are associated with the communication ID of the initiating terminal requesting the start of a call and are managed. These destination terminal candidates are updated by being added or deleted in response to an addition or deletion request from the administrator terminal 4 to the communication management device 5.

[0055] The destination list is an example of destination information, and the destination information may not be in a list format, but may instead be a list of information about the destinations, such as communication IDs.

[0056] (Session management table) 8(B) is a conceptual diagram showing a session management table. A session management DB 5004 configured with a session management table such as that shown in FIG. 8(B) is constructed in the storage unit 5000. In this session management table, for each communication session ID for identifying a session for mutual communication between each terminal and the relay device 3, the relay device ID of the relay device 3 used, the communication ID of the initiating terminal, the communication ID of the destination terminal, the reception delay time (ms) when the image data is received at the destination terminal, and the reception date and time when the delay information indicating this delay time is sent from the destination terminal and received by the communication management device 5 are associated and managed.

[0057] (Communication information management table) Fig. 8(C) is a conceptual diagram showing a communication information management table. A communication information management DB 5005 configured with a communication information management table such as that shown in Fig. 8(C) is constructed in the storage unit 5000. In this communication information management table, for each communication ID for identifying mutual communication performed between each terminal and the relay device 3, the IP address of the relay device 3 used and the communication ID of the terminal with which the mutual communication is performed are associated and managed.

[0058] (Image information management table) Fig. 8(D) is a conceptual diagram showing an image information management table. An image information management DB 50006 configured with an image information management table such as that shown in Fig. 8(D) is constructed in the storage unit 5000. In this image information management table, landmark IDs, file names of images of tourist spots, accommodation information, file names of images of the exteriors of accommodation facilities, room grades (ranks) and fees, room IDs, and file names of images of rooms are associated and managed.

[0059] Of these, the landmark ID is tourist attraction identification information for identifying landmarks, which will be described later. The accommodation information is accommodation identification information for identifying accommodation facilities, such as the hotel name. The room ID is room identification information for identifying the grade of the room. The room grade and price, as well as the room ID, are part of the room information. The types of image data are still image data, video image data, and spherical image data. In this embodiment, when assisting in the creation of travel plans, it is assumed that spherical image data (including still images and videos) will be used as background images to create a sense of realism as if the user were actually at the actual location.

[0060] Here, the data of the tourist spot images, the exterior images of the accommodation facilities, and the room images are stored in the storage unit 5000. Note that the data of the tourist spot images, the exterior images of the accommodation facilities, and the room images may be stored in another device (such as a server) instead of the communication management device 5, and the communication management device 5 may acquire the data from the other device.

[0061] (Functional configuration of the communication management device) Next, we will explain in detail each functional configuration of the communication management device 5. In explaining each functional configuration of the communication management device 5, we will also explain the relationship between each component shown in Fig. 3 and the main components that realize each functional configuration of the communication management device 5.

[0062] The transmission / reception unit 51 of the communication management device 5 shown in Figure 5 is realized by instructions from the CPU 101 shown in Figure 3 and the network I / F 109, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 100.

[0063] The selection unit 53 is implemented by an instruction from the CPU 101 shown in FIG. 3, and selects the optimum relay device for intercommunication based on the IP address of each terminal.

[0064] The creation unit 54 is realized by a command from the CPU 101 shown in Fig. 3, and creates communication information in response to a request from each terminal to start mutual communication. This communication information includes the IP address of the relay device selected by the selection unit 53, the above-mentioned communication ID, etc.

[0065] The determination unit 55 is realized by instructions from the CPU 101 shown in FIG. 3, and performs various determinations.

[0066] The storage / readout processing unit 59 is realized by instructions from the CPU 101 and the HDD controller 105 shown in FIG. 3, and performs processing such as storing various data in the storage unit 5000 and reading out various data stored in the storage unit 5000.

[0067] <Service information management device> Next, the functional configuration of the service information management device 7 will be described with reference to Figures 3 and 5. As shown in Figure 5, the service information management device 7 has a transmission / reception unit 71, a determination unit 75, and a storage / readout processing unit 79. Each of these units is a function or a means for performing a function that is realized when any of the components shown in Figure 3 operates in response to an instruction from the CPU 101 in accordance with a program loaded from the HD 104 onto the RAM 103. The service information management device 7 also has a storage unit 7000 constructed by the RAM 103 and HD 104 shown in Figure 3. A service information management DB 7001 is constructed in this storage unit 7000. The service information management DB 7001 contains the tables shown in Figures 9 to 11.

[0068] (System update information management table) 9(A) is a conceptual diagram showing a system update information management table. In this system update information management table, the terminal name, latest version, update application date and time, and download URL (Uniform Resource Locator) are associated and managed for each terminal ID. In the client name column, the store terminal app is an application for the store terminal 1, and the presenter terminal app is an application for the presenter terminal 9. Each application is managed by its download URL.

[0069] (Business management table) 9(B) is a conceptual diagram showing a business operator management table. In this business operator management table, business operator IDs and business operator names are associated and managed. Business operators are, for example, travel agencies, leasing companies, etc.

[0070] (Group management table) 9(C) is a conceptual diagram showing a group management table. In this group management table, a group name (area name) and a service provider ID are associated with each group ID (area ID) and managed. The groups represent areas such as area E1 in FIG. 1.

[0071] (Terminal management table) FIG. 10A is a conceptual diagram showing a terminal management table. In this terminal management table, a destination name, role, login ID, password, provider ID, group ID, and communication ID are associated and managed for each account ID. The account ID is identification information for identifying the store, the presenter, and the administrator. In the case of a store, the account of the store itself, not the customer, is managed. The destination name is also managed in the terminal status management table of FIG. 7B, but is the display name shown on the destination selection screen displayed on the store terminal 1 as shown in FIG. 23C. The role indicates the role of the terminal user, etc. The login ID and password are used for authentication when the administrator ad1 accesses the service information management device 7 from the administrator terminal 4. The "password" in this terminal management table is the same as the "password" in the authentication management table of FIG. 7A, but may be a different password. The communication ID is also managed in the terminal status management table (see FIG. 7A) in the communication management device 5.

[0072] (Skill management table) Fig. 10(B) is a conceptual diagram showing a skill management table. In this skill management table, for each skill ID, a skill name and the account ID of the presenter who possesses this skill are associated and managed. As shown in Fig. 10(B), it is also possible to manage the fact that the same presenter (001) possesses multiple skills, such as Specialty 1 and Specialty 2.

[0073] (Login information management table) 11(A) is a conceptual diagram showing a login information management table. In this login information management table, the version, account ID, and last login date and time are associated and managed for each serial number ID. The version is the version of the client application (each application of the store terminal 1 and the presenter terminal 9) at the time of login. This makes it possible to know which account is using which version when an application is updated.

[0074] (Subgroup management table) FIG. 11(B) is a conceptual diagram showing a subgroup management table. In this subgroup management table, the subgroup name, group ID, and account ID of the presenter belonging to the subgroup are associated and managed for each subgroup ID. As shown in FIG. 11(B), it is also possible to manage the fact that the same presenter (account ID "001") has multiple skills such as Specialty 1 and Specialty 2. In the case of the account ID of the store terminal, as shown in FIG. 1, the subgroup indicates a staffed store or an unstaffed store. Furthermore, in the case of the presenter's account ID, the subgroup indicates a staffed store priority response organization or an unstaffed store priority response organization.

[0075] According to the subgroup management table, for example, the store terminal (Tokyo main store No. 1) with account ID "003" will preferentially transmit connection request information indicating a connection request for a communication session for a video call to the presenter terminals used by the presenters with account IDs "001" and "002" simultaneously (first transmission) as a preliminary step to a video call start request (see S101 in FIG. 25) (see S84-1 and S85-1 in FIG. 24 described later). Then, if there is no response from the presenter terminals of the presenters with account IDs "001" and "002" within a certain time period (see S204 in FIG. 25 described later), the store terminal (Tokyo main store No. 1) with account ID "003" will transmit a call start request simultaneously (second transmission) to predetermined presenter terminals that are online (available for calls) among the presenter terminals of presenters other than those with account IDs "001" and "002" and who have the same skills (see S206 in FIG. 25 described later).

[0076] (User update information management table) 11C is a conceptual diagram showing a user update information management table. In this user update information management table, the account ID to be updated, the scheduled update version, the update application date and time, and the download URL are associated and managed for each update ID.

[0077] (Functional configuration of the service information management device) Next, a detailed description will be given of each functional configuration of the service information management device 7. In the following, when describing each functional configuration of the service information management device 7, the relationship between each component shown in Fig. 3 and the main components for realizing each functional configuration of the service information management device 7 will also be described.

[0078] The transmission / reception unit 11 of the service information management device 7 shown in Figure 5 is realized by instructions from the CPU 101 shown in Figure 3 and the network I / F 109, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 100.

[0079] The determination unit 75 is realized by instructions from the CPU 101 shown in FIG. 3, and makes various determinations.

[0080] The storage / readout processing unit 79 is realized by instructions from the CPU 101 and the HDD controller 105 shown in FIG. 3, and performs processing such as storing various data in the storage unit 7000 and reading out various data stored in the storage unit 7000.

[0081] [Processing or Operation of the Embodiment] Next, the processing or operation in the communication system according to this embodiment will be described with reference to Figures 12 to 26. In the following, the store terminal 1 as the request source requesting the start of a video call may be referred to as the "start terminal," and the presenter terminal 9 as the destination (relay destination) to which the request is made may be referred to as the "destination terminal."

[0082] <Service information registration process> 12 to 19, a process for registering various types of service information from the administrator terminal 4 to the service information management device 7 will be described. The service information is managed in the tables shown in FIGS.

[0083] Fig. 12 is a sequence diagram showing the service information registration process. Fig. 13 is a diagram showing a login screen displayed on the administrator terminal. Fig. 14 is a diagram showing a store list screen displayed on the administrator terminal. Fig. 15 is a diagram showing a presenter list screen displayed on the administrator terminal. Fig. 16 is a diagram showing a group list screen displayed on the administrator terminal. Fig. 17 is a diagram showing a presenter information editing screen displayed on the administrator terminal. Fig. 18 is a diagram showing a group information editing screen displayed on the administrator terminal. Fig. 19 is a diagram showing a store information editing screen displayed on the administrator terminal.

[0084] 12, the administrator ad1 uses the administrator terminal 4 to access the service information management device 7 via the communication network 100, causing the web browser (display control unit 44) of the administrator terminal 4 to display the screens shown in FIGS. 13 to 19 on the display 106, and the transmitting / receiving unit 41 to transmit each piece of service information input or selected by the administrator ad1 on each screen to the service information management device 7 (S1). Then, in the service information management device 7, the transmitting / receiving unit 71 receives each piece of service information, and the storage / readout processing unit 79 stores each piece of service information in the service information management DB 7001. As a result, each piece of information is managed in each table shown in FIGS. 9 to 11.

[0085] The screens are displayed in the order of Fig. 13 to Fig. 19. For example, on the group information editing screen shown in Fig. 18, the administrator ad1 inputs the account ID of the presenter of the presenter terminal 9 to which the administrator wants to give priority in transmitting a call start request in the "Presenter" field of the subgroup (for unmanned store). Furthermore, on the store information editing screen shown in Fig. 19, the administrator inputs the subgroup name (here, for unmanned store) in the "Subgroup to call with priority" field. This allows the store terminal 1 to acquire service information from the service information management device 7 (see S19 in Fig. 20 described later), thereby managing the presenter terminal 9 to which the store terminal 1 should give priority in transmitting a call start request (see S20).

[0086] <Video call preparation process> Next, the process of preparing for a video call for remote customer service will be described with reference to Fig. 20 to Fig. 23. Fig. 20 is a sequence diagram showing the process of preparing for a call by the store terminal.

[0087] First, when a customer who visits a store turns on the power switch of the store terminal 1, the reception unit 12 receives the power-on command and starts up the store terminal 1 (S11). When the store terminal 1 starts up, the communication app installed in the store terminal 1 starts up (S12). Thereafter, processing in the store terminal 1 is executed according to commands from the communication app. Then, the storage / readout processing unit 19 reads local setting information from the storage unit 1000 (S13). This completes the store terminal 1's preparation for setting the background image of the display screen, the login ID and password, etc.

[0088] Next, the transmitter / receiver 11 of the store terminal 1 transmits a login request to the service information management device 7 via the communication network 100 (S14). This login request includes the login ID and password read in step S13. As a result, the transmitter / receiver 71 of the service information management device 7 receives the login request. By transmitting the login request from the store terminal 1 to the service information management device 7, the service information management device 7, which is the receiving side, can obtain the IP address of the store terminal 1, which is the sending side.

[0089] Next, the judgment unit 75 of the service information management device 7 searches the terminal management table (FIG. 10(A)) of the service information management DB 7001 using the login ID and password included in the login request as search keys, and performs authentication by determining whether the same login ID and password are managed in this terminal management table (S15).

[0090] Then, the transmitting / receiving unit 71 of the service information management device 7 transmits authentication result information indicating the authentication result to the store terminal 1 (S16). As a result, the transmitting / receiving unit 11 of the store terminal 1 receives the authentication result information.

[0091] Next, the transmitting / receiving unit 11 of the store terminal 1 transmits a request for the service information to the service information management device 7 (S17). As a result, the transmitting / receiving unit 71 of the service information management device 7 receives the request for the service information.

[0092] Next, in the service information management device 7, the storage / readout processing unit 79 reads out the service information from the service information management DB 7001 (S18). Then, the transmission / reception unit 71 transmits the service information to the store terminal 1 (S19). As a result, the transmission / reception unit 11 of the store terminal 1 receives the service information. Next, in the store terminal 1, the storage / readout processing unit 19 sets the service information (S20).

[0093] <Video call start process> First, the creation unit 54 of the store terminal 1 creates a session ID for identifying its own communication session (S22). Next, the transmitting / receiving unit 11 transmits login request information indicating a login request to the communication management device 5 via the communication network 100 (S23). This login request information includes the session ID created in step S22, and the communication ID and password of the store terminal itself in the service information acquired in step S19. Note that here, the communication ID is a start terminal ID for identifying the store terminal 1 that transmitted the login request in step S23.

[0094] Next, the storage / readout processing unit 59 of the communication management device 5 searches the authentication management table (see FIG. 7(A)) using the communication ID and password included in the login request information received via the transmission / reception unit 51 as search keys, and determines whether the same communication ID and the same password are managed, thereby performing terminal authentication (S24). Here, the following description will be given assuming that the same communication ID and the same password are managed by the storage / readout processing unit 59.

[0095] If the storage / readout processing unit 59 determines that the login request is from a terminal with valid usage authority, assuming that the same communication ID and the same password are managed, the storage / readout processing unit 59 changes the operation status field of the record of the communication ID received in step S23 to "Online (available for calls)" in the terminal status management table (see FIG. 7(B)), and stores the reception date and time when the login request was received in step S23 in the reception date and time field (S25). As a result, for example, the terminal status management table manages the communication ID "01aa" in association with the operation status "Online (available for calls)," the reception date and time "2019.4.10.13:40," and the IP address "1.2.1.4." Note that the IP address of the terminal does not need to be registered in advance, but may be transmitted from the store terminal 1 in step S23.

[0096] Next, the storage / readout processing unit 59 adds a new record including the communication ID and password of the store terminal 1 received in step S23 to the session management table (see FIG. 8(B)) and manages it (S26). Then, the transmitting / receiving unit 51 of the communication management device 5 transmits authentication result information indicating the authentication result obtained by the processing of step 24 to the store terminal 1 that made the login request via the communication network 100 (S27).

[0097] When the transmitting / receiving unit 11 of the login request terminal (store terminal 1) receives authentication result information indicating that the terminal has been determined to be a terminal with valid usage authority, the transmitting / receiving unit 11 transmits destination list request information indicating a request for a destination list to the communication management device 5 via the communication network 100 (S28). As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the destination list request information.

[0098] Next, the storage / readout processing unit 59 searches the destination list management table (see FIG. 8(A)) using the communication ID "01ab" of the login request terminal (store terminal 1) as a search key, reads out the communication IDs of destination candidates that can communicate with the login request terminal (store terminal 1), and reads out the destination names corresponding to these communication IDs from the terminal status management table (see FIG. 7(B)) (S29). Here, the communication IDs of the destination candidates that correspond to the communication ID "01ab" of the login request terminal (store terminal 1) and the destination names corresponding to these are extracted.

[0099] Next, the storage and readout processing unit 59 of the communication management device 5 reads out the data of the destination list frame from the memory unit 5000 (S30). Then, the transmitting and receiving unit 51 transmits "destination list information (destination list frame, communication ID, destination name)" including this destination list frame and the communication ID and destination name read out by the storage and readout processing unit 59 to the login request terminal (store terminal 1) (S31). As a result, in the login request terminal (store terminal 1), the transmitting and receiving unit 11 receives the destination list information, and the storage and readout processing unit 19 stores the destination list information in the memory unit 1000 (S32).

[0100] As described above, in this embodiment, rather than each terminal managing the destination list information, the communication management device 5 centrally manages the destination list information for all terminals. Therefore, even if a new terminal is added to the communication system, a new model of terminal is added to replace an existing terminal, or the appearance of the destination list frame needs to be changed, this can be handled collectively on the communication management device 5 side, eliminating the need to change the destination list information on each terminal side.

[0101] In addition, the storage / readout processing unit 59 of the communication management device 5 searches the terminal status management table (see Figure 7(B)) using the communication ID of the destination candidate read out above as a search key, and obtains each operating status of the communication terminal as a destination candidate by reading out the corresponding operating status for each of the communication IDs above (S33).

[0102] Next, the transmitter / receiver 51 transmits "terminal status information" including each communication ID used as a search key in step S26 above and the operating status of each corresponding destination terminal to the starting terminal (store terminal 1) via the communication network 100 (S34).

[0103] Next, the storage / readout processing unit 19 of the starting terminal (store terminal 1) sequentially stores the terminal status information received from the communication management device 5 in the storage unit 1000 (S35). Thus, by receiving the status information of each of the above terminals, the starting terminal (store terminal 1) can obtain the current operating status of each of the presenter terminals 9 and other terminals that are potential destinations that can communicate with the starting terminal (store terminal 1).

[0104] Next, the display control unit 14 of the login request terminal (store terminal 1) creates a destination list that reflects the status of the terminal as a destination candidate based on the destination list information and terminal status information stored in the storage unit 1000 (S36). This destination list is later displayed on the store terminal 1 as shown in FIG. 23(C).

[0105] Meanwhile, the storage / readout processing unit 59 of the communication management device 5 searches the destination list management table (see FIG. 8(A)) based on the communication ID of the login request terminal (store terminal 1) to extract the communication IDs of other terminals that have registered the communication ID of the login request terminal (store terminal 1) as a destination candidate (S37).

[0106] Next, the storage / readout processing unit 59 of the communication management device 5 searches the terminal status management table (see FIG. 7(B)) based on the communication ID of the login request terminal (store terminal 1) and acquires the operating status of the login request terminal (store terminal 1) (S38).

[0107] Then, the transmitter / receiver 51 transmits "terminal status information" including the communication ID and operating status of the login request terminal (store terminal 1) acquired in step S38 to terminals having an operating status of "Online" in the terminal status management table (see FIG. 7(B)) among the terminals associated with the communication IDs extracted in step S37 (S39). When transmitting the terminal status information to the presenter terminal 9, the transmitter / receiver 51 references the IP addresses of the communication terminals managed in the terminal status management table (see FIG. 7(B)) based on each communication ID. This allows the communication ID and operating status of the login request terminal (store terminal 1) to be communicated to each of the other destination terminals that can communicate with the login request terminal (store terminal 1) as a destination candidate. This allows the destination candidate (presenter terminal 9) to display the status of the destination candidate as shown in FIG. 23(C) (S40).

[0108] <Processing for determining the operating status of presenter terminals> Next, a process in which the store terminal 1 determines the operating status of the group of presenter terminals for each skill will be described with reference to FIG. 22. FIG. 22 is a flowchart showing the preparation process for a video call of the store terminal, and in particular the process of determining the operating status of the group of presenter terminals for each skill. The operating status of a skill group, "Online," indicates that at least one of the destination candidates of this skill group has a presence of "Online" that allows communication. The operating status of a skill group, "Chat," indicates that none of the destination candidates of this skill group have an operating status of "Online," and that at least one destination candidate has a presence of "Chat" that is currently in communication. Furthermore, the operating status of a skill group, "Offline," indicates that all of the destination candidates of this skill group have an operating status of "Offline." An example of the process of determining the operating status of a skill group will be described below.

[0109] 21, the storage / readout processing unit 19 of the store terminal 1 acquires information recorded in the updated record each time the operating status of the destination management table is updated (S51). The acquired information includes the communication ID of the presenter terminal 9ug11, the operating status indicating the latest status of the presenter terminal 9ug11 that is a destination candidate, and the skill name indicating the skill of the presenter who uses the presenter terminal 9ug11 that is a destination candidate.

[0110] The storage and readout processing unit 19 of the store terminal 1 updates each record in the skill group management DB 1002 that includes the skill name acquired in step S51. That is, if the acquired skill name is "Specialty 1, Specialty 2," the storage and readout processing unit 19 updates each record in the skill group management DB 1002 that includes the skill name "Specialty 1, Specialty 2."

[0111] As an example, the following describes the processing when operating status information (operating status "Online", skill name (Specialty 1, Specialty 2), communication ID "01ba") is sent from the presenter terminal 9ug11 to the store terminal 1 and this information is acquired in step S51 (S52; Online).

[0112] First, the process of updating a record in which the skill name "Specialty 1" is recorded in a table of the skill group management DB 1002 in the loop processing will be described. The storage / read processing unit 19 of the shop terminal 1 references the field of the communication ID that can be used for calls in the record in which the skill name "Specialty 1" is recorded in the skill group management DB 1002 (see (B) of FIG. 6). As a result, the judgment unit 15 judges whether the communication ID "01ba" acquired in step S51 is included in the field of the communication ID that can be used for calls (S53).

[0113] If it is determined that the communication ID "01ba" acquired in step S51 is not included in the call-enabled field in the skill group management DB 1002 (S53; NO), the storage / readout processing unit 19 adds the acquired communication ID "01ba" to the call-enabled field (S54).

[0114] When the processing of step S54 is completed, or when the answer is YES in step S53, the storage / read processing unit 19 of the store terminal 1 refers to the in-call field of the record in which the skill name "Specialty 1" is recorded in the skill group management DB 1002 (see FIG. 6B). As a result, the judgment unit 15 judges whether the in-call field contains the communication ID "01ba" acquired in step S51 (S55).

[0115] If it is determined that the communication ID "01ba" acquired in step S51 is included in the in-call field in the skill group management DB1002 (S55; YES), the storage / read processing unit 19 deletes the acquired communication ID from the in-call field (S56). Through the processing up to this point, whether the operating status of the presenter terminal 9ug11 transitions from "Chat" to "Online" or from "Offline" to "Online", the communication ID of the presenter terminal 9ug11 is recorded only in the available-for-call field in the record in the skill group management DB1002 in which the skill name "Specialty 1" is recorded.

[0116] When the processing of step S56 is completed, or if the answer is NO in step S55, the storage / readout processing unit 19 of the store terminal 1 refers to the operating status of the group in the record in which the skill name "Specialty 1" is recorded in the skill group management DB 1002 (see FIG. 6B). As a result, the determination unit 15 determines whether the operating status of the group is "Online" (S57).

[0117] If it is determined that the operating status of the group in the record in which the skill name "Specialty 1" is recorded is not "Online" (S57; NO), the storage and readout processing unit 19 updates this operating status to "Online" (S58). As a result, when displaying the screen shown in FIG. 23(A), the display control unit 14 performs a normal display that clearly appears, for example, "Domestic Travel." On the other hand, if it is determined that the operating status of the group in the record in which the skill name "Specialty 1" is recorded is "Online" (S57; YES), the storage and readout processing unit 19 does not update the operating status of the group in the record in which the skill name "Specialty 1" is recorded. Through the processing up to this point, the operating status of the group in the record in which the skill name "Specialty 1" is recorded in the skill group management DB 1002 becomes "Online."

[0118] Since the skill names "Specialty 1, Specialty 2" are acquired in step S51, the storage / read processing unit 19 similarly updates each record in the skill group management DB 1002 in which the skill name "Specialty 2" is recorded in the loop processing.

[0119] Next, the operating status information (operating status "Chat", skill name (Specialty 1, Specialty 2), communication ID "01ba") is sent from the presenter terminal 9ug11 to the store terminal 1, and the processing when this operating status information is acquired in step S51 will be explained (S52; Chat).

[0120] First, the process of updating the record in the skill group management DB 1002 in which the skill name "Specialty 1" is recorded in the loop process will be described. The storage / read processing unit 19 of the shop terminal 1 references the "Call Allowed" field in the record in which the skill name "Specialty 1" is recorded in the table of the skill group management DB 1002. As a result, the determination unit 15 determines whether the "Call Allowed" field contains the communication ID "01ba" acquired in step S51 (S63). If it is determined that the communication ID "01ba" acquired in step S51 is included in the call-enabled field in the skill group management DB 1002 (S63; YES), the storage / readout processing unit 19 deletes the acquired communication ID "01ba" from the call-enabled field (S64).

[0121] When the processing of step S64 is completed, or when the result of step S63 is NO, the storage / read processing unit 19 of the store terminal 1 refers to the field of the call in progress in the record in which the skill name "Specialty 1" is recorded in the skill group management DB 1002. As a result, the determination unit 15 determines whether the communication ID "01ba" acquired in step S51 is included in the field of the call in progress (S65).

[0122] If it is determined that the communication ID "01ba" acquired in step S51 is not included in the in-call field in the skill group management DB 1002 (S65; NO), the storage / read processing unit 19 adds the acquired communication ID "01ba" to the in-call field (S66). By the processing up to this point, the communication ID of the presenter terminal 9ug11 is recorded only in the in-call field of the record in which the skill name "Specialty 1" is recorded in the skill group management DB 1002.

[0123] When the processing of step S66 is completed, or when the answer is YES in step S65, the storage and readout processing unit 19 of the shop terminal 1 refers to the "call available" field of the record in which the skill name "Specialty 1" is recorded in the skill group management DB 1002. As a result, the storage and readout processing unit 19 determines whether the number of communication IDs recorded in the "call available" field of the record in which the skill name "Specialty 1" is recorded is 0 (zero) (S67).

[0124] If the number of communication IDs recorded in the "Call Allowed" field in the record in which the skill name "Specialty 1" is recorded is 0 (zero) (S67; YES), the storage and readout processing unit 19 updates the operating status of the group in the record in which the skill name "Specialty 1" is recorded to "Chat" (S68). As a result, when displaying the screen shown in FIG. 23(A), the display control unit 14 performs a semi-transparent display such as "Wedding." On the other hand, if the number of communication IDs recorded in the "Call Allowed" field in the record in which the skill name "Specialty 1" is recorded is not 0 (zero) (S67; NO), the storage and readout processing unit 19 does not update the operating status of the group in the record in which the skill name "Specialty 1" is recorded. In other words, if a communication ID is recorded in the "Call Allowed" field in the record in which the skill name "Specialty 1" is recorded, the corresponding operating status remains "Online" and is not updated.

[0125] Since the skill names "Specialty 1, Specialty 2" are acquired in step S51, the storage / read processing unit 19 similarly updates each record in the skill group management DB 1002 in which the skill name "Specialty 2" is recorded in the loop processing.

[0126] Next, we will explain the case where operating status information (operating status "Offline", skill names "Specialty 3, Specialty 4", communication ID "01ca") is sent from the presenter terminal 9ug21 to the store terminal 1, and this operating status information is acquired in step S51 (S52; Offline).

[0127] First, the process of updating the record in the skill group management DB 1002 in which the skill name "Specialty 3" is recorded in the loop process will be described. The storage / read processing unit 19 of the store terminal 1 references the "Call Allowed" field in the record in which the skill name "Specialty 3" is recorded in the table of the skill group management DB 1002. As a result, the determination unit 15 determines whether the "Call Allowed" field contains the communication ID "01ca" acquired in step S51 (S73).

[0128] If it is determined that the communication ID "01ca" acquired in step S51 is included in the call-allowed field in the skill group management DB 1002 (S63; YES), the storage / readout processing unit 19 deletes the acquired communication ID "01ca" from the call-allowed field (S74).

[0129] When the processing of step S74 is completed, or when the answer is NO in step S73, the storage / read processing unit 19 of the store terminal 1 refers to the field of the call in progress in the record in which the skill name "Specialty 3" is recorded in the table of the skill group management DB 1002. As a result, the determination unit 15 determines whether the communication ID "01ca" acquired in step S51 is included in the field of the call in progress (S75).

[0130] If it is determined that the communication ID "01ca" acquired in step S51 is included in the in-call field in the skill group management DB 1002 (S75; YES), the storage / read processing unit 19 deletes the acquired communication ID "01ca" from the in-call field (S76). By the processing up to this point, the communication ID of the presenter terminal 9ug21 is deleted from the record in the skill group management DB 1002 in which the skill name "Specialty 3" is recorded.

[0131] When the processing of step S76 is completed, or when the answer is NO in step S75, the storage / readout processing unit 19 of the shop terminal 1 refers to the available to call and in-call fields of the record in which the skill name "Specialty 3" is recorded in the skill group management DB 1002. As a result, the storage / readout processing unit 19 determines whether conditions IF1 and IF2 are met (S77). Condition IF1 is when the number of communication IDs recorded in the available to call field in the record in which the skill name "Specialty 1" is recorded is 0 (zero), and the number of communication IDs recorded in the in-call field is not 0 (zero). Condition IF2 is when the number of communication IDs recorded in the available to call field in the record in which the skill name "Specialty 3" is recorded is 0 (zero), and the number of communication IDs recorded in the in-call field is 0 (zero).

[0132] If the number of communication IDs recorded in the "Available for Call" field in the record in which the skill name "Specialty 3" is recorded is 0 (zero) and the number of communication IDs recorded in the "In Call" field is not 0 (zero) (S77; IF1), the storage and readout processing unit 19 updates the operating status of the group in the record in which the skill name "Specialty 3" is recorded in the skill group management DB 1002 to "Chat" (S78). If the number of communication IDs recorded in the "Available for Call" field in the record in which the skill name "Specialty 3" is recorded is 0 (zero) and the number of communication IDs recorded in the "In Call" field is 0 (zero) (S77; IF2), the storage and readout processing unit 19 updates the operating status of the group in the record in which the skill name "Specialty 3" is recorded in the skill group management DB 1002 to "Offline" (S79). As a result, when displaying the screen shown in FIG. 23(A), the display control unit 14 performs a semi-transparent display such as "Wedding." On the other hand, if neither IF1 nor IF2 applies in step S77 (S77; NO), the storage / readout processing unit 19 does not update the operating status of the group in the record in which the skill name "Specialty 3" is recorded, and leaves it as "Online."

[0133] Since the skill names "Specialty 3, Specialty 4" are acquired in step S51, the storage / readout processing unit 19 similarly updates each record in the skill group management DB 1002 in which the skill name "Specialty 4" is recorded in the loop processing.

[0134] <Video communication start process> Next, the process of starting video communication for remote customer service will be described with reference to Fig. 23 and Fig. 24. Fig. 23 is a diagram showing a screen displayed on a store terminal. Fig. 24 is a sequence diagram showing the process of starting video communication.

[0135] At the store terminal 1, the display control unit 14 displays one of the screens shown in Figures 23(A) to 23(D) on the display 106 of the store terminal 1, and the reception unit 12 accepts the customer's selection of a skill group or the designation of an instructor (S82).

[0136] Specifically, the storage / readout processing unit 19 acquires a set of each skill name stored in the table (see FIG. 6(B)) of the skill group management DB 1002 and the operating status of the skill group corresponding to this skill name. As a result, the display control unit 14 displays the skill selection screen shown in FIG. 23(A) on the display 106 of the store terminal 1.

[0137] On the right side of the skill selection screen, skill display buttons (domestic travel, international travel, etc.) for selecting a skill and a presenter list display button for displaying a list of all presenter destinations are displayed. Here, the "domestic travel" skill display button is normally displayed and can be selected by the customer. In contrast, the "international travel" skill display button is displayed translucently and cannot be selected by the customer. For example, when a customer selects the "domestic travel" skill display button, the reception unit 12 accepts the selection, and the display control unit 14 displays the confirmation screen shown in FIG. 23(B) on the display 106. When the customer presses the "OK" button, the reception unit 12 accepts the press, and the transmission / reception unit 11 transmits connection request information indicating a connection request for a communication session for a video call to the desired presenter terminal 9 via the communication management device 5 (see S84-1 and S85-1 described later).

[0138] On the other hand, when the customer presses the presenter list display button on the skill selection screen shown in FIG. 23(A), the reception unit 12 accepts the press, and the display control unit 14 displays a destination list of presenters as shown in FIG. 23(C) on the display 106 based on the presenter names, the operating status of the presenter terminals, and the skill names stored in the table of the presenter management DB 1001 (see FIG. 6(A)). Here, the customer selects the name of the presenter who uses the desired presenter terminal from among the presenter terminals available for communication and presses the "OK" button. As a result, the reception unit 12 accepts the selection of the presenter, and the display control unit 14 displays the confirmation screen shown in FIG. 23(D) on the display 106. Then, when the customer presses the "OK" button, the reception unit 12 accepts the press, and the transmission / reception unit 11 transmits a request to start a video call to the designated presenter terminal 9 via the communication management device 5 (see the processing of S90, described later, in FIG. 25).

[0139] Next, in the store terminal 1, when the customer selects a desired skill group (e.g., an organization that can explain domestic travel) on the skill selection screen of FIG. 23(A), the determination unit 15 determines whether to send a request to start a video call to an online (call-enabled) presenter terminal in a specific skill group (e.g., domestic travel) of the unmanned-store priority organization UG1, or to send a request to start a video call to an online (call-enabled) presenter terminal in the same specific skill group (e.g., domestic travel) of the manned-store priority organization MG1 (S83). In this case, the store terminal 1 makes the determination based on the subgroup name to which its own account ID belongs in the subgroup management table (see FIG. 11(B)) in the service information received in step S19. If its own terminal (store terminal 1) belongs to an unmanned store, the determination unit 15 determines the destination of the start request to be the specific skill group of the unmanned-store priority organization UG1. On the other hand, if the terminal (store terminal 1) belongs to a staffed store, the determination unit 15 determines the destination of the start request to be a specific skill group of the staffed store priority response organization MG1.

[0140] Next, in Figure 1, when the store terminal 1 is the store terminal 1c2 installed in the unmanned store US, we will explain the case where a request to start a video call is sent (first time) to the online (call-enabled) presenter terminals 9ug11, 9ug12 of the organization SR21 responsible for skill SK1 of the unmanned store priority response organization UG1.

[0141] First, the transmitter / receiver 11 of the store terminal 1c2 simultaneously transmits connection request information indicating a connection request for a communication session for a video call (S84-1) for the first time as a preliminary step to a request to start a video call (see S101 in FIG. 25). This connection request information includes the communication IDs whose corresponding operating status is "Online" among the communication IDs read out in step S83, the communication ID of the store terminal 1c2 that originated the start request, and the skill name indicating the skill selected in step S82.

[0142] Next, when the transmitter / receiver unit 51 of the communication management device 5 receives the connection request information sent by the store terminal 1c2, it sends this connection request information to the presenter terminal 9ug11 identified by the communication ID included in the received connection request information (S84-2).

[0143] If the operating status is updated in step S34 (see FIG. 21) between when the store terminal 1c2 transmits the connection request information and when it receives a response, the store terminal 1c2 repeats the processes of steps S82 and S83. As a result, the operating status of the presenter terminal 9 is updated to "Online", and if a communication ID is extracted as a communication ID whose corresponding operating status is "Online", the transmitter / receiver 11 of the store terminal 1c2 transmits connection request information including the communication ID to the communication management device 5 (S85-1). Note that the connection request information transmitted here also includes the skill name indicating the skill selected in step S82 and the communication ID of its own terminal.

[0144] When the transmitter / receiver unit 51 of the communication management device 5 receives the connection request information sent by the store terminal 1, it sends this connection request information to the presenter terminal 9ug12 identified by the communication ID included in the received connection request information (S85-2).

[0145] The transmitter / receiver 91 of each presenter terminal 9ug11, 9ug12 receives connection request information transmitted from the store terminal 1c2 via the communication management device 5. When this connection request information is received, the display control unit 94 of each presenter terminal 9ug11, 9ug12 displays on the display 106 of each presenter terminal 9ug11, 9ug12 a message including request source information corresponding to the communication ID included in the connection request information and a skill name (skill information) corresponding to the skill name included in the call start request. By indicating the skill in the message, the presenter at each presenter terminal 9ug11, 9ug12 can understand in advance what kind of travel product the customer wants.

[0146] Next, when each presenter ug11, ug12 performs an operation input for response permission at each presenter terminal 9ug11, 9ug12, the reception unit 92 of each presenter terminal 9ug11, 9ug12 receives the operation input for response permission (S86-1, S86-2). When the operation input for response permission is received, the transmission / reception unit 91 of each presenter terminal 9ug11, 9ug12 transmits response information indicating connection permission to the communication management device 5 (S87-1, S87-2). This response information includes the communication ID of the sender (store terminal 1c2) and the communication ID of the terminal itself (9ug11, 9ug12). As a result, the transmission / reception unit 51 of the communication management device 5 receives the response information transmitted by each presenter terminal 9ug11, 9ug12. The above process is an example, and the transmitter / receiver unit 91 of each presenter terminal 9ug11, 9ug12 may automatically transmit response information to the store terminal 1c2 when it receives connection request information from the store terminal 1c2. Furthermore, when it receives multiple pieces of connection request information from multiple store terminals 1, the transmitter / receiver unit 11 of each presenter terminal 9ug11, 9ug12 may transmit response information to the store terminal 1 that sent the connection request information that was received first.

[0147] Next, the transmitting / receiving unit 51 of the communication management device 5 transmits the received response information to the store terminal 1c2 specified by the communication ID included in this response information (S87-2, S88-2).

[0148] When the transmitter / receiver 11 of the store terminal 1c2 receives multiple pieces of response information, the determination unit 15 determines the presenter terminal 9 that transmitted the earliest response information (S89). Hereinafter, in this embodiment, a case where the presenter terminal 9ug11 responds first will be described.

[0149] The transmitter / receiver unit 51 of the communication management device 5 controls the establishment of a communication session for transmitting image data and audio content data via the relay device 30 between the presenter terminal 9ug11 and the store terminal 1, identified by the communication IDs ("store terminal communication ID, presenter communication ID") included in the communication establishment request, to conduct a video call (S90). This step S90 will be described in detail later with reference to FIG. 25. Note that the communication session is not limited to the above and may be a session for directly transmitting content data between the presenter terminal 9ug11 and the store terminal 1c2 via the communication network 100. Once communication between the store terminal 1c2 and the presenter terminal 9ug11 is established, each store terminal 1c2 and the presenter terminal 9ug11 transmits image data of an image captured at the respective terminal and audio data collected at the respective terminal to the other terminal via the relay device 3. This allows a video call to be initiated between the customer and the presenter, as shown in FIG. 2.

[0150] When communication is established in step S90, the transmitter / receiver unit 91 of the presenter terminal 9ug11 transmits to the communication management device 5 status information including the operating status "Chat", the skill name "Specialty 1", and the communication ID of the presenter terminal 9ug11.

[0151] Furthermore, when communication is established in step S90, the transmitter / receiver 11 of the store terminal 1c2 transmits cancellation information to the communication management device 5 in order to cancel the call start request for the sender of the response other than the sender of the first received response (S91-1). The cancellation information includes the communication ID of its own terminal (store terminal 1c2) and the communication ID of the presenter terminal 9 (here, presenter terminal 9ug12) of the sender of the response other than the sender of the first received response.

[0152] When the transmitting / receiving unit 51 of the communication management device 5 receives the cancellation information, it transmits the cancellation information to the presenter terminal 9ug12 identified by the communication ID included in the cancellation information (S91-2). The transmitting / receiving unit 11 of the presenter terminal 9ug12 receives the cancellation information transmitted by the communication management device 5. This allows the presenter terminal 9ug12 to understand that it is no longer necessary to respond to the call start request from the store terminal 1. <Communication establishment control process> Next, the communication establishment control process (see S90) of Fig. 24 will be described in detail with reference to Fig. 25. Fig. 25 is a sequence diagram showing the detailed communication establishment control process.

[0153] First, the store terminal 1c2 transmits start request information indicating a request to start video communication to the communication management device 5 (S101). This start request information includes the communication ID of the start terminal (here, the store terminal 1c2) and the communication ID of the presenter ug11 of the destination terminal (here, the presenter terminal 9ug11). As a result, the communication management device 5 receives the start request information.

[0154] Next, in the communication management device 5, the selection unit 53 selects the nearest relay device based on the IP addresses of the starting terminal and the destination terminal (S102). The IP addresses of each terminal are managed in a terminal management table, and the IP addresses of each relay device are pre-stored in the storage unit 5000. Next, the creation unit 54 creates communication information including the IP address of the relay device and the above-mentioned communication ID (S103). Then, the storage / readout processing unit 59 manages the communication information by storing the communication ID and the IP address of the relay device, which are the contents of the communication information, and the communication IDs of the presenters of the starting terminal and the destination terminal received in step S101 in the communication information management DB 5005 (see FIG. 8(C)), in association with each other. Thereafter, the transmission / reception unit 51 transmits start request information to the destination terminal (here, the presenter terminal 9ug11) (S105). This start request information includes the communication information in addition to the communication IDs sent from the starting terminal in step S101. As a result, the transmission / reception unit 91a of the destination terminal receives the start request information.

[0155] Next, at the destination terminal, the determination unit 95 of the presenter terminal 9ug11 determines whether the communication ID of the initiating terminal (store terminal 1c2) received in step S105 is the same as the communication ID of the store terminal 1c2 already received in the processing of step S84-2 in FIG. 24 (S106). In this case, since they are basically the same, the transmission / reception unit 91a transmits response possible information to the communication management device 5 (S107). In this case, the response possible information indicates that a response is possible and includes the communication ID of the initiating terminal. As a result, the transmission / reception unit 51 of the communication management device 5 receives the response possible information.

[0156] Next, the storage / read processing unit 59 of the communication management device 5 searches the communication information management DB 5005 using the communication ID of the starting terminal received in step S107 as a search key, and reads out the corresponding communication information (communication ID, relay device IP address) (S108). Then, the transmitting / receiving unit 51 transmits response possible information to the starting terminal. The response possible information in this case includes the communication information read out in step S108. As a result, the transmitting / receiving unit 11 of the starting terminal receives the response possible information.

[0157] Next, the transmitting / receiving unit 91 of the destination terminal transmits establishment request information indicating a request to establish a mutual communication session to the IP address of the relay device included in the communication information received in step S105 (S110). This establishment request information includes the communication ID of the initiating terminal and the communication ID of the presenter of the destination terminal. The IP address of the destination terminal, which is the sender, is also transmitted. Meanwhile, the transmitting / receiving unit 11 of the initiating terminal transmits establishment request information indicating a request to establish a mutual communication session to the IP address of the relay device included in the communication information received in step S109 (S111). This establishment request information includes the communication ID of the initiating terminal and the communication ID of the presenter of the destination terminal. The IP address of the initiating terminal, which is the sender, is also transmitted. As a result, a communication session for mutual communication of images and sounds is established between the relay device 3 and the initiating terminal (S112-1), and a communication session for mutual communication of images and sounds is also established between the relay device 3 and the destination terminal (S112-2).

[0158] Next, the transmitting / receiving unit 11 of the initiating terminal transmits status information of its own terminal (initiating terminal) to the communication management device 5 (S113-1). This status information includes the communication ID and operating status information (here, "in communication") of the initiating terminal. As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the status information of the initiating terminal. Meanwhile, the transmitting / receiving unit 91 of the destination terminal transmits status information of its own terminal (destination terminal) to the communication management device 5 (S113-2). This status information includes the communication ID and operating status information (here, "Chat (in communication)") of the destination terminal. As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the status information of the destination terminal. In this way, each terminal notifies the communication management device 5 of its own operating status. Then, in the communication management device 5, the storage / read processing unit 59 changes the operating status associated with the communication ID of the starting terminal to "Chat (communicating)" in the terminal management table (see FIG. 10), and also changes the operating status associated with the communication ID of the destination terminal to "Chat (communicating)" (S114). As a result, the starting terminal (store terminal 1c2) and the destination terminal (presenter terminal 9ug11) can perform a video call through mutual communication, as shown in FIG. 2. The above is the processing when a skill group such as "domestic travel" is selected in FIG. 23(A) in step S82.

[0159] <Request to start a video call when a presenter is designated> Next, a process of starting a video call when a presenter is designated in step S82 in Fig. 23(C) will be described. In this case, in Fig. 24, only the process of step S90 is executed among the processes after step S84.

[0160] Furthermore, when a presenter is designated, the processes of steps S84-2 and S85-2 in Fig. 24 are not performed, and therefore in step S106 in Fig. 25, the display control unit 94 displays the communication ID of the starting terminal, thereby allowing the designated presenter to manually select whether or not to respond. Then, when the designated presenter selects whether or not to respond, the reception unit 92 receives the selection of whether or not to respond.

[0161] Then, in steps S107 and S109, response availability information is sent instead of response availability information. In this case, the communication ID of the initiating terminal is also included. The other processing is the same as when a skill group is selected, so a description thereof will be omitted.

[0162] <Processing for changing the destination of a video call start request> Next, using Fig. 26, a process for changing the destination of a video call start request when all presenter terminals do not respond to the start request in steps S86-1 and S86-2 of Fig. 24 will be described. Fig. 26 is a flowchart showing various processes for starting a call. Specifically, Fig. 26 is a flowchart showing three patterns: a start request after a skill group is selected, a change in the destination of the start request, and a start request when a presenter is designated.

[0163] The determination by the determination unit 15 described below is made based on the information managed in the presenter management DB 1001 and the skill group management DB 1002 of the storage unit 1000.

[0164] First, in the store terminal 1c2 of the unmanned store US, the determination unit 15 determines whether a skill group is selected or a presenter is designated as a result of the processing of step S82 shown in FIG. 24 (S201). If a skill group is selected (S201; SKILL), the determination unit 15 determines whether there is an online (call-enabled) presenter terminal of an unmanned store priority organization in the same area E1 for the accepted skill group (S202). If there is an online (call-enabled) presenter terminal of an unmanned store priority organization (S202; YES), the transceiver 11 of the store terminal 1c2 transmits connection request information indicating a connection request for a communication session for a video call to each online (call-enabled) presenter terminal of the unmanned store priority organization simultaneously as the first video call start request (see S101 in FIG. 25) (S203). The processing of step S203 corresponds to the processing of steps S84-1 and S85-1 in FIG. 24.

[0165] Next, after the processing of steps S84-1 and S85-1, the judgment unit 15 judges whether or not response information has been received from each of the presenter terminals 9ug11 and 9ug12 within a certain period of time (for example, 30 seconds) (S87-2, S88-2) (S204). If response information has been received within the certain period of time (S204; YES), the store terminal 1c2 performs communication establishment control with the presenter terminal that responded first (here, the presenter terminal 9ug11) (S205). The processing of this step S205 corresponds to the processing of steps S89 and S90 in FIG. 24. In this case, the processing of FIG. 26 ends.

[0166] On the other hand, if the store terminal 1c2 receives response information indicating a connection failure (NO) in step S202, or if the store terminal 1c2 does not receive response information indicating a connection failure within a predetermined time (e.g., 30 seconds) after transmitting the connection request information in step S204 (NO), the transceiver 11 of the store terminal 1c2 simultaneously transmits the connection request information for the second time to the presenter terminals 9mg11 and 9mg12 of the Online (available for calls) organization with staffed store priority (S206). After the second processing of steps S84-1 and S85-1, the determination unit 15 determines whether the store terminal 1c2 has received response information (S87-2, S88-2) from the presenter terminals 9mg11 and 9mg12 within the predetermined time (S207; YES). If the store terminal 1c2 receives response information within the predetermined time (S207; YES), the store terminal 1c2 performs communication establishment control for the presenter terminal that responded first (here, the presenter terminal 9mg11) in step S205. If no response information is received within the predetermined time (S207; NO), the display control unit 14 displays an error dialog indicating that there is no response on the display 106 of the terminal (S208). Then, the process returns to step S201.

[0167] In the above step S201, if a presenter is designated (S201; designation), the display control unit 14 displays a destination list of presenters shown in FIG. 23(C) on the display 106 of the terminal (S209). Then, the reception unit 12 receives the designation (selection) of a presenter from the customer c2 (S210). The processing of steps S209 and S210 corresponds to the processing of step S82 in FIG. 24.

[0168] As a result, the store terminal 1c2 executes the process of communication establishment control only for the designated presenter terminal (here, the presenter terminal 9ug11) (S211). The process of this step S211 corresponds to the process of step S90 in FIG.

[0169] <Example of a screen during a video call between the presenter's device and the store's device> Fig. 27 is an example of a call screen displayed on the presenter terminal. Fig. 27 is a diagram showing the call screen of the presenter terminal. The call screen 930 of the presenter terminal includes a main image field 940, a sub-image field 950, and a button display field 960. The button display field 960 displays a hold button 961, a travel plan button 962, a display switch button 963, a camera button 964, a microphone button 965, and a hang up button 966. The main image field 940 is displayed across the entire call screen 930 of the presenter terminal 9, and the sub-image field 950 is displayed in a part of the call screen 930 (here, the bottom right) smaller than the main image field 940.

[0170] The hold button 961 is a button that stops the terminal device from transmitting images and audio while maintaining communication. It is used when the presenter will be away from their seat for a short period of time. A message that the hold button 961 has been pressed is sent to the store terminal 1. When the reception unit 92 of the presenter terminal 9 detects that the hold button has been pressed, it reads out a predetermined image from the storage unit and displays it on the display of the presenter terminal 9. For example, by displaying a message such as "The presenter has left their seat," it is possible to prevent customers from feeling uncomfortable. The image to be displayed can be set by store staff, so advertisements, videos, etc. can also be displayed.

[0171] The itinerary planning button 962 is a function that activates a mode for creating an itinerary while listening to the customer's requests.

[0172] Display switch button 963 can hide the image of the customer displayed in main image field 940 and switch to displaying only the background image. When the presenter creates a travel plan with the customer after pressing travel plan button 962, this function temporarily hides the image of the customer so that the customer can see an image of an appropriate location when displaying local image (spherical image) 941 as in FIG. 30 .

[0173] The camera button 964 and microphone button 965 are buttons for the presenter to switch the camera of his / her presenter terminal 9 and mute the microphone.

[0174] The disconnect button 966 is a button for disconnecting communication between terminals (presence changes from Chat to Online). Only the presenter can disconnect communication.

[0175] Next, a call screen displayed by the store terminal will be described with reference to FIG. 28. FIG. 28 is an example of a call screen displayed by the store terminal. As shown in FIG. 28, the call screen 130 displayed on the store terminal 1 includes a main image field 140, a sub-image field 150, and a button display field 160. The button display field 160 displays a display switch button 163, a camera button 164, a microphone button 165, and a hang-up button 166. The display switch button 163, the camera button 164, the microphone button 165, and the hang-up button 166 perform the same functions as the display switch button 963, the camera button 964, the microphone button 965, and the hang-up button 966 in FIG. 27. The customer can only display the sub-image field 150 (where the customer's own face is displayed), switch between displaying and hiding the presenter displayed in the main image field 140, and operate the microphone and camera. The main image field 140 is displayed over the entire call screen 180 of the store terminal 1, and the sub-image field 150 is displayed smaller than the main image field 140 in a part of the call screen 180 (here, the bottom right).

[0176] <Travel plan suggestions> Next, we will explain the process in which the presenter proposes a travel plan to a customer via remote communication. When serving customers, the presenter asks the customer (guest) about their desired travel destination, and then inputs the country or region from a world map, or the tourist spot specified by the customer (guest). After inputting, the map is displayed on the screen. When a tourist spot within the corresponding map area is selected, an image (spherical video) is displayed in the background image, allowing the visitor to create a travel plan while experiencing the local atmosphere. This will be explained in detail below.

[0177] Fig. 29 is a sequence diagram showing the process of creating a travel plan when a presenter terminal and a store terminal start a video call from a dedicated application. Fig. 30 is an example of a call screen (including an input window and a map window) displayed on the presenter terminal. Fig. 31 is an example of a call screen (including an input window and a map window) displayed on the presenter terminal.

[0178] First, as shown in FIGS. 27 and 28, a video call is being conducted between the store terminal 1 and the presenter terminal 9. When the presenter creates a travel plan with a customer, the presenter presses the Travel Plan button 962 displayed on the call screen of the presenter terminal shown in FIG. 27. The reception unit 92 then accepts the request to enter the travel plan mode and activates the travel plan creation mode. When this "travel plan creation mode" is activated, as shown in FIG. 30, the display control unit 94 displays an input window 970 in the upper left corner of the main image field 940 of the call screen 930, allowing the customer to input desired tourist destinations, accommodations, dates of stay, number of people, number of rooms, etc., and a map window 980 in the lower left corner, showing landmarks of the tourist destinations entered in the input window 970. The presenter then asks the customer about the desired tourist destinations via remote call and enters the desired conditions in the fields displayed in the input window 970. The reception unit 92 then accepts the input of the desired conditions. For example, the presenter enters "Okinawa," "Okinawa Prefecture," and "Remote Island 1" as the tourist destinations where the customer will stay.

[0179] This input causes the display control unit 94 to display a map corresponding to remote island 1 in the map window 980, as shown in Fig. 31. This map displays a number of landmarks indicating areas that the presenter can introduce. In this case, each hotel on the remote islands, centered around Miyako Island in Okinawa Prefecture, is displayed as a landmark.

[0180] Next, the presenter selects a specific landmark 981 in the map window 980 while listening to the customer's request, and the reception unit 92 accepts the selection of the tourist attraction related to the specific landmark 981 (S302). When the specific landmark 981 is selected, the transmission / reception unit 91 of the presenter terminal 9 transmits a request for an image of the tourist attraction to the communication management device 5 via the communication network 100 (S303). At this time, the request transmitted from the presenter terminal 9 includes a landmark ID for identifying the specific landmark 981. As a result, the transmission / reception unit 51 of the communication management device 5 receives the request transmitted in step S317.

[0181] Next, in the communication management device 5, the storage / readout processing unit 59 reads out the file name of the tourist spot image corresponding to the landmark ID stored in the image information management DB 5006 (see FIG. 8(D)) based on the landmark ID included in the request received from the presenter terminal 9 by the transmission / reception unit 51, and reads out the image data of the tourist spot with this file name from the storage unit 5000 (S304). Then, the transmission / reception unit 51 of the communication management device 5 transmits the image data of the tourist spot to the presenter terminal 9 (S305) and also to the store terminal 1 (S306). As a result, the transmission / reception unit 91 of the presenter terminal 9 and the transmission / reception unit 11 of the store terminal 1 each receive the image data of the tourist spot.

[0182] Fig. 32 is an example of a call screen (including an input window, a map window, and an image of a tourist spot) displayed on the presenter terminal. Fig. 33 is an example of a call screen (including an image of a tourist spot) displayed on the store terminal.

[0183] As a result of step S305, as shown in FIG. 32, in the presenter terminal 9, the display control unit 94 displays the image of the tourist spot as a background image on the display 106 of the presenter terminal 9 (S307). In this case, the display control unit 94 displays the image not only as a background image in the main image field 940 in which the image of the customer is displayed, but also as a background image in the sub-image field 950 in which the presenter himself is displayed. Meanwhile, as a result of step S306, as shown in FIG. 33, in the store terminal 1, the display control unit 14 displays the image of the tourist spot as a background image on the display 106 of the store terminal 1 (S308). In this case, the display control unit 14 displays the image not only as a background image in the main image field 140 in which the image of the presenter is displayed, but also as a background image in the sub-image field 150 in which the customer himself is displayed.

[0184] This allows the presenter and the customers to share the same background image during a video call, allowing them to have lively conversations about tourist spots.

[0185] Furthermore, the presenter asks the customer via remote call about the desired accommodation (here, "XX△ Hotel") and inputs the conditions according to the customer's requests into the items displayed in the input window 970, and the reception unit 92 accepts the input of the conditions according to the customer's requests (here, accommodation) (S309).

[0186] Next, the transmitting / receiving unit 91 of the presenter terminal 9 transmits a request for an image of the exterior of the accommodation facility to the communication management device 5 via the communication network 100 (S310). At this time, the request transmitted from the presenter terminal 9 includes the accommodation facility information (here, "XXX Hotel") as the condition accepted in step S309. As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the request transmitted in step S310.

[0187] Next, in the communication management device 5, based on the accommodation facility information included in the request received from the presenter terminal 9 by the transmission / reception unit 51, the storage / readout processing unit 59 reads out the file name of the image of the exterior of the accommodation facility corresponding to the accommodation facility information stored in the image information management DB 5006 (see FIG. 8(D)), reads out image data of the exterior of the accommodation facility with this file name from the storage unit 5000, and also reads out room information corresponding to the accommodation facility information received in step S310 from the image information management DB 5006 (S311). Then, the transmission / reception unit 51 of the communication management device 5 transmits the image data of the exterior of the accommodation facility and the room information read out in step S311 to the presenter terminal 9 (S312), and also transmits it to the store terminal 1 (S313). As a result, the transmission / reception unit 91 of the presenter terminal 9 and the transmission / reception unit 11 of the store terminal 1 receive the image data of the exterior of the accommodation facility and the room information, respectively.

[0188] Fig. 34 is an example of a call screen (including an input window, a map window, an image of the exterior of the accommodation facility, and a price list) displayed on the presenter terminal. Fig. 35 is an example of a call screen (including an image of the exterior of the accommodation facility and a price list) displayed on the store terminal.

[0189] As a result of step S312, as shown in FIG. 34, in the presenter terminal 9, the display control unit 94 displays an image of the exterior of the accommodation facility as a background image on the display 106 of the presenter terminal 9, and also displays a price list 990 based on the room information (S314). In this case, the display control unit 94 displays not only the image of the customer in the main image field 940 where the image of the customer is displayed, but also the image of the sub-image field 950 where the presenter himself is displayed. Meanwhile, as a result of step S313, as shown in FIG. 35, in the store terminal 1, the display control unit 14 displays an image of the exterior of the accommodation facility as a background image on the display 106 of the store terminal 1, and also displays a price list 190 based on the room information (S315). In this case, the display control unit 14 displays not only the image of the customer in the main image field 140 where the image of the presenter is displayed, but also the image of the customer in the sub-image field 150 where the customer himself is displayed.

[0190] This allows the presenter and the users acting as customers to share the same background image while making a video call, allowing them to have a lively conversation about accommodations.

[0191] Furthermore, the presenter asks the customer via remote communication for the grade (rank) of the desired room (here, "Deluxe") and accepts the selection of the room grade (here, "Deluxe") to be displayed on the price list 990 (S316).

[0192] Next, the transmitting / receiving unit 91 of the presenter terminal 9 transmits a request for an image of the room accepted in step S316 to the communication management device 5 via the communication network 100 (S317). At this time, the request transmitted from the presenter terminal 9 includes a room ID indicating the grade of the room accepted in step S316. As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the request transmitted in step S317.

[0193] Next, in the communication management device 5, the storage / readout processing unit 59 reads out the file name of the image of the room corresponding to the image ID stored in the image information management DB 5006 (see FIG. 8(D)) based on the room ID included in the request received from the presenter terminal 9 by the transmission / reception unit 51, and reads out the image data of the room with this file name from the storage unit 5000 (S318). Then, the transmission / reception unit 51 of the communication management device 5 transmits the image data of the room read out in step S318 to the presenter terminal 9 (S318) and also to the store terminal 1 (S320). As a result, the transmission / reception unit 91 of the presenter terminal 9 and the transmission / reception unit 11 of the store terminal 1 each receive the image data of the room.

[0194] Fig. 36 is an example of a call screen (including an input window, a map window, a room image, and a price list) displayed on the presenter terminal. Fig. 37 is an example of a call screen (including a room image and a price list) displayed on the store terminal.

[0195] 36, in the presenter terminal 9, the display control unit 94 displays an image of the room (here, "Deluxe") as a background image on the display 106 of the presenter terminal 9 (S321). In this case, the display control unit 94 displays the image not only as a background image in the main image field 940 in which an image of the customer is displayed, but also as a background image in the sub-image field 950 in which the presenter himself is displayed. Meanwhile, in the store terminal 1, in the above step S320, the display control unit 14 displays the image of the room as a background image on the display 106 of the store terminal 1 (S322), as shown in FIG. 36. In this case, the display control unit 14 displays the image of the room as a background image not only as a background image in the main image field 140 in which an image of the presenter is displayed, but also as a background image in the sub-image field 150 in which the customer himself is displayed.

[0196] This allows the presenter and the customer to share the same background image during a video call, allowing them to have a lively conversation about the room.

[0197] [Major Effects of the Present Embodiment] As described above, according to this embodiment, the presenter allows the customer (guest) to virtually experience the atmosphere of the local area where the trip is planned. Furthermore, since the local area images can be viewed from various viewpoints using spherical video, the customer can feel the local area more realistically than with a photograph and make plans, which provides a sense of security. This allows the presenter and the customer users to share the same background image displayed across the entire main image field 140, 940 during a video call, allowing them to have a lively conversation about the trip.

[0198] Furthermore, even if a first call start request is made from a store terminal US in an unmanned store to each presenter terminal 9ug11, 9ug12 in an organization SK12 responsible for a desired skill SK1 in an unmanned-store-priority-supporting organization UG1, and no response is received from any of the presenter terminals, a second call start request can be made to each presenter terminal 9mg11, 9mg12 in an organization SR11 responsible for the same skill SK1 in a manned-store-priority-supporting organization MG1. This makes it possible to take advantage of the unmanned store's advantage of streamlining and streamlining store operations while reducing customer exits from the unmanned store by installing presenter terminals 9ug11, 9ug12 that are unmanned-store-priority-supporting. Furthermore, even if the presenter terminals 9ug11, 9ug12 that are unmanned-store-priority-supporting cannot respond, by enabling a store terminal to make a call start request to the presenter terminals 9mg11, 9mg12 that are manned-store-priority-supporting. This has the effect of further reducing customer loss.

[0199] 〔supplement〕 In the above embodiment, the processes of steps S302, S309, and S316 are performed, but at least two of these processes may be performed in the same step. Also, any one of steps S302, S309, and S316 may be omitted.

[0200] The store terminal 1 is an example of a communication terminal. The store terminal 1 includes not only PCs but also smart watches, game consoles, and devices dedicated to video calls.

[0201] Furthermore, in the communication between the communication management device 5 and the store terminal 1 and the presenter terminal 9, other servers, routers, etc. may relay data. [Explanation of symbols]

[0202] 1. Store terminal (an example of a second communication terminal) 3. Relay Device 4. Administrator terminal 5 Communication management device 7 Service information management device 9. Presenter terminal (an example of a first communication terminal) 11 Transmitter / Receiver 12 Reception 13 Communication control section 14 Display control unit 15 Judgment Department 51 Transmitting / receiving unit (an example of a transmitting means, an example of a receiving means) 5000 Memory unit (an example of a memory means) 100 Communication Network

Claims

1. A communication management device that mediates video data of a subject at a first location transmitted from a first communication terminal so that the video data of the subject at a first location can be received by a second communication terminal, a management means for managing data of a predetermined background image and predetermined identification information for identifying the predetermined background image in association with each other; a receiving means for receiving a transmission request for the predetermined background image transmitted by the first communication terminal, the transmission request including the predetermined identification information; a readout means for reading out data of the predetermined background image corresponding to the predetermined identification information included in the transmission request from the management means; transmitting the data of the predetermined background image read by the reading means to the second communication terminal so that the second communication terminal displays the predetermined background image as a background image for the subject at the first location while the second communication terminal is displaying the subject at the first location, and the predetermined background image as a background image for the subject at the second location while the second communication terminal is displaying video data of the subject at the second location photographed by the second communication terminal; a transmitting means for transmitting the data of the predetermined background image read by the reading means to the first communication terminal so that the first communication terminal displays the predetermined background image as a background image for the subject at the second location while the first communication terminal is displaying the subject at the second location, and the predetermined background image as a background image for the subject at the first location while the first communication terminal is displaying video data of the subject at the first location photographed by the first communication terminal; A communication management device comprising:

2. the video data transmitted by the first communication terminal is video data of a first subject photographed by the first communication terminal, the receiving means receives the video data of the subject at the second location, the video data being captured by the second communication terminal; the transmitting means transmits the video data of the subject at the second location so that the first communication terminal receives the video data of the subject at the second location; The transmitting means transmits the data of the predetermined background image based on the transmission request so that the first communication terminal receives video data of the subject at the second location and displays the predetermined background image as a background image for the subject at the second location being displayed.

2. The communication management device according to claim 1.

3. The communication management device described in claim 2, characterized in that the transmitting means transmits data of the specified background image based on the transmission request so as to display the specified background image as a background image for the subject at the first location while the first communication terminal is displaying video data of the subject at the first location photographed by the first communication terminal.

4. 4. The communication management device according to claim 1, The receiving means receives a request to send a predetermined background image based on information input or selected from a call screen displayed on the first communication terminal.

5. The communication management device according to claim 1 , wherein the background image is a spherical image.

6. The communication device according to claim 5 , wherein the spherical image is a still image or a video image.

7. A communication management method executed by a communication management device including a management means for mediating video data of a subject at a first location transmitted by a first communication terminal so that the video data of the subject at the first location can be received by a second communication terminal, and for managing data of a predetermined background image and predetermined identification information for identifying the predetermined background image in association with each other, a receiving step of receiving a transmission request for the predetermined background image transmitted by the first communication terminal, the transmission request including the predetermined identification information; a reading step of reading, from the management means, data of the predetermined background image corresponding to the predetermined identification information included in the transmission request; transmitting the data of the predetermined background image read in the reading step to the second communication terminal so that the second communication terminal displays the predetermined background image as a background image for the subject at the first location while the second communication terminal is displaying the subject at the first location, and the predetermined background image as a background image for the subject at the second location while the second communication terminal is displaying video data of the subject at the second location photographed by the second communication terminal; a transmitting step of transmitting the data of the predetermined background image read in the reading step to the first communication terminal so that the first communication terminal displays the predetermined background image as a background image for the subject at the second location while the first communication terminal is displaying the subject at the second location, and the predetermined background image as a background image for the subject at the first location while the first communication terminal is displaying video data of the subject at the first location photographed by the first communication terminal; A communication management method comprising:

8. A program causing a computer to execute the method according to claim 7.

9. A communication management device that manages a video call between a first communication terminal and a second communication terminal so that video data of a subject at a first location transmitted by the first communication terminal is received by the second communication terminal, and video data of a subject at a second location transmitted by the second communication terminal is received by the first communication terminal, a management means for managing data of a predetermined background image and predetermined identification information for identifying the predetermined background image in association with each other; a receiving means for receiving a transmission request for the predetermined background image transmitted by the first communication terminal, the transmission request including the predetermined identification information; a readout means for reading out data of the predetermined background image corresponding to the predetermined identification information included in the transmission request from the management means; displaying the predetermined background image as a background image for the subject at the first location while the second communication terminal is displaying the subject at the first location, and transmitting the data of the predetermined background image read by the read means to the second communication terminal so that the second communication terminal can display the predetermined background image as a background image for the subject at the second location while the second communication terminal is displaying video data of the subject at the second location photographed by the second communication terminal; a transmitting means for transmitting the data of the predetermined background image read by the reading means to the first communication terminal so that the first communication terminal displays the predetermined background image as a background image for the subject at the second location while the first communication terminal is displaying the subject at the second location, and the predetermined background image as a background image for the subject at the first location while the first communication terminal is displaying video data of the subject at the first location taken by the first communication terminal; and A communication management device characterized in that the transmitting means transmits data of the specified background image based on the transmission request so that the first communication terminal receives video data of a subject at the second location and displays the specified background image as a background image for the subject at the second location that is being displayed.

Images