Communication systems, communication methods, and programs

Abstract

Compared to face-to-face customer service, remote customer service has the challenge of lacking the ability to build excitement in the customer when planning a trip. In traditional face-to-face customer service, the customer and the presenter can share travel brochures and exchange opinions while developing a travel plan. In that case, the conversation becomes more engaging as the customer hears the presenter's personal experiences, and their expectations for the trip grow. On the other hand, while remote customer service also involves sharing electronic data of travel brochures via screen sharing to develop a travel plan, it is not a face-to-face format, making it difficult to build excitement in the customer for the trip. This invention aims to convey the atmosphere of the destination where the customer is planning to travel to the customer in a way that enhances their sense of presence. [Solution] Upon instruction from the presenter's terminal, the presenter's image is displayed in the main image field 140 and the customer's own image is displayed in the sub-image field 150, and images of the travel destination, etc., are displayed as background images for each of the fields 140 and 150.

Description

Technical Field

[0004] , , , , , , ,

[0005] , , , ,

[0001] The present disclosure relates to a communication system, a communication method, and a program.

Background Art

[0002] In recent years, from the perspective of streamlining and improving the efficiency of store operations, remote concierge services have come to be provided. As a result, for example, even when a customer (guest) has to wait in line for customer service because an employee in the store is busy serving other customers, the customer (guest) can access an explainer terminal of an explainer (concierge) with specialized skills at a remote service center via the Internet or the like from a store terminal and receive customer service by having a video call with the explainer (see Patent Document 1).

[0003] The introduction of remote concierge has been progressing in travel agencies in recent years. With the introduction of this service, customers (guests) can receive customer service without waiting at the store, and it has started to spread as a service that can prevent travel agencies from losing opportunities to acquire customers.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, several challenges have also emerged. Compared to face-to-face customer service, remote customer service has the challenge of not being able to build excitement in the customer's (guest's) mood when planning a trip. In traditional face-to-face customer service, the customer and the presenter can share travel brochures and exchange opinions while developing a travel plan. In that case, the conversation becomes more engaging as the customer hears the presenter's personal experiences directly, and their expectations for the trip grow. On the other hand, while remote customer service also involves sharing electronic data of travel brochures via screen sharing to develop a travel plan, it is not a face-to-face format, making it difficult to build excitement in the customer for the trip. Therefore, when using video calls, it becomes more difficult to make the conversation engaging compared to face-to-face interactions. [Means for solving the problem]

[0006] The invention according to claim 1 is a communication management device that mediates video data of a subject at a first location transmitted by a first communication terminal so that a second communication terminal can receive the video data of the subject at the first location transmitted by the first communication terminal, the communication management device comprising: receiving means for receiving a transmission request for a specific background image transmitted by the first communication terminal; and transmitting means for transmitting data of the specific background image to the second communication terminal based on the transmission request, so that the second communication terminal displays the specific background image as a background image for the subject at the first location while the second communication terminal is displaying video data of a subject at a second location captured by the second communication terminal, so that the second communication terminal can display the specific background image as a background image for the subject at the second location while the second communication terminal is displaying the video data of the subject at the second location captured by the second communication terminal. [Effects of the Invention]

[0007] As explained above, according to the present invention, even in video calls, each user can share the same background image with each other, which has the effect of making the conversation more engaging. [Brief explanation of the drawing]

[0008] [Figure 1] This is an overall configuration diagram of the communication system according to this embodiment. [Figure 2] This is an illustrative diagram showing a situation where a customer at a store and a representative at a service center are having a remote video call. [Figure 3] This is a hardware configuration diagram of each terminal and device that make up the communication system of this embodiment. [Figure 4] This is a functional block diagram of the communication system according to this embodiment. [Figure 5] This is a functional block diagram of the communication system according to this embodiment. [Figure 6] (A) is a conceptual diagram showing the presenter management table, and (B) is a conceptual diagram showing the skill group management table. [Figure 7] (A) is a conceptual diagram showing the authentication management table, and (B) is a conceptual diagram showing the terminal state management table. [Figure 8] (A) is a conceptual diagram showing the destination list management table, (B) is a conceptual diagram showing the session management table, and (C) is a conceptual diagram showing the communication information management table. [Figure 9] (A) is a conceptual diagram of the system update information management table, (B) is a conceptual diagram of the business operator management table, and (C) is a conceptual diagram of the area management table. [Figure 10] (A) is a conceptual diagram of the terminal management table, and (B) is a conceptual diagram of the skill management table. [Figure 11] (A) is a conceptual diagram of the login information management table, (B) is a conceptual diagram of the subgroup management table, and (C) is a conceptual diagram of the user update information management table. [Figure 12] This is a sequence diagram showing the process of registering service information. [Figure 13] This diagram shows the login screen displayed on the administrator's terminal. [Figure 14] This diagram shows the store list screen displayed on the administrator's terminal. [Figure 15] This diagram shows the list of presenters displayed on the administrator's terminal. [Figure 16] This diagram shows the group list screen displayed on the administrator's terminal. [Figure 17] It is a diagram showing an explainer information editing screen displayed on the administrator terminal. [Figure 18] It is a diagram showing a group information editing screen displayed on the administrator terminal. [Figure 19] It is a diagram showing a store information editing screen displayed on the administrator terminal. [Figure 20] It is a sequence diagram showing the preparation process of a video call on a store terminal, particularly showing the process of acquiring service information. [Figure 21] It is a sequence diagram showing the preparation process of a video call on a store terminal, particularly showing the process of acquiring the operating status of each terminal. [Figure 22] It is a flowchart showing the preparation process of a video call on a store terminal, particularly showing the process of determining the operating status of the group of explainer terminals for each skill. [Figure 23] It is a diagram showing a screen displayed on a store terminal. [Figure 24] It is a sequence diagram showing the process of starting a video call. [Figure 25] It is a sequence diagram showing the detailed process of communication establishment control. [Figure 26] It is a flowchart showing various processes for starting a call. [Figure 27] It is an example of a call screen displayed on an explainer terminal. [Figure 28] It is an example of a call screen displayed on a store terminal. [Figure 29] It is a sequence diagram showing the process in which an explainer terminal and a store terminal start a video call from a dedicated application to create a travel plan. [Figure 30] It is an example of a call screen (including an input window and a map window) displayed on an explainer terminal. [Figure 31] It is an example of a call screen (input window, map window) displayed on an explainer terminal. [Figure 32] It is an example of a call screen (including an input window, a map window, and an image of a tourist destination) displayed on an explainer terminal. [Figure 33]This is an example of a call screen displayed on a store terminal (including images of tourist destinations). [Figure 34] This is an example of a call screen displayed on the presenter's terminal (including input window, map window, image of the accommodation's exterior, and price list). [Figure 35] This is an example of a call screen displayed on a store terminal (including an image of the hotel's exterior and a price list). [Figure 36] This is an example of a call screen displayed on the presenter's terminal (including the input window, map window, room image, and price list). [Figure 37] This is an example of a call screen displayed on a store terminal (including images of the room and a price list). [Modes for carrying out the invention]

[0009] [Overall configuration of the communication system] The overall configuration of the communication system will be explained using Figure 1. Figure 1 is an overall configuration diagram of a communication system according to an embodiment of the present invention. In this system, a customer, a travel agency that sells travel products to this customer, and a system provider that provides a system for remote concierge (remote customer service) to the travel agency are involved.

[0010] The travel agency operates manned stores, unmanned stores, and service centers in multiple areas such as Kanto and Kansai. Figure 1 shows that the travel agency operates a manned store MS, an unmanned store US, and a service center SC1 in area E1. A manned store MS is a store with employees who provide customer service. An unmanned store is a store without employees who provide customer service. Note that although only one manned store MS and one unmanned store US are shown in Figure 1, there may be multiple of each.

[0011] In manned stores (MS), employees typically provide information about travel products to customers who visit the store. However, if an employee is busy serving another customer, they may not be able to assist a newly arrived customer. Furthermore, unmanned stores (US) do not have any employees to provide customer service. In such cases, customers can use a store terminal 1c1 in a manned store (MS) or a store terminal 1c2 in an unmanned store (US) to access the presenter's terminal at the service center (SC1) via a communication network 100 such as the internet, and receive remote customer service regarding travel products from the presenter. The presenter is, for example, the remote concierge mentioned above. Each presenter basically uses their own dedicated presenter terminal, so a specific presenter is linked to a specific presenter terminal.

[0012] Store terminals 1c1 and 1c2 are each composed of a PC (personal computer). Multiple store terminals may be installed in manned stores (MS) and unmanned stores (US). Hereafter, when referring to multiple store terminals such as 1c1 and 1c2 collectively, the term "store terminal 1" will be used.

[0013] Service Center SC1 has a manned store priority response organization MG1, which prioritizes receiving call initiation requests for remote customer service from store terminal 1c1 of manned stores MS, and an unmanned store priority response organization UG1, which prioritizes receiving call initiation requests for remote customer service from store terminal 1c2 of unmanned stores US. Note that there may be three or more priority response organizations.

[0014] Furthermore, the staffed store priority organization MG1 includes the organization SR11 responsible for skill SK1 and the organization SR12 responsible for skill SK2. For example, skill SK1 is the ability of an explainer to make suggestions regarding domestic travel, and skill SK2 is the ability of an explainer to make suggestions regarding international travel. Similarly, the unstaffed store priority organization UG1 includes the organization SR21 responsible for skill SK1 and the organization SR22 responsible for skill SK2. In other words, both the staffed store priority organization MG1 and the unstaffed store priority organization UG1 have organizations SR11 and SR21 (or organizations SR12 and SR22) that employ explainers with the same skills. Note that there may be three or more skills and skill-responsible organizations.

[0015] Furthermore, within the staffed store priority organization MG1, the organization SR11 responsible for skill SK1 has two presenters, mg11 and mg12, who possess skill SK1. Presenter mg11 uses presenter terminal 9mg11, and presenter mg12 uses presenter terminal 9mg12.

[0016] Furthermore, the organization SR12, which is responsible for skill SK2, has two presenters, mg21 and mg22, who possess skill SK2. Presenter mg21 uses presenter terminal 9mg21, and presenter mg22 uses presenter terminal 9mg22.

[0017] Similarly, in the unmanned store priority organization UG1, the organization SR21 responsible for skill SK1 has presenters ug11 and ug12 who possess skill SK1. Presenter ug11 uses presenter terminal 9ug11, and presenter ug12 uses presenter terminal 9ug12. In addition, the organization SR22 responsible for skill SK2 has presenters ug21 and ug22 who possess skill SK2. Presenter ug21 uses presenter terminal 9ug21, and presenter ug22 uses presenter terminal 9ug22.

[0018] Note that each assigned organization (SR11, SR12, SR21, SR22) may have three or more presenters. In this case, each of the three or more presenters will be assigned a presenter terminal. Furthermore, from now on, when referring to multiple presenter terminals such as 9mg11, 9mg12, 9mg21, 9mg22, 9ug12, 9ug12, 9ug21, 9ug22, etc., the collective term will be written as "presenter terminal 9".

[0019] Furthermore, each travel agency has an administrator ad1 who manages the terminals in each service area. Administrator ad1 uses the administrator terminal 4 to access the service information management device 7 via the communication network 100 and configure various service information settings. This service information concerns the processing and settings of each terminal in the travel agency, and the administrator terminal 4 is configured as a PC.

[0020] Customer c1 and customer c2 visited the staffed store MS and the unstaffed store US, respectively.

[0021] The system provider manages and operates the relay device 3, the communication management device 5, and the service information management device 7. The relay device 3, the communication management device 5, and the service information management device 7 are computers such as servers. The relay device 3 relays image data and audio data when video calls are made between any of the store terminals 1 and any of the presenter terminals 9. The communication management device 5 centrally manages login authentication from store terminals 1 and presenter terminals 9, management of the communication status of store terminals 1 and presenter terminals 9, management of the destination list, and the communication status of the relay device 30. The service information management device 7 manages various configured service information. The relay device 3, the communication management device 5, and the service information management device 7 will be explained in detail later.

[0022] Furthermore, the store terminal 1, administrator terminal 4, presenter terminal 9, relay device 3, communication management device 5, and service information management device 7 can be connected to a communication network 100 such as the Internet.

[0023] The relay device 3, communication management device 5, and service information management device 7 may be constructed by a single computer, or they may be constructed by multiple computers, each part (function or means) being divided and assigned to a different computer as desired.

[0024] Figure 2 is an illustrative diagram showing a scenario where a customer at a store and a service center representative are remotely communicating via video call. In Figure 2, customer c2, who has visited the unmanned store US, is using the store terminal 1c2, while representative ug11 from the unmanned store priority support organization UG11 is using representative terminal 9ug11. In this way, customer c2 can receive explanations about travel products through remote customer service.

[0025] [Hardware configuration of the communication system] Next, using Figure 3, we will describe the hardware configuration of each terminal (store terminal 1, administrator terminal 4, presenter terminal 9) and each device (relay device 3, communication management device 5, service information management device 7) that make up the communication system shown in Figure 1. In this embodiment, we will assume that all terminals and devices have the same configuration, and will describe the hardware configuration of store terminal 1, omitting the description of the hardware configurations of the other terminals and devices.

[0026] As shown in Figure 3, the store terminal 1 is composed of a computer and includes a CPU 101, ROM 102, RAM 103, HD 104, HDD (Hard Disk Drive) controller 105, display 106, external device connection I / F (Interface) 108, network I / F 109, data bus 110, keyboard 111, pointing device 112, DVD-RW (Digital Versatile Disk Rewritable) drive 114, and media I / F 116.

[0027] Of these components, the CPU 101 controls the operation of the entire computer. The ROM 102 stores programs used to drive the CPU 101, such as the IPL. The RAM 103 is used as the work area for the CPU 101. The HD 104 stores various data, such as programs. The HDD controller 105 controls the reading or writing of various data to the HD 104 according to the control of the CPU 101. The display 106 displays various information such as cursors, menus, windows, characters, or images. The external device connection I / F 108 is an interface for connecting various external devices. In this case, external devices include, for example, USB (Universal Serial Bus) memory and printers. The network I / F 109 is an interface for data communication using the communication network 100. The bus lines 110 are address buses, data buses, etc., for electrically connecting each component, such as the CPU 101 shown in Figure 3.

[0028] The keyboard 111 is a type of input means equipped with multiple keys for inputting characters, numbers, and various instructions. The pointing device 112 is a type of input means for selecting and executing various instructions, selecting processing targets, and moving the cursor. The DVD-RW drive 114 controls the reading or writing of various data to the DVD-RW 113, which is an example of a removable recording medium. Note that it is not limited to DVD-RW, but may also be DVD-R or Blu-ray Disc. The media I / F 116 controls the reading or writing (storage) of data to the recording medium 115, such as flash memory.

[0029] Furthermore, the external device connection interface is connected to a microphone 121 as an example of a sound collection device, a speaker 122 as an example of a sound output device, and a camera 123 as an example of an imaging device. In Figure 3, the microphone 121, speaker 122, and camera 123 are shown as external devices, but they may also be built-in devices without using the external device connection interface.

[0030] [Functional Configuration of the Communication System] Next, the functional configurations of the communication system of this embodiment will be described using Figures 3 to 11. Figures 4 and 5 are functional block diagrams of the communication system according to this embodiment.

[0031] <Functional Configuration of Store Terminals> First, the functional configuration of the store terminal 1 will be explained using Figures 3 and 4. As shown in Figure 4, the store terminal 1 has a transmitting / receiving unit 11, a receiving unit 12, a communication control unit 13, a display control unit 14, a judgment unit 15, and a storage / reading processing unit 19. Each of these units is a function or means of functioning, realized by any of the components shown in Figure 3 operating according to instructions from the CPU 101 that follow a program deployed from the HD 104 onto the RAM 103. The store terminal 1 also has a storage unit 1000 constructed from the RAM 103 and HD 104 shown in Figure 3. The storage unit 1000 contains a presenter management DB 1001 and a skill group management DB 1002. The information managed in the presenter management DB 1001 and the skill group management DB 1002 is included in the client information sent from the service information management device 7 in step S19, which will be described later. The store terminal 1 then uses the information managed in the presenter management DB 1001 and the skill group management DB 1002 to display the screen shown in Figure 23, which will be described later.

[0032] (User management table) Figure 6(A) is a conceptual diagram of the presenter management table. The memory unit 1000 has a presenter management DB 1001 constructed, which consists of the presenter management table shown in Figure 6(A). In this presenter management table, various information of the destination candidate of store terminal 1 (in this case, presenter terminal 9) is associated and managed, including the communication ID, presenter name, the operational status (presence) of the destination candidate, and the skill name indicating the skills of the presenter who has exclusive use of the presenter terminal.

[0033] The communication ID is identification information used to identify the communication destination in communication system 1. While there are no particular limitations on the communication ID, examples include a user ID to identify the presenter using presenter terminal 9, and a communication ID to identify a store terminal. The presenter name in Figure 6(A) is the same presenter name shown in Figure 23(C), described later. The operational status indicates the operational status of the terminal on the other end of the video call. There are three types of operational status: Online (available for calls), Chat (on a call), and Offline (unavailable for calls). The operational status is reflected in the display in Figures 23(A) and (C). Skills, as mentioned above, indicate the names of the skills possessed by the presenter. As shown in Figure 6(A), a single presenter may possess multiple skills (Specialty 1, Specialty 2).

[0034] (Skill group management table) Figure 6(B) is a conceptual diagram showing the skill group management table. The storage unit 1000 has a skill group management DB 1002 constructed using the skill group management table shown in Figure 6(B). In this skill group management table, the skill name, the operational status (presence) of the skill group, the communication ID of the presenter terminal that the store terminal can call, and the communication ID of the presenter terminal that is currently on a call are associated and managed.

[0035] The operational status of a skill group indicates the overall operational status of each presenter terminal within a group of presenters possessing the same skill, such as Specialty 1. For example, if there are multiple presenter terminals within a group of presenters possessing the same skill, and even one of these terminals is online, the operational status of the Specialty 1 skill group will be managed as online. Furthermore, even if none of the multiple presenter terminals are online, if even one of them is in Chat mode, the operational status of the Specialty 1 skill group will be managed as Chat. Additionally, if none of the multiple presenter terminals are online or in Chat mode, i.e., if all are offline, the operational status of the Specialty 1 skill group will be managed as offline.

[0036] (Functional configuration of each store terminal) Next, the components of the store terminal 1 will be described. The transmitting / receiving unit 11 is implemented by commands from the CPU 101 shown in Figure 3, as well as the external device connection I / F 108 and the network I / F 109, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 100.

[0037] The reception unit 12 is mainly implemented by commands from the CPU 101 shown in Figure 3, as well as the keyboard 111 and pointing device 112, and accepts various inputs from the user.

[0038] The communication control unit 13 is implemented by instructions from the CPU 101 shown in Figure 3 and receives image data obtained from an external camera connected to the external device connection interface 108, which captures images of a subject. The communication control unit 13 also receives sound data obtained from an external microphone connected to the external device connection interface 108. Furthermore, the communication control unit 13 outputs the image data received from the video call partner and the image data obtained by its own terminal to the display control unit 14. The communication control unit 13 also outputs the sound data received from the video call partner and the sound data obtained by its own terminal to an external speaker connected to the external device connection interface 108.

[0039] The display control unit 14 is mainly implemented by instructions from the CPU 101 shown in Figure 3, and displays an image by outputting image data to the display 106 or an external display connected to the external device connection I / F.

[0040] The decision unit 15 is primarily implemented by instructions from the CPU 101 shown in Figure 3, and performs various decisions. The details of these decisions will be described later.

[0041] The memory and read processing unit 19 is mainly executed by instructions from the CPU 101 and the HDD controller 105 shown in Figure 3, and performs processing such as storing various data in the memory unit 1000 and reading various data stored in the memory unit 1000.

[0042] Furthermore, the memory unit 1000 stores image data and sound data received during communication with other terminals, overwriting the existing data each time it is received. Of these, the image data before overwriting is used to display an image on the display 106 or an external display, and the sound data before overwriting is used to output sound from an external speaker.

[0043] <Administrator terminal> Next, the functional configuration of the administrator terminal 4 will be explained using Figures 3 and 4. As shown in Figure 4, the administrator terminal 4 has a transmitting / receiving unit 41, a receiving unit 42, a display control unit 44, and a storage / reading processing unit 49. The administrator terminal 9 also has a storage unit 4000 constructed from the RAM 103 and HD 104 shown in Figure 3.

[0044] Furthermore, the transmitting / receiving unit 41, receiving unit 42, display control unit 44, and storage / reading processing unit 49 of the administrator terminal 4 have the same functions as the transmitting / receiving unit 11, receiving unit 12, display control unit 14, and storage / reading processing unit 49 of the store terminal 1, respectively, so their descriptions will be omitted.

[0045] <Functional configuration of the presenter's terminal> Next, the functional configuration of the presenter terminal 9 will be explained using Figures 3 and 4. As shown in Figure 4, the presenter terminal 9 has a transmitting / receiving unit 91, a receiving unit 92, a communication control unit 93, a display control unit 94, a decision unit 95, and a storage / reading processing unit 99. The presenter terminal 9 also has a storage unit 9000 constructed from the RAM 103 and HD 104 shown in Figure 3. The storage unit 9000 contains a presenter management DB 9001 and a skill group management DB 9002.

[0046] Furthermore, the transmitting / receiving unit 91, receiving unit 92, communication control unit 93, display control unit 94, judgment unit 95, and storage / reading processing unit 99 of the presenter terminal 9 have the same functions as the transmitting / receiving unit 11, receiving unit 12, communication control unit 13, display control unit 14, judgment unit 15, and storage / reading processing unit 19 of the store terminal 1, respectively, so their explanations are omitted. In addition, the presenter management DB 9001 and skill group management DB 9002 of the presenter terminal 9 have the same data structure as the presenter management DB 1001 and skill group management DB 1002 of the store terminal 1, so their explanations are omitted.

[0047] <Functional configuration of the relay device> Next, the functional configuration of the relay device 3 will be explained using Figures 3 and 5. As shown in Figure 5, the relay device 3 has a transmitting / receiving unit 31 that also serves as a transfer unit, a determination unit 32, and a storage / reading processing unit 39. Each of these units is a function or means of functioning that is realized by any of the components shown in Figure 3 operating according to instructions from the CPU 101 that follow a program deployed from the HD 104 onto the RAM 103. The relay device 3 also has a storage unit 3000 constructed from the RAM 103 and HD 104 shown in Figure 3.

[0048] (Functional configuration of each relay device) Next, the functional configurations of the relay device 3 will be described in detail. The transmitting / receiving unit 31 of the relay device 3, shown in Figure 5, is implemented by commands from the CPU 101 shown in Figure 3 and the network I / F 109 shown in Figure 3, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 100. The transmitting / receiving unit 31 also acts as a transfer unit, transferring image data and sound data transmitted from a predetermined terminal to other terminals.

[0049] The determination unit 32 is implemented by instructions from the CPU 101 shown in Figure 3, and performs tasks such as determining the data delay state.

[0050] The memory and read processing unit 39 is implemented by instructions from the CPU 101 shown in Figure 3 and the HDD controller 105 shown in Figure 3, and performs processing such as storing various data in the memory unit 3000 and reading various data stored in the memory unit 3000.

[0051] <Functional Configuration of Communication Management Device> Next, the functional configuration of the communication management device 5 will be explained using Figures 3 and 5. As shown in Figure 5, the communication management device 5 has a transmitting / receiving unit 51, a selection unit 53, a creation unit 54, a determination unit 55, and a storage / reading processing unit 59. Each of these units is a function or means of functioning, realized by any of the components shown in Figure 3 operating according to instructions from the CPU 101 that follow a program deployed from the HD 104 onto the RAM 103. The communication management device 5 also has a storage unit 5000 constructed from the RAM 103 and HD 104 shown in Figure 3.

[0052] (Authentication management table) Figure 6(A) is a conceptual diagram showing the authentication management table. The storage unit 5000 has an authentication management DB 5001 constructed, which consists of the authentication management table shown in Figure 6(A). In this authentication management table, each password is associated with and managed for each communication ID of all terminals 1,9 managed by the communication management device 5. Note that the password is an example of authentication information, and authentication information also includes access tokens.

[0053] (Terminal status management table) Figure 7(B) is a conceptual diagram showing the terminal status management table. The memory unit 5000 has a terminal status management DB 5002 constructed, which is composed of the terminal status management table shown in Figure 7(B). In this terminal status management table, for each communication ID of each terminal 1,9, the destination name when each terminal 1,9 is the destination, the operating status of each terminal 1,9, the date and time when the login request information described later was received by the communication management device 5, and the IP address of the store terminal 1,9 are associated and managed. The communication ID, destination name, and terminal IP address are stored when each terminal 1,9 is pre-registered to receive services from the communication management device 5.

[0054] (Recipient list management table) Figure 8(A) is a conceptual diagram showing the destination list management table. The memory unit 5000 has a destination list management DB 5003 constructed, which consists of the destination list management table shown in Figure 8(A). In this destination list management table, all communication IDs of destination terminals registered as candidate destination terminals are associated and managed with the communication ID of the initiating terminal that requests the start of a call. These candidate destination terminals are updated by being added or deleted in response to requests for addition or deletion from the administrator terminal 4 to the communication management device 5.

[0055] Note that the recipient list is just one example of recipient information; the recipient information may not be in list format, but rather a list of recipient-related information such as communication IDs.

[0056] (Session management table) Figure 8(B) is a conceptual diagram showing the session management table. The memory unit 5000 has a session management DB 5004 constructed, which consists of the session management table shown in Figure 8(B). In this session management table, for each communication session ID used to identify a session for mutual communication between each terminal and the relay device 3, the relay device ID of the relay device 3 used, the communication ID of the initiating terminal, the communication ID of the destination terminal, the reception delay time (ms) when image data is received at the destination terminal, and the reception date and time when delay information indicating this delay time is sent from the destination terminal and received by the communication management device 5 are associated and managed.

[0057] (Communication Information Management Table) Figure 8(C) is a conceptual diagram showing the communication information management table. The storage unit 5000 has a communication information management DB 5005 constructed, which is composed of the communication information management table shown in Figure 8(C). In this communication information management table, for each communication ID used to identify mutual communication between each terminal and the relay device 3, the IP address of the relay device 3 being used and the communication ID of the terminal communicating with it are associated and managed.

[0058] (Image information management table) Figure 8(D) is a conceptual diagram showing the image information management table. The memory unit 5000 has an image information management DB 50006 constructed, which is composed of the image information management table shown in Figure 8(D). In this image information management table, landmark ID, file name of the tourist destination image, accommodation information, file name of the exterior image of the accommodation, room grade (rank) and price, room ID, and file name of the room image are associated and managed.

[0059] Of these, the Landmark ID is tourist destination identification information used to identify the landmark described later. The Accommodation Information is accommodation identification information used to identify the accommodation, such as the hotel name. The Room ID is room identification information used to identify the room grade. The room grade, price, and Room ID are part of the room information. The types of image data are still image data, moving image data, and 360-degree image data. In this embodiment, in order to provide a sense of presence as if you were actually at the location when assisting in the creation of travel plans, it is assumed that 360-degree image data (including still images and moving images) will be used as the background image.

[0060] Here, the data for images of tourist attractions, the exterior of accommodations, and the rooms are stored in the storage unit 5000. Alternatively, the data for images of tourist attractions, the exterior of accommodations, and the rooms could be stored in another device (such as a server) instead of the communication management device 5, and the communication management device 5 could retrieve the data from the other device.

[0061] (Functional configuration of the communication management device) Next, we will explain in detail the functional configurations of the communication management device 5. In addition, when explaining the functional configurations of the communication management device 5, we will also explain the relationship between each component shown in Figure 3 and the main components necessary to realize each functional configuration of the communication management device 5.

[0062] The transmitting / receiving unit 51 of the communication management device 5 shown in Figure 5 is implemented by commands from the CPU 101 shown in Figure 3 and the network I / F 109, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 100.

[0063] The selection unit 53 is implemented by instructions from the CPU 101 shown in Figure 3, and selects the relay device best suited for mutual communication based on the IP address of each terminal.

[0064] The creation unit 54 is implemented by instructions from the CPU 101 shown in Figure 3, and creates communication information in response to a request to start mutual communication from each terminal. This communication information includes the IP address of the relay device selected by the selection unit 53, and the aforementioned communication ID, etc.

[0065] The decision unit 55 is implemented by instructions from the CPU 101 shown in Figure 3 and performs various decisions.

[0066] The memory and read processing unit 59 is implemented by instructions from the CPU 101 and the HDD controller 105 shown in Figure 3, and performs processing such as storing various data in the memory unit 5000 and reading various data stored in the memory unit 5000.

[0067] <Service Information Management Device> Next, the functional configuration of the service information management device 7 will be explained using Figures 3 and 5. As shown in Figure 5, the service information management device 7 has a transmitting / receiving unit 71, a determination unit 75, and a storage / reading processing unit 79. Each of these units is a function or means of functioning, realized by any of the components shown in Figure 3 operating according to instructions from the CPU 101 that follow a program deployed from the HD 104 onto the RAM 103. The service information management device 7 also has a storage unit 7000 constructed from the RAM 103 and HD 104 shown in Figure 3. A service information management DB 7001 is constructed in this storage unit 7000. The service information management DB 7001 has the tables shown in Figures 9 to 11 constructed in it.

[0068] (System update information management table) Figure 9(A) is a conceptual diagram showing the system update information management table. In this system update information management table, the terminal name, latest version, update application date and time, and download URL (Uniform Resource Locator) are associated and managed for each terminal ID. In the client name column, the store terminal application is the application for store terminal 1, and the presenter terminal application is the application for presenter terminal 9. Each application is managed by its respective download URL.

[0069] (Business Management Table) Figure 9(B) is a conceptual diagram showing a business management table. In this business management table, business IDs and business names are managed in association. Businesses are, for example, travel agencies, leasing companies, etc.

[0070] (Group management table) Figure 9(C) is a conceptual diagram showing the group management table. In this group management table, the group name (area name) and business ID are associated and managed for each group ID (area ID). The groups are shown as Area E1 in Figure 1, etc.

[0071] (Terminal management table) Figure 10(A) is a conceptual diagram showing the terminal management table. In this terminal management table, the destination name, role, login ID, password, business ID, group ID, and communication ID are associated and managed for each account ID. The account ID is identification information used to identify stores, presenters, and administrators. In the case of stores, the account for the store itself, not the customer, is managed. The destination name is also managed in the terminal status management table in Figure 7(B), but as shown in Figure 23(C), it is the display name shown on the destination selection screen displayed on store terminal 1. The role indicates the role of the terminal user, etc. The login ID and password are used for authentication when administrator ad1 accesses the service information management device 7 from administrator terminal 4. Note that the "password" in this terminal management table is the same as the "password" in the authentication management table in Figure 7(A), but a different password may be used. The communication ID is also managed in the terminal status management table in the communication management device 5 (see Figure 7(A)).

[0072] (Skill Management Table) Figure 10(B) is a conceptual diagram showing a skill management table. In this skill management table, the skill name and the account ID of the presenter who possesses that skill are associated and managed for each skill ID. As shown in Figure 10(B), it is also possible to manage cases where the same presenter (001) possesses multiple skills, such as Specialty 1 and Specialty 2.

[0073] (Login information management table) Figure 11(A) is a conceptual diagram showing the login information management table. In this login information management table, the version, account ID, and last login date and time are associated and managed for each reference number ID. The version is the version of the client application (the application for store terminal 1 and the presenter terminal 9) at the time of login. This makes it possible to know which account is using which version when an application is updated.

[0074] (Subgroup management table) Figure 11(B) is a conceptual diagram showing a subgroup management table. In this subgroup management table, the subgroup name, group ID, and account IDs of the presenters belonging to the subgroup are associated and managed for each subgroup ID. As shown in Figure 10(B), it is also possible to manage cases where the same presenter (account ID "001") possesses multiple skills, such as Specialty 1 and Specialty 2. In the case of a store terminal account ID, as shown in Figure 1, the subgroup indicates whether it is a staffed store or an unstaffed store. In the case of a presenter's account ID, the subgroup indicates whether it is an organization that prioritizes staffed stores or an organization that prioritizes unstaffed stores.

[0075] According to the subgroup management table, for example, the store terminal with account ID "003" (Tokyo Main Store Unit 1) will prioritize sending connection request information (first transmission) indicating a request to connect a communication session for a video call to the presenter terminals used by presenters with account IDs "001" and "002" as a preliminary step to requesting the start of a video call (see S101 in Figure 25) (see S84-1 and S85-1 in Figure 24 below). If there is no response from the presenter terminals of the presenters with account IDs "001" and "002" within a certain period of time (see S204 in Figure 25 below), the store terminal with account ID "003" (Tokyo Main Store Unit 1) will send a call start request to a designated presenter terminal of a presenter other than account IDs "001" and "002" who has the same skills and is online (capable of making a call). This will result in a simultaneous transmission (second transmission) (see S206 in Figure 25, described later).

[0076] (User update information management table) Figure 11(C) is a conceptual diagram showing the user update information management table. In this user update information management table, the account ID to be updated, the planned update version, the date and time of the update application, and the download URL are associated and managed for each update ID.

[0077] (Functional configuration of the service information management device) Next, we will explain in detail the functional configurations of the service information management device 7. In addition, when explaining the functional configurations of the service information management device 7, we will also explain the relationship between each component shown in Figure 3 and the main components necessary to realize each functional configuration of the service information management device 7.

[0078] The transmitting / receiving unit 11 of the service information management device 7 shown in Figure 5 is implemented by commands from the CPU 101 shown in Figure 3 and the network I / F 109, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 100.

[0079] The decision unit 75 is implemented by instructions from the CPU 101 shown in Figure 3 and performs various decisions.

[0080] The memory and read processing unit 79 is implemented by instructions from the CPU 101 and the HDD controller 105 shown in Figure 3, and performs processing such as storing various data in the memory unit 7000 and reading various data stored in the memory unit 7000.

[0081] [Processing or operation of the embodiment] Next, the processing or operation in the communication system according to this embodiment will be explained using Figures 12 to 26. Hereinafter, the store terminal 1, which requests the initiation of a video call, will be referred to as the "initiating terminal," and the presenter terminal 9, which is the destination (relay destination), will be referred to as the "destination terminal."

[0082] <Service information registration process> First, using Figures 12 to 19, we will explain the process of registering various service information from the administrator terminal 4 to the service information management device 7. The service information is managed in the tables shown in Figures 9 to 11.

[0083] Figure 12 is a sequence diagram showing the service information registration process. Figure 13 is a diagram showing the login screen displayed on the administrator terminal. Figure 14 is a diagram showing the store list screen displayed on the administrator terminal. Figure 15 is a diagram showing the presenter list screen displayed on the administrator terminal. Figure 16 is a diagram showing the group list screen displayed on the administrator terminal. Figure 17 is a diagram showing the presenter information editing screen displayed on the administrator terminal. Figure 18 is a diagram showing the group information editing screen displayed on the administrator terminal. Figure 19 is a diagram showing the store information editing screen displayed on the administrator terminal.

[0084] As shown in Figure 12, administrator ad1 uses administrator terminal 4 to access service information management device 7 via communication network 100. The web browser (display control unit 44) of administrator terminal 4 displays the screens shown in Figures 13 to 19 on the display 106, and the transmitting / receiving unit 41 transmits the service information entered or selected on each screen by administrator ad1 to the service information management device 7 (S1). The service information management device 7 then receives the service information via the transmitting / receiving unit 71, and the storage / reading processing unit 79 stores the service information in the service information management DB 7001. As a result, the information is managed in the tables shown in Figures 9 to 11.

[0085] The screens are displayed in the order shown in Figures 13 to 19. For example, in the group information editing screen shown in Figure 18, administrator ad1 enters the account ID of the presenter at the presenter terminal 9 to whom the call initiation request should be sent as priority in the "Presenter" field of the subgroup (for unmanned stores). Furthermore, in the store information editing screen shown in Figure 19, the subgroup name (in this case, "for unmanned stores") is entered in the "Subgroup to call as priority" field. As a result, store terminal 1 can manage the presenter terminal 9 to which the call initiation request should be sent as priority by obtaining service information from the service information management device 7 (see S19 in Figure 20 described later) (see S20).

[0086] <Preparation process for video calls> Next, Figures 20 to 23 will be used to explain the process of preparing for video calls for remote customer service. Figure 20 is a sequence diagram showing the process of preparing a call on a store terminal.

[0087] First, when a customer enters the store and turns on the power switch of store terminal 1, the reception unit 12 receives the power-on signal and starts up store terminal 1 (S11). Once store terminal 1 starts up, the communication application installed on store terminal 1 starts up (S12). From this point onward, processing on store terminal 1 is executed according to the instructions of the communication application. Then, the storage / reading processing unit 19 reads local configuration information from the storage unit 1000 (S13). As a result, store terminal 1 completes the preparation of settings such as the background image of the display screen, login ID, and password.

[0088] Next, the transmitting / receiving unit 11 of the store terminal 1 sends a login request to the service information management device 7 via the communication network 100 (S14). This login request includes the login ID and password read in step S13. As a result, the transmitting / receiving unit 71 of the service information management device 7 receives the login request. When a login request is sent from the store terminal 1 to the service information management device 7, the receiving device, the service information management device 7, can obtain the IP address of the transmitting device, the store terminal 1.

[0089] Next, the determination unit 75 of the service information management device 7 searches the terminal management table (Figure 10(A)) of the service information management DB 7001 using the login ID and password included in the login request as search keys, and performs authentication by determining whether the same login ID and password are managed in this terminal management table (S15).

[0090] Then, the transmitting / receiving unit 71 of the service information management device 7 transmits authentication result information indicating the authentication result to the store terminal 1 (S16). As a result, the transmitting / receiving unit 11 of the store terminal 1 receives the authentication result information.

[0091] Next, the transmitting / receiving unit 11 of the store terminal 1 sends a request for service information to the service information management device 7 (S17). As a result, the transmitting / receiving unit 71 of the service information management device 7 receives the request for service information.

[0092] Next, in the service information management device 7, the storage / reading processing unit 79 reads service information from the service information management DB 7001 (S18). Then, the transmitting / receiving unit 71 transmits the service information to the store terminal 1 (S19). As a result, the transmitting / receiving unit 11 of the store terminal 1 receives the service information. Next, in the store terminal 1, the storage / reading processing unit 19 sets the service information (S20).

[0093] <Starting a video call> First, the creation unit 54 of the store terminal 1 creates a session ID to identify its own communication session (S22). Next, the transmitting / receiving unit 11 transmits login request information to the communication management device 5 via the communication network 100 (S23). This login request information includes the session ID created in step S22, the communication ID of the terminal obtained in step S19, and the password. Here, the communication ID is the start terminal ID used to identify the store terminal 1 that sent the login request in step S23.

[0094] Next, the storage and retrieval processing unit 59 of the communication management device 5 uses the communication ID and password contained in the login request information received via the transmitting and receiving unit 51 as search keys to search the authentication management table (see Figure 7(A)) and performs terminal authentication by determining whether the same communication ID and password are being managed (S24). Here, we will continue the explanation assuming that the same communication ID and password are being managed by the storage and retrieval processing unit 59.

[0095] If the memory / read processing unit 59 determines that the same communication ID and password are being managed and that the login request is from a terminal with legitimate usage rights, the memory / read processing unit 59 changes the operational status field of the communication ID record received in step S23 to "Online (Call Available)" in the terminal state management table (see Figure 7(B)) and stores the reception date and time in step S23 when the login request was received in step S23 in the reception date and time field (S25). As a result, for example, in the terminal state management table, the communication ID "01aa" will be managed in association with the operational status "Online (Call Available)", reception date and time "2019.4.10.13:40", and IP address "1.2.1.4". Note that the terminal's IP address is not registered in advance, but may be sent from the store terminal 1 in step S23.

[0096] Next, the storage and retrieval processing unit 59 adds and manages the new record containing the communication ID and password of the store terminal 1 received in step S23 in the session management table (see Figure 8(B)) (S26). Then, the transmitting and receiving unit 51 of the communication management device 5 transmits the authentication result information, which indicates the authentication result obtained through the processing in step 24, to the store terminal 1 that made the login request via the communication network 100 (S27).

[0097] When the transmitting / receiving unit 11 of the login request terminal (store terminal 1) receives authentication result information indicating that it has been determined to be a terminal with legitimate usage rights, the transmitting / receiving unit 11 transmits destination list request information indicating that it requests a destination list to the communication management device 5 via the communication network 100 (S28). As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the destination list request information.

[0098] Next, the memory and read processing unit 59 searches the destination list management table (see Figure 8(A)) using the communication ID "01ab" of the login request terminal (store terminal 1) as the search key, and reads the communication IDs of candidate destinations that can communicate with the login request terminal (store terminal 1), and also reads the destination names corresponding to these communication IDs from the terminal status management table (see Figure 7(B)) (S29). Here, the communication IDs of each candidate destination corresponding to the communication ID "01ab" of the login request terminal (store terminal 1) and the corresponding destination names are extracted.

[0099] Next, the storage / read processing unit 59 of the communication management device 5 reads the destination list frame data from the storage unit 5000 (S30). Then, the transmitting / receiving unit 51 transmits the destination list information (destination list frame, communication ID, destination name), which includes this destination list frame and the communication ID and destination name read by the storage / read processing unit 59, to the login request terminal (store terminal 1) (S31). As a result, the transmitting / receiving unit 11 of the login request terminal (store terminal 1) receives the destination list information, and the storage / read processing unit 19 stores the destination list information in the storage unit 1000 (S32).

[0100] Thus, in this embodiment, instead of each terminal managing destination list information, the communication management device 5 centrally manages the destination list information for all terminals. Therefore, even if a new terminal is added to the communication system, a new model of terminal is added to replace an existing terminal, or the appearance of the destination list frame is changed, the communication management device 5 handles everything in one place, eliminating the need for each terminal to change its destination list information.

[0101] Furthermore, the storage and read processing unit 59 of the communication management device 5 searches the terminal status management table (see Figure 7(B)) using the communication ID of the destination candidate read above as a search key, and reads the corresponding operating status for each communication ID, thereby obtaining the operating status of each communication terminal as a destination candidate (S33).

[0102] Next, the transmitting / receiving unit 51 transmits "terminal status information," which includes each communication ID used as a search key in step S26 and the operating status of each corresponding destination terminal, to the starting terminal (store terminal 1) via the communication network 100 (S34).

[0103] Next, the memory and read processing unit 19 of the starting terminal (store terminal 1) sequentially stores the terminal status information received from the communication management device 5 in the storage unit 1000 (S35). Therefore, by receiving the status information of each of the above terminals, the starting terminal (store terminal 1) can obtain the current operating status of each of the destination candidates, such as the presenter terminal 9, which can communicate with the starting terminal (store terminal 1).

[0104] Next, the display control unit 14 of the login request terminal (store terminal 1) creates a destination list that reflects the status of the terminal as a destination candidate, based on the destination list information and terminal status information stored in the storage unit 1000 (S36). This destination list will be displayed on the store terminal 1 later, as shown in Figure 23(C).

[0105] Meanwhile, the storage and read processing unit 59 of the communication management device 5 searches the destination list management table (see Figure 8(A)) based on the communication ID of the login request terminal (store terminal 1) to extract the communication IDs of other terminals that have registered the communication ID of the login request terminal (store terminal 1) as a destination candidate (S37).

[0106] Next, the storage and read processing unit 59 of the communication management device 5 searches the terminal status management table (see Figure 7(B)) based on the communication ID of the login request terminal (store terminal 1) and obtains the operating status of the login request terminal (store terminal 1) (S38).

[0107] Then, the transmitting / receiving unit 51 transmits "terminal status information" (S39) containing the communication ID and operating status of the login request terminal (store terminal 1) obtained in step S38 to the terminals whose operating status is "Online" in the terminal status management table (see Figure 7(B)) among the terminals related to the communication ID extracted in step S37. When the transmitting / receiving unit 51 transmits the terminal status information to the presenter terminal 9, it refers to the IP address of the communication terminal managed in the terminal status management table (see Figure 7(B)) based on each communication ID. This allows the communication ID and operating status of the login request terminal (store terminal 1) to be conveyed to each of the other destination terminals that can communicate with the login request terminal (store terminal 1) as a destination candidate. As a result, the destination candidate (presenter terminal 9) can also display the status of the destination candidate as shown in Figure 23(C).

[0108] <Processing to determine the operational status of the presenter's terminals> Next, using Figure 22, we will explain the process by which store terminal 1 determines the operational status of the presenter terminals for each skill. Figure 22 is a flowchart showing the preparation process for video calls on the store terminal, and in particular, the process of determining the operational status of the presenter terminals for each skill. Note that the operational status of a skill group "Online" indicates that at least one of the destination candidates in this skill group has a call-enabled presence of "Online". The operational status of a skill group "Chat" indicates that none of the destination candidates in this skill group are in the "Online" state, and that at least one destination candidate has a call-enabled presence of "Chat". Furthermore, the operational status of a skill group "Offline" indicates that all of the destination candidates in this skill group are in the "Offline" state. Below, we will explain an example of the process for determining the operational status of a skill group.

[0109] In step S34 of Figure 21, the memory and read processing unit 19 of the store terminal 1 retrieves the information recorded in the updated record whenever the operational status of the destination management table is updated (S51). The retrieved information includes the communication ID of the presenter terminal 9ug11, the operational status indicating the latest state of the presenter terminal 9ug11 which is a destination candidate, and the skill name indicating the skill of the presenter who will be using the presenter terminal 9ug11 which is a destination candidate.

[0110] The memory and read processing unit 19 of the store terminal 1 updates each record in the skill group management DB 1002 that contains the skill name obtained in step S51. That is, if the obtained skill name is "Specialty 1, Specialty 2", the memory and read processing unit 19 updates each record in the skill group management DB 1002 that contains the skill name "Specialty 1, Specialty 2".

[0111] The following describes, as an example, the process when operational status information (operating status "Online", skill name (specialist 1, specialist 2), communication ID "01ba") is transmitted from the presenter terminal 9ug11 to the store terminal 1, and this information is acquired in step S51 (S52; Online).

[0112] First, we will explain the process of updating the record in the Skill Group Management DB 1002 table where the skill name "Specialty 1" is recorded, using a loop process. The storage / reading processing unit 19 of the store terminal 1 refers to the "Available for Call" communication ID field in the record where the skill name "Specialty 1" is recorded in the Skill Group Management DB 1002 (see (B) in Figure 6). Based on this, the determination unit 15 determines whether the "Available for Call" field contains the communication ID "01ba" obtained in step S51 (S53).

[0113] If the communication ID "01ba" obtained in step S51 is not included in the "Callable" field in the skill group management DB 1002 (S53; NO), the storage / reading processing unit 19 adds the obtained communication ID "01ba" to the "Callable" field (S54).

[0114] If the processing in step S54 is completed, or if the answer in step S53 is YES, the storage / reading processing unit 19 of the store terminal 1 refers to the "on call" field in the record where the skill name "Specialty 1" is recorded in the skill group management DB 1002 (see (B) in Figure 6). Based on this, the determination unit 15 determines whether the "on call" field contains the communication ID "01ba" obtained in step S51 (S55).

[0115] If it is determined that the "On Call" field in the Skill Group Management DB 1002 contains the communication ID "01ba" obtained in step S51 (S55; YES), the storage / reading processing unit 19 deletes the obtained communication ID from the "On Call" field (S56). As a result of the processing up to this point, whether the operating status of the presenter terminal 9ug11 transitions from "Chat" to "Online" or from "Offline" to "Online", the record in the Skill Group Management DB 1002 where the skill name "Specialist 1" is recorded will only have the communication ID of the presenter terminal 9ug11 recorded in the "On Call" field.

[0116] If the processing in step S56 is completed, or if the result in step S55 is NO, the storage / reading processing unit 19 of the store terminal 1 refers to the operational status of the group in the record where the skill name "Specialty 1" is recorded in the skill group management DB 1002 (see (B) in Figure 6). Based on this, the determination unit 15 determines whether the operational status of the group is "Online" (S57).

[0117] If the operational status of the group in the record containing the skill name "Specialty 1" is determined not to be "Online" (S57; NO), the storage / read processing unit 19 updates this operational status to "Online" (S58). As a result, when the display control unit 14 displays the screen shown in Figure 23(A), it displays a normal display that is clearly visible, such as "Domestic Travel". On the other hand, if the operational status of the group in the record containing the skill name "Specialty 1" is determined to be "Online" (S57; YES), the storage / read processing unit 19 does not update the operational status of the group in the record containing the skill name "Specialty 1". Through these processes, the operational status of the group in the record containing the skill name "Specialty 1" in the skill group management DB 1002 becomes "Online".

[0118] In step S51, the skill names "Specialty 1, Specialty 2" have been retrieved, so the memory / read processing unit 19, in a loop, similarly updates each record in the skill group management DB 1002 where the skill name "Specialty 2" is recorded.

[0119] Next, operational status information (operational status "Chat", skill name (specialist 1, specialist 2), communication ID "01ba") is sent from the presenter terminal 9ug11 to the store terminal 1, and the process when this operational status information is obtained in step S51 will be explained (S52; Chat).

[0120] First, we will explain the process of updating the record containing the skill name "Specialty 1" in the skill group management DB 1002 using a loop. The storage / reading processing unit 19 of the store terminal 1 refers to the "Call Available" field in the record containing the skill name "Specialty 1" in the table of the skill group management DB 1002. Based on this, the determination unit 15 determines whether the "Call Available" field contains the communication ID "01ba" obtained in step S51 (S63). If it is determined that the communication ID "01ba" obtained in step S51 is included in the "Available to Call" field in the skill group management DB1002 (S63; YES), the storage / reading processing unit 19 deletes the obtained communication ID "01ba" from the "Available to Call" field (S64).

[0121] If the processing in step S64 is completed, or if the result in step S63 is NO, the storage / reading processing unit 19 of the store terminal 1 refers to the "on call" field in the record of the skill name "Specialist 1" in the skill group management DB 1002. Based on this, the determination unit 15 determines whether the "on call" field contains the communication ID "01ba" obtained in step S51 (S65).

[0122] If the Skill Group Management DB 1002 determines that the "In Call" field does not contain the communication ID "01ba" obtained in step S51 (S65; NO), the storage / read processing unit 19 adds the obtained communication ID "01ba" to the "In Call" field (S66). As a result of these processes, the record in the Skill Group Management DB 1002 that records the skill name "Specialty 1" will now only record the communication ID of the presenter terminal 9ug11 in the "In Call" field.

[0123] If the processing in step S66 is completed, or if the answer in step S65 is YES, the storage / reading processing unit 19 of the store terminal 1 refers to the "Callable" field in the record where the skill name "Specialty 1" is recorded in the skill group management DB 1002. Based on this, the storage / reading processing unit 19 determines whether the number of communication IDs recorded in the "Callable" field in the record where the skill name "Specialty 1" is recorded is 0 (zero) (S67).

[0124] If the number of communication IDs recorded in the "Available for Calls" field in the record containing the skill name "Specialist 1" is 0 (zero) (S67; YES), the storage / read processing unit 19 updates the group's operational status in the record containing the skill name "Specialist 1" to "Chat" (S68). As a result, when the display control unit 14 displays the screen shown in Figure 23(A), it displays it semi-transparently, for example, as "Wedding". On the other hand, if the number of communication IDs recorded in the "Available for Calls" field in the record containing the skill name "Specialist 1" is not 0 (zero) (S67; NO), the storage / read processing unit 19 does not update the group's operational status in the record containing the skill name "Specialist 1". In other words, if a communication ID is recorded in the "Available for Calls" field in the record containing the skill name "Specialist 1", the corresponding operational status remains "Online" and is not updated.

[0125] In step S51, the skill names "Specialty 1, Specialty 2" have been retrieved, so the memory / read processing unit 19, in a loop, similarly updates each record in the skill group management DB 1002 where the skill name "Specialty 2" is recorded.

[0126] Next, the presenter terminal 9ug21 transmits operational status information (operating status "Offline", skill name "Specialist 3, Specialist 4", communication ID "01ca") to the store terminal 1, and the case in step S51 in which this operational status information is acquired will be explained (S52; Offline).

[0127] First, we will explain the process of updating the record in the skill group management DB 1002 where the skill name "Specialty 3" is recorded, using a loop process. The storage / reading processing unit 19 of the store terminal 1 refers to the "Call Available" field in the record in the skill group management DB 1002 where the skill name "Specialty 3" is recorded. Based on this, the determination unit 15 determines whether the "Call Available" field contains the communication ID "01ca" obtained in step S51 (S73).

[0128] If it is determined that the communication ID "01ca" obtained in step S51 is included in the "Available for Calls" field in the skill group management DB1002 (S63; YES), the storage / read processing unit 19 deletes the obtained communication ID "01ca" from the "Available for Calls" field (S74).

[0129] If the processing in step S74 is completed, or if the answer in step S73 is NO, the storage / reading processing unit 19 of the store terminal 1 refers to the "on call" field in the record of the skill name "Specialist 3" in the table of the skill group management DB 1002. Based on this, the determination unit 15 determines whether the "on call" field contains the communication ID "01ca" obtained in step S51 (S75).

[0130] If it is determined that the "In Call" field in the Skill Group Management DB 1002 contains the communication ID "01ca" obtained in step S51 (S75; YES), the storage / read processing unit 19 deletes the obtained communication ID "01ca" from the "In Call" field (S76). As a result of the processing up to this point, the communication ID of the presenter terminal 9ug21 is deleted from the record in the Skill Group Management DB 1002 where the skill name "Specialist 3" is recorded.

[0131] If the processing in step S76 is completed, or if the answer in step S75 is NO, the storage / read processing unit 19 of the store terminal 1 refers to the "Available for Call" and "On Call" fields in the record where the skill name "Specialty 3" is recorded in the skill group management DB 1002. Based on this, the storage / read processing unit 19 determines whether conditions IF1 and IF2 are met (S77). Condition IF1 is when the number of communication IDs recorded in the "Available for Call" field in the record where the skill name "Specialty 1" is recorded is 0, and the number of communication IDs recorded in the "On Call" field is not 0. Condition IF2 is when the number of communication IDs recorded in the "Available for Call" field in the record where the skill name "Specialty 3" is recorded is 0, and the number of communication IDs recorded in the "On Call" field is 0.

[0132] If the number of communication IDs recorded in the "Available to Call" field in the record where the skill name "Specialist 3" is recorded is 0 (zero), and the number of communication IDs recorded in the "In Call" field is not 0 (S77; IF1), the storage / read processing unit 19 updates the operational status of the group in the record where the skill name "Specialist 3" is recorded in the skill group management DB 1002 to "Chat" (S78). If the number of communication IDs recorded in the "Available to Call" field in the record where the skill name "Specialist 3" is recorded is 0 (zero), and the number of communication IDs recorded in the "In Call" field is 0 (zero) (S77; IF2), the storage / read processing unit 19 updates the operational status of the group in the record where the skill name "Specialist 3" is recorded in the skill group management DB 1002 to "Offline" (S79). As a result, when the display control unit 14 displays the screen shown in Figure 23(A), it displays it semi-transparently, for example, as "Wedding". On the other hand, if neither IF1 nor IF2 applies in step S77 (S77; NO), the storage / reading processing unit 19 does not update the group's operational status in the record where the skill name "Specialty 3" is recorded, leaving it as "Online".

[0133] In step S51, the skill names "Specialty 3, Specialty 4" have been retrieved, so the memory / read processing unit 19, in a loop, similarly updates each record in the skill group management DB 1002 where the skill name "Specialty 4" is recorded.

[0134] <Initiating video communication> Next, we will explain the process of starting video communication for remote customer service using Figures 23 and 24. Figure 23 is a diagram showing the screen displayed on the store terminal. Figure 24 is a sequence diagram showing the process of starting video communication.

[0135] In the store terminal 1, the display control unit 14 displays one of the screens shown in Figures 23(A) to 23(D) on the display 106 of the store terminal 1, and the reception unit 12 receives a selection of a skill group or a designation of an instructor from the customer (S82).

[0136] Specifically, the memory / retrieval processing unit 19 retrieves each skill name stored in the table of the skill group management DB 1002 (see Figure 6(B)) and the operating status of the skill group corresponding to that skill name. As a result, the display control unit 14 displays the skill selection screen shown in Figure 23(A) on the display 106 of the store terminal 1.

[0137] On the right side of the skill selection screen, there are skill display buttons for selecting skills (e.g., domestic travel, international travel) and a presenter list display button for displaying a complete list of presenters' destinations. Here, the "Domestic Travel" skill display button is normally displayed and is selectable by the customer. In contrast, the "International Travel" skill display button is semi-transparent and is not selectable by the customer. For example, when a customer selects the "Domestic Travel" skill display button, the reception unit 12 accepts the selection, and the display control unit 14 displays the confirmation screen shown in Figure 23(B) on the display 106. When the customer presses the "OK" button, the reception unit 12 accepts the press, and the transmitting / receiving unit 11 transmits connection request information indicating a request to connect a communication session for a video call to the desired presenter terminal 9 via the communication management device 5 (see S84-1 and S85-1 below).

[0138] On the other hand, on the skill selection screen shown in Figure 23(A), when a customer presses the button to display the list of presenters, the reception unit 12 receives the press, and the display control unit 14 displays a list of presenters on the display 106 as shown in Figure 23(C), based on the presenter name, the operating status of the presenter terminal, and the skill name stored in the table of the presenter management DB 1001 (see Figure 6(A)). Here, the customer selects the name of the presenter who will use the desired presenter terminal from among the presenter terminals that can make calls, and presses the "OK" button. The reception unit 12 receives the presenter nomination, and the display control unit 14 displays the confirmation screen shown in Figure 23(D) on the display 106. When the customer presses the "OK" button, the reception unit 12 receives the press, and the transmitting / receiving unit 11 sends a request to start a video call to the nominated presenter terminal 9 via the communication management device 5 (see the process in S90, Figure 25, described later).

[0139] Next, in the store terminal 1, when the customer selects a desired skill group (for example, an organization that can explain domestic travel) on the skill selection screen in Figure 23(A), the decision unit 15 determines whether to send a video call start request to an online (voice-enabled) presenter terminal in a specific skill group (for example, domestic travel) of the unmanned store priority support organization UG1, or to send a video call start request to an online (voice-enabled) presenter terminal in the same specific skill group (for example, domestic travel) of the manned store priority support organization MG1 (S83). In this case, the store terminal 1 makes the decision based on the subgroup name to which its account ID belongs in the subgroup management table (see Figure 11(B)) within the service information received in step S19. If the terminal (store terminal 1) belongs to an unmanned store, the decision unit 15 sets the destination of the start request to a specific skill group of the unmanned store priority support organization UG1. On the other hand, if the terminal (store terminal 1) belongs to a staffed store, the determination unit 15 sets the destination of the start request to a specific skill group of the staffed store priority response organization MG1.

[0140] Next, we will explain the case in Figure 1 where, if store terminal 1 is store terminal 1c2 installed in unmanned store US, a request to start a video call is preferentially sent to the online (voice-enabled) presenter terminals 9ug11 and 9ug12 of the organization SR21 responsible for skill SK1 of the unmanned store priority response organization UG1 (first time).

[0141] First, the transmitting / receiving unit 11 of the store terminal 1c2, as a preliminary step to the video call initiation request (see S101 in Figure 25), simultaneously transmits connection request information indicating a request to connect a communication session for video calls as the first transmission (S84-1). This connection request information includes the communication ID whose corresponding operating status is "Online" among the communication IDs read in step S83, the communication ID of the store terminal 1c2 that initiated the initiation request, and the skill name indicating the skill selected in step S82.

[0142] Next, when the transmitting / receiving unit 51 of the communication management device 5 receives connection request information transmitted by the store terminal 1c2, it transmits this connection request information to the presenter terminal 9ug11, which is identified by the communication ID contained in the received connection request information (S84-2).

[0143] If the operational status is updated in step S34 (see Figure 21) between the time the store terminal 1c2 sends connection request information and receives a response, the store terminal 1c2 repeats the processing in steps S82 and S83. As a result, the operational status of the presenter terminal 9 is updated to "Online," and if a communication ID is extracted as a communication ID corresponding to the operational status "Online," the transmit / receive unit 11 of the store terminal 1c2 sends connection request information including the communication ID to the communication management device 5 (S85-1). The connection request information sent here also includes the skill name indicating the skill selected in step S82 and the communication ID of the terminal itself.

[0144] When the transmitting / receiving unit 51 of the communication management device 5 receives connection request information transmitted by the store terminal 1, it transmits this connection request information to the presenter terminal 9ug12, which is identified by the communication ID contained in the received connection request information (S85-2).

[0145] Each of the transmitting and receiving units 91 in the presenter terminals 9ug11 and 9ug12 receives connection request information transmitted from the store terminal 1c2 via the communication management device 5. Upon receiving this connection request information, the display control units 94 in the presenter terminals 9ug11 and 9ug12 display a message on the display 106 of each presenter terminal 9ug11 and 9ug12 that includes information about the requesting party corresponding to the communication ID included in the connection request information, and the skill name (skill information) corresponding to the skill name included in the call initiation request. By indicating the skill in the message, the presenter in each presenter terminal 9ug11 and 9ug12 can understand in advance what kind of travel product the customer wants.

[0146] Next, when each presenter terminal 9ug11, 9ug12 performs an operation input to grant permission to respond, the receiving unit 92 of each presenter terminal 9ug11, 9ug12 receives the operation input to grant permission to respond (S86-1, S86-2). Once the operation input to grant permission to respond is received, the transmitting / receiving unit 91 of each presenter terminal 9ug11, 9ug12 transmits response information indicating that connection is possible to the communication management device 5 (S87-1, S87-2). This response information includes the communication ID of the source (store terminal 1c2) and the communication ID of the terminal itself (9ug11, 9ug12). As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the response information transmitted by each presenter terminal 9ug11, 9ug12. The above process is just one example, and the transmitting / receiving unit 91 of each presenter terminal 9ug11, 9ug12 may automatically send response information to the store terminal 1c2 when it receives connection request information from the store terminal 1c2. In addition, the transmitting / receiving unit 11 of each presenter terminal 9ug11, 9ug12 may send response information to the store terminal 1 that first received connection request information when it receives multiple connection request information from multiple store terminals 1.

[0147] Next, the transmitting / receiving unit 51 of the communication management device 5 transmits the received response information to the store terminal 1c2, which is identified by the communication ID contained in the response information (S87-2, S88-2).

[0148] When the transmitting / receiving unit 11 of the store terminal 1c2 receives multiple response messages, the determination unit 15 determines which of the two is the presenter terminal 9 that sent the earliest response message (S89). Hereafter, this embodiment will describe the case where the presenter terminal 9ug11 is the first to respond.

[0149] The transmitting / receiving unit 51 of the communication management device 5 controls the establishment of a communication session for video calls by transmitting image data and sound data content data via the relay device 30 between the presenter terminal 9ug11 and the store terminal 1, which are identified by the communication IDs "communication ID of the store terminal, communication ID of the presenter" included in the communication establishment request (S90). This step S90 will be explained in detail later with reference to Figure 25. Note that the communication session is not limited to the above, and may also be a session for directly transmitting content data via the communication network 100 between the presenter terminal 9ug11 and the store terminal 1c2. Once communication is established between the store terminal 1c2 and the presenter terminal 9ug11, each store terminal 1c2 and presenter terminal 9ug11 transmits image data of images captured on their own terminal and sound data collected on their own terminal to the other terminal via the relay device 3. As a result, a video call can be started between the customer and the presenter, as shown in Figure 2.

[0150] When communication is established in step S90, the transmitting / receiving unit 91 of the presenter terminal 9ug11 transmits status information including the operating status "Chat", the skill name "Specialist 1", and the communication ID of the presenter terminal 9ug11 to the communication management device 5.

[0151] Furthermore, once communication is established in step S90, the transceiver unit 11 of the store terminal 1c2 sends cancellation information to the communication management device 5 to cancel the call initiation request to all response sources other than the source of the first received response (S91-1). The cancellation information includes the communication ID of the store terminal 1c2 and the communication ID of the presenter terminal 9 (in this case, presenter terminal 9ug12) of the response source other than the source of the first received response.

[0152] When the transmitting / receiving unit 51 of the communication management device 5 receives cancellation information, it transmits the cancellation information to the presenter terminal 9ug12, which is identified by the communication ID contained in the cancellation information (S91-2). The transmitting / receiving unit 11 of the presenter terminal 9ug12 receives the cancellation information transmitted by the communication management device 5. As a result, the presenter terminal 9ug12 can understand that it is no longer necessary to respond to the call initiation request from the store terminal 1. <Communication establishment control process> Next, we will explain in detail the communication establishment control process shown in Figure 24 (see S90) using Figure 25. Figure 25 is a sequence diagram showing the detailed communication establishment control process.

[0153] First, the store terminal 1c2 transmits start request information to the communication management device 5, indicating a request to start video communication (S101). This start request information includes the communication ID of the initiating terminal (in this case, the store terminal 1c2) and the communication ID of the presenter ug11 at the destination terminal (in this case, the presenter terminal 9ug11). The communication management device 5 then receives the start request information.

[0154] Next, in the communication management device 5, the selection unit 53 selects the nearest relay device based on the IP addresses of the starting terminal and the destination terminal (S102). The IP addresses of each terminal are managed in the terminal management table, and the IP addresses of each relay device are pre-stored in the storage unit 5000. Next, the creation unit 54 creates communication information including the IP address of the relay device and the communication ID mentioned above (S103). Then, the storage and reading processing unit 59 manages the communication information by storing the communication ID and the IP address of the relay device, which are the contents of the communication information, as well as the communication IDs of the presenters of the starting terminal and the destination terminal received in step S101, in association with each other in the communication information management DB 5005 (see Figure 8(C)). After that, the transmitting and receiving unit 51 sends start request information to the destination terminal (in this case, the presenter terminal 9ug11) (S105). This start request information includes the communication IDs sent from the starting terminal in step S101, as well as the communication information. As a result, the transmitting and receiving unit 91a of the destination terminal receives the start request information.

[0155] Next, at the destination terminal, the determination unit 95 of the presenter terminal 9ug11 determines whether the communication ID of the start terminal (store terminal 1c2) received in step S105 is the same as the communication ID of store terminal 1c2 that was already received in the process of step S84-2 in Figure 24 (S106). In this case, they are basically the same, so the transmitting / receiving unit 91a transmits a response-ready information to the communication management device 5 (S107). The response-ready information in this case indicates that a response is possible and includes the communication ID of the start terminal. As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the response-ready information.

[0156] Next, the storage and retrieval processing unit 59 of the communication management device 5 searches the communication information management DB 5005 using the communication ID of the initiating terminal received in step S107 as the search key, and retrieves the corresponding communication information (communication ID, relay device IP address) (S108). Then, the transmitting and receiving unit 51 transmits response-ready information to the initiating terminal. In this case, the response-ready information includes the communication information retrieved in step S108. As a result, the transmitting and receiving unit 11 of the initiating terminal receives the response-ready information.

[0157] Next, the transceiver 91 of the destination terminal transmits establishment request information to the IP address of the relay device included in the communication information received in step S105, indicating a request to establish a mutual communication session (S110). This establishment request information includes the communication ID of the initiating terminal and the communication ID of the presenter at the destination terminal. The IP address of the destination terminal, which is the source, is also transmitted. Meanwhile, the transceiver 11 of the initiating terminal transmits establishment request information to the IP address of the relay device included in the communication information received in step S109, indicating a request to establish a mutual communication session (S111). This establishment request information includes the communication ID of the initiating terminal and the communication ID of the presenter at the destination terminal. The IP address of the initiating terminal, which is the source, is also transmitted. As a result, a communication session for mutual communication of images and sound is established between the relay device 3 and the initiating terminal (S112-1), and a communication session for mutual communication of images and sound is established between the relay device 3 and the destination terminal (S112-2).

[0158] Next, the transmitting / receiving unit 11 of the initiating terminal transmits status information of its own terminal (initiating terminal) to the communication management device 5 (S113-1). This status information includes the communication ID of the initiating terminal and operational status information (here, "communicating"). As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the status information of the initiating terminal. Meanwhile, the transmitting / receiving unit 91 of the destination terminal transmits status information of its own terminal (destination terminal) to the communication management device 5 (S113-2). This status information includes the communication ID of the destination terminal and operational status information (here, "Chat (communicating)"). As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the status information of the destination terminal. In this way, each terminal notifies the communication management device 5 of its own operational status. Then, in the communication management device 5, the storage and read processing unit 59 changes the operational status associated with the communication ID of the starting terminal to "Chat (communicating)" in the terminal management table (see Figure 10), and also changes the operational status associated with the communication ID of the destination terminal to "Chat (communicating)" (S114). As a result, the starting terminal (store terminal 1c2) and the destination terminal (presenter terminal 9ug11) can make a video call through mutual communication, as shown in Figure 2. The above describes the processing when a skill group such as "Domestic Travel" is selected in step S82 in Figure 23(A).

[0159] <Request to initiate a video call when a presenter is designated> Next, we will explain the process of starting a video call when a presenter is designated in Figure 23(C) in step S82. In this case, in Figure 24, only the process in step S90 is executed from steps S84 onwards.

[0160] Furthermore, if a presenter is designated, the processes in steps S84-2 and S85-2 in Figure 24 are not performed. Therefore, in step S106 in Figure 25, the display control unit 94 displays the communication ID of the starting terminal, allowing the designated presenter to manually select whether or not to respond. Once the designated presenter has selected whether or not to respond, the reception unit 92 accepts the selection.

[0161] Then, in steps S107 and S109, response availability information is sent instead of response availability information. In this case as well, the communication ID of the initiating terminal is included. The other processing is the same as when a skill group is selected, so the explanation is omitted.

[0162] <Processing to change the recipient of the video call initiation request> Next, using Figure 26, we will explain the process of changing the destination of the video call start request when all presenter terminals do not respond to the start request in steps S86-1 and S86-2 of Figure 24. Figure 26 is a flowchart showing various processes for starting a call. Specifically, Figure 26 is a flowchart showing three patterns: a start request after selecting a skill group, a change in the destination of the start request, and a start request when a presenter is designated.

[0163] The decisions made by the decision unit 15, as described below, are based on the information managed in the presenter management DB 1001 and the skill group management DB 1002 of the storage unit 1000.

[0164] First, in the store terminal 1c2 of the unmanned store US, as a result of the processing in step S82 shown in Figure 24, the determination unit 15 determines whether it is a selection of a skill group or a designation of an explainer (S201). If it is a selection of a skill group (S201; skill), the determination unit 15 determines whether there is an explainer terminal of an unmanned store priority support organization that is online (callable) for the accepted skill group within the same area E1 (S202). If there is an explainer terminal of an unmanned store priority support organization that is online (callable) (S202; YES), the transmitting / receiving unit 11 of the store terminal 1c2 sends connection request information indicating a request to connect a communication session for video call to each explainer terminal of the unmanned store priority support organization that is online (callable), as a preliminary step to the first simultaneous request to start a video call (see S101 in Figure 25) (S203). The processing in step S203 corresponds to the processing in steps S84-1 and S85-1 in Figure 24.

[0165] Next, after processing in steps S84-1 and S85-1, the determination unit 15 determines whether or not it has received response information from the presenter terminals 9ug11 and 9ug12 within a certain time period (for example, 30 seconds) (S87-2, S88-2) (S204). If response information is received within the certain time period (S204; YES), the store terminal 1c2 performs communication establishment control for the presenter terminal that responded first (in this case, presenter terminal 9ug11) (S205). This process in step S205 corresponds to the processes in steps S89 and S90 in Figure 24. In this case, the process in Figure 26 is completed.

[0166] On the other hand, if in step S202 a response indicating that connection is not possible is received (NO), or if in step S204 a response indicating whether connection is possible is not received within a predetermined time (for example, 30 seconds) after sending the connection request information (NO), the transmitting / receiving unit 11 of the store terminal 1c2 changes the destination to each of the presenter terminals 9mg11 and 9mg12 of the organization that prioritizes online (call-enabled) manned stores, and sends the connection request information to them all simultaneously as a second attempt (S206). After processing in steps S84-1 and S85-1 in the second attempt, the determination unit 15 determines whether or not it has received response information (S87-2 and S88-2) from the presenter terminals 9mg11 and 9mg12, respectively, within a predetermined time (S207). If response information is received within a predetermined time (S207; YES), in step S205 the store terminal 1c2 performs communication establishment control for the presenter terminal that responded first (in this case, presenter terminal 9mg11). Furthermore, if no response information is received within a predetermined time (S207; NO), the display control unit 14 displays an error dialog indicating that there is no response on the terminal's display 106 (S208). After that, the process returns to step S201.

[0167] By the way, in step S201 above, if the presenter is designated (S201; designation), the display control unit 14 displays the list of presenters shown in Figure 23(C) on the display 106 of its own terminal (S209). Then, the reception unit 12 receives the designation (selection) of the presenter from customer c2 (S210). The processing in steps S209 and S210 corresponds to the processing in step S82 in Figure 24.

[0168] Furthermore, as a result, the store terminal 1c2 performs the communication establishment control process only for the designated presenter terminal (in this case, presenter terminal 9ug11) (S211). The process in step S211 corresponds to the process in step S90 in Figure 24.

[0169] <Example of a screen during a video call between the presenter's terminal and the store's terminal> Figure 27 is an example of a call screen displayed on the presenter's terminal. The presenter's terminal's call screen 930 includes a main image area 940, a sub-image area 950, and a button display area 960. The button display area 960 displays a hold button 961, a travel plan button 962, a display switching button 963, a camera button 964, a microphone button 965, and a disconnect button 966. The main image area 940 is displayed across the entire call screen 930 of the presenter's terminal 9, while the sub-image area 950 is displayed in a smaller area than the main image area 940 in a portion of the call screen 930 (in this case, the lower right).

[0170] The hold button 961 is a button that stops the terminal device from transmitting images and audio while maintaining communication. It is used when the presenter is away from their seat for a short time. The store terminal 1 receives a notification that the hold button 961 has been pressed. When the reception unit 92 of the presenter terminal 9 detects that the hold button has been pressed, it reads a predetermined image from its storage unit and displays it on the display of the presenter terminal 9. For example, by displaying a message such as "The presenter is away from their seat," it helps to prevent the customer from feeling uneasy. The image to be displayed can be set by the store staff, so advertisements or videos can also be displayed.

[0171] The travel plan button 962 is a function that activates a mode for creating a travel plan while listening to the customer's requests.

[0172] The display switching button 963 allows you to hide the customer's image displayed in the main image field 940 and switch to displaying only the background image. When the presenter presses the travel plan button 962 and creates a travel plan with the customer, this function temporarily hides the customer's image when displaying local images (360° panoramic video) 941, as shown in Figure 30, in order to show the customer the appropriate area of ​​the location.

[0173] The camera button 964 and microphone button 965 are buttons used by the presenter to switch the camera and mute the microphone on their presenter terminal 9.

[0174] The disconnect button 966 is used to disconnect communication between terminals (the presence status changes from Chat to Online). Only the presenter can disconnect the communication.

[0175] Next, we will explain the call screen displayed by the store terminal using Figure 28. Figure 28 is an example of a call screen displayed on a store terminal. As shown in Figure 28, the call screen 130 displayed on store terminal 1 includes a main image area 140, a sub-image area 150, and a button display area 160. The button display area 160 displays a display switching button 163, a camera button 164, a microphone button 165, and a disconnect button 166. The display switching button 163, camera button 164, microphone button 165, and disconnect button 166 perform the same functions as the display switching button 963, camera button 964, microphone button 965, and disconnect button 966 in Figure 27. The customer can only view the sub-image area (where the customer's own face is displayed) 150, switch the display / hide of the presenter displayed in the main image area 140, and operate the microphone and camera. The main image area 140 is displayed across the entire call screen 180 of the store terminal 1, while the sub-image area 150 is displayed in a smaller size than the main image area 140, on a portion of the call screen 180 (in this case, the lower right).

[0176] <Travel plan proposal> Next, we will explain the process by which the presenter proposes a travel plan to the customer via remote call. During the customer interaction, the presenter listens to the customer's (guest's) desired travel destination and inputs countries and regions from a world map, or inputs tourist spots specified by the customer (guest). After input, a map is displayed on the screen. When a tourist spot within the area of ​​the map is selected, an image (360-degree video) is displayed on the background image, allowing the customer to plan their trip while experiencing the atmosphere of the place. A detailed explanation follows below.

[0177] Figure 29 is a sequence diagram showing the process of the presenter terminal and store terminal initiating a video call from a dedicated application and creating a travel plan. Figure 30 is an example of the call screen (including the input window and map window) displayed on the presenter terminal. Figure 31 is an example of the call screen (input window and map window) displayed on the presenter terminal.

[0178] First, as shown in Figures 27 and 28, the store terminal 1 and the presenter terminal 9 are making a video call. When the presenter creates a travel plan with the customer, they press the travel plan button 962 displayed on the call screen of the presenter terminal shown in Figure 27, and the reception unit 92 accepts the disclosure of the travel plan mode and activates the travel plan creation mode. When this "travel plan creation mode" is activated, as shown in Figure 30, the display control unit 94 displays an input window 970 in the upper left of the main image field 940 of the call screen 930, where the customer can input desired tourist destinations, accommodations, dates of stay, number of people, number of rooms, etc., and a map window 980 in the lower left that includes landmarks of the tourist destinations entered in the input window 970. Here, the presenter hears the customer's desired tourist destinations via remote call and enters the conditions according to the customer's requests into the items displayed in the input window 970, and the reception unit 92 accepts the input of the conditions according to the customer's requests. For example, the presenter enters Okinawa, Okinawa Prefecture, and Island 1 as the tourist destinations where the customer will stay.

[0179] Upon receiving this input, the display control unit 94 displays a map corresponding to the remote island 1 in the map window 980, as shown in Figure 31. This map displays multiple landmarks indicating the area that the presenter can introduce. In this case, each hotel on the remote islands, centered around Miyako Island in Okinawa Prefecture, is displayed as a landmark.

[0180] Next, the presenter selects a specific landmark 981 on the map window 980 while listening to the customer's request, and the reception unit 92 accepts the selection of the tourist destination related to the specific landmark 981 (S302). Once the specific landmark 981 is selected, the transmitting / receiving unit 91 of the presenter terminal 9 sends a request for an image of the tourist destination to the communication management device 5 via the communication network 100 (S303). At this time, the request sent from the presenter terminal 9 includes a landmark ID to identify the specific landmark 981. As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the request that was sent in step S317.

[0181] Next, in the communication management device 5, based on the landmark ID included in the request received from the presenter terminal 9 by the transmitting / receiving unit 51, the storage / reading processing unit 59 reads the file name of the tourist destination image corresponding to the landmark ID stored in the image information management DB 5006 (see Figure 8(D)), and reads the image data of the tourist destination with this file name from the storage unit 5000 (S304). Then, the transmitting / receiving unit 51 of the communication management device 5 transmits the image data of the tourist destination to the presenter terminal 9 (S305) and to the store terminal 1 (S306). As a result, the transmitting / receiving unit 91 of the presenter terminal 9 and the transmitting / receiving unit 11 of the store terminal 1 each receive the image data of the tourist destination.

[0182] Figure 32 shows an example of a call screen (including input window, map window, and tourist destination image) displayed on the presenter's terminal. Figure 33 shows an example of a call screen (including tourist destination image) displayed on the store's terminal.

[0183] In step S305, as shown in Figure 32, the display control unit 94 of the presenter terminal 9 displays an image of the tourist destination as a background image on the display 106 of the presenter terminal 9 (S307). In this case, the display control unit 94 displays the image not only as a background image for the main image field 940 where the customer's image is displayed, but also as a background image for the sub-image field 950 where the presenter's image is displayed. On the other hand, in step S306, as shown in Figure 33, the display control unit 14 of the store terminal 1 displays an image of the tourist destination as a background image on the display 106 of the store terminal 1 (S308). In this case, the display control unit 14 displays the image not only as a background image for the main image field 140 where the presenter's image is displayed, but also as a background image for the sub-image field 150 where the customer's image is displayed.

[0184] This allows both the presenter and the customer to share the same background image during video calls, enabling them to engage in lively conversations about tourist destinations.

[0185] Furthermore, the presenter remotely communicates with the customer to hear their preferred accommodation (in this case, "Hotel XX△"), and enters the conditions corresponding to the customer's request into the fields displayed in the input window 970. The reception unit 92 then accepts the input of the conditions corresponding to the customer's request (in this case, accommodation). (S309).

[0186] Next, the transmitting / receiving unit 91 of the presenter terminal 9 transmits a request for an image of the exterior of the accommodation facility to the communication management device 5 via the communication network 100 (S310). At this time, the request transmitted from the presenter terminal 9 includes the accommodation facility information (here, "〇×△ Hotel") as a condition accepted in step S309. As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the request sent in step S310.

[0187] Next, in the communication management device 5, based on the accommodation facility information included in the request received from the presenter terminal 9 by the transceiver unit 51, the storage / reading processing unit 59 reads the file name of the exterior image of the accommodation facility corresponding to the accommodation facility information stored in the image information management DB 5006 (see Figure 8(D)), reads the exterior image data of the accommodation facility with this file name from the storage unit 5000, and also reads the room information corresponding to the accommodation facility information received in step S310 from the image information management DB 5006 (S311). Then, the transceiver unit 51 of the communication management device 5 transmits the exterior image data and room information of the accommodation facility read in step S311 to the presenter terminal 9 (S312) and to the store terminal 1 (S313). As a result, the transceiver unit 91 of the presenter terminal 9 and the transceiver unit 11 of the store terminal 1 receive the exterior image data and room information of the accommodation facility, respectively.

[0188] Figure 34 shows an example of a call screen displayed on the presenter's terminal (including an input window, map window, image of the accommodation's exterior, and price list). Figure 35 shows an example of a call screen displayed on the store's terminal (including an image of the accommodation's exterior and price list).

[0189] In step S312, as shown in Figure 34, the display control unit 94 of the presenter terminal 9 displays an image of the exterior of the accommodation facility as a background image on the display 106 of the presenter terminal 9, and also displays a price list 990 based on room information (S314). In this case, the display control unit 94 displays the image not only as a background image for the main image field 940 where the customer's image is displayed, but also as a background image for the sub-image field 950 where the presenter's image is displayed. On the other hand, in step S313, as shown in Figure 35, the display control unit 14 of the store terminal 1 displays an image of the exterior of the accommodation facility as a background image on the display 106 of the store terminal 1, and also displays a price list 190 based on room information (S315). In this case, the display control unit 14 displays the image not only as a background image for the main image field 140 where the presenter's image is displayed, but also as a background image for the sub-image field 150 where the customer's image is displayed.

[0190] This allows both the presenter and the customer to share the same background image during video calls, enabling them to engage in lively conversations about accommodations.

[0191] Furthermore, the presenter remotely asks the customer for their desired room grade (rank) (in this case, "Deluxe") and accepts their selection of the room grade (in this case, "Deluxe") as displayed in price list 990 (S316).

[0192] Next, the transmitting / receiving unit 91 of the presenter terminal 9 transmits the request for room images received in step S316 to the communication management device 5 via the communication network 100 (S317). At this time, the request transmitted from the presenter terminal 9 includes the room ID indicating the room grade received in step S316. As a result, the transmitting / receiving unit 51 of the communication management device 5 receives the request sent in step S317.

[0193] Next, in the communication management device 5, based on the room ID included in the request received from the presenter terminal 9 by the transmitting / receiving unit 51, the storage / reading processing unit 59 reads the file name of the room image corresponding to the image ID stored in the image information management DB 5006 (see Figure 8(D)), and reads the image data of the room with this file name from the storage unit 5000 (S318). Then, the transmitting / receiving unit 51 of the communication management device 5 transmits the room image data read in step S318 to the presenter terminal 9 (S318) and to the store terminal 1 (S320). As a result, the transmitting / receiving unit 91 of the presenter terminal 9 and the transmitting / receiving unit 11 of the store terminal 1 each receive the room image data.

[0194] Figure 36 shows an example of a call screen displayed on the presenter's terminal (including an input window, map window, room image, and price list). Figure 37 shows an example of a call screen displayed on the store's terminal (including a room image and price list).

[0195] In step S319, as shown in Figure 36, the display control unit 94 of the presenter terminal 9 displays an image of the room (in this case, "Deluxe") as a background image on the display 106 of the presenter terminal 9 (S321). In this case, the display control unit 94 displays the image not only as a background image for the main image field 940 where the customer's image is displayed, but also as a background image for the sub-image field 950 where the presenter's image is displayed. On the other hand, in step S320, as shown in Figure 36, the display control unit 14 of the store terminal 1 displays an image of the room as a background image on the display 106 of the store terminal 1 (S322). In this case, the display control unit 14 displays the image not only as a background image for the main image field 140 where the presenter's image is displayed, but also as a background image for the sub-image field 150 where the customer's image is displayed.

[0196] This allows both the presenter and the customer to share the same background image during video calls, enabling them to engage in lively conversations about the room.

[0197] [Main effects of this embodiment] As described above, this embodiment allows the presenter to virtually experience the atmosphere of the destination where the customer (guest) is planning their trip. Furthermore, because the destination can be viewed from various perspectives using 360-degree video, it provides a sense of security as the customer can plan their trip while feeling the location more realistically than with photographs. As a result, both the presenter and the customer can share the same background image displayed across the entire main image area 140,940 and conduct video calls, allowing them to have engaging conversations about their trip.

[0198] Furthermore, even if a call initiation request (first attempt) is made from the store terminal US of an unmanned store to the respective presenter terminals 9ug11 and 9ug12 of the organization SK12 responsible for the desired skill SK1 of the unmanned store priority support organization UG1, and no response is received from any of the presenter terminals, a call initiation request (second attempt) can be made to the respective presenter terminals 9mg11 and 9mg12 of the organization SR11 responsible for the same skill SK1 of the manned store priority support organization MG1. This allows the store terminal to leverage the advantages of unmanned stores, such as streamlining and improving the efficiency of store operations, while also reducing customer departures from unmanned stores by installing the unmanned store priority support presenter terminals 9ug11 and 9ug12. Moreover, even if the unmanned store priority support presenter terminals 9ug11 and 9ug12 are unresponsive, the store terminal can still initiate a call to the manned store priority support presenter terminals 9mg11 and 9mg12, thereby further reducing customer loss.

[0199] 〔supplement〕 In the above embodiment, steps S302, S309, and S316 are performed, but at least two of these processes may be performed within the same step. Also, any one of steps S302, S309, or S316 may be omitted.

[0200] Furthermore, store terminal 1 is an example of a communication terminal. Store terminal 1 includes not only PCs, but also smartwatches, game consoles, and dedicated video call devices.

[0201] Furthermore, in communication between the communication management device 5 and the store terminal 1 and the presenter terminal 9, other servers, routers, etc., may relay the data. [Explanation of symbols]

[0202] 1. Store terminal (an example of a second communication terminal) 3. Relay device 4. Administrator terminal 5 Communication management device 7. Service Information Management Device 9. Presenter terminal (an example of the first communication terminal) 11 Transmitter / Receiver 12 Reception Department 13 Communication Control Unit 14 Display Control Unit 15 Judgment Department 51 Transmitting and receiving unit (Example of transmitting means, example of receiving means) 5000 Memory Unit (An example of a memory means) 100 Communication Networks

Claims

1. A communication management device that mediates video data of a subject at a first location transmitted by a first communication terminal so that a second communication terminal can receive the video data of the subject at the first location, Receiving means for receiving a request to transmit a specific background image transmitted by the first communication terminal, A transmission means that, while the second communication terminal is displaying the subject of the first location, displays the specific background image as a background image for the subject of the first location, and transmits the data of the specific background image to the second communication terminal based on the transmission request, in order to display the specific background image as a background image for the subject of the second location while the second communication terminal is displaying video data of the subject of the second location captured by the second communication terminal, A communication management device characterized by having the following features.

2. The video data transmitted by the first communication terminal is video data of a first subject captured by the first communication terminal. The second communication terminal transmits video data of the subject at the second location, which has been captured by the second communication terminal, and the receiving means receives the video data of the subject at the second location. In order for the first communication terminal to receive video data of the subject at the second location, the transmitting means transmits video data of the subject at the second location. The first communication terminal receives video data of the subject at the second location and, in order to display the specific background image as a background image for the second subject being displayed, the transmitting means transmits the data of the specific background image based on the transmission request. The communication management device according to feature 1.

3. The communication management device according to claim 2, characterized in that the transmission means transmits data of the specific background image based on the transmission request in order to display the specific background image as a background image for the subject at the first location while the first communication terminal is displaying video data of the subject at the first location captured by the first communication terminal.

4. A communication management device according to any one of claims 1 to 3, It has a management means for managing a predetermined background image data and predetermined identification information for identifying the predetermined background image in association with each other. The receiving means receives specific identification information as a request to transmit the specific background image, The communication management device is characterized in that the transmitting means transmits data of a specific background image corresponding to the specific identification information.

5. The communication management device according to any one of claims 1 to 4, characterized in that the background image is a 360-degree spherical image.

6. The communication device according to claim 5, characterized in that the aforementioned 360-degree image is a still image or a video.

7. A communication management method performed by a communication management device that mediates video data of a subject at a first location, so that a second communication terminal can receive video data of a subject at a first location transmitted by a first communication terminal, A receiving step of receiving a request to transmit a specific background image transmitted by the first communication terminal, A transmission step in which the second communication terminal displays the specific background image as a background image for the subject at the first location while the second communication terminal is displaying the subject at the first location, and transmits the data of the specific background image to the second communication terminal based on the transmission request in order to display the specific background image as a background image for the subject at the second location while the second communication terminal is displaying video data of the subject at the second location captured by the second communication terminal, A communication management method characterized by execution.

8. A program that causes a computer to perform the method described in claim 7.

9. A communication management device that manages video calls between a first communication terminal and a second communication terminal, such that a second communication terminal receives video data of a subject at a first location transmitted by a first communication terminal, and a first communication terminal receives video data of a subject at a second location transmitted by the second communication terminal, Receiving means for receiving a request to transmit a specific background image transmitted by the first communication terminal, The video data of the subject at the second location, captured by the second communication terminal, is transmitted to the second communication terminal. A transmission means that transmits data of the specific background image to the second communication terminal based on the transmission request, in order to display the specific background image as a background image for the subject of the second location while the communication terminal is displaying, It has, A communication management device characterized in that the first communication terminal receives video data of a subject at the second location and displays the specific background image as a background image for the second subject being displayed, and the transmitting means transmits data of the specific background image based on the transmission request.

Images