Information processing system and information processing method

Abstract

This invention provides an information processing system and method for automatically generating an avatar when needed. [Solution] In the information processing system, multiple customer service monitors are provided in the store. Specific actions by customers (CUS) within the store, such as touching a product or taking a catalog, are detected as trigger events. Upon detecting this trigger event (S3), the system asks the customer whether they consent to displaying an avatar (AVT) on the customer service monitor (S5). If the customer gives consent, the avatar is displayed on the customer service monitor (S9).

Description

Technical Field

[0001] This invention relates to an information processing system and an information processing method, and particularly, for example, to an information processing system and an information processing method in a store that remotely receives customers by displaying an operator's avatar.

Background Art

[0002] An example of this type of information processing system is disclosed in Patent Document 1. In the information processing system disclosed in this Patent Document 1, as an example, when a user sits in a booth in a bank, an operator's avatar appears on a display so as to face the user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Such a remote customer service is effective when face-to-face customer service is the norm, such as in a bank. However, in a store that sells goods, for example, self-checkout machines are also widespread, so an avatar is not necessarily required. That is, it is sufficient for the avatar to appear when needed. [[ID=!]]

[0005] Therefore, the main object of this invention is to provide a novel information processing system and an information processing method.

[0006] Another object of this invention is to provide an information processing system and an information processing method that can automatically make an avatar appear when needed.

Means for Solving the Problems

[0007] It seems there is an error in the original text where the "!" character is present in the ID=! line. Please check and correct the original text for a proper translation. The above translation is based on the provided text as much as possible.To address the above-mentioned issues, this disclosure employs the following configuration. Reference numerals and supplementary explanations in parentheses indicate correspondences with embodiments described to aid understanding of this disclosure and do not limit this disclosure in any way.

[0008] The first embodiment is an information processing system operated in a store where customer service is provided through an avatar image displayed on a customer service monitor, and comprises a detection means for detecting a specific customer action as a trigger event, and a display means for displaying an avatar on the customer service monitor in response to the detection of the trigger event.

[0009] In the first embodiment, the information processing system (10: a reference numeral used to illustrate a part corresponding to the embodiment, but not intended to limit it; the same applies hereinafter) is operated in a store (100) where, for example, an operator provides customer service through an avatar (AVT) displayed on a customer service monitor (102, 102A, 102B). The detection means (20, 54f, S11, S12, S13) detects a specific action of a customer as a trigger event. The display means (20, 54g, S9) displays the avatar (AVT) on the customer service monitor in response to the detection of the trigger event.

[0010] According to the first embodiment, when a specific action by the customer is detected, that is, when the customer needs it, the avatar is automatically displayed, so effective customer service can be expected through the avatar.

[0011] The second embodiment is an information processing system dependent on the first embodiment, further comprising a determination means that determines whether the person who detected a trigger event is a store employee based on camera footage from a camera installed in the store, and the display means displays an avatar when the determination means determines that the person is not a store employee.

[0012] In the second embodiment, the determination means (20, 54h, S7A) determines whether the person who detected the trigger event is an employee, based on video data from cameras (104, 120) installed in the store (100). In other words, it determines whether the person who triggered the event is a customer. If the person is not an employee, the display means displays an avatar on the customer service monitor.

[0013] According to the second embodiment, if the target of the trigger event is not a store employee, i.e., a customer, an avatar is displayed, thus ensuring that customer service is provided via avatar.

[0014] A third embodiment is an information processing system dependent on the first embodiment, further comprising an inquiry means for inquiring with a customer whether or not to allow the display of an avatar on a customer service monitor in response to the detection of a trigger event, wherein the display means displays the avatar on the customer service monitor when the customer expresses permission in response to the inquiry.

[0015] In the third embodiment, the inquiry means (20, 54g, S5) inquires the customer (CUS) whether to display an avatar (AVT) on the customer service monitor (102, 102A, 102B) by displaying a message, for example, on the customer service monitor (102, 102A, 102B) in response to detecting a trigger event. The display means (20, 54g, S9) displays an avatar (AVT) on the customer service monitor when the customer operates the permission icon (132Y), for example.

[0016] According to the third embodiment, the avatar is automatically displayed when needed by the customer, so effective customer service can be expected through the avatar.

[0017] The fourth embodiment is an information processing system dependent on the first embodiment, wherein the detection means detects a customer's product touching behavior as a trigger event.

[0018] In the fourth embodiment, when a customer (CUS) touches, for example, a product (126) on a product display stand (122), this is detected as a trigger event.

[0019] The fifth embodiment is an information processing system subordinate to the first embodiment, wherein the detection means detects, as a trigger event, a customer's catalog extraction behavior.

[0020] In the fifth embodiment, when a customer (CUS) extracts a catalog (136) from a catalog case (134) on a product display stand (122) for example, this is detected as a trigger event.

[0021] The sixth embodiment is an information processing system subordinate to the first embodiment, wherein the detection means detects, as a trigger event, the failure of a customer's action or operation on a terminal in the store that is assumed to be operated by the customer himself.

[0022] In the sixth embodiment, when an example of a terminal in the store that is assumed to be operated by the customer himself is a cash register, the failure of the cash register operation or cash register operation, such as the customer (CUS) repeating the cash register operation or interrupting the cash register operation, is detected as a trigger event.

[0023] The seventh embodiment is an information processing method applied in a store that serves customers through an avatar image displayed on a customer service monitor, which detects a specific action of a customer as a trigger event, and in response to detecting the trigger event, displays an avatar on the customer service monitor.

[0024] The eighth embodiment is an information processing system applied in a store where an operator serves customers through an avatar image displayed on a movable customer service monitor, comprising a detection means for detecting a trigger event, a moving means for moving the customer service monitor to a location related to the customer in response to detecting the trigger event, and a display means for displaying an avatar on the moved customer service monitor.

[0025] In the eighth embodiment, the information processing system (10) is operated in a store (100) where an operator serves customers through an avatar (AVT) displayed on a movable customer service monitor (10B). The detection means (20, 54f, S3) detects, for example, a trigger event such as a specific action of a customer or the opening of an elevator door. The moving means (20, 54h, S8) moves the customer service monitor (102B) to a location related to the customer, such as near a product display stand or in front of an elevator, in response to the detection means detecting a trigger event. When the customer service monitor is moved to the location, the display means (20, 54g, S9) displays an avatar (AVT) on the moved customer service monitor.

[0026] Also according to the eighth embodiment, when a specific action of a customer is detected, that is, when it is necessary for the customer, the avatar is automatically displayed, so effective customer service through the avatar can be expected.

Advantages of the Invention

[0027] According to this invention, when a specific action of a customer is detected, that is, when it is necessary for the customer, the avatar is automatically displayed, so effective customer service through the avatar can be expected.

[0028] The above object, other objects, features, and advantages of this invention will become more apparent from the following detailed description of the embodiments made with reference to the drawings.

Brief Description of the Drawings

[0029] [Figure 1] FIG. 1 is a diagram showing an information processing system according to an embodiment of this invention. [Figure 2] FIG. 2 is a diagram showing an example of a store using the information processing system according to an embodiment of this invention. [Figure 3] FIG. 3 is a block diagram showing an example of the electrical configuration of the store terminal shown in FIG. 2. [Figure 4] FIG. 4 is a block diagram showing an example of the electrical configuration of the operator terminal shown in FIG. 2. [Figure 5]Figure 5 is an illustrative diagram showing an example of a customer's product handling behavior in the store shown in Figure 2. [Figure 6] Figure 6 is an illustration illustrating another example of a customer's product handling behavior in the store shown in Figure 2. [Figure 7] Figure 7 is an illustrative diagram showing an example of a customer's catalog-taking behavior in the store shown in Figure 2. [Figure 8] Figure 8 shows an example of the RAM memory map of the store terminal shown in Figure 3. [Figure 9] Figure 9 is a flowchart illustrating one example of the overall information processing performed by the CPU of the store terminal shown in Figure 3. [Figure 10] Figure 10 is a flowchart showing an example of the trigger event detection process in the overall information processing shown in Figure 6. [Figure 11] Figure 11 is a flowchart showing an example of customer product touch motion detection, which is an example of the trigger event detection process shown in Figure 10. [Figure 12] Figure 12 is a flowchart showing another example of customer product touch motion detection, which is an example of the trigger event detection process shown in Figure 10. [Figure 13] Figure 13 is a flowchart showing an example of the trigger event detection process shown in Figure 10, which is the detection of a tag removal operation. [Figure 14] Figure 14 is a flowchart showing customer action detection, which is an example of the trigger event detection process shown in Figure 10. [Figure 15] Figure 15 shows another example of the RAM memory map of the store terminal shown in Figure 3. [Figure 16] Figure 16 is a flowchart illustrating another example of the overall information processing performed by the CPU of the store terminal shown in Figure 3. [Figure 17] Figure 17 is an illustrative diagram showing an example of a robot (avatar) that can be used in other embodiments of this invention. [Figure 18] Figure 18 shows an example of the RAM memory map of the store terminal shown in Figure 3 in another embodiment. [Figure 19] Figure 19 is a flowchart showing an example of operation in another embodiment. [Figure 20] Figure 20 is a flowchart showing an example of operation in another embodiment. [Modes for carrying out the invention]

[0030] Referring to Figure 1, the information processing system 10 of this embodiment includes a store terminal 12, which is communicated to an operator terminal 16 via a network 14.

[0031] For example, using WebRTC (Web Real-Time Communication) technology, the store terminal 12 and the operator terminal 16 can communicate via P2P (Peer to Peer) and send and receive video and / or audio in real time via a web browser.

[0032] Although not shown in the diagram, WebRTC can be used in combination with multiple servers. Specifically, the multiple servers used for WebRTC include a signaling server, a STUN server, and a TURN server. Although not explained in detail, the signaling server is a server that obtains information about the communication partner via WebRTC. The STUN server and TURN server are servers that perform so-called NAT (Network Address Translation) traversal when the communication partner is on a different network.

[0033] In this embodiment, one store terminal 12 and one operator terminal 16 are shown, but in reality, multiple store terminals 12 and multiple operator terminals 16 are connected to the network 14, and a connection state is established for one of the store terminals 12 and one operator terminal 16 that is configured to conduct video calls or web conferences.

[0034] Furthermore, if the operator terminal 16 to be connected to the store terminal 12 is predetermined, and the store terminal 12 has connection information for the operator terminal 16, and the operator terminal 16 has connection information for the store terminal 12, then the connection status between the store terminal 12 and the operator terminal 16 can be established without using the server described above.

[0035] The store terminal 12 is basically installed in a store 100 (Figure 2) to which the information processing system 10 is applied, for example, in a supermarket, and is used by a customer (CUS) (Figure 2) or an employee of the store. The operator terminal 16 is used by an operator (not shown).

[0036] The store terminal 12 is an information processing device, and is either a notebook PC or a desktop PC, but other devices such as smartphones or tablet PCs can also be used. However, when using a notebook PC, smartphone, or tablet PC, a relatively large display device for displaying avatar images may be prepared separately and connected to the notebook PC, smartphone, or tablet PC.

[0037] Network 14 consists of an IP network (or IP network) including the Internet, and an access network (or access network) for accessing this IP network. The access network can include public telephone networks, mobile phone networks, wired LANs, wireless LANs, CATV (Cable Television), etc.

[0038] The operator terminal 16 is an information processing device different from the store terminal 12. For example, it may be a notebook PC or a desktop PC, but other devices such as a smartphone or tablet PC can also be used.

[0039] Now, referring to Figure 2, we will explain the parts necessary to describe the embodiment in store 100.

[0040] Inside the store 100, a customer service monitor 102 is provided. In this embodiment, the customer service monitor 102 is a touch display with a touch panel, and when necessary, the customer service monitor 102 displays an avatar image AVT corresponding to the operator's avatar, and the operator interacts with the customer CUS through this avatar image AVT.

[0041] The operator interacts with the customer user (CUS) by controlling the avatar through the operator terminal 16, the network 14, and the store terminal 12, that is, by controlling the avatar's movements and speech. In other words, the customer user (CUS) is a user of the customer service provided by the operator at the operator terminal 16 using the avatar.

[0042] A customer service camera 104 is provided in a predetermined location related to the customer service monitor 102, for example, on the top edge of the customer service monitor 102. The customer service camera 104 is positioned to capture the face of the customer CUS in front of the customer service monitor 102, i.e., facing the customer service monitor 102. A microphone 106 is provided in the same position as the customer service camera 104 (or a different position). The microphone 106 is positioned to detect the voice of the customer CUS in front of the customer service monitor 102.

[0043] Furthermore, a speaker 108 is provided at the top of the customer service monitor 102, for example. The speaker 108 is positioned to deliver sound to the customer user (CUS) in front of the customer service monitor 102. Therefore, the operator, or avatar, interacts with the customer user (CUS) through the microphone 106 and speaker 108 while providing customer service. For example, the operator can answer questions from the customer user (CUS) or provide information to the customer user (CUS).

[0044] A payment machine 110 is also provided within the store 100. The payment machine 110 is a so-called self-checkout machine and is equipped with a scanner unit 112 for reading barcodes (not shown) attached to products, a payment unit 114 for settling payments with cards (prepaid cards, credit cards, etc.) or cash, and a display 116 for displaying the status of operations such as scanning barcodes with the scanner unit 112 and making payments using the payment unit 114.

[0045] Multiple surveillance cameras 120 are installed in predetermined locations to photograph the area around the payment machine 110 and other areas. These surveillance cameras 120 are used to monitor the payment status of customers (CUS) and to detect the faces and behavior of users (customers).

[0046] In such a store 100, the operator interacts with the customer (CUS) through an avatar AVT displayed on the customer service monitor 102 as needed. At that time, the avatar AVT's voice is output from the speaker 108, and the operator interacting with the customer (CUS) can view the video footage captured by the customer service camera 104 and the surveillance camera 120, and listen to the customer's voice detected by the microphone 106. In other words, the operator interacts with the customer (CUS) by viewing the customer's (face) video and listening to their voice through the avatar AVT.

[0047] The avatar's actions when the operator interacts with a customer (CUS) are controlled by the operator. For example, video data that performs pre-set actions is played. In this case, the operator instructs the avatar to move, and the store terminal 12 plays video data corresponding to the instructed action, and the video is displayed on the customer service monitor 102. In another example, the avatar's action data can be generated based on the operator's actions, and the avatar's actions can be controlled by making it move according to the action data, thereby controlling the avatar's image (AVT) displayed on the customer service monitor 102. The action data can be generated from captured images of the operator using an image processing library such as MediaPipe Holistic. This action data is information (data) about the position and orientation of each part of the avatar to control its gestures, facial orientation, and facial expressions.

[0048] However, if the operator is not attending to a customer, the customer service monitor 102 may either display nothing, or it may display information about the store or information about the products sold in the store.

[0049] Figure 3 is a block diagram showing an example of the electrical configuration of the store terminal 12 shown in Figure 1. As shown in Figure 3, the store terminal 12 includes a CPU 20, which is connected to RAM 22, a communication interface (communication I / F) 24, and an input / output interface (input / output I / F) 26 via an internal bus.

[0050] The CPU 20 is responsible for the overall control of the store terminal 12. However, instead of the CPU 20, a System-on-a-chip (SoC) that includes multiple functions such as CPU functionality and GPU (Graphics Processing Unit) functionality may be provided.

[0051] RAM22 is the main memory and is used as the work area or buffer area for the CPU20. Although not shown in the diagram, the store terminal 12 is equipped with an HDD and ROM as auxiliary storage devices. However, non-volatile memory such as an SSD may be used instead of the HDD, or in addition to the HDD.

[0052] The communication interface 24 is a wired interface for sending and receiving control signals and data to and from an external computer, such as an operator terminal 16, via the network 14 under the control of the CPU 20. However, a wireless interface such as Wi-Fi or Bluetooth® can also be used as the communication interface 24.

[0053] The input / output interface 26 is connected to an input device 28, a customer service monitor 102, a customer service camera 104, a microphone 106, a speaker 108, and a surveillance camera 120.

[0054] The input device 28 includes a keyboard, a computer mouse, and a touch panel. However, the touch panel is provided on the display surface of the customer service monitor 102.

[0055] The customer service monitor 102 is, for example, a liquid crystal display. The microphone 106 is a sound-collecting microphone. The speaker 108 is a stereo speaker. The customer service camera 104 and the surveillance camera 120 are CCD cameras, respectively.

[0056] Furthermore, the input / output interface 26 outputs operation data (or operation information) received from the input device 28 to the CPU 20, and outputs image data generated by the CPU 20 to the customer service monitor 102, displaying a screen or image corresponding to the image data on the customer service monitor 102. However, image data received from an external computer (for example, the operator terminal 16) may also be output by the CPU 20.

[0057] Furthermore, the input / output I / F 26 converts customer voices detected by the microphone 106 into digital audio data (customer voice data) and outputs it to the CPU 20, converts the audio data output by the CPU 20 into an analog audio signal and outputs it from the speaker 108, outputs video data of the footage including customer CUS captured (detected) by the customer service camera 104 to the CPU 20, and outputs video data of the usage status of the payment machine 110 captured (detected) by the surveillance camera 120 to the CPU 20.

[0058] However, in this embodiment, the audio data output from the CPU 20 is either the operator's audio data received from the operator terminal 16 or audio data obtained by converting the operator's audio into the avatar's audio. Hereafter, these audio data will be referred to as operator audio data.

[0059] The payment machine 110 is connected to the CPU 20 via a bus and a payment machine interface (payment machine I / F) 30. The payment machine I / F 30, although not shown in the figure, includes a computer or processor that controls various operations in the payment machine 110. For example, barcode data read by the scanner unit 112 is sent to the CPU 20, stored in the RAM 22, and displayed on the display 116 (Figure 2). Similarly, card data read by the settlement unit 114 is sent to the CPU 20, stored in the RAM 22, and displayed on the display 116.

[0060] Note that the electrical configuration of the store terminal 12 shown in Figure 3 is just one example and is not limited to this configuration.

[0061] For example, the customer service monitor 102 (including customer service monitors 102A and 102B described later) may not be installed on the store terminal 12, but may be independently connected to the network 14, and may directly receive video data (and audio data) from the store terminal 12 and the operator terminal 16 through the network 14.

[0062] In addition, although not shown in the diagram, environmental sensors such as cameras other than the aforementioned cameras 104 and 120, and microphones other than microphone 106 may be installed within the store 100.

[0063] Figure 4 is a block diagram showing the electrical configuration of the operator terminal 16 shown in Figure 1. As shown in Figure 4, the operator terminal 16 includes a CPU 32, which is connected to the RAM 34, communication I / F 36, and input / output I / F 38 via an internal bus.

[0064] The CPU 32 is responsible for the overall control of the operator terminal 16. However, instead of the CPU 32, an SoC (System on a Chip) with multiple functions such as CPU and GPU capabilities may be provided.

[0065] RAM34 is the main memory and is used as the work area or buffer area for the CPU32. Although not shown in the diagram, the operator terminal 16 is equipped with an HDD and ROM as auxiliary storage devices. However, non-volatile memory such as an SSD may be used instead of the HDD, or in addition to the HDD.

[0066] Communication I / F36 is a wired interface that, under the control of the CPU 32, transmits and receives control signals and video and audio data to and from an external computer such as the operator terminal 16 via the network 14. However, a wireless interface such as Wi-Fi or Bluetooth® can also be used as Communication I / F36.

[0067] The input / output interface 38 is connected to an input device 40, a display device 42, a microphone 44, a speaker 46, and a camera 48.

[0068] The input device 40 is a keyboard and a computer mouse. A touch panel may also be provided. However, if a smartphone or tablet PC is used as the operator terminal 16, the input device 40 is a touch panel and hardware buttons.

[0069] Display device 42 is, for example, a liquid crystal display device. Microphone 44 is a sound-collecting microphone. Speaker 46 is a stereo speaker. Camera 48 is a CCD camera.

[0070] Furthermore, the input / output interface 38 outputs operation data (or operation information) received from the input device 40 to the CPU 32, and also outputs image data generated by the CPU 32 to the display device 42, causing the display device 42 to display a screen corresponding to the image data. However, image data received from an external computer (for example, the store terminal 12) may also be output by the CPU 32. The image data received from the store terminal 12 consists of customer video data and screen video data.

[0071] Furthermore, the input / output interface 38 converts the operator's voice detected by the microphone 44 into digital audio data and outputs it to the CPU 32, converts the audio data output by the CPU 32 into an analog audio signal and outputs it from the speaker 46, and outputs video data of the operator's video captured (detected) by the camera 48 to the CPU 32. However, in this embodiment, the audio data output from the CPU 32 is the audio data of the customer CUS received from the store terminal 12. Also, the operator's video is a moving image.

[0072] Note that the electrical configuration of the operator terminal 16 shown in Figure 3 is just one example and is not limited to this configuration.

[0073] Furthermore, if the operator terminal 16 is a smartphone, it is equipped with a call circuit for making calls via a mobile phone network, or a mobile phone network and a public telephone network. However, in this embodiment, such calls are not made, so they are omitted from the illustration.

[0074] Here, with reference to Figures 5-7, we will explain an example of how an avatar is automatically displayed in response to a predetermined action of the customer's CUS (Customer User System).

[0075] In the embodiment shown in Figure 5, as shown in Figure 5(A), a product 126, such as a smartphone, is placed on a product display stand 124 fixedly attached to a display stand 122, so that customers (CUS) can freely pick it up. A mooring cord 128 is provided on the product display stand 124, and the product 126 is moored to the product display stand 124 by the mooring cord 128. However, this mooring cord 128 is formed to be expandable and retractable, for example, by winding a plastic wire into a coil.

[0076] For the product display stand 124, for example, an anti-theft device such as Utility Model Registration No. 3188377 or an anti-theft display stand such as the "Jmax_V2 series" (product name) can be used. In such a product display stand 124, a tag is connected to a cable corresponding to a mooring cord 128, and the tag is configured to detach if the cable is pulled strongly. In other words, if the cable is not pulled, the tag will not detach, and in that state, a product, such as product 126, can be removed from the product display stand 124 and picked up for examination.

[0077] However, the product display stand 124 is not limited to this; it can be any stand that allows the product 126 to be removed from the product display stand 124 while being secured by means such as the mooring cord 128.

[0078] The system detects, for example, that a customer (CUS) has removed product 126 from the product display stand 124, based on footage from a surveillance camera 120 (Figure 3).

[0079] As a result, as shown in Figure 5(B), the customer service monitor 102A displays a message 130 asking whether to allow the display of an avatar, such as "Do you want to summon an avatar?", along with a YES button (allowance icon) 132Y indicating permission and a NO button (rejection icon) 132N indicating rejection.

[0080] In Figure 5(B), when the customer CUS touches the permission icon 132Y, the avatar AVT is displayed on the customer service monitor 102A, as shown in Figure 5(C), and the operator interacts with the customer CUS through the avatar AVT. In other words, the operator performs remote customer service through the avatar AVT.

[0081] As shown in the embodiment in Figure 5, when a customer user (CUS) touches product 126, this is detected, and icons 132Y and 132N are displayed on the customer service monitor 102A to ask the customer user whether or not to display the avatar. The avatar AVT is then displayed on the customer service monitor 102A only when the customer user operates the permission icon 132Y, that is, only when the customer user wants to call up the avatar AVT. In other words, the avatar AVT can be called up only when necessary.

[0082] Furthermore, as shown in Figure 5(B), methods for detecting when a customer CUS touches product 126 include, as described above, the method based on the video from the surveillance camera 120, as well as methods using a switch or sensor that activates when the product is removed from the product display stand 124.

[0083] Furthermore, the system can ask the customer user (CUS) whether or not to allow the display of the avatar AVT, and as a way for the customer user to indicate their consent or refusal, an icon 132 is displayed on the customer service monitor 102A. In addition, by identifying the voice spoken by the customer user, such as "yes" or "no," it is possible to determine whether the customer user consents to or does not consent to the summoning of the avatar AVT. It should be noted in advance that this is also the case in the embodiments shown in Figure 6 and Figure 7, which will be described later.

[0084] Furthermore, as a method for inquiring with the customer (CUS) whether or not to display the avatar (AVT), the embodiment employs displaying message 130 on the customer service monitor 102A. However, it is also possible to inquire using a voice message, such as "Shall I display your avatar?". It should be noted in advance that this is also the case in the embodiment shown in Figure 6 and Figure 7, which will be described later.

[0085] In the embodiment shown in Figure 5, when product 126 is displayed on a product display stand 124, the system detects that a customer (CUS) has touched the product when product 126 is removed from the product display stand 124.

[0086] In contrast, the embodiment shown in Figure 6 applies when the product 126 is displayed on the display stand 122 by itself. In the embodiment shown in Figure 6, as shown in Figure 6(A), the product 126, for example, a laptop computer, is displayed directly on the display stand 122.

[0087] Then, as shown in Figure 6(B), when the customer CUS touches the product 126, this is detected, for example, based on the video from the surveillance camera 120, similar to the previous embodiment. At that time, as shown in Figure 6(B), a message 130 is displayed on the customer service monitor 102A asking for permission or denial of whether to call the avatar AVT, along with a permission icon 132Y and a denial icon 132N.

[0088] Similar to the embodiment in Figure 5, when the customer CUS touches the permission icon 132Y, the avatar AVT is displayed on the customer service monitor 102A as shown in Figure 6(C).

[0089] In this embodiment shown in Figure 6, when a customer user (CUS) touches product 126, this is detected, and icons 132Y and 132N are displayed on the customer service monitor 102A to ask the customer user whether or not to display the avatar. The avatar AVT is then displayed on the customer service monitor 102A only when the customer user operates the permission icon 132Y, that is, only when the customer user user indicates that it is okay to display the avatar AVT. In other words, the avatar AVT can be invoked only when necessary.

[0090] Furthermore, as shown in Figure 6(B), methods for detecting when a customer CUS touches product 126 include, as described above, the method based on the video from the surveillance camera 120, as well as methods using a switch or sensor that activates when product 126 is lifted from the display stand 122.

[0091] The embodiment shown in Figure 7 is one in which, when a customer (CUS) takes a catalog 136 from a catalog case 134 placed on a product display stand 122, an avatar (AVT) is summoned to the customer service monitor 102A. However, it should be noted that this catalog case 134 is placed on the product display stand 122 near the product (not shown in Figure 7), and the catalog 136 includes leaflets, brochures, etc., and is a document that describes the product in detail.

[0092] As shown in Figure 7(B), when a customer (CUS) takes the catalog 136 out of the catalog case 134, this is detected based on the video feed from the surveillance camera 120, or based on a signal from a switch or sensor installed in the catalog case 134 that activates when the catalog 136 is removed. Accordingly, as shown in Figure 7(B), a message 130 asking for permission or denial of calling the avatar (AVT) is displayed on the customer service monitor 102A, along with a permission icon 132Y and a denial icon 132N.

[0093] Similar to the embodiments in Figures 5 and 6, when the customer CUS touches the permission icon 132Y, the avatar AVT is displayed on the customer service monitor 102A as shown in Figure 7(C).

[0094] In this embodiment shown in Figure 7, when the customer user (CUS) takes out catalog 136, icons 132Y and 132N are displayed on the customer service monitor 102A to ask the customer user whether or not to display the avatar. The avatar AVT is then displayed on the customer service monitor 102A only when the customer user operates the permission icon 132Y, that is, only when the customer user user indicates that it is okay to display the avatar AVT. In other words, the avatar AVT can be called up only when necessary.

[0095] Figure 8 is an illustrative diagram showing an example of the memory map of the RAM 22 of the store terminal 12 shown in Figure 3. As shown in Figure 8, this RAM 22 includes a program storage area 50 and a data storage area 52. The control program of the store terminal 12 is stored in the program storage area 50.

[0096] The control program executed on the store terminal 12 includes a main processing program 54a, an operation detection program 54b, a communication program 54c, a camera image acquisition program 54d, an audio acquisition program 54e, a trigger event detection program 54f, an avatar display program 54g, and the like.

[0097] The main processing program 54a is a program for executing the main routine of the control processing of the store terminal 12 in this embodiment. This main processing program 54a also includes a program that executes processing related to the payment machine 110 via the payment machine I / F 30, but this program is not directly related to the embodiment of this disclosure, so its description is omitted.

[0098] The operation detection program 54b is a program that detects operation data input from the input device 28 and stores it in the operation data storage area 56a (described later) of the data storage area 52.

[0099] The communication program 54c is a program for communicating with external devices, such as the operator terminal 16.

[0100] The camera image acquisition program 54d is a program for acquiring images of customers (CUS) and the inside of the store 100 that are being captured by cameras such as the customer service camera 104 and the surveillance camera 120, which are installed in conjunction with the store terminal 12.

[0101] Such camera images are stored as video data in the camera video data storage area 56c (described later) of the data storage area 52, and are transmitted to the operator terminal 16 by the communication program 54c as needed.

[0102] The voice acquisition program 54e is a program for acquiring customer CUS and voices from within the store 100, which are picked up by microphones such as microphone 106 that are provided in conjunction with the store terminal 12.

[0103] The voice acquired in this manner is stored as voice data in the voice data storage area 56d (described later) of the data storage area 52, and is transmitted to the operator terminal 16 by the communication program 54c as needed.

[0104] The trigger event detection program 54f is a program for detecting specific actions of the customer (CUS) as trigger events that trigger the display (calling) of the avatar (AVT).

[0105] In this embodiment, three types of trigger events are assumed, including, for example, product touching actions as described with reference to Figures 5 and 6, catalog removal actions as described with reference to Figure 7, and customer CUS actions, particularly checkout actions.

[0106] The product touch detection program 54f1 is a program that detects whether a customer CUS has touched product 126 based on, for example, camera footage (or signals from a switch or sensor) in the camera footage data storage area 56c.

[0107] The catalog removal detection program 54f2 is a program that detects that a customer CUS has removed catalog 136, for example, based on camera images (or signals from a switch or sensor) in the camera image data storage area 56c.

[0108] The customer behavior detection program 54f3 is a program that detects the behavior of a customer (CUS) attempting to make a payment using, for example, the payment machine 110 (Figure 2), based on camera footage and other information, and specifically detects predetermined customer behaviors, such as whether the customer is having difficulty with the payment process. In other words, this customer behavior detection program 54f3 is a program that detects unsuccessful payment attempts by customers as trigger events.

[0109] The avatar display program 54g is a program for displaying the avatar image AVT on the customer service monitor 102A when a trigger event is detected by the trigger event detection program 54f and the customer CUS operates the permission icon 132Y.

[0110] Although not shown in the diagram, the program storage area 50 stores the operating system and middleware of the store terminal 12, as well as a browser and other application programs other than the control program disclosed herein. However, since these other application programs are separate programs from the control program, i.e., the information processing program, in this embodiment, they are not included in the information processing program.

[0111] The RAM22's data storage area 52 stores operation data storage area 56a, avatar display data storage area 56b, camera video data storage area 56c, audio data storage area 56d, system setting data storage area 56e, and trigger event detection flag 56f, among others.

[0112] The operation data storage area 56a temporarily stores operation data for the store terminal 12 detected according to the operation detection program 54b. The operation data is used for processing by the CPU 20 and transmitted to the operator terminal 16 as needed.

[0113] As described above, the avatar display data storage area 56b stores control data for controlling the avatar image AVT displayed on the customer service monitor 102A, display data for the message 130 displayed on the customer service monitor 102A, and display data for icons 132Y and 132N, etc.

[0114] The camera image data storage area 56c stores the camera image data acquired by the camera image acquisition program 54d.

[0115] The audio data storage area 56d stores the audio data acquired by the audio acquisition program 54e.

[0116] The system configuration data storage area 56e is pre-configured with the necessary settings data for operating the information processing system of this embodiment within the store 100.

[0117] The trigger event detection flag 56f is, for example, a 3-bit trigger event detection flag that stores three types of trigger events detected by the trigger event detection program 54f described above in their respective corresponding bits. For example, the trigger event detection flag 56f stores the event detected by the product touch detection program 54f1 (the event that customer CUS touched product 126) in the first bit, the event detected by the catalog removal detection program 54f2 (the event that customer CUS removed catalog 136 from catalog case 134) in the second bit, and the event that a predetermined customer action was detected by the customer action detection program 54f3 in the third bit.

[0118] Although not shown in the diagram, the data storage area 52 stores other data necessary for the execution and operation of the control program of the store terminal 12, as well as flags and other counters (timers) necessary for the execution of the control program.

[0119] The operation of the information processing system 10 in this embodiment will now be described. In this information processing system 10, the operator basically interacts with customers (CUS) via an avatar image (AVT) displayed on a customer service monitor (AVT) in the store (Store 100) from the operator terminal (Operator Terminal) (16) when necessary. However, customer service itself is not a crucial part of this embodiment, so its explanation will be omitted.

[0120] Referring to Figure 9, the CPU 20 of the store terminal 12 constantly monitors whether a trigger event is detected in step S1. The specific method for detecting a trigger event will be explained later with reference to Figures 10-13.

[0121] Note that the processing operations shown in the flowcharts of Figures 9-13 are similar to the processing operations shown in the flowcharts of Figures 16, 19, and 20 described later, but they are executed repeatedly, for example, every frame.

[0122] Then, in step S3, the CPU 20 determines whether a trigger event has been detected. In this step S3, the CPU 20 can determine whether a trigger event has been detected by referring to the trigger event detection flag 56f in the data storage area 52.

[0123] When "YES" is determined in step S3, that is, when some trigger event is detected, in step S5, the customer CUS is inquired about the display of the avatar AVT. Specifically, in step S5, the CPU 20 uses the avatar display data stored in the avatar display data storage area 56b, as already explained with reference to Figures 5-7, for example, to display a message 130 asking whether to allow the display of the avatar AVT, along with an allow icon 132Y and a reject icon 132N on the customer service monitor 102A.

[0124] Then, in the next step S7, the CPU 20 determines whether the customer CUS has authorized the summoning of the avatar AVT, that is, the display of the avatar AVT. Specifically, in step S7, the CPU 20 refers to the operation data stored in the operation data storage area 56a and determines whether the authorization icon 132Y displayed on the customer service monitor 102A has been operated by the customer CUS.

[0125] When the customer user (CUS) responds to the permission / denial inquiry by voice, the CPU 20 should determine whether the customer user (CUS) has granted or denied permission based on the voice data stored in the voice data storage area 56d.

[0126] When "YES" is determined in step S7, the CPU 20 displays an avatar AVT on the customer service monitor 102A in the next step S9, as shown in Figures 5-7. When the avatar AVT is displayed by the display means in this way, the operator receives a notification from the store terminal 12 to the operator terminal 16 that their avatar AVT has been displayed, and thereafter the operator will respond to the customer CUS through that avatar AVT.

[0127] Step S9 in Figure 9 is the same as step S9 in Figures 16 and 19, which will be described later, and functions as a display means.

[0128] After the avatar AVT is displayed, as explained earlier, the operator operates the operator terminal 16 and interacts with the customer CUS through the avatar AVT to provide customer service.

[0129] According to this embodiment, the avatar AVT is only invoked when the customer user (CUS) consents, so the operator does not need to operate the avatar AVT at all times, which is expected to improve the operator's work efficiency. On the other hand, the customer user (CUS) is also freed from the annoyance of receiving customer service from the avatar AVT when it is not needed. In other words, the avatar is automatically displayed when the customer needs it, so effective customer service can be expected through the avatar.

[0130] As shown in Figure 10, in this embodiment, three types of trigger events are set, which are detected by the product touch detection program 54f1, the catalog sampling detection program 54f2, and the customer action detection program 54f3, respectively, as shown in Figure 8.

[0131] Referring to Figure 11, in the product touch detection process shown in Figure 5, one of the two types of product touching actions, in step S111, the surveillance camera 120 is constantly filming the product display stand 122 (Figure 5) on which the product (smartphone) 126 is displayed. The CPU 20 monitors the video from this surveillance camera 120. Then, in step S112, the CPU 20 determines, based on the video data from the surveillance camera 120, whether the customer CUS has removed the product (smartphone) 126 from the product display stand 124. In order to determine, based on the video data, whether the customer CUS has removed the product (smartphone) 126 from the product display stand 124, it is sufficient to learn a large amount of video data according to the AI ​​method. However, if a switch or sensor (not shown) is provided to detect that the product 126 has been removed from the product display stand 124, the CPU 20 performs the determination in step S112 based on the signal from such a switch or sensor.

[0132] When the CPU 20 determines "YES" in step S112, it detects that the customer CUS has touched the product and sets the trigger event detection flag 56f. As a result, the CPU 20 determines "YES" in step S3 in Figure 9.

[0133] In another embodiment for detecting product touching behavior, as shown in Figure 12, at step S114, the surveillance camera 120 is constantly filming the product display stand 122 (Figure 6) on which the product (computer) 126 is displayed. The CPU 20 monitors the video from this surveillance camera 120. Then, at step S115, the CPU 20 determines, based on the video data from the surveillance camera 120, whether the customer (CUS) has touched the product (computer) 126 on the product display stand 122. In order to determine whether the customer (CUS) has touched the product (computer) 126 based on the video data, it is sufficient to learn a large amount of video data according to the AI ​​method. However, if a switch or sensor (not shown) is provided to detect when the product 126 is picked up from the product display stand 122, the CPU 20 performs the determination at step S115 based on the signal from such a switch or sensor.

[0134] When the CPU 20 determines "YES" in step S115, it detects that the customer CUS has touched the product and sets the trigger event detection flag 56f. As a result, the CPU 20 determines "YES" in step S3 in Figure 9.

[0135] Figure 13 shows the operation when the customer CUS detects that the catalog has been removed as a trigger event, corresponding to the embodiment in Figure 7.

[0136] In the first step S121 of Figure 13, the surveillance camera 120 is constantly filming the product display stand 122 (Figure 7) where the products (personal computers) are displayed. The CPU 20 monitors the video from this surveillance camera 120. Then, in step S122, the CPU 20 determines, based on the video data from the surveillance camera 120, whether the customer CUS has taken a catalog 136 from the catalog case 134 on the product display stand 122. In order to determine whether the customer CUS has taken a catalog 136 based on the video data, it is sufficient to learn a large amount of video data according to the AI ​​method. However, if the catalog case 134 is equipped with a switch or sensor (not shown) for detecting that the catalog 136 has been taken, the CPU 20 performs the determination in step S122 based on the signal from such a switch or sensor.

[0137] When the CPU 20 determines "YES" in step S122, it detects that the customer CUS has taken a catalog and sets the trigger event detection flag 56f. As a result, the CPU 20 determines "YES" in step S3 in Figure 9.

[0138] In the first step S131 of Figure 13, the surveillance camera 120 is constantly filming the payment machine 110 (Figure 2). The CPU 20 monitors the video from this surveillance camera 120. Then, in step S132, the CPU 20 determines, based on the video data from the surveillance camera 120, whether the customer CUS appears to be having trouble with the payment process. In order to determine whether the customer CUS is having trouble with the payment process based on the video data, the AI ​​can be trained on a large amount of video data according to AI methods so that it can detect actions such as repeating the payment process or interrupting the payment process.

[0139] When "YES" is determined in step S132, the CPU 20 detects that the customer CUS has performed a predetermined action and sets the trigger event detection flag 56f. As a result, the CPU 20 determines "YES" in step S3 in Figure 9.

[0140] Figure 15 is an illustrative diagram showing an example of a memory map of the memory 22 of the store terminal 12 shown in Figure 3, according to this first other embodiment. As shown in Figure 15, this embodiment differs from the embodiment in Figure 8 in that a face recognition program 54h is further stored in the program storage area 50 of the RAM of memory 22.

[0141] This facial recognition program 54h is a program that determines (identifies) whether a person who has detected a trigger event, such as in the previous embodiment, is a store employee, based on camera footage (video data) from the customer service camera 104 and / or surveillance camera 120. For this purpose, for example, facial image data (2D and 3D) of all store employees is pre-stored in the system setting data storage area 56e.

[0142] Then, in order for the facial recognition program 54h to determine whether the person who detected the trigger event is an employee, it is sufficient to compare the camera images (video data) from the customer service camera 104 and / or surveillance camera 120, etc., stored in the camera video data storage area 56c, with the facial images of those employees.

[0143] In an embodiment equipped with a facial recognition program 54h, as shown in Figure 16, in step S5A, it is identified whether the person who detected the trigger event is a store employee.

[0144] However, steps S1 and S3 are the same as in the embodiment shown in Figure 9.

[0145] Then, in step S7A, the CPU 20 determines whether the result of face recognition in step S5A indicates that the person who detected the trigger event is a store employee. If the CPU 20 determines "NO" in step S7A, that is, if the person who detected the trigger event is not a store employee, the CPU 20 displays the avatar AVT on the customer service monitor 102A in step S9.

[0146] Step S7A, which determines whether the person who detected the trigger event is a store employee, functions as a determination means.

[0147] In the embodiment shown in Figure 9, the avatar AVT was displayed on the customer service monitor 102A only when a trigger event was detected and the person who detected the trigger event gave permission.

[0148] In contrast, in the embodiment shown in Figure 16, when a trigger event is detected, the avatar AVT is displayed on the customer service monitor 102A only when it is determined that the person who detected the trigger event is not a store employee. If the person is a store employee, the avatar AVT does not need to be displayed, but if the person is a customer CUS, the avatar AVT can be displayed to provide customer service.

[0149] Figure 17 shows another embodiment of the customer service monitor 102 available in the embodiment of this invention. In this embodiment, the customer service monitor 102B can be an autonomously mobile, so-called telepresence (or tele-existence) robot, such as the "Double 3" manufactured by Double Robotics, Inc.

[0150] However, the system is not limited to these specific robots; robots equipped with displays, such as those disclosed in Japanese Patent Publication No. 2020-004182, and the humanoid robot "PEPPER" (registered trademark) provided by SoftBank Robotics Corp., can also be used as customer service monitors 102B.

[0151] This customer service monitor 102B includes a base 138, on which an ultrasonic sensor 140, one of the movement sensors, is provided on the front. In addition, left and right movement wheels 142L and 142R are rotatably provided at both ends of the base 138.

[0152] A pole 144 extending upward is provided on the upper surface of the base 138, and a touch display 146 capable of displaying the operator's avatar AVT is provided at the upper end of the pole 144. This touch display (hereinafter simply referred to as "display") 146 is equipped with a camera 104, microphone 106, speaker 108, etc., as shown in Figure 2, but details are omitted from the illustration.

[0153] Furthermore, this customer service monitor 102B is a battery-powered robot, and the battery is housed, for example, in the pole 144.

[0154] In this embodiment, the customer service monitor 102B can communicate with the store terminal 12 and the operator terminal 16 connected to the network 14. The operator terminal 16 is a general-purpose computer used by an operator who remotely controls the customer service monitor 102A. For example, the operator uses the operator terminal 16 to transmit video and audio to the customer service monitor 102A, displays their avatar AVT on the display 146 of the customer service monitor 102A, interacts with the customer CUS, and provides services.

[0155] As is well known, the display 146 has a touch panel on its surface (or built into it) that displays images, and a location on the display 146 can be pointed to by touching it.

[0156] In one embodiment utilizing such a mobile customer service monitor 102B, the customer service monitor 102B moves to the front of the elevator and displays an avatar AVT on the display 146. For example, when the elevator stops on a floor where products (such as smartphones or personal computers, or any other product) 126 are displayed and sold, and the elevator doors open, the customer service monitor 102B moves to the front of the elevator and displays an avatar AVT on the display 146.

[0157] In this embodiment, as shown in Figure 16, the program storage area 50 is set with an elevator open detection program 54f4 as the trigger event detection program 54f, and also with a customer service monitor movement program 54i.

[0158] The elevator door open detection program 54f4 detects when the elevator door is open by receiving a signal from a switch or sensor (not shown) that detects the opening and closing of the elevator door (not shown) via wired or wireless connection. Alternatively, a surveillance camera 120 may be installed to monitor the opening and closing of the elevator door, and the program may detect when the door is open based on the video feed from the camera.

[0159] As explained earlier, the customer service monitor movement program 54i is a program that moves the movable customer service monitor 102B, as shown in Figure 17, to a customer service-ready location, for example, near the front of the elevator, in response to the elevator door opening detection program 54f4 detecting that the elevator door has opened.

[0160] However, in order to move the customer service monitor 102B to the customer service location, which is the destination, it is sufficient to pre-set two-dimensional map data in the system setting data storage area 56e, for example. Then, the customer service monitor movement program 54i calculates a movement path from the coordinate position indicating the current location of the customer service monitor 102B to the coordinate position indicating the destination on that map data, and moves the customer service monitor 102B according to that movement path.

[0161] Furthermore, in embodiments where the movable customer service monitor 102B is kept on standby at a predetermined location (platform), the customer service monitor 102B can be programmed in advance to move from the platform to in front of or near the elevator. In this case, it becomes unnecessary to calculate the movement path each time the customer service monitor 102B moves.

[0162] Referring to Figure 19, the CPU 20 of the store terminal 12 constantly monitors whether a trigger event is detected in step S1.

[0163] When the elevator door is detected to have opened by the elevator open detection program 54f4, the trigger event detection flag 56f in the data storage area 52 is set, so in step S3 the CPU 20 determines "YES".

[0164] Then, in the next step S8, the CPU 20 moves the customer service monitor 102B to the customer service location (for example, in front of the elevator) according to the customer service monitor movement program 54i, as described earlier, and then displays the avatar AVT in step S9.

[0165] As described above, in the embodiment using the movable customer service monitor 102B as shown in Figure 17, when a trigger event is detected, the customer service monitor 102B is moved to the customer service location (a place or location related to the customer CUS) and the avatar AVT is displayed at that location. Therefore, compared to the case where customer service monitors 102 are fixedly installed as in customer service monitor 102A, it is possible to handle the situation with fewer customer service monitors 102.

[0166] Such a portable customer service monitor 102B can also be applied to the embodiments shown in Figures 9 and 16 above.

[0167] In the embodiment shown in Figure 20, when a trigger event such as a product touching motion as shown in Figure 11 is detected, in step S8, the monitor is moved to a location (a place or position related to the customer CUS) near the customer CUS who is in front of the product display stand 122 shown in Figure 5. However, the location of the customer CUS, which is the destination of the customer service monitor 102B, can be calculated based on video data from the surveillance camera, but it may also be set in advance to a location near the product display stand 124.

[0168] The operation from step S5 onward in Figure 20 is the same as in the embodiment shown in Figure 9, so redundant explanations will be omitted.

[0169] In the embodiment shown in Figure 20, when a trigger event such as a product touching motion as shown in Figure 12 is detected, in step S8, the device is moved to a location near the customer CUS who is standing in front of the product display stand 122 shown in Figure 6. However, the location of the customer CUS, which is the destination of the customer service monitor 102B, can be calculated based on video data from a surveillance camera, but it may also be pre-set to be near the location where the product 126 is displayed.

[0170] In the embodiment shown in Figure 20, when a trigger event such as a catalog removal operation as shown in Figure 13 is detected, in step S8, the monitor is moved to the vicinity of the customer CUS who is near the catalog case 134 on the product display stand 122 shown in Figure 7. However, the location of the customer CUS, which is the destination of the customer service monitor 102B, can be calculated based on video data from the surveillance camera, but it may also be set in advance to a location near where the catalog case 134 is installed.

[0171] In the embodiment shown in Figure 20, when a trigger event such as a customer action as shown in Figure 16 is detected, in step S8, the system moves to the vicinity of the customer, for example, near the payment machine 110. However, the location of the customer CUS, which is the destination of the customer service monitor 102B, can be calculated based on video data from a surveillance camera, but it may also be pre-set to be near the location where the payment machine 110 is installed.

[0172] In the embodiment shown in Figure 20, the customer CUS is inquired about displaying the avatar AVT in step S5. However, as in the embodiment shown in Figure 19, the customer service monitor 102B may be moved to its destination, and the avatar AVT may be displayed on the customer service monitor 102B in step S9 without inquiring about the customer CUS.

[0173] In the above embodiment, the trigger event to detect a customer action is the unsuccessful action or operation of a customer user (CUS) towards a self-checkout machine. In other words, it detects the unsuccessful action or operation of a customer user (CUS) towards a terminal (device or equipment) in the store that is expected to be operated by the customer user themselves. Another example of such a terminal (device or equipment) in the store that is expected to be operated by the customer user is a coffee server that the customer user operates to pour coffee into a purchased cup.

[0174] In other words, the customer action detection program 54f3 described above is a program for detecting a customer's actions or unsuccessful attempts to operate a terminal within a store that is intended to be operated by the customer themselves.

[0175] Furthermore, while the above embodiment uses five types of trigger events, it is not limited to these. One or more of the five types of trigger events may be used, or additional trigger events may be added.

[0176] Furthermore, if the same result can be obtained from each step in the flowchart shown in the above embodiment, the order in which the processes are performed can be changed.

[0177] Furthermore, the various screens and specific numerical values ​​mentioned in the above-described examples are merely illustrative and can be modified as needed. [Explanation of Symbols]

[0178] 10. Information Processing Systems 12 ... Store terminal 14…Network 16 ... Operator terminal 20, 34 ...CPU 22, 34…memory 24, 36 ... Communication I / F 100 stores 102, 102A, 102B... Customer service monitors 110…Payment machine 120 ... Surveillance cameras 122…Product display stand 126…product 130 ... Message 132Y ...Permission icon 132N ... Reject icon 134... Catalog case 136…Catalog

Claims

1. An information processing system used in stores where customer service is provided through avatar images displayed on customer service monitors, A detection means for detecting specific customer behavior as a trigger event, and An information processing system comprising display means for displaying an avatar on the customer service monitor in response to the detection of the aforementioned trigger event.

2. The store further includes a determination means for determining whether the person who detected the trigger event is a store employee, based on the camera footage from a camera installed in the store. The information processing system according to claim 1, wherein the display means displays an avatar when the determination means determines that the subject is not a store employee.

3. The system further includes an inquiry means for inquiring with the customer whether or not to allow the display of an avatar on the customer service monitor in response to the detection of the trigger event. The information processing system according to claim 1, wherein the display means displays an avatar when the customer expresses permission in response to the inquiry regarding permission or denial.

4. The information processing system according to claim 1, wherein the detection means detects a customer's action of touching a product as a trigger event.

5. The information processing system according to claim 1, wherein the detection means detects a customer's act of removing a catalog as a trigger event.

6. The information processing system according to claim 1, wherein the detection means detects a customer's actions or unsuccessful operations on a terminal within the store that the customer is expected to operate themselves as a trigger event.

7. An information processing method used in stores where customer service is provided through avatar images displayed on customer service monitors, Detect specific customer behavior as a trigger event, and An information processing method that displays an avatar on the customer service monitor in response to the detection of the aforementioned trigger event.

8. An information processing system used in stores where customer service is provided through images of avatars displayed on a portable customer service monitor, Detection means for detecting trigger events, A means for moving the customer service monitor to a location related to the customer in response to the detection of the trigger event, and An information processing system comprising display means for displaying an avatar on the aforementioned moved customer service monitor.

Images