System and information processing method

Abstract

To have communication between users in a more realistic sense.SOLUTION: An information processing apparatus that can communicate with a first user terminal operated by a first user and a second user terminal operated by a second user comprises: delivery data creation means that creates delivery data for displaying a first imaged image of a first camera in an imaging direction selected by the first user and a second imaged image of a second camera in an imaging direction selected by the second user on the first user terminal; communication means that transmits the created delivery data to the first user terminal; and communication function provision means that provides a bi-directional communication function to the first user and the second user.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention provides system and an information processing method. [Background technology]

[0002] A sales support system has been known for some time that, in response to a request from a mobile application on a smartphone terminal, displays live video of a product on the smartphone terminal, and allows users to post text comments or make transactions for each product (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0003] However, conventional technologies do not allow users to check the surrounding conditions, such as the atmosphere of the sales floor, or the products displayed on the sales floor, while also being able to check product details, such as texture and freshness, as needed. An object of the present disclosure is to make communication between users more realistic. [Means for solving the problem]

[0004] In one embodiment of the present invention, a first user terminal operated by a first user who is a purchaser and a second user terminal operated by a second user who is a seller are connected to each other. And the camera, Information processing device and a system including And, The information processing device includes: installed in the sales area The aforementioned a distribution data generating means for generating distribution data for displaying a display screen that displays a first captured image captured by a camera; and a communication means for transmitting the generated distribution data to the first user terminal. a communication function providing means for providing a two-way communication function between the first user who operates the first user terminal and the second user who operates the second user terminal when an input operation on the first user terminal is received; the distribution data generating means receives a message from the first user when the first captured image is displayed on the display screen. The aforementionedand when an input operation is received, the distribution data is generated to display a second photographed image captured by a camera worn by the second user or a camera of the second user terminal on the display screen displaying the first photographed image. system is. [Effects of the Invention]

[0005] According to one embodiment of the present invention, communication between users can be made more realistic. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a configuration diagram of an example of an information processing system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to the present embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a smartphone according to the present embodiment. [Figure 4] 1 is a functional configuration diagram of an example of an information processing system according to an embodiment of the present invention. [Figure 5] FIG. 10 is a diagram illustrating an example of distribution setting information. [Figure 6] 3A and 3B are diagrams illustrating an example of first and second display areas of a display screen according to the present embodiment. [Figure 7] FIG. 1 is a diagram illustrating an example of a usage scene of an information processing system according to an embodiment of the present invention. [Figure 8] FIG. 10 is a diagram illustrating an example of a usage scene of the information processing system according to the present embodiment in another sales floor. [Figure 9] FIG. 10 is a diagram illustrating an example of another usage scene in a transaction between traders in the information processing system according to the present embodiment. [Figure 10] 10 is a flowchart illustrating an example of processing of the information processing system according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, each embodiment of the present invention will be described with reference to the accompanying drawings. Note that in this embodiment, an example will be described in which the first user is a purchaser or prospective purchaser of a product, and the second user is a seller of the product, but this example is not limiting. For example, the first user is not required to purchase the product. The first user may be a user who communicates with the second user while displaying, on the first user terminal, a second image taken by the second user using a wearable camera or a camera built into a smartphone.

[0008] [First embodiment] <System configuration> Fig. 1 is a configuration diagram of an example of an information processing system according to this embodiment. The information processing system 1 shown in Fig. 1 includes a first user terminal 10, a second user terminal 12, an information processing device 14, a first camera 16, a second camera 18, a data storage system 20, and a payment system 22, and communicates via a network 28. The second user terminal 12, the first camera 16, and the second camera 18 constitute a distribution system 24. The network 28 is the Internet, a LAN (Local Area Network), or the like.

[0009] The first user terminal 10 (first user terminal using the information processing system 1) is an information processing terminal operated by a purchaser or prospective purchaser of a product (hereinafter simply referred to as a "purchaser, etc."). The first user terminal 10 is, for example, a PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game console, or a PDA (Personal Digital Assistant).

[0010] The second user terminal (second user terminal using the information processing system 1) 12 is an information processing terminal operated by a seller of a product, such as a salesperson. The second user terminal 12 is, for example, a PC, a mobile phone, a smartphone, a tablet terminal, a game console, a PDA, or the like.

[0011] The information processing device 14 is an example of an information providing system that provides the service according to this embodiment. The information processing device 14 is, for example, a PC or a workstation. The information processing device 14 may be realized by a server device, an ASP (Application Service Provider), or cloud computing.

[0012] The first camera 16 is installed in a product sales area or the like. The first camera 16 captures a 360-degree image of the surroundings as a captured image and distributes it to the first user terminal 10 as a live video of the product sales area. The first camera 16 may be, for example, a digital camera that has recently become available and that can capture a 360-degree spherical panoramic image with a single capture. Note that the first camera 16 is not limited to a camera that captures 360-degree images, and may be, for example, a camera that allows panning, which moves the camera left and right, and tilting, which moves the camera up and down, from the first user terminal 10.

[0013] The second camera 18 may be a wearable camera worn by the seller, or a built-in camera of the second user terminal 12 operated by the seller. A wearable camera is a camera worn by the seller on the head or shoulder, etc., and captures an image in the direction the seller is looking (eye-level image) in a hands-free manner. The second camera 18 may also be a built-in camera of various smart devices such as a smartphone or smart glasses. The second camera 18 communicates directly or via the second user terminal 12. The second camera 18 captures an image from the seller's eye level as a captured image, and distributes it to the first user terminal 10 as a live video of the product.

[0014] The first user terminal 10 and the second user terminal 12 provide buyers and sellers with two-way communication functions using voice functions, such as call functions, packet communication using voice data, WebRTC (Web Real-Time Communication), or two-way communication functions using characters (text) using chat functions.

[0015] The data storage system 20 stores images captured by the first camera 16, images captured by the second camera 18, and information exchanged between buyers and sellers via the communication function. The payment system 22 provides a payment function for products.

[0016] The first user terminal 10 displays the captured image distributed from the first camera 16. For example, as described above, the first user terminal 10 displays an image captured from a shooting direction selected by a purchaser or the like from a 360-degree image of the sales floor using an image captured by a digital camera or the like that captures a 360-degree panoramic image. The purchaser or the like can check the products lined up on the sales floor of a store such as a department store from the captured image displayed on the first user terminal 10.

[0017] If a purchaser finds a product that interests them, they can call the second user terminal 12 operated by the seller from the first user terminal 10 and begin two-way communication with the seller. The first user terminal 10 displays an image of the product taken from the seller's point of view, etc., delivered from the second camera 18. While viewing the image of the product displayed on the first user terminal 10, the purchaser can receive an explanation of the product from the seller and consider purchasing the product.

[0018] For example, when actually going to a sales floor in a department store or other store to purchase an item, a purchaser moves around while looking at the items lined up on the sales floor, and if there is an item that catches their eye, they approach it to check it out and ask the seller for an explanation of the item. According to this embodiment, a purchaser can check the items lined up on the sales floor using images captured by the first camera 16, and after finding an item that catches their eye, they can begin two-way communication with the seller. Furthermore, according to this embodiment, after starting two-way communication with the seller, the purchaser can listen to an explanation of the product from the seller while looking at an image of the product captured by the second camera 18, and then consider purchasing the product.

[0019] The configuration of the information processing system 1 shown in Fig. 1 is an example. For example, at least some of the functions of the information processing device 14 may be provided in the first user terminal 10. At least some of the functions of the first user terminal 10 and the second user terminal 12 may be provided in the information processing device 14. At least some of the functions of the data storage system 20 and the payment system 22 may be provided in the information processing device 14. The configuration of the data storage system 20 may be omitted.

[0020] <Hardware configuration> "computer" The information processing device 14, data storage system 20, and payment system 22 in Fig. 1 are realized by, for example, a computer 500 having a hardware configuration as shown in Fig. 2. Furthermore, in the case of a PC, the first user terminal 10 and second user terminal 12 in Fig. 1 are realized by, for example, a computer 500 having a hardware configuration as shown in Fig. 2. Fig. 2 is a diagram showing the hardware configuration of an example of a computer according to this embodiment.

[0021] The computer 500 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, a HD 504, a HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a data bus 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0022] Of these, the CPU 501 controls the overall operation of the computer 500 in accordance with a program. The ROM 502 stores programs used to drive the CPU 501, such as the IPL. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading and writing of various data from and to the HD 504 in accordance with the control of the CPU 501.

[0023] The display 506 displays various types of information such as a cursor, menus, windows, characters, or images. The external device connection I / F 508 is an interface for connecting various types of external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory. The network I / F 509 is an interface for data communication using the network 28. The data bus 510 is an address bus, data bus, or the like for electrically connecting each component such as the CPU 501.

[0024] The keyboard 511 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 514 controls reading and writing of various data from a DVD-RW 513, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R, etc. The media I / F 516 controls reading and writing (storing) of data from a recording medium 515, such as a flash memory.

[0025] The hardware configuration shown in FIG. 2 is an example, and it is not necessary to include all of the components shown in FIG. 2, or it may include components other than those shown in FIG.

[0026] Smartphone The first user terminal 10 and the second user terminal 12 in Fig. 1 are realized by, for example, a smartphone 700 having the hardware configuration shown in Fig. 3. Fig. 3 is a diagram showing the hardware configuration of an example of a smartphone according to this embodiment. As shown in Fig. 3, the smartphone 700 includes a CPU 701, a ROM 702, a RAM 703, an EEPROM 704, a CMOS sensor 705, an image sensor I / F 706, an acceleration / direction sensor 707, a media I / F 709, and a GPS receiver 711.

[0027] The CPU 701 controls the overall operation of the smartphone 700. The ROM 702 stores the CPU 701 and programs used to drive the CPU 701, such as the IPL. The RAM 703 is used as a work area for the CPU 701. The EEPROM 704 reads and writes various data, such as programs for the smartphone 700, under the control of the CPU 701.

[0028] The CMOS sensor 705 is a type of built-in imaging means that captures an image of a subject (mainly a self-portrait) under the control of the CPU 701 to obtain image data. Note that instead of the CMOS sensor 705, an imaging means such as a CCD sensor may be used. The imaging element I / F 706 is a circuit that controls the driving of the CMOS sensor 705. The acceleration / azimuth sensor 707 is one of various sensors such as an electronic magnetic compass that detects geomagnetism, a gyrocompass, or an acceleration sensor.

[0029] A media I / F 709 controls reading and writing (storing) of data from and to a recording medium 708 such as a flash memory. A GPS receiving unit 711 receives GPS signals from GPS satellites.

[0030] The smartphone 700 also includes a long-distance communication circuit 712, a CMOS sensor 713, an image sensor I / F 714, a microphone 715, a speaker 716, an audio input / output I / F 717, a display 718, an external device connection I / F 719, a short-distance communication circuit 720, an antenna 720a of the short-distance communication circuit 720, and a touch panel 721.

[0031] The long-distance communication circuit 712 is a circuit that communicates with other devices via the network 28. The CMOS sensor 713 is a type of built-in imaging means that captures an image of a subject and obtains image data under the control of the CPU 701. The image sensor I / F 714 is a circuit that controls the driving of the CMOS sensor 713. The microphone 715 is a built-in circuit that converts sound into an electrical signal. The speaker 716 is a built-in circuit that converts electrical signals into physical vibrations to produce sounds such as music and voice.

[0032] The audio input / output I / F 717 is a circuit that processes input and output of audio signals between the microphone 715 and the speaker 716 under the control of the CPU 701. The display 718 is a type of display unit such as a liquid crystal display or organic electroluminescence (EL) display that displays an image of a subject, various icons, etc. The external device connection I / F 719 is an interface that connects various external devices. The short-range communication circuit 720 is a communication circuit such as NFC or Bluetooth. The touch panel 721 is a type of input unit that allows the user to operate the smartphone 700 by pressing the display 718.

[0033] The smartphone 700 also includes a bus line 710. The bus line 710 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 701 shown in FIG.

[0034] <Functional configuration> The information processing system 1 according to this embodiment is realized by, for example, a functional configuration as shown in Fig. 4. Fig. 4 is a functional configuration diagram of an example of the information processing system according to this embodiment. Note that the functional configuration diagram in Fig. 4 omits components that are not necessary for explaining this embodiment. The first user terminal 10, the second user terminal 12, the information processing device 14, the first camera 16, the second camera 18, the data storage system 20, and the payment system 22 realize, for example, the functional configuration of Fig. 4 by executing programs such as an OS and applications.

[0035] The first camera 16 has an imaging unit 70 and a communication unit 72. The imaging unit 70 captures a 360-degree image of the surroundings. The communication unit 72 transmits the image captured by the imaging unit 70 to the information processing device 14. The communication unit 72 transmits the captured image to the information processing device 14, for example, by sending the captured image to a registered URL (a URL defined in the distribution settings).

[0036] The second camera 18 has an imaging unit 80 and a communication unit 82. The imaging unit 80 captures an image in the imaging direction of the second camera 18. The communication unit 82 transmits the image captured by the imaging unit 80 to the information processing device 14. Note that the communication unit 82 transmits the captured image to the information processing device 14, for example, by sending the captured image to a registered URL. The second camera 18 may transmit the captured image to the information processing device 14 via the second user terminal 12. Furthermore, the second camera 18 may be built into the second user terminal 12.

[0037] The information processing device 14 also has a distribution management unit 50, a distribution data generation unit 52, a communication unit 54, a communication function providing unit 56, a purchase processing unit 58, a memory control unit 60, a distribution setting information memory unit 62, a script management information memory unit 64, and a data memory unit 66.

[0038] The distribution management unit 50 performs processing related to the distribution of the captured images received from the first camera 16 and the second camera 18 to the first user terminal 10. The distribution data generation unit 52 generates distribution data (image data) for a display screen for displaying the captured images of the first camera 16 and the second camera 18 on the first user terminal 10.

[0039] For example, the distribution data generation unit 52 generates distribution data based on an input operation by a purchaser, such as an operation by the purchaser to call a seller, to add an image captured by the second camera 18 to a display screen displaying an image captured by the first camera 16. Furthermore, the distribution data generation unit 52 may generate distribution data based on an input operation by a purchaser, such as an operation by the purchaser to call a seller, to switch the display screen displaying the image captured by the first camera 16 to a display screen displaying an image captured by the second camera 18. The distribution data generation unit 52 transmits the generated distribution data to the first user terminal 10 via the communication unit 54.

[0040] The communication unit 54 communicates with the first user terminal 10, the second user terminal 12, the first camera 16, the second camera 18, the data storage system 20, and the payment system 22. The communication function providing unit 56 provides a voice communication function using a call function or a text communication function using a chat function to a purchaser using the first user terminal 10 and a seller using the second user terminal 12.

[0041] The purchase processing unit 58 receives a purchase instruction for a product shown in an image captured by the second camera 18 from the first user terminal 10 operated by the purchaser, etc., and executes a purchase process for the product in accordance with the purchase instruction. The storage control unit 60 stores in the data storage unit 66 or the data storage system 20 at least one of the image captured by the first camera 16, the image captured by the second camera 18, and information exchanged between the purchaser, etc., and the seller via the communication function.

[0042] The images taken by the first camera 16 and the second camera 18 stored in the data storage unit 66 or the data storage system 20, and information exchanged between the purchaser, etc. and the seller via the communication function may be used as big data. The images taken by the first camera 16 and the second camera 18 stored in the data storage unit 66 or the data storage system 20, and information exchanged between the purchaser, etc. and the seller via the communication function can be used for, for example, training for sellers or secondary use of data.

[0043] The distribution setting information storage unit 62 stores distribution setting information, which will be described later. The script management information storage unit 64 manages scripts. A script is a program for displaying 360-degree images on a first user terminal 10 that does not have an application for displaying 360-degree images installed. The data storage unit 66 stores at least one of the images captured by the first camera 16, the images captured by the second camera 18, and information exchanged between a buyer or the like and a seller via the communication function.

[0044] The first user terminal 10 has a display control unit 30, a first communication execution unit 32, a purchase acceptance unit 34, and a communication unit 36. The display control unit 30 displays a display screen including an image captured by the first camera 16 and an image captured by the second camera 18 based on the distribution data for the display screen received from the information processing device 14.

[0045] The first communication execution unit 32 executes processing to allow a purchaser or the like operating the first user terminal 10 to engage in two-way communication with a seller operating the second user terminal 12. The purchase acceptance unit 34 accepts an instruction from a purchaser or the like to purchase a product shown in an image captured by the second camera 18. The communication unit 36 communicates with the information processing device 14.

[0046] The second user terminal 12 has a second communication execution unit 40 and a communication unit 42. The second communication execution unit 40 executes processing to allow the seller operating the second user terminal 12 to carry out two-way communication with the purchaser or the like operating the first user terminal 10. The communication unit 42 communicates with the information processing device 14.

[0047] The data storage system 20 has a storage control unit 90, a communication unit 92, and a data storage unit 94. The storage control unit 90 controls the storage of images captured by the first camera 16, images captured by the second camera 18, and information exchanged between a purchaser, etc. and a seller via the communication function, which are received from the information processing device 14. The communication unit 92 communicates with the information processing device 14. The data storage unit 94 stores at least one of the images captured by the first camera 16, images captured by the second camera 18, and information exchanged between a purchaser, etc. and a seller via the communication function.

[0048] The payment system 22 has a payment processing unit 100, a communication unit 102, and a payment information storage unit 104. The payment processing unit 100 executes payment processing in response to a product purchase instruction from a purchaser or the like who operates the first user terminal 10. The communication unit 102 communicates with the information processing device 14. The payment information storage unit 104 stores payment information related to the executed payment processing.

[0049] 4 is an example, and some of the functions of the information processing device 14 may be provided in the first user terminal 10. For example, the distribution data generation unit 52 of the information processing device 14 may be provided in the first user terminal 10. The functional configuration of the information processing device 14 may be realized by multiple server devices.

[0050] A first user terminal 10 equipped with an application for displaying 360-degree images can display 360-degree images without using a script. Therefore, if there is no first user terminal 10 that does not have an application for displaying 360-degree images equipped, the information processing device 14 can omit the script management information storage unit 64.

[0051] The distribution setting information storage unit 62 of the information processing device 14 shown in Fig. 4 stores, for example, distribution setting information as shown in Fig. 5. Fig. 5 is a configuration diagram of an example of distribution setting information. The distribution setting information in Fig. 5 is configured to set a URL to which an image captured by the first camera 16 is transmitted.

[0052] FIG. 5(A) has the following items: floor, sales floor type, sales corner, device ID, and room URL. Floor is an example of information that groups product sales floors, and represents a floor such as basement 1 (B1) or the first floor. Sales floor type is an example of information that represents the type of product sales floor, and represents the type of sales floor such as restaurant, special cosmetics, or food. Sales floor corner is an example of information that represents a sales floor, and is, for example, a sales floor that separates special cosmetics by brand, or a sales floor that separates food by ingredient. Device ID is an example of information that identifies the first camera 16 installed in the sales floor. Room URL is an example of a registered URL to which captured images are sent.

[0053] 5(B) has items such as a shelf image ID, a device ID, and a room URL. The shelf image ID is an example of information that indicates a sales floor for a product, and is a sales floor divided into shelves on which products are displayed. The room URL is an example of a registered URL to which a captured image is sent.

[0054] According to the distribution setting information in FIG. 5, by specifying a sales floor of a product, it is possible to identify a room URL, which is an example of a URL to which an image captured by first camera 16 of that sales floor is transmitted.

[0055] <First and second display areas> 6 is a diagram illustrating an example of a first and second display area on a display screen according to this embodiment. The first display area is an example of a display area for displaying an image captured by the first camera 16. The second display area is an example of a display area for displaying an image captured by the second camera 18. The display screen displayed on the first user terminal 10 displays the first display area and the second display area side by side as shown in FIG. 6(A) or 6(B).

[0056] FIG. 6(A) is a display example in which the first display area and the second display area are arranged vertically. The vertical positions of the first display area and the second display area may be reversed. FIG. 6(B) is a display example in which the first display area and the second display area are arranged horizontally. The horizontal positions of the first display area and the second display area may be reversed.

[0057] While Fig. 6 shows an example in which the first display area and the second display area are rectangular, they may have other shapes. Although Fig. 6 shows an example in which the first display area and the second display area are the same size, they may have different sizes. Although Fig. 6 shows an example in which the first display area and the second display area are displayed side by side, an image captured by the first camera 16 may be displayed first in the first display area, and then an image captured by the second camera 18 may be displayed in the second display area. Furthermore, although Fig. 6 shows an example in which the first display area and the second display area are displayed side by side, the first display area displaying the image captured by the first camera 16 may be switched to the second display area displaying the image captured by the second camera 18.

[0058] <Processing> The processing procedure of the information processing system 1 according to this embodiment will be described. Fig. 7 is a diagram illustrating an example of a usage scene of the information processing system according to this embodiment. Fig. 7 shows an example of transition of a display screen displayed on the first user terminal 10.

[0059] The display screen 1000 is a screen image of logging in from the first user terminal 10 to a service provided by the information processing device 14. The display screen 1000 is an example in which the service is for members. If the service is not for members but is available to anyone, the display screen 1000 may be omitted.

[0060] Display screens 1020 and 1040 are screen image diagrams showing the selection of a sales floor for a product by a purchaser or the like operating the first user terminal 10. In the example of Fig. 7, when a purchaser or the like operating the first user terminal 10 selects "1F Special Selection Floor" on the display screen 1020, a display screen 1040 of sales floors where special selection cosmetics are categorized by brand is displayed.

[0061] A purchaser or the like operating the first user terminal 10 selects from the display screen 1040 the sales floor for which the purchaser or the like wishes to view an image captured by the first camera 16, for example, by clicking. When the sales floor is selected from the display screen 1040, the first user terminal 10 receives from the information processing device 14 the image captured by the first camera 16 installed in the sales floor selected by the purchaser or the like, and displays a display screen 1060 in which the image captured by the first camera 16 is displayed in the first display area. The purchaser or the like operating the first user terminal 10 can select, for example, the shooting direction of the image captured by the first camera 16 on the display screen 1060. Note that the process of displaying a partial image from the shooting direction selected by the purchaser or the like from the 360-degree image can use a known technique for creating a celestial sphere panoramic image from an image (Mercator image) captured by a digital camera or the like that obtains a 360-degree celestial sphere panoramic image as described above.

[0062] A purchaser or the like operating the first user terminal 10 can check the atmosphere of the sales floor and the products lined up in the sales floor from the image captured by the first camera 16 displayed on the display screen 1060. For example, if there is a product that interests the purchaser or the like and the purchaser or the like wants to hear an explanation about it from a seller, the purchaser or the like operating the first user terminal 10 calls the second user terminal 12 operated by the seller. There are various possible methods for calling the second user terminal 12 operated by the seller from the display screen 1060. For example, a call button or the like may be provided on the display screen 1060 and operated by the purchaser or the like, or a call from the purchaser or the like may be recognized by voice recognition or character recognition.

[0063] Display screen 1080 is a screen image diagram after a purchaser or the like operating first user terminal 10 calls a seller operating second user terminal 12 and starts communication between the purchaser or the like and the seller. First user terminal 10 receives a captured image, such as an eye-level image of the seller who has started communication, from second camera 18, and displays display screen 1080 on which the captured image by second camera 18 is displayed in the second display area. Second camera 18 may be associated with the seller or may be associated with second user terminal 12.

[0064] After starting two-way communication with the seller, the buyer can receive an explanation of the product from the seller while viewing an image of the product captured by the second camera 18 included in the display screen 1080. When the buyer finds a product they want to purchase, the buyer, operating the first user terminal 10, operates the display screen 1100 to display the display screen 1100 and issues a purchase instruction for the product from the display screen 1100. The display screen 1100 is a screen image that accepts a purchase instruction from the buyer for the product captured in the image captured by the second camera 18. Upon receiving the purchase instruction from the display screen 1100, the first user terminal 10 executes a payment process in accordance with the purchase instruction from the buyer via the information processing device 14. While FIG. 7 illustrates an example in which the product is cosmetics, it may also be food, as shown in FIG. 8. Furthermore, when purchasing a product, if an e-commerce site for purchasing the product exists, a transition button or the like with an embedded URL to the e-commerce site may be displayed on the first user terminal 10, allowing the first user terminal 10 to access and purchase the product from the e-commerce site.

[0065] Fig. 8 is a diagram illustrating an example of a usage scene of the information processing system according to this embodiment in another sales area. This embodiment is an example of another sales area such as a food sales area, and Fig. 8 shows an example of transition of the display screen displayed on the first user terminal 10, similar to Fig. 7.

[0066] The display screen 1200 is a screen image for logging in from the first user terminal 10 to a service provided by the information processing device 14. The display screen 1200 is an example in which the service is for members. If the service is not for members but is available to anyone, the display screen 1200 may be omitted.

[0067] Display screens 1220 and 1240 are screen image diagrams in which a purchaser or the like operating the first user terminal 10 selects a product sales area. In the example of Fig. 8, when a purchaser or the like operating the first user terminal 10 selects "B1 Foods" on display screen 1220, display screen 1240 is displayed, which shows sales areas in which foods are categorized by ingredients. Note that display screen 1240 may clearly display sales areas that should attract the purchaser's attention, such as sales areas that are currently on limited-time sales.

[0068] A purchaser or the like operating the first user terminal 10 selects, by an operation such as clicking, a sales floor for which the purchaser or the like wishes to view an image captured by the first camera 16 from the display screen 1240. When, for example, the "Fresh Fish" sales floor is selected from the display screen 1240, the first user terminal 10 receives from the information processing device 14 images captured by the first camera 16 installed in the sales floor selected by the purchaser or the like, and displays a display screen 1260 in which the images captured by the first camera 16 are displayed in the first display area. A purchaser or the like operating the first user terminal 10 can select, for example, the shooting direction of the images captured by the first camera 16 on the display screen 1260.

[0069] A purchaser or the like operating the first user terminal 10 can check the atmosphere of the sales floor and the products lined up in the sales floor from the image captured by the first camera 16 displayed on the display screen 1260. For example, if there is a product that interests the purchaser or the like and the purchaser or the like wants to receive an explanation from the seller, the purchaser or the like operating the first user terminal 10 calls the second user terminal 12 operated by the seller.

[0070] Display screen 1280 is a screen image diagram after a purchaser or the like operating first user terminal 10 calls a seller operating second user terminal 12 and starts communication with the purchaser or the like and the seller. First user terminal 10 receives from second camera 18 a captured image such as an eye-level image of the seller who has started communication, and displays display screen 1280 on which the captured image by second camera 18 is displayed in the second display area.

[0071] After starting two-way communication with the seller, the buyer or the like can receive an explanation of the product from the seller while viewing an image of the product captured by the second camera 18 included in the display screen 1280. When the buyer or the like finds a product they want to purchase, the buyer or the like, operating the first user terminal 10, performs an operation to display the display screen 1300 and performs an operation to instruct the buyer or the like to purchase the product from the display screen 1300. The display screen 1300 is a screen image that accepts a purchase instruction from the buyer or the like for the product captured in the image captured by the second camera 18. Note that the display screen 1300 shows that the price of the product has been changed from "¥4,980" to "¥2,580." For example, the price of the product may be automatically changed and displayed on the display screen 1300 based on information correlating the elapsed time with the discount rate, or the seller may change it manually. In addition, although this embodiment shows an example of displaying the price of a product on the display screen 1300, it is also possible to display the price of the product superimposed on each product on the display screen 1280 based on the product information (reading of barcodes, product images, etc.) read from the image captured by the second camera 18, and to accept product purchase instructions from purchasers, etc.

[0072] When a product purchase instruction is received from the display screen 1300, the first user terminal 10 executes a settlement process according to the product purchase instruction from the purchaser, etc. via the information processing device 14. While Figures 7 and 8 show examples of B2C (transactions between businesses and general consumers), they may also be B2B (transactions between businesses), as shown in Figure 9.

[0073] FIG. 9 is a diagram illustrating another example of a usage scenario for transactions between traders in the information processing system according to this embodiment. In this embodiment, a buyer, such as a stockist at a fish and vegetable wholesale market or a direct sales store, is a purchaser, etc., who operates the first user terminal 10. Also, in this embodiment, a seller, such as a middleman or wholesaler, is a seller, etc., who operates the second user terminal 12. Here, an example that can be applied to transactions between a buyer, such as a stockist at a fish and vegetable wholesale market or a direct sales store (hereinafter simply referred to as a "buyer, etc."), and a seller, such as a middleman or wholesaler (hereinafter simply referred to as a "seller"), is described. Like FIG. 7 and other figures, FIG. 9 illustrates an example of transitions of display screens displayed on the first user terminal 10.

[0074] The display screen 1400 is a screen image for logging in from the first user terminal 10 to a service provided by the information processing device 14. The display screen 1400 is an example in which the service is for members. If the service is not for members but is available to anyone, the display screen 1400 may be omitted.

[0075] The display screen 1420 is a screen image diagram showing a screen in which a purchaser or the like operating the first user terminal 10 selects a sales area for a product (such as a wholesale market for fresh fish and vegetables or a direct sales store). In the example of FIG. 8, a purchaser or the like operating the first user terminal 10 selects a sales area for a product from the display screen 1420. FIG. 8 shows an example of a sales area being a shelf at a central wholesale market for fresh fish and vegetables or a direct sales store. When a purchaser or the like selects a shelf or island on which they want to view a product on the display screen 1420, the first user terminal 10 displays a sales area display screen 1440. Note that the display screen 1440 may clearly display sales areas that should be noted by purchasers or the like (such as by superimposing a red frame) as a measure to prevent food waste. Furthermore, the display screen 1440 may be capable of displaying sales areas for products that have been on display for a predetermined time as a measure to prevent food waste.

[0076] A purchaser or the like operating the first user terminal 10 selects, by an operation such as clicking, from the display screen 1420, the sales floor for which the purchaser or the like wishes to view an image captured by the first camera 16. The first user terminal 10 receives from the information processing device 14 the image captured by the first camera 16 installed in the sales floor selected by the purchaser or the like, and displays a display screen 1440 on which the image captured by the first camera 16 is displayed in the first display area. The purchaser or the like operating the first user terminal 10 can select, for example, on the display screen 1440, the direction in which the image captured by the first camera 16 is to be captured.

[0077] A purchaser or the like operating the first user terminal 10 can check the atmosphere of the sales floor and the products lined up in the sales floor from the image captured by the first camera 16 displayed on the display screen 1440. For example, if there is a product that interests the purchaser or the like and the purchaser or the like wants to receive an explanation from the seller, the purchaser or the like operating the first user terminal 10 calls the second user terminal 12 operated by the seller.

[0078] Display screen 1460 is a screen image diagram after a purchaser or the like operating first user terminal 10 calls a seller operating second user terminal 12 and starts communication with the purchaser or the like and the seller. First user terminal 10 receives from second camera 18 a captured image such as an eye-level image of the seller who has started communication, and displays display screen 1460 on which the captured image by second camera 18 is displayed in the second display area.

[0079] After starting two-way communication with the seller, the buyer can receive an explanation of the product from the seller while viewing an image of the product captured by the second camera 18 included in the display screen 1460. Once the buyer finds a product they want to purchase, the buyer, operating the first user terminal 10, operates the display screen 1480 to display the display screen 1480 and issues a purchase instruction for the product from the display screen 1480. The display screen 1480 is a screen image that accepts a purchase instruction from the buyer for the product captured in the image captured by the second camera 18. The display screen 1480 shows the price of the product changed from "¥4,980" to "¥2,580." Upon receiving the purchase instruction from the display screen 1480, the first user terminal 10 executes a settlement process in accordance with the purchase instruction from the buyer via the information processing device 14. Another example of business-to-business transactions may be applied to regional development, such as purchasing from various local (local) stores and associations.

[0080] Here, the processing executed by the information processing device 14 of the information processing system 1 will be described using the usage scene shown in Fig. 7 as an example. Fig. 10 is a flowchart showing an example of processing of the information processing system according to this embodiment.

[0081] In step S10, the information processing device 14 accepts login from the first user terminal 10. If the login is successful, the information processing device 14 executes the processes from step S12 onwards. If the login is unsuccessful, the information processing device 14 does not execute the processes from step S12 onwards. Here, the explanation will continue assuming that the login is successful. It is also possible to allow the information processing system 1 to be used without logging in. In that case, the processing of step S10 is omitted, and the first user terminal 10 is permitted to be used as a guest. For a purchaser or the like permitted to use as a guest, the information processing system 1 may request login in one of the following steps and execute the processing of step S10 (for example, a purchase instruction in step 30), or may be configured to prompt the purchaser to register as a member before executing the processing.

[0082] In step S12, the distribution management unit 50 of the information processing device 14 accepts a selection of a sales floor for the product from a purchaser or the like operating the first user terminal 10. In step S14, the distribution management unit 50 refers to the distribution setting information in Fig. 5, for example, and connects to the room URL corresponding to the selected sales floor.

[0083] In step S16, the distribution data generation unit 52 acquires the captured image received from the first camera 16. In step S18, the distribution data generation unit 52 generates distribution data for a display screen that displays the captured image by the first camera 16 in a first display area. The distribution management unit 50 transmits the generated distribution data for the display screen to the first user terminal 10, and causes the first user terminal 10 to display, for example, display screen 1060 in FIG. 7 , in which the captured image by the first camera 16 is displayed in the first display area.

[0084] 7, a purchaser or the like operating the first user terminal 10 checks the atmosphere of the sales floor, the products lined up in the sales floor, etc., while selecting the shooting direction of the image captured by the first camera 16 displayed in the first display area of the display screen 1060. For example, if there is a product that interests the purchaser or the like and the purchaser or the like would like to receive an explanation from the seller, the purchaser or the like operating the first user terminal 10 calls the second user terminal 12 operated by the seller.

[0085] In step S20, the distribution management unit 50 waits until it receives, from the first user terminal 10, an input operation by a purchaser or the like operating the first user terminal 10 to call a seller operating the second user terminal 12. When the input operation by the purchaser or the like to call a seller is received from the first user terminal 10, the process proceeds to step S22, and the distribution management unit 50 connects to the second camera 18 of the seller called by the purchaser or the like.

[0086] In step S24, the distribution data generation unit 52 acquires the captured image received from the second camera 18. In step S26, the distribution data generation unit 52 generates distribution data for a display screen that displays the captured image by the second camera 18 in the second display area. The distribution management unit 50 transmits the generated distribution data for the display screen to the first user terminal 10, and causes the first user terminal 10 to display, for example, display screen 1080 in FIG. 7 , in which the captured image by the first camera 16 is displayed in the first display area and the captured image by the second camera 18 is displayed in the second display area.

[0087] In step S28, the communication function providing unit 56 provides a voice communication function using the call function or a text communication function using the chat function to a purchaser using the first user terminal 10 and a seller using the second user terminal 12.

[0088] For example, on the display screen 1080 of FIG. 7 , after starting two-way communication with a seller, a purchaser operating the first user terminal 10 can receive an explanation of the product from the seller while viewing an image of the product (e.g., an image from the seller's perspective) captured by the second camera 18 included in the display screen 1080. Thus, according to this embodiment, a purchaser can check the surrounding circumstances, such as the atmosphere of the sales floor, and the products displayed on the sales floor, and, if necessary, check product details, such as the texture and freshness of the product. Furthermore, according to this embodiment, a purchaser can receive a realistic explanation of the product from the seller while viewing, for example, a captured image of the product. Once a product they wish to purchase is found, a purchaser operating the first user terminal 10 can, for example, perform an operation to display the display screen 1100 and issue a purchase instruction for the product from the display screen 1100.

[0089] In step S30, the purchase processing unit 58 of the information processing device 14 determines whether or not a purchase instruction has been received from the first user terminal 10. If the purchase processing unit 58 receives a purchase instruction from the first user terminal 10, the process proceeds to step S32, and performs payment processing for the product in accordance with the purchase instruction in cooperation with the payment system 22. If the purchase processing unit 58 has not received a purchase instruction from the first user terminal 10, the payment processing of step S32 is skipped.

[0090] In step S34, the distribution management unit 50 determines whether or not a termination operation by a purchaser or the like operating the first user terminal 10 has been accepted from the first user terminal 10. If a termination operation has not been accepted, the distribution management unit 50 returns to step S28 and repeats the processes of steps S28 to S34. If a termination operation has been accepted, the distribution management unit 50 ends the process of the flowchart in FIG. 10.

[0091] [Other embodiments] In this embodiment, an example has been shown in which first camera 16 is installed on the sales floor, but first camera 16 may also be mobile. For example, first camera 16 may be carried by a sales floor employee, may be attached to an employee patrolling the sales floor, or may be attached to a self-propelled robot patrolling the sales floor.

[0092] Furthermore, in this embodiment, a purchaser or the like operating the first user terminal 10 selects a sales floor and displays the image captured by the first camera 16 in the first display area of the display screen, but this is not limiting. For example, area coordinates within a predetermined range from the position of the first camera 16 may be managed, and when the position of the second user terminal 12 enters the area coordinates of the first camera 16, the image captured by the first camera 16 may begin to be displayed in the first display area. The image captured by the second camera 18 may begin to be displayed simultaneously with the image captured by the first camera 16, or may begin to be displayed before the image captured by the first camera 16 is displayed.

[0093] For example, by associating a buyer or the like with a seller through login, the information processing system 1 can display a display screen including images captured by the first camera 16 and the second camera 18 on the first user terminal 10 of the buyer or the like when the second camera 18 of the moving seller enters the area coordinates of the first camera 16. Note that this can also be achieved by acquiring the location information of each device such as the first camera 16 and the second camera 18 using a beacon as an indoor sensor, short-range wireless communication such as RFID (Radio Frequency Identification) or Bluetooth (registered trademark).

[0094] As described above, according to this embodiment, a purchaser, who is an example of a first user, and a seller, who is an example of a second user, communicate with each other, and the purchaser, who is an example of a second user, can check the images captured by the second camera 18 (the shooting direction is determined by the direction the seller is facing / aiming, etc.).Therefore, by the purchaser, etc. conveying their requests to the seller, it is easy to display the images captured by the second camera 18 from the seller's perspective as the purchaser, etc. desires.

[0095] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a conventional circuit module designed to perform each function described above.

[0096] The devices described in the examples merely represent one of multiple computing environments for implementing the embodiments disclosed herein. The present invention is not limited to these embodiments, and the components in these embodiments include those that would be easily conceived by a person skilled in the art, those that are substantially identical, and those that are within the scope of what is called equivalents. Furthermore, various omissions, substitutions, modifications, and combinations of the components can be made without departing from the spirit of the present embodiments. [Explanation of symbols]

[0097] 1. Information Processing Systems 10. First user terminal 12 Second user terminal 14 Information processing equipment 16 First Camera 18 Second Camera 20 Data Storage System 22 Payment Systems 28 Network 50 Distribution Management Department 52 Distribution data generation unit 54 Communications Department 56 Communication Functions Department 58 Purchase Processing Department 60 Memory control unit 62 Distribution setting information storage unit 64 Script management information storage unit 66 Data storage unit 1060, 1080 display screen [Prior art documents] [Patent documents]

[0098] [Patent Document 1] Japanese Patent Application Publication No. 2019-197427

Claims

1. A system including a first user terminal operated by a first user who is a purchaser, a second user terminal operated by a second user who is a seller, a camera, and an information processing device, The information processing device includes: a distribution data generating means for generating distribution data for displaying a display screen that displays a first photographed image photographed by the camera installed in the sales floor; a communication means for transmitting the generated distribution data to the first user terminal; a communication function providing means for providing a two-way communication function between the first user who operates the first user terminal and the second user who operates the second user terminal when an input operation on the first user terminal is received; and The system is characterized in that, when the distribution data generation means receives the input operation from the first user while the first captured image is being displayed on the display screen, it generates the distribution data that adds and displays a second captured image taken by a camera worn by the second user or a camera of the second user terminal on the display screen displaying the first captured image.

2. a distribution management unit that accepts a selection of one sales floor by the first user from among a plurality of sales floors; and The distribution data generating means generates the distribution data for displaying the display screen including the first captured image taken by the camera installed in the one sales floor selected by the first user. The system of claim 1 .

3. The input operation is an operation by the purchaser to call the seller.

3. The system according to claim 1 or 2.

4. The display screen displays the first captured image and the second captured image side by side, either vertically or horizontally.

4. A system according to any one of claims 1 to 3.

5. The camera installed in the sales area is a camera that can capture 360-degree images of the surrounding area.

5. A system according to any one of claims 1 to 4.

6. The distribution data generating means generates the first captured image from the 360-degree image in the capturing direction selected by the first user. The system of claim 5.

7. a purchase processing means for receiving a purchase instruction relating to a subject included in the second photographed image from the first user terminal and executing a purchase process in accordance with the purchase instruction; 7. The system of any one of claims 1 to 6, further comprising:

8. The first user terminal includes an execution unit that executes processing for two-way communication with the second user who operates the second user terminal. and The second user terminal includes an execution unit that executes a process for performing two-way communication with the first user who operates the first user terminal. have 8. A system according to any one of claims 1 to 7.

9. An information processing method executed by a system including a first user terminal operated by a first user who is a purchaser, a second user terminal operated by a second user who is a seller, a camera, and an information processing device, The information processing device, a distribution data generating step of generating distribution data for displaying a display screen that displays a first captured image captured by a camera installed in the sales floor; a communication step of transmitting the generated distribution data to the first user terminal; a communication function providing step of providing a two-way communication function between the first user who operates the first user terminal and the second user who operates the second user terminal when an input operation on the first user terminal is received; Run The distribution data generating step, when receiving the input operation from the first user while the first photographed image is being displayed on the display screen, generates the distribution data for additionally displaying a second photographed image captured by a camera worn by the second user or a camera of the second user terminal on the display screen displaying the first photographed image. Information processing methods.

Images