Terminal device, server device, selling system, order placement method, order acceptance method, and program

Abstract

To provide a terminal device and so on capable of reducing the time it takes for a customer to enter a store and order a product displayed in a showcase.SOLUTION: The terminal device according to the present disclosure stores first face information which is the face information of a customer using a terminal device as a store customer and taste information showing the customer taste, and transmits it to a server device. The portable device receives the order information including the product information generated for the product indicated by the taste information and indicating the information of the product displayed in a store showcase, and including displayed position information indicating the displayed position, from the server device when the customer entry into the store is detected by the server device through matching second face information which is the face information obtained by photographing the face of a person entering a store by an imaging device installed in the store and the first face information. The terminal device displays the received ordering information, accepts user operations to specify an order product based on the ordering information, and transmits the ordering information representing the specified product to the server device.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a terminal device, a server device, a sales system, an ordering method, an order receiving method, and a program.

Background Art

[0002] Patent Document 1 discloses a customer management system aimed at executing efficient one-to-one marketing by identifying a customer and acquiring customer data through natural customer service sales behavior. The customer management system includes an imaging means for imaging a customer's face image, a face authentication system for recognizing a customer based on the face image data imaged by the imaging means, a customer information presenting means for presenting customer information recognized by the face authentication system, and a customer information storage means. The customer information storage means stores customer information including customer attribute information and purchase information. The face authentication system has a face image storage means in which the face image data of the customer is registered, and stores the face image data of the customer registered in the face image data storage means in association with the customer information stored in the customer information storage means. The face authentication system presents the customer information associated with the recognized face image data to the customer information presenting means as a result of recognizing the customer based on the face image data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the customer management system described in Patent Document 1 cannot shorten the time required from when a customer enters a store to when the customer places an order for a product displayed in a showcase.

[0005] An object of the present disclosure is to provide a terminal device, a server device, a sales system, an ordering method, an order receiving method, and a program capable of shortening the time required from when a customer enters a store until the customer orders a product displayed in a showcase.

Means for Solving the Problems

[0006] The terminal device according to the present disclosure includes a storage unit that stores first face information, which is face information of a customer who is a customer of the store and uses the terminal device, and preference information indicating the preferences of the customer; a first transmission unit that transmits the first face information and the preference information to a server device; a first reception unit that receives, from the server device, order information including product information indicating products displayed in a showcase of the store and display position information indicating display positions, the order information being generated for products indicated by the preference information when entry of the customer into the store is detected by the server device by collating second face information, which is face information obtained by imaging the face of a person entering the store with an imaging device installed in the store, with the first face information; a display unit that displays the received order information; an operation unit that receives a user operation for designating a product to be ordered based on the order information; and a second transmission unit that transmits order information indicating the product designated by the user operation to the server device.

[0007] The server device according to the present disclosure includes: a first storage unit that stores, in association with each other, product information, which is information indicating a product, and display position information, which indicates a display position, regarding the products displayed in the showcase, as a result of performing image recognition processing on an image of the showcase captured by a first imaging device installed so as to be able to image the showcase of the store; a second storage unit that stores first face information, which is the face information of the customer, and preference information, which indicates the preference of the customer, transmitted from a terminal device used by the customer; a detection unit that detects the entry of the customer into the store by collating second face information, which is face information obtained by imaging the face of a person entering the store with a second imaging device installed in the store, with the first face information; a first transmission unit that transmits, to the terminal device, order information including the product information and the display position information regarding the products indicated by the preference information, based on the preference information, when the entry of the customer into the store is detected; and a first reception unit that receives, from the terminal device, order information indicating a product designated as a product to be ordered based on the order information in the terminal device.

[0008] The sales system according to the present disclosure includes a terminal device used by a customer and a server device. The server device stores, in association with each other, product information, which is information indicating a product, and display position information, which indicates a display position, for the products displayed in the showcase, as a result of performing image recognition processing on an image of the showcase captured by a first imaging device installed so as to be able to image the showcase of the store. The server device also includes a second storage unit that stores first face information, which is the face information of the customer, and preference information, which indicates the preferences of the customer, transmitted from the terminal device; a detection unit that detects the entry of the customer into the store by comparing second face information, which is face information obtained by imaging the face of a person entering the store with a second imaging device installed in the store, with the first face information; a first transmission unit that, when the entry of the customer into the store is detected, transmits order information including the product information and the display position information for the products indicated by the preference information to the terminal device based on the preference information; and a first reception unit that receives order information indicating the product designated as the product to be ordered based on the order information from the terminal device. The terminal device includes a storage unit that stores the first face information and the preference information; a second transmission unit that transmits the first face information and the preference information to the server device; a second reception unit that receives the order information from the server device when the entry of the customer into the store is detected by the server device by comparing the second face information with the first face information; a display unit that displays the received order information; an operation unit that accepts a user operation for designating a product to be ordered based on the order information; and a third transmission unit that transmits the order information indicating the product designated by the user operation to the server device.

[0009] The ordering method according to the present disclosure is such that a terminal device stores first face information that is the face information of a customer of a store who uses the terminal device, and preference information indicating the preferences of the customer, transmits the first face information and the preference information to a server device, and when the server device detects the entry of the customer into the store by collating second face information, which is face information obtained by imaging the face of an incoming customer with an imaging device installed in the store, with the first face information, order information including product information, which is information indicating the products displayed in the showcase of the store and generated for the products indicated by the preference information, and display position information indicating the display positions, is received from the server device, the received order information is displayed, a user operation for designating a product to be ordered based on the order information is received, and order information indicating the product designated by the user operation is transmitted to the server device.

[0010] The order receiving method according to the present disclosure is such that a computer stores, in association with each other, product information, which is information indicating products, and display position information indicating display positions, for the products displayed in a showcase as a result of performing image recognition processing on an image of the showcase captured by a first imaging device installed so as to be able to image the showcase of the store, stores first face information, which is the face information of a customer, and preference information indicating the preferences of the customer, transmitted from a terminal device used by the customer, detects the entry of the customer into the store by collating second face information, which is face information obtained by imaging the face of an incoming customer with a second imaging device installed in the store, with the first face information, and when the entry of the customer into the store is detected, transmits order information including the product information and the display position information for the products indicated by the preference information, based on the preference information, to the terminal device, and receives, from the terminal device, order information indicating the product designated as the product to be ordered based on the order information.

[0011] The program according to the present disclosure causes a computer to store, for a customer of a store, first face information that is the face information of the customer who uses the computer, and preference information indicating the customer's preferences, transmit the first face information and the preference information to a server device, and collate second face information, which is face information obtained by imaging the face of a person entering the store with an imaging device installed in the store, with the first face information. When the server device detects that the customer has entered the store, order information including product information, which is information indicating products displayed in a showcase of the store and generated for products indicated by the preference information, and display position information indicating display positions is received from the server device, the received order information is displayed, a user operation for designating a product to be ordered based on the order information is received, and order information indicating the product designated by the user operation is transmitted to the server device. It is a program for executing an order processing.

[0012] Another program according to the present disclosure causes a computer to store, as a result of performing image recognition processing on an image of a showcase imaged by a first imaging device installed so as to be able to image the showcase of a store, product information, which is information indicating products displayed in the showcase, and display position information indicating display positions in association with each other, store first face information, which is the face information of the customer, and preference information indicating the customer's preferences, transmitted from a terminal device used by the customer, collate second face information, which is face information obtained by imaging the face of a person entering the store with a second imaging device installed in the store, with the first face information to detect that the customer has entered the store, and when it is detected that the customer has entered the store, transmit order information including the product information and the display position information for products indicated by the preference information to the terminal device based on the preference information, and receive, from the terminal device, order information indicating a product designated as a product to be ordered based on the order information. It is a program for executing an order receiving processing.

Effects of the Invention

[0013] According to the present disclosure, it is possible to provide a terminal device, a server device, a sales system, an order receiving method, an order placing method, and a program capable of shortening the time required from when a customer enters a store until the customer places an order for a product displayed in a showcase.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Mode for Carrying Out the Invention

[0015] Hereinafter, embodiments will be described with reference to the drawings. In the embodiments, the same or equivalent elements may be denoted by the same reference numerals, and overlapping descriptions will be omitted as appropriate.

[0016] <Embodiment 1> Using FIG. 1, a configuration example of the terminal device 1 will be described. FIG. 1 is a block diagram showing a configuration example of a terminal device according to the present disclosure.

[0017] The terminal device 1 communicates with a server device described later and is used to perform processing for a user to order products displayed in a showcase of a store. Examples of products displayed in the showcase include foods such as cakes in a sweets shop, but may also be items other than foods. Since the system including the terminal device 1 and this server device is a system for selling products, it can be referred to as a sales system. The user of the terminal device 1 becomes a customer of the store.

[0018] The terminal device 1 can be an information processing device having a communication function such as a smartphone, a tablet terminal, or a mobile PC (Personal Computer). As shown in FIG. 1, the terminal device 1 can include a storage unit 1a, a first transmission unit 1b, a first reception unit 1c, a display unit 1d, an operation unit 1e, and a second transmission unit 1f.

[0019] The storage unit 1a includes, for example, a storage device using a hard disk drive, a solid state drive, or other memory. The storage unit 1a stores first face information that is the face information of a customer of a store who uses the terminal device 1, and preference information indicating the preferences of the customer. The first face information can be a face image obtained by imaging the face with an imaging unit (not shown) provided in the terminal device 1, or information obtained by extracting features from the image. The preference information may include information indicating preferences for products sold in a store that performs order processing in the server device, information indicating preferences unrelated to the store, or both types of information.

[0020] The first transmission unit 1b transmits the first face information and the preference information stored in the storage unit 1a to the server device. Note that even if the transmission timing of the first face information and the transmission timing of the preference information are different, it is sufficient if the server device can determine that the information is from the same terminal device 1. That is, the transmission timing of the preference information is not limited to being simultaneous with the transmission of the first face information. The terminal device 1 only needs to transmit the preference information at least once to the server device 2 together with information indicating that the information is from the terminal device 1. In addition, the terminal device 1 can accept modification of the preference information by a user operation, and when the preference information is modified, the modified preference information may be transmitted to the server device.

[0021] The first reception unit 1c receives order information from the server device. The order information can be a user interface (UI) image for placing an order or information for generating the UI image on the terminal device 1 side.

[0022] Then, when the order information is transmitted from the server device when the server device detects that the customer corresponding to the first face information has entered the store by comparing the first face information with the second face information, which is the face information obtained by imaging the face of the person entering the store with an imaging device installed in the store. Hereinafter, this imaging device is referred to as the first imaging device. The order information is information generated for the products indicated by the preference information of the customer, and includes product information indicating the products displayed in the store showcase and display position information indicating the display positions of the products.

[0023] The display unit 1d is a part for displaying information. The display unit 1d can display, as at least a part of this information, the order information received by the first receiving unit 1c. The display unit 1d is constituted by, for example, a display device such as a liquid crystal display or an organic EL display. The display unit 1d can also have a configuration including a display and a drive circuit for driving the display.

[0024] The operation unit 1e is a part for receiving operations and inputs by the user, and can also be referred to as an operation reception unit. The operation unit 1e receives a user operation for designating a product to be ordered based on the order information. The operation unit 1e can receive an operation on the UI image displayed on the display unit 1d. The operation unit 1e can be realized by, for example, any one or a plurality of physical buttons, a touch panel or a touch sensor mounted on the display unit 1d, a pointing device, and a keyboard. In a configuration where the operation unit 1e includes a touch panel or a touch sensor, the display unit 1d and the touch panel can also be referred to as the operation panel of the terminal device 1.

[0025] The second transmission unit 1f transmits order information indicating the product designated by the user operation received by the operation unit 1e to the server device. In this way, the terminal device 1 can confirm an order based on the order information.

[0026] The terminal device 1 can include a control unit (not shown). This control unit controls the entire terminal device 1. This control unit can be configured to include, for example, an arithmetic processing device such as a CPU and a GPU, a working memory, and a storage device that stores control programs, parameters, and the like. Note that CPU is an abbreviation for Central Processing Unit, and GPU is an abbreviation for Graphics Processing Unit. This control unit can be configured to store a control program in an executable state. However, this control unit can also be configured to store a control program as a circuit configuration like an FPGA (field-programmable gate array), or configured as a dedicated circuit.

[0027] The above program can include a program for realizing the functions of the terminal device 1 while cooperating with hardware such as the storage device included in the storage unit 1a, the display unit 1d, the operation unit 1e, and a communication unit (not shown). The functions of the terminal device 1 refer to the functions described by the storage unit 1a, the first transmission unit 1b, the first reception unit 1c, the display unit 1d, the operation unit 1e, and the second transmission unit 1f. This program can be installed on the terminal device 1 as order application software. Hereinafter, this order application software is simply referred to as an order application. Since the order application has a function of collecting information for orders, it can also be referred to as a POS (Point of Sale) application.

[0028] Next, a configuration example of the server device 2 will be described with reference to FIG. 2. FIG. 2 is a block diagram showing a configuration example of the server device according to the present disclosure.

[0029] The server device 2 is a computer for receiving orders for products from customers and can also be referred to as a server computer. As shown in FIG. 2, the server device 2 can include a first storage unit 2a, a second storage unit 2b, a detection unit 2c, a first transmission unit 2d, and a first reception unit 2e.

[0030] The first storage unit 2a includes, for example, a storage device using a hard disk drive, a solid state drive, or other memories. The first storage unit 2a stores by associating product information and display position information.

[0031] Here, the product information and the display position information are input to the server device 2 as a result of performing image recognition processing on an image of the showcase captured by a first imaging device installed so as to be able to image the showcase of the store. Alternatively, the server device 2 inputs an image of the showcase captured by the first imaging device, and performs image recognition processing on the image to generate product information and display position information.

[0032] In this way, the image recognition processing may be performed by the server device 2, or may be performed by the first imaging device. Further, the image recognition processing may be performed by another server device connected to the first imaging device. The image recognition processing may be any processing that can recognize a product and its position from an image, regardless of the method.

[0033] The product information is information indicating a product displayed in the showcase, and examples include the product name and its abbreviation. The display position information is information indicating the display position of the product displayed in the showcase. The display position of a certain product indicates the position in the showcase where the product is displayed, and can be indicated, for example, by a display position number of the product. For example, if an image of the showcase is divided in advance into a plurality of image areas with a determined size and a display position number, which is a number for distinguishing each image area, is attached, the display position information of a certain product can be obtained as the display position number of the product. At this time, if the first imaging device fixes the imaging range toward the showcase, such division of the image area becomes easy. However, this imaging range does not have to be fixed. Therefore, for example, the first imaging device may be used for security or other purposes without fixing the imaging range, or may be used as the second imaging device described later.

[0034] The second storage unit 2b includes, for example, a storage device using a hard disk drive, a solid state drive, or other memories. Of course, the first storage unit 2a and the second storage unit 2b may use a common storage device. The second storage unit 2b stores the first face information, which is the face information of the customer transmitted from the terminal device 1 used by the customer, and the preference information indicating the preferences of the customer. The second storage unit 2b may store the first face information and the preference information at the stage where they are used by the detection unit 2c and the first transmission unit 2d respectively, for example, it may be a temporary storage until use.

[0035] The detection unit 2c detects the entry of the customer into the store by comparing the second face information, which is the face information obtained by imaging the face of the person entering the store with the second imaging device installed in the above store, with the first face information. The second face information can be a face image obtained by imaging the face of the person entering the store with the second imaging device, or information obtained by extracting features from the image. The detection unit 2c detects the entry of the terminal device 1 into the store when this comparison is successful. The method of comparison itself is not limited. If the second storage unit 2b stores the first face information before detection by the detection unit 2c, the detection unit 2c can detect the entry of the customer indicated by the first face information.

[0036] When the entry of the customer into the store is detected, the first transmission unit 2d transmits order information including product information and display position information about the product indicated by the preference information to the terminal device 1 used by the customer, based on the preference information indicating the preferences of the customer stored in the second storage unit 2b. As described above, the order information can be a UI image for placing an order or information for generating the UI image on the terminal device 1 side. If the first storage unit 2a updates the product information and the display position information so that they are always the latest product information and display position information, the first transmission unit 2d can generate and transmit the order information based on the latest display status.

[0037] The first receiving unit 2e receives order information indicating a product designated as a product to be ordered based on the order information in the terminal device 1 from the terminal device 1. Based on this order information, the store side can make preparations to deliver the product indicated by the order information to the customer.

[0038] Note that the server device 2 can include a control unit (not shown). This control unit performs overall control of the server device 2. This control unit can be configured to include, for example, an arithmetic processing device such as a CPU or GPU, a working memory, and a storage device that stores control programs, parameters, etc. This control unit can be configured to store the control program in an executable state. However, this control unit can also be configured to store the control program as a circuit configuration like an FPGA, or configured as a dedicated circuit. The above program can include an order receiving server program for realizing the functions of the server device 2 while cooperating with the storage devices provided in the first storage unit 2a and the second storage unit 2b, and hardware such as a communication unit (not shown). Here, the functions of the server device 2 refer to the functions described in the first storage unit 2a, the second storage unit 2b, the detection unit 2c, the first transmission unit 2d, and the first receiving unit 2e.

[0039] Also, the server device 1 can be constructed as a distributed system in which its functions are distributed among a plurality of server devices.

[0040] Next, an example of the order placement process in the terminal device 1 will be described with reference to FIG. 3. FIG. 3 is a flowchart for explaining an example of the order placement method according to the present disclosure.

[0041] First, the terminal device 1 stores the first face information and preference information (step S1). Next, the terminal device 1 transmits the stored first face information and preference information to the server device 2 (step S2).

[0042] Step S2, or steps S1 and S2, may be executed, for example, when a customer who owns the terminal device 1 enters the store, or may be executed before entering the store. The stage of entering the store can be detected, for example, when the terminal device 1 can connect to a Wi-Fi (registered trademark; the same applies hereinafter) access point included in the network to which the server device 2 belongs. Alternatively, the stage of entering the store can be detected when the customer operates the terminal device 1 to connect to this network, such as by starting an order application.

[0043] Next, the terminal device 1 receives, from the server device 2, order information including product information and display position information about products displayed in the store's showcase, which is generated for the products indicated by the preference information (step S3). The order information is generated in the server device 2 when the server device 2 detects that a customer using the terminal device 1 has entered the store by comparing the second face information with the first face information.

[0044] Next, the terminal device 1 displays the received order information (step S4). Then, the terminal device 1 accepts a user operation for designating a product to be ordered based on the order information. The terminal device 1 determines whether such a user operation has been accepted (step S5). If YES in step S5, order information indicating the product designated by the user operation is transmitted to the server device 2 (step S6). If NO in step S5 or after the processing of step S6, the terminal device 1 ends the processing.

[0045] Note that the program described above for the terminal device 1 can include a program for causing a computer to execute such an order processing.

[0046] Next, an example of the order receiving process in the server device 2 will be described with reference to FIG. 4. FIG. 4 is a flowchart for explaining an example of the order receiving method according to the present disclosure.

[0047] The server device 2 stores, in association with each other, product information and display position information as a result of performing image recognition processing on an image of a showcase captured by a first imaging device (step S11). Next, the server device 2 stores the first face information and preference information transmitted from the terminal device 1 used by the customer (step S12). Note that the process of step S11 only needs to be executed before step S15 described later. Also, among the processes of step S12, the process of storing the preference information only needs to be executed before step S15 described later. Further, among the processes of step S12, the process of storing the first face information only needs to be executed before step S13 described later.

[0048] The server device 2 collates second face information, which is face information obtained by imaging the face of a person entering the store with a second imaging device, with the first face information (step S13). Then, based on this collation, the server device 2 detects the customer's entry into the store. The server device 2 determines whether or not the entry has been detected (step S14). Note that, after a YES determination in step S14 or the like, after the process of step S13, an image of the showcase may be captured by the first imaging device, and the process of step S11 may be executed. Thereby, the server device 2 can obtain product information and display position information that reflect the inventory status of products after the entry has been detected.

[0049] If the determination in step S14 is YES, the server device 2 generates order information including product information and display position information about products indicated by the preference information based on the preference information indicating the customer's preferences, and transmits the order information to the terminal device 1 (step S15).

[0050] Next, the server device 2 receives order information indicating a product specified as a product to be ordered based on the order information from the terminal device 1 (step S16). If the determination in step S14 is NO or after the process of step S16, the server device 2 ends the process.

[0051] Note that the server program described above for the server device 2 can include a program for causing a computer to execute such an order processing.

[0052] As described above, in the present embodiment, a customer can place an order for a product in the showcase using the terminal device 1, and in particular, can place an order while viewing order information based on preference information, and can also immediately recognize the actual display position based on the order information. Further, in the present embodiment, the store side automatically transmits order information to the terminal device in cooperation with face authentication processing, and the customer can place an order based on the order information. Further, in the present embodiment, since the order information is displayed on the terminal device by automatically transmitting the order information to the terminal device by face authentication, the customer can save the trouble of launching the order application and can smoothly place an order for the product.

[0053] Therefore, according to the present embodiment, it becomes easier for a customer to place an order for a product. Such an effect will be specifically described. In a store where orders are placed while looking at a showcase such as a sweets shop, the storefront is likely to be crowded, there are many people waiting for orders, and it tends to be difficult to place an order. In particular, at the timing when there are many customers visiting the store, a waiting queue is likely to form, and there may be people who give up buying and go home after confirming the presence of the queue. In addition, the difficulty of placing an order also occurs because it is embarrassing to specify a long or difficult product name for an order. In that case, there are also customers who give up ordering such products because they want to avoid such conversations. Further, even if a customer has a product they wish to purchase, they may want to confirm the actual product before purchasing, but it may be difficult to immediately find where the product is in the showcase. It can be said that the difficulty of placing an order also occurs in this regard.

[0054] However, in this embodiment, the customer can place an order while checking the order information based on the preference information on the terminal device 1, and can also immediately recognize the actual display position from the order information. Therefore, according to this embodiment, it is possible to quickly place an order without waiting for the order, to reduce the shyness during the order placement with little need for conversation during the order placement, and to reduce the difficulty of finding the product in the showcase, and to place the order smoothly. Thus, according to this embodiment, it is possible to improve the efficiency of the customer's order placement work and improve the situation where it is difficult for the customer to place an order, and the customer can easily place an order for the product.

[0055] Furthermore, according to this embodiment, since the product can be easily ordered on the terminal device 1, not only the waiting time for the order of the customer who places the order using the terminal device 1 is eliminated, but it can also be said that the waiting time for the order of the customer who places the order without using the terminal device 1 can be shortened. Thus, according to this embodiment, it is possible to shorten the time required from when the customer enters the store to when the customer places an order for the product displayed in the showcase.

[0056] Furthermore, according to this embodiment, since the time required from when the customer enters the store to when the customer places an order for the product displayed in the showcase can be shortened, from the perspective of the store side, the time required until the order for such a product is received can be shortened. That is, according to this embodiment, it is possible to improve the efficiency of the order receiving work on the store side, particularly to shorten the time required from when the customer enters the store until the store side receives the order for the product displayed in the showcase. And for the store, by shortening such time, it is possible to shorten the time the customer stays in the store, reduce the congestion at the storefront, and suppress the omission of customers who purchase products, that is, to suppress the situation where the customer gives up the order and goes back. Therefore, according to this embodiment, it is possible to increase the number of sales per unit time in the store.

[0057] <Embodiment 2> Regarding Embodiment 2, the differences from Embodiment 1 will be mainly described. However, in Embodiment 2, the various examples and effects described in Embodiment 1 can be applied. Using FIG. 5, a configuration example of the sales system 100 will be described. FIG. 5 is a block diagram showing a configuration example of the sales system according to the present disclosure.

[0058] The sales system 100 illustrated in FIG. 5 includes a terminal device 10 which is an example of the terminal device 1, a store server device 20 which is an example of the server device 2, a first camera 30 which is an example of the first imaging device, a second camera 40 which is an example of the second imaging device, and a settlement server device 50. In the sales system 100, the first camera 30 and the second camera 40 are connected to the store server device 20 via the network N. Also, the terminal device 10 can be connected to the store server device 20 via the network N. The settlement server device 50 can be connected to the terminal device 10 and the store server device 20 via the network N or another network.

[0059] The terminal device 10 will be described. The terminal device 10 can include a control unit 11, a storage unit 12, a communication unit 13, an operation unit 14, a display unit 15, and an imaging unit 16.

[0060] The control unit 11 is a part described as the control unit of the terminal device 1 in Embodiment 1. That is, the control unit 11 can be realized by a processor, a working memory, and a non-volatile storage device storing a control program including an order application. The order application is preferably coordinated with the POS function provided in the store server device 20.

[0061] The storage unit 12 is an example of the storage device included in the storage unit 1a. The storage unit 12 stores the first face information 12a and the preference information 12b. The storage unit 12 can also be the storage device provided in the control unit 11. The communication unit 13 is a part that communicates with the store server device 20 and the settlement server device 50 via the network N. The communication unit 13 can be configured by a wireless communication interface capable of such communication.

[0062] The operation unit 14 is an example of the operation unit 1e and is a part that receives operations by the user of the terminal device 10. The display unit 15 is an example of the display unit 1d and is a part that displays a UI image for operating the terminal device 10.

[0063] The operation unit 14 receives an input of preference information 12b indicating the user's preferences from the user of the terminal device 10 and passes the preference information 12b to the control unit 11. The control unit 11 stores the received preference information 12b in the storage unit 12. The input of the preference information 12b can be received with the UI image displayed on the display unit 15.

[0064] The imaging unit 16 is an imaging device that captures moving images and still images. The imaging unit 16 can also be referred to as a camera. The imaging unit 16 is used to obtain first face information that is the face information of the user of the terminal device 10. The first face information can be a face image obtained by the imaging unit 16 capturing the user's face, or information obtained by extracting features from the face image. An operation for performing such imaging is also received by the operation unit 14. An instruction for capturing a face image may be received with the UI image displayed on the display unit 15. The imaging unit 16 passes the face image obtained by imaging to the control unit 11. The control unit 11 stores the face image in the storage unit 12 as the first face information 12a, or extracts features from the face image and stores the extracted information in the storage unit 12 as the first face information 12a.

[0065] As described as the function of the first transmission unit 1b, the control unit 11 transmits the first face information 12a and the preference information 12b stored in the storage unit 12 to the store server device 20 via the communication unit 13.

[0066] As described as the function of the first reception unit 1c, the control unit 11 receives order information from the store server device 20 via the communication unit 13. The order information can be a UI image for placing an order or information for generating the UI image on the terminal device 1 side, but an example of a UI image for placing an order will be given below. Hereinafter, this UI image is referred to as an order UI image.

[0067] As described as the function of the display unit 1d, the control unit 11 causes the display unit 15 to display the order UI image received. As described as the function of the operation unit 1e, the operation unit 14 receives a user operation for designating an item to be ordered based on the order UI image and passes it to the control unit 11. As described as the function of the second transmission unit 1f, the control unit 11 transmits order information indicating the item designated by the user operation received by the operation unit 14 to the store server device 20 via the communication unit 13. Further, the control unit 11 can also transmit payment information to the store server device 20 via the communication unit 13.

[0068] Next, the store server device 20 will be described. The store server device 20 can include a control unit 21, a storage unit 22, and a communication unit 23.

[0069] The control unit 21 is the part described as the control unit of the server device 2 in Embodiment 1. That is, the control unit 21 can be realized by a processor, a working memory, and a non-volatile storage device storing a control program including an order receiving server program.

[0070] The storage unit 22 is an example of the storage device included in the first storage unit 2a and the second storage unit 2b. The storage unit 22 stores display information 22a and customer information 22b. Here, information associating product information and display position information is described as display information 22a, and information associating the first face information 12a and the preference information 12b is described as customer information 22b. The storage unit 22 can also be a storage device provided in the control unit 21.

[0071] The communication unit 23 is a part that communicates with the terminal device 10, the first camera 30, the second camera 40, and the payment server device 50 via the network N. The communication unit 23 can be configured by a wireless communication interface or a wired communication interface capable of such communication. Note that the network N may be provided with a communication control device such as an access point for performing wireless communication with the terminal device 10.

[0072] The control unit 21 receives, via the communication unit 23, an image of the showcase captured by the first camera 30 installed so as to be able to image the showcase of the store. The control unit 21 performs image recognition processing on the image of the showcase, and as a result, generates product information and display position information. Here, an example is given in which the product information is the product name of the product displayed in the showcase, and the display position information is the display position number of the product. Then, as described as the function of the first storage unit 2a, the control unit 21 associates the product name and the display position number and stores them in the storage unit 22 as display information 22a. The display information 22a can be stored, for example, in a database format by associating the product name and the display position number.

[0073] Also, the control unit 21 receives, via the communication unit 23, the first face information 12a, which is the face information of the customer, and the preference information 12b, which indicates the preference of the customer, transmitted from the terminal device 10 used by the customer. Then, as described as the function of the second storage unit 2b, the control unit 21 associates the received first face information 12a and preference information 12b and stores them in the storage unit 22 as customer information 22b. The customer information 22b can be stored, for example, in a database format by associating the first face information 12a and the preference information 12b. As described for the second storage unit 2b, it may be stored at the stage where the first face information 12a and the preference information 12b are used by the detection unit 2c and the first transmission unit 2d, respectively. Therefore, the control unit 21 may associate the information received later among the first face information 12a and the preference information 12b with the information received earlier and store it in the storage unit 22 as customer information 22b.

[0074] Also, the control unit 21 receives, via the communication unit 23, the second face information, which is the face information obtained by imaging the face of the person entering the store from the second camera 40 installed in the store. The second face information can be a face image obtained by imaging the face of the person entering the store with the second camera 40, or information obtained by extracting features from the face image. Such extraction may be performed by the control unit 21 instead of the second camera 40.

[0075] Then, as described in the function of the detection unit 2c, the control unit 21 collates the second face information with the first face information included in the customer information 22b, and when the collation is successful, it detects the entry of the customer indicated by the first face information into the store. When using a face image as the second face information, the first face information to be collated may also use a face image. On the other hand, when using information obtained by extracting features from a face image as the second face information, the first face information to be collated may also use information obtained by extracting features from a face image. When the first face information is included as a face image in the customer information 22b, the control unit 21 may execute the extraction of features from this face image. Hereinafter, the description will be made on the premise that both the first face information and the second face information are face images.

[0076] As described in the function of the first transmission unit 2d, when the control unit 21 detects the entry of a customer into the store, it generates an order UI image based on the preference information included in the customer information 22b and the display information 22a. The generated order UI image is a UI image including the product name and the display position number of the product indicated by the preference information. Then, the control unit 21 transmits it to the terminal device 10 used by the customer via the communication unit 23. For a customer whose entry has been detected once, the control unit 21 may be set not to execute the generation and transmission of such an order UI image again when a face image is captured by the second camera 40 and the entry is detected again.

[0077] After transmitting such an order UI image, the control unit 21 performs the control as described in the function of the first reception unit 2e. That is, the control unit 21 receives, from the terminal device 10 via the communication unit 23, order information indicating the product designated as the product to be ordered based on the order UI image in the terminal device 10. Based on this order information, the store side can prepare to deliver the product indicated by the order information to the customer. In addition, the control unit 21 can receive settlement information from the terminal device 10 via the communication unit 23 and process this settlement information as part of the POS function.

[0078] Next, the settlement server device 50 will be briefly described. The settlement server device 50 can be a computer that receives an electronic settlement request based on settlement information from the store server device 20 via the network N, performs electronic settlement, and returns the result. The settlement server device 50 is a server device operated by a settlement company. The electronic settlement request may be transmitted from the terminal device 10 or the POS terminal device to the settlement server device 50 via the store server device 20. The timing of the electronic settlement may be before the control unit 21 receives the order information or after the receipt of the order information. Basically, it is sufficient if the electronic settlement is completed when the goods are delivered to the customer. Note that the settlement server device 50 may directly receive an electronic settlement request from the terminal device 10 or the POS terminal device via the network N.

[0079] Here, the POS terminal device will be described. When the sales system 100 includes a POS terminal device, the POS terminal device may accept settlement other than electronic settlement, such as cash settlement. In this case as well, regardless of the timing of the settlement, it is basically sufficient if the settlement is completed when the goods are delivered to the customer.

[0080] Next, a processing example of the sales system 100 will be described with reference to FIGS. 6 to 18. Hereinafter, for the sake of simplicity of explanation, the processing executed by the store server device 20 and the terminal device 10 will be described as being executed by the store server device 20 and the terminal device 10, respectively, but in fact, the control unit 21 and the control unit 11 execute while controlling parts in other devices.

[0081] First, an example of a sales method implemented in a store will be schematically described with reference to FIG. 6. FIG. 6 is a flowchart for explaining an example of a sales method implemented in a store using the sales system according to the present disclosure.

[0082] When a customer enters the store, the second camera 40 captures the face image of the entering customer and transmits it to the store server device 20. The store server device 20 acquires it by receiving the face image of the entering customer captured by the second camera 40 (step S21). Next, the store server device 20 collates the acquired second face information, which is the face image, with the first face information included in the customer information 22b stored for each of the plurality of customers (step S22).

[0083] The store server device 20 determines whether there is first face information that has succeeded in collation with the second face information through the collation in step S22, that is, whether the second face information is the face information registered as the first face image, thereby detecting the entering customer (step S23).

[0084] If the result in step S23 is NO, the store server device 20 displays a notification recommending the download and installation of the order application on the store display device, which is a display device provided in the store, to perform an application introduction notification (step S24). In this case, such a notification may include a QR code (registered trademark; the same shall apply hereinafter) indicating the download source of the order application. Instead of step S24, or together with step S24, the store server device 20 may notify a terminal device or a POS terminal device used by a store clerk, not shown in the figure, that a customer whose face information is not registered has visited the store. In that case, it is advisable for the store clerk to introduce the order application to the visiting customer. Alternatively, the application introduction notification in step S24 may be a notification from the store server device 20 to the terminal device 10 recommending that the order application be installed when the terminal device 10 of the entering customer is connected to the network N, for example.

[0085] After step S24, the process returns to step S22, and the face image of the entering customer captured just now is collated with the face image of the customer registered after the installation of the order application. Alternatively, after step S24, the process may return to step S22 to acquire the face image of the entering customer again.

[0086] If the answer is YES in step S23, the store server device 20 generates an order UI image based on the preference information of the shopper and the display information 22a, and pushes and notifies the order UI image to the terminal device 10 of the shopper (step S25). Prior to generating the order UI image, the store server device 20 performs image recognition processing on the image of the showcase captured by the first camera 30 as described above, and as a result, generates the display information 22a. Such push notification causes the order UI image to be displayed on the terminal device 10 of the shopper. In step S25, the store server device 20 may transmit information for granting in-store points to the terminal device 10 of the shopper.

[0087] Next, by specifying the products to be ordered from the terminal device 10 by the shopper and the quantity thereof, the terminal device 10 transmits order information to the store server device 20. As a result, the store server device 20 receives the order information from the terminal device 10 (step S26).

[0088] The store server device 20 performs preparation processing for delivering the products indicated by the received order information to the customer using the terminal device 10 (step S27). This preparation processing may include processing for printing the order details of the products using a store printing device, which is a printing device used by the store and not shown in the figure.

[0089] Next, the store server device 20 determines whether or not registration for external payment has been made with reference to the customer information 22b for the customer of the terminal device 10 that transmitted the order information (step S28). For such determination, the store server device 20 may receive, in advance from the terminal device 10, payment information indicating whether to perform external payment or on-site payment, and include and register it in the customer information 22b. In the case of performing external payment, information such as credit card information used for performing payment by the payment server device 50 may also be registered in the customer information 22b. As a result, the store server device 20 can also identify the payment server device 50 of the payment destination.

[0090] If the answer in step S28 is YES, the store server device 20 transmits settlement information based on the settlement information and order information of the terminal device 10 to the settlement server device 50 to perform electronic settlement (step S29). The terminal device 10 receives an operation for performing electronic settlement for the customer to pay the price of the product at a timing before the transmission of the order information in step S26 or at a timing after the transmission. Then, the terminal device 10 transmits information indicating electronic settlement for the price of the product to the store server device 20. Alternatively, the terminal device 10 may be regarded as having given permission to the store server device 20 to perform electronic settlement with the transmission of the order information in step S26. The terminal device 10 receives settlement information, which is the result of the settlement, from the settlement server device 50 via the store server device 20. The settlement server device 50 also transmits settlement information with the same content as this settlement information to the store server device 20 of the store that is the payment destination of the price. Thereby, the store server device 20 can receive the result of the electronic settlement in step S29. After the process of step S29, the product is handed over to the customer and the sale is completed.

[0091] If the answer in step S28 is NO, settlement is made using a POS terminal device (not shown) (step S30). The POS terminal device transmits accounting information indicating this settlement to the store server device 20, and the store server device 20 performs accounting processing as part of the POS function. The customer who placed the order using the terminal device 10 may pay the price to the store clerk between step S26 and step S27 or after step S27. For this payment, the store clerk inputs the payment amount into the POS terminal device, and the POS terminal device generates accounting information and transmits the accounting information to the store server device 20. Here, the payment may be QR code settlement, electronic settlement using the settlement server device 50, or the like. Thereby, the store server device 20 can receive the accounting information in step S30. For the sake of convenience, although distinguished, the accounting information can also be treated as information synonymous with the settlement information. After the process of step S30, the product is handed over to the customer and the sale is completed.

[0092] Next, with reference to FIGS. 7 and 8, a specific processing example executed after customer U who holds terminal device 10 enters store 200 will be described while giving an example of a process including different payment methods shown in steps S29 and S30. FIG. 7 is a schematic diagram showing a processing example of the sales system according to the present disclosure. FIG. 8 is a schematic diagram showing another processing example of the sales system according to the present disclosure. Note that in FIGS. 7 and 8, the installation location of store server device 20 is not limited to the inside of store 200, and may be outside store 200. Also, in FIGS. 7 and 8, the numbers indicating the procedure of the process are shown in Roman numerals, but for convenience, some processes such as the process in store server device 20 are not numbered.

[0093] In explaining each processing example of FIGS. 7 and 8, FIGS. 9 to 18 are further used. FIG. 9 is a diagram showing an example of a UI image displayed on the terminal device according to the present disclosure. FIGS. 10, 14, 16, and 18 are all diagrams showing other examples of UI images displayed on the terminal device according to the present disclosure. FIG. 11 is a schematic diagram showing an example of a store showcase in the sales system according to the present disclosure. FIG. 12 is a diagram showing an example of display information stored in the sales system according to the present disclosure. FIG. 13 is a diagram showing another example of display information stored in the sales system according to the present disclosure. FIG. 15 is a diagram showing an example of a display image on an in-store display device in the sales system according to the present disclosure. FIG. 17 is a diagram showing an example of a slip printed by an in-store printing device in the sales system according to the present disclosure.

[0094] In FIG. 7, a processing example in which customer U uses the terminal device 10 to perform an electronic payment with respect to the payment server device 50 will be described. As shown in FIG. 7, in store 200, a store server device 20, a first camera 30, a second camera 40, a showcase 60 in which products are displayed, an in-store display device 70, and an in-store printing device 80 are installed. The first camera 30 is arranged at a position and in a direction that can capture the entire showcase 60 so as to be able to grasp the display positions of the products in the showcase 60. The second camera 40 is arranged at a position and in a direction that can capture the faces of the customers entering the store. Also, FIG. 7 shows a situation where customer U enters store 200 while holding the terminal device 10. Although not shown, a POS terminal device operated by store clerk P is installed in store 200.

[0095] After entering store 200 or before entering the store, the user of the terminal device 10 performs a startup operation to start the order application, and starts the order application. The terminal device 10 communicates with the store server device 20 at the stage when the order application is started.

[0096] When the order application is started, as shown in FIG. 9, a UI image 151 of the order application is displayed on the display unit 15 of the terminal device 10. The UI image 151 can include, for example, a menu image 151a including menus according to the user's preferences, recommended menus of store 200, and other normal menus. In addition, the UI image 151 can include a cart button 151b, a favorite registration button 151c, and a member information button 151d. The cart button 151b is a button for displaying a list of menus selected from the menu image 151a and proceeding to the order operation.

[0097] The membership information button 151d is a button for displaying a UI image for displaying and editing the user's information as the membership information of the store 200. When the membership information button 151d is selected, for example, as shown in FIG. 10, a UI image 152 including information input fields such as the name input field 152a of personal information is displayed. Then, the terminal device 10 may store the information input by the user from the information input field in the storage unit 12. Note that the UI image 152 can be displayed in a state where the already registered personal information can be changed.

[0098] When the face image is not registered, the UI image 152 can include a shooting frame 152a for registering the face image of the user of the terminal device 10. The captured face image by the imaging unit 16 may be superimposed and displayed on the shooting frame 152b. Then, when the user selects the shooting button 152c included in the UI image 152, the terminal device 10 may store the face image captured at that stage as the first face information 12a. Note that when the face image is already registered, the UI image 152 may include the registered face image instead of the shooting frame 152b.

[0099] The UI image 152 can include check boxes that allow the user to select one or more preferences from a list 152d of favorite tastes. Then, when the user selects one or more check boxes indicating preferences in the UI image 152, the terminal device 10 may store the information indicating the selected preferences as preference information 12b. Note that when the preference information is already registered, the UI image 152 may display the information indicating the preferences in a selected state, or since the preferences may change depending on the time, such check boxes may be displayed.

[0100] In addition, the UI image 152 can include payment information indicating whether to perform an electronic payment or an in - person payment from the terminal device 10 to the payment server device 50 as payment information. Alternatively, if it is before the registration of the payment information, the UI image 152 can include a checkbox or a pull - down menu or the like that allows the selection of the payment information. The payment information can include credit card information or the like when performing an electronic payment.

[0101] In this way, the user can input the first face information 12a, the preference information 12b, and other personal information, etc., and by selecting the registration button 152e, the registration is made. The registration is executed by the terminal device 10 transmitting the first face information 12a, the preference information 12b, the payment information, and other input information stored by the input to the store server device 20. The store server device 20 receives these information and stores them as customer information 22b to register the customer information 22b. In the case of not registering, the user may select the cancel button 152f even after inputting various information. Thereby, the terminal device 10 may not transmit the various information to the store server device 20.

[0102] The favorite registration button 151c is a button for displaying a UI image for registering the user's preferences. This UI image may be the UI image 152 in FIG. 10, or may be a UI image including only the list 152d, the registration button 152e, and the cancel button 152f. The input of the user's preference information 12b and the registration to the store server device 20 are as described above.

[0103] As described above, the user of the terminal device 10 uses the order application to register information after entering or before entering the store 200.

[0104] As shown in FIG. 7, when customer U enters store 200, first, second camera 40 installed at the entrance or the storefront of store 200 captures a face image of customer U (process i), and transmits it to store server device 20 (process ii). Store server device 20 collates the face image received from second camera 40 with the face image included in customer information 22b. In the following, the explanation will be made on the premise that customer information 22b about customer U is registered. Store server device 20 can detect the entry of customer U through this collation. In this way, in store 200, face authentication is performed using second camera 40 at the storefront.

[0105] After processes i and ii, or prior to processes i and ii, generation or modification of display information 22a is executed as follows. First, first camera 30 captures showcase 60 (process iii). Here, an example where showcase 60 has three tiers is given, but it is not limited to this, and it may have one tier, two tiers, four tiers or more. Note that the showcase is not limited to being divided into tiers in this way, and it may be one in which a plurality of products are hung vertically. Next, first camera 30 transmits the captured image to store server device 20 (process iv). The captured image is, for example, an image that does not include the dashed line indicating the area and the numbers indicating each area in image 60g of FIG. 11.

[0106] Then, store server device 20 divides the received image into a plurality of image areas with a predetermined size, for example, and attaches a display position number, which is a number for distinguishing each image area. Here, an example is given in which store server device 20 extracts 12 image areas with a predetermined size and position, and attaches product position numbers 1 to 12 as display position numbers to each image area. The image areas of product position numbers 1 to 12 extracted in this way can be as shown in image 60g of FIG. 11, for example. Store server device 20 performs image recognition processing on each such image area by comparing features with, for example, a group of pre-stored product images, associates product position numbers 1 to 12 of each image area with the product name, and generates display information 22a as exemplified in FIG. 12.

[0107] As a result, the positions of the products on each shelf of the showcase 60 are stored as display information 22a. The display positions of the products may be fixed for each product, but since image recognition processing is performed on the image captured by the first camera 30, the store can arrange the products freely without any problem. If display information 22a already exists for the showcase 60, it may be overwritten with the generated display information 22a.

[0108] Also, the display information 22a can include an image of the showcase 60 captured by the first camera 30. The store server device 20 may store the image of the showcase 60 captured by the first camera 30 associated with the product information and the display position information as the display information 22a.

[0109] Moreover, although an example where one showcase 60 exists in the store 200 is given, if there are two or more, one or more first cameras 30 are installed so that all the showcases can be imaged, and display information 22a may be generated for each showcase. As a result, for example, as illustrated in FIG. 13, display information 22aa including the showcase number is generated.

[0110] When the display information 22a is stored and the face image matching is successful, the store server device 20 generates an order UI image based on the preference information included in the customer information 22b of the entering customer and the display information 22a. Then, the store server device 20 sends the order UI image to the terminal device 10 of the entering customer by push notification (process v). By such a push notification, the order UI image is displayed on the terminal device 10 of the entering customer. The display of this order UI image can be executed even if the customer U has not launched the order application on the terminal device 10. When there are multiple showcases, information such as the display information 22aa is used for generating the order UI image instead of the display information 22a.

[0111] The order UI image generated by the store server device 20 and displayed on the terminal device 10 can include an image in which display position information 153b about the products indicated by the preference information is superimposed on the image of the showcase 60, as in the UI image 153a of FIG. 14. This image may further include a map that shows the route from the entrance to the showcase 60 or the like, so that the customer U can visually recognize which showcase the product is in, especially when there are multiple showcases.

[0112] In addition, since the UI image 153a is push-notified, it will be displayed on the terminal device 10 as, for example, a pop-up image. Alternatively, the content of the UI image 153a may be displayed on the terminal device 10 as a notification from an order application or the like, and may be displayed when the notification is selected. In FIG. 14, for the product corresponding to the preference selected in the list 152d of favorite tastes in FIG. 10, display position information 153b about the product indicated by the preference information is exemplified, where the product within the image area corresponding to the product position number 12 is surrounded by a circle. However, the display position information 153b to be superimposed is not limited to such an example, and any information that can distinguish the display position of the product indicated by the preference information from other products may be used.

[0113] In this way, the terminal device 10 may display the UI image 151 including the menu of the products sold in the store 200 on the display unit 15, and when receiving order information based on the preference information, superimpose and display the UI image 153a based on the order information on this UI image 151.

[0114] Also, as described above, the UI image 153a may be an image that shows the position of the product registered as a favorite by the customer U at the product display position stored in the store server device 20. The UI image 153a may be able to guide the customer U to the display position of the product using, for example, augmented reality (AR) technology. In addition, the UI image 153a may include information indicating the remaining quantity in addition to the display position of the product.

[0115] Further, the UI image 153a can include an order button 153c for ordering the product indicated by the preference information and a button 153d for closing the UI image 153a. When the order button 153c is selected by the customer U, order information indicating that this product is to be ordered is to be transmitted to the store server device 20.

[0116] Also, the order UI image generated by the store server device 20 and displayed on the terminal device 10 may be an image including the product indicated by the preference information in the menu that matches the user's preference included in the menu image 151a. And when the user selects this menu, the UI image 153a may be displayed. Note that, in this case, the UI image 153a to be displayed by push notification may be an image that does not include the order button 153c.

[0117] This product can be selected based on not only the preference information but also, or as part of the preference information, the past purchase history. For the selection of products using such purchase history, for example, existing marketing analysis can be applied. Thereby, the store server device 20 can select the products that seem to be to the taste of the customer U and can include the selected products in the menu image 151a. By displaying the products in the menu image 151a in the order of the customer U's preference, it is possible to stimulate the customer U's willingness to purchase and encourage additional orders, and increase the average order value.

[0118] Further, when the product indicated by the preference information of the customer U is displayed in the showcase 60, the store server device 20 may transmit the order UI image to the terminal device 10. That is, the terminal device 10 may receive the order UI image from the store server device 20 when the product indicated by the preference information of the customer U is displayed in the showcase 60. Thereby, in the terminal device 10, when the product that matches the preference information of the customer U is not displayed, the existing order UI image can be kept displayed.

[0119] Customer U, who is a customer entering the store, specifies the products to be ordered and their quantities from the terminal device 10 based on the order UI image displayed in process v, and the terminal device 10 transmits order information to the store server device 20 (process vi). In the example of FIG. 7, it shows that the payment information is used for electronic payment by the payment server device 50. Therefore, the store server device 20 receives this order information and transmits payment information to the payment server device 50 based on the payment information included in the customer information 22b (process vii). As a result, the payment server device 50 will receive the payment information from the store server device 20.

[0120] The payment server device 50 performs payment based on the payment information. If there is no problem with the payment information, the payment is completed, and the store server device 20 is notified of the completion of the payment (process viii). The store server device 20 notified of the completion of the payment notifies the terminal device 10 of the completion of the payment (process ix).

[0121] Based on the order information received from the terminal device 10, the store server device 20 outputs to the in-store display device 70 by adding an order number or the like for identifying the order to the order information (process x), and causes the in-store display device 70 to display a UI image as shown in FIG. 15, for example. The UI image shown in FIG. 15 can include information indicating the preparation status of the products. The in-store display device 70 can be a display such as digital signage, but it is preferable to be equipped with a touch panel. At this time, the store server device 20 may output a UI image indicating that it is in preparation to the terminal device 10 based on the order information received from the terminal device 10. In this case as well, it is preferable to add an order number to the UI image. As a result, the terminal device 10 receives and displays on the display unit 15 a UI image 154 including a character string 154a indicating that it is in preparation and an order number 154b from the store server device 20 as shown in FIG. 16, for example.

[0122] In this way, the store server device 20 may transmit status information indicating the status of preparations to deliver the products indicated by the order information received from the terminal device 10 from the store 200 to the customer U to the in-store display device 70 installed in the store 200. As a result, the store 200 side can streamline the operation of delivering the product, and the receipt of the product after the order can be expedited. The in-store display device 70 can be installed at a position where not only the store clerk P but also the customer U can view it, or two or more devices can be installed, one for the store clerk P and one for the customer.

[0123] Further, based on the order information received from the terminal device 10, the store server device 20 adds an order number or the like to the order information and outputs it to the in-store printing device 80 (process xi). For example, it causes the in-store printing device 80 to print the order sheet 81 shown in FIG. 17. The order sheet 81 can be used to be handed to the customer U together with the packed products. Note that regarding process ix, process x, process xi, and the output process of images such as the image 154 to the terminal device 10, their order is not limited.

[0124] The store clerk P in the store 200 checks the order details using the in-store display device 70 or the order sheet 81, picks up the products, packs them, and when the packing is completed, selects the packing completion button 71 indicating the completion of product preparation for the corresponding order number on the in-store display device 70 (process xii). By process xii, the in-store display device 70 notifies the store server device 20 that the packing completion button 71 has been selected. The above-described process xi may be executed when the store server device 20 receives a notification to the effect that the packing completion button 71 has been selected. As a result, the store clerk P can surely prepare to hand the order sheet 81 to the customer U together with the packed products.

[0125] Also, when the store server device 20 receives a notification that the packing completion button 71 has been selected, it transmits notification information indicating that preparations are complete for delivering the merchandise to the terminal device 10 (process xiii). The terminal device 10 that has received this information causes, for example, the UI image 155 shown in FIG. 18 to be displayed (process xiv). At this time, the terminal device 10 may output a notification sound. The UI image 155 can include a character string 155a indicating that preparations are complete, an order number 155b, and a receipt completion button 155c.

[0126] In this way, when the store 200 is ready to deliver the merchandise indicated by the received order information to the customer U, the store server device 20 may transmit notification information indicating that preparations are complete to the terminal device 10. That is, when the store 200 is ready to deliver the merchandise indicated by the order information to the customer U, the terminal device 10 may receive the notification information indicating that preparations are complete from the store server device 20. In this case, the display unit 15 displays this notification information. Also, instead of or in addition to displaying the notification information, the terminal device 10 may be configured to output it as audio. Thereby, the work of the store 200 side to deliver the merchandise to the customer U can be made more efficient, and the receipt of the merchandise after the order can be expedited.

[0127] Next, the customer U checks the UI image 155 and heads to the delivery location to pick up the merchandise (process xv). Also, after the process of process xiii or after the process of process xi performed after packing is complete, the store clerk P who delivers the merchandise may look at the order sheet 81 and call out the order number to the delivery location.

[0128] When the customer U receives the merchandise from the store clerk P, it is advisable to select the receipt completion button 155c (process xvi). Also, when the receipt completion button 155c is selected, the UI image 155 can be switched to an image including a gratitude character string such as "Thank you" for example.

[0129] Also, when the receipt completion button 155c is selected, the terminal device 10 transmits information indicating that fact to the store server device 20 (process xvii). As a result, the store server device 20 detects that the delivery of the product has been completed. Based on this detection, the store server device 20 can execute a process of storing or updating the purchase history of the customer U, or a process of updating the inventory management information. Note that the purchase history can be stored in the storage unit 22 as part of the customer information 22b. Also, the store server device 20 can manage the points that the customer U can use in the store 200, and it is preferable to update these points based on the above detection.

[0130] In FIG. 8, an example of a process in which the customer U pays the consideration to the store clerk P of the store 200 will be described. As shown in FIG. 8, in the store 200, the store server device 20, the first camera 30, the second camera 40, the showcase 60, the in-store display device 70, the in-store printing device 80, and the POS terminal device 90 operated by the store clerk P, which were described in FIG. 7, are installed. Also, in FIG. 8 as well, similar to FIG. 7, a situation where the customer U enters the store 200 with the terminal device 10 is shown. Although not shown, a settlement server device 50 may be connected to the store server device 20.

[0131] As shown in FIG. 8, when the customer U enters the store 200, the same processes as processes i to vi in FIG. 7 are executed (processes I to VI). Note that in processes I to VI as well, the same processes as the processes executed between processes i to vi will be executed in the store server device 20 and the like. By the process of process VI, the order information is transmitted from the terminal device 10 to the store server device 20.

[0132] In the example of FIG. 8, the settlement information indicates that on-site settlement is to be performed. Therefore, the store server device 20 receives the order information transmitted in process VI from the terminal device 10, determines the POS terminal device 90 as the transmission destination based on the settlement information included in the customer information 22b, and transmits the order information to the POS terminal device 90 (process VII). Note that process VII does not have to be executed as long as the order information can be referred to at the time of checkout from the POS terminal device 90.

[0133] In addition, if the payment information has not been registered at this time, the store server device 20 may send a notification to the terminal device 10 to prompt registration, and output a UI image indicating that it is in preparation including the order number at the timing when the electronic payment is completed or the on-site payment is selected.

[0134] Also, after Process VI, similar to Process x, the store server device 20 adds an order number or the like for identifying the order to the order information received from the terminal device 10 based on the order information, and outputs it to the in-store display device 70 (Process VIII). Thereby, a UI image as shown in FIG. 15, for example, is displayed on the in-store display device 70. At this time, the store server device 20 may output a UI image indicating that it is in preparation to the terminal device 10 based on the order information received from the terminal device 10. Also in this case, it is advisable to add an order number as in the UI image 154 of FIG. 16.

[0135] Also, similar to Process xi, the store server device 20 adds an order number or the like to the order information based on the order information received from the terminal device 10, and outputs it to the in-store printing device 80 (Process IX). Thereby, for example, an order sheet 81 as shown in FIG. 17 is printed by the in-store printing device 80. Note that regarding Process VII, Process VIII, Process IX, and the image output process such as the image 154 to the terminal device 10, their order does not matter.

[0136] The store clerk P of the store 200 checks the order details on the store display device 70 or the order sheet 81 in the same way as in Process xii, picks up the goods, packs them, and when the packing is completed, selects the packing completion button 71 of the corresponding order number on the store display device 70 (Process X). By Process X, the store display device 70 notifies the store server device 20 that the packing completion button 71 has been selected. The above-mentioned Process IX may be executed when the store server device 20 receives a notification that the packing completion button 71 has been selected.

[0137] In addition, when the store server device 20 receives a notification that the packing completion button 71 has been selected, similar to process xiii, it transmits notification information indicating that the preparation for delivering the product is complete to the terminal device 10 (process XI). When the terminal device 10 receives this information, it displays a UI image 155 shown in FIG. 18, for example, similar to process xiv (process XII).

[0138] Next, customer U checks the UI image 155 and heads to the delivery location to pick up the product for accounting and delivery (process XIII). Also, after the process of process XI or after the process of process IX performed after packing is completed, the store clerk P who performs accounting and delivery for the product may look at the order sheet 81 and call out the order number to the delivery location.

[0139] Customer U pays the store clerk P the price of the product in cash or the like, and the store clerk P performs accounting processing using the POS terminal device 90 (process XIV). Next, or simultaneously with process XIV, customer U receives the product from the store clerk P. Then, when customer U receives the product from the store clerk P, similar to process xvi, it is advisable to select the receipt completion button 155c (process XV). Also, when the receipt completion button 155c is selected, the UI image 155 can be switched to an image including a thank-you message such as "Thank you" for example. Further, when the receipt completion button 155c is selected, the terminal device 10 transmits information indicating that fact to the store server device 20, similar to process xvii (process XVI). As a result, the store server device 20 detects that the delivery of the product is complete and performs processes such as storing or updating the purchase history of customer U and updating points.

[0140] As described above, also in this embodiment, the customer can order the products in the showcase using the terminal device, and in particular, can place an order while viewing the order information based on the preference information, and can also immediately recognize the actual display position based on the order information. Further, in this embodiment, the store side automatically notifies the terminal device of the order information by push notification in cooperation with the face authentication process, and the customer can place an order based on the order information. Further, in this embodiment, by the push notification based on face authentication, the customer can save the trouble of launching the order application and can smoothly place an order for the product.

[0141] Accordingly, according to this embodiment, similar to the effect of Embodiment 1, it becomes easier for the customer to place an order for the product, and the same accompanying effects are achieved. Further, according to this embodiment, in addition to the effect of Embodiment 1, the following effects are achieved.

[0142] In this embodiment, the order information can be provided by superimposing the display position information of the products indicated by the preference information on the image of the showcase. Therefore, according to this embodiment, it becomes easier for the customer to recognize the display position.

[0143] Further, in this embodiment, the order information can be provided when the products indicated by the preference information are displayed in the showcase. Therefore, according to this embodiment, when only the products that do not match the customer's preferences are displayed, the provision of such order information can be avoided, and it is not necessary to provide unnecessary information to the customer.

[0144] Further, in this embodiment, a UI image including the menu of the products sold in the store is displayed, and when the order information is received, the order information can be superimposed on the UI image. Therefore, according to this embodiment, it is possible to provide the information of the products that match the preferences after presenting the information of the products that do not match the customer's preferences.

[0145] In addition, in the present embodiment, when preparations are made to deliver a product to a customer, a notification can be sent to the terminal device. Therefore, according to the present embodiment, the waiting time for delivery can be shortened.

[0146] In addition, in the present embodiment, the status of preparations for delivering a product to a customer can be displayed on the in-store display device. Therefore, according to the present embodiment, the store staff can efficiently perform their work while checking the status. Also, by installing the in-store display device in a location where customers can view it, customers can also wait for delivery while checking the status including preparations for other customers.

[0147] Also, in stores where customers order while looking at a showcase in a sweets shop or the like, the storefront is likely to be crowded, and not only waiting for orders but also waiting for payment often occurs, leading to a situation where it is difficult to place an order. However, according to the present embodiment, since it is possible to perform pre-payment according to the customer's wishes, the waiting time for payment can also be reduced.

[0148] <Other Embodiments> In each embodiment, the sales system, the order placement method, and the order reception method have been described. However, the system configuration, the configuration of each device constituting the system, the processing procedure, etc. are not limited to those illustrated, and it is sufficient if they include the functions of the devices and the steps of the methods. Also, each device described in each embodiment may have the following hardware configuration. FIG. 19 is a diagram showing an example of the hardware configuration of the device according to the present disclosure.

[0149] The device 1000 shown in FIG. 19 can include a processor 1001, a memory 1002, and an interface (I / F) 1003. The processor 1001 may be, for example, a microprocessor, an MPU (Micro Processor Unit), or a CPU. The processor 1001 may include a plurality of processors. The memory 1002 is configured, for example, by a combination of a volatile memory and a non-volatile memory. The functions in each device described in each embodiment are realized by the processor 1001 reading and executing a program stored in the memory 1002. At this time, for example, the transmission and reception of information with other devices can be performed via a communication interface as the I / F 1003. Also, the input / output for the operation of the device 1000 can be performed via an input / output interface as the I / F 1003.

[0150] In the above example, the program can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic recording media (such as flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (such as magneto-optical disks). Further, this example includes CD-ROM (Read Only Memory), CD-R, CD-R / W. Further, this example includes semiconductor memories (such as mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). Also, the program may be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer readable media can supply the program to the computer via wired communication channels such as electric wires and optical fibers, or wireless communication channels.

[0151] As described above, the present disclosure has been described with reference to the embodiments, but the present disclosure is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Note that the present disclosure is not limited to the above-described embodiments, and can be appropriately changed without departing from the gist. And each embodiment can be combined with other embodiments as appropriate.

[0152] The figures are merely examples for explaining one or more embodiments. Each figure may be associated with not only one specific embodiment but also one or more other embodiments. As can be understood by those skilled in the art, various features or steps described with reference to any one figure can be combined with the features or steps shown in one or more other figures to create, for example, embodiments that are not explicitly illustrated or described. Not all of the features or steps shown in any one figure for explaining exemplary embodiments are necessarily essential, and some features or steps may be omitted. The order of the steps described in any figure may be changed as appropriate.

[0153] Some or all of the above embodiments may be described as follows, but are not limited thereto. (Appendix 1) A terminal device, a storage unit that stores first face information, which is face information of a customer of a store who uses the terminal device, and preference information indicating the customer's preferences; a first transmission unit that transmits the first face information and the preference information to a server device; when the server device detects that the customer has entered the store by collating second face information, which is face information obtained by imaging the face of an entering person with an imaging device installed in the store, with the first face information, order information including product information indicating the products displayed in the showcase of the store and display position information indicating the display positions, which are generated for the products indicated by the preference information, is received from the server device by a first reception unit; a display unit that displays the received order information; an operation unit that receives a user operation for designating a product to be ordered based on the order information; a second transmission unit that transmits order information indicating the product designated by the user operation to the server device; A terminal device comprising the above. (Appendix 2) The order information includes an image obtained by imaging the showcase, with the display position information about the products indicated by the preference information superimposed on the image of the showcase. The terminal device according to Supplementary Note 1. (Supplementary Note 3) When the product indicated by the customer's preference information is displayed in the showcase, the first receiving unit receives the order information from the server device. The terminal device according to Supplementary Note 1 or 2. (Supplementary Note 4) The display unit displays a user interface image including a menu of products sold in the store, and when receiving the order information, superimposes and displays the order information on the user interface image. The terminal device according to any one of Supplementary Notes 1 to 3. (Supplementary Note 5) When the store is ready to deliver the product indicated by the order information to the customer, it includes a second receiving unit that receives notification information indicating that it is ready from the server device. The display unit displays the notification information. The terminal device according to any one of Supplementary Notes 1 to 4. (Supplementary Note 6) A first storage unit that stores, in association with each other, product information indicating the product and display position information indicating the display position about the products displayed in the showcase, as a result of performing image recognition processing on the image of the showcase imaged by a first imaging device installed so as to be able to image the store's showcase. A second storage unit that stores the first face information, which is the face information of the customer, and the preference information indicating the customer's preferences, transmitted from the terminal device used by the customer. A detection unit that detects the customer's entry into the store by collating the second face information, which is the face information obtained by imaging the face of the person entering the store with a second imaging device installed in the store, with the first face information. When the entry of the customer into the store is detected, based on the preference information, a first transmission unit that transmits order information including the product information and the display position information about the product indicated by the preference information to the terminal device; A first reception unit that receives order information indicating a product designated as a product to be ordered based on the order information from the terminal device; A server device comprising the same. (Appendix 7) The first storage unit stores the image of the showcase captured by the first imaging device in association with the product information and the display position information. The order information includes an image in which the display position information about the product indicated by the preference information is superimposed on the image of the showcase. The server device according to Appendix 6. (Appendix 8) When the product indicated by the preference information of the customer is displayed in the showcase, the first transmission unit transmits the order information to the terminal device. The server device according to Appendix 6 or 7. (Appendix 9) When the store is ready to deliver the product indicated by the order information received by the first reception unit to the customer, it comprises a second transmission unit that transmits notification information indicating that it is ready to the terminal device. The server device according to any one of Appendices 6 to 8. (Appendix 10) It comprises a third transmission unit that transmits status information indicating the status of the preparation to deliver the product indicated by the order information received by the first reception unit from the store to the customer to a display device installed in the store. The server device according to any one of Appendices 6 to 9. (Appendix 11) Comprising a terminal device used by a customer and a server device, The server device is Product information, which is information indicating a product, and display position information indicating a display position, associated with each other and stored in a first storage unit as a result of performing image recognition processing on an image of the showcase captured by a first imaging device installed so as to be able to image the showcase of the store. A second storage unit that stores the first face information, which is the face information of the customer, and the preference information indicating the preferences of the customer, transmitted from the terminal device. A detection unit that detects the entry of the customer into the store by collating second face information, which is face information obtained by imaging the face of a person entering the store with a second imaging device installed in the store, with the first face information. A first transmission unit that, when the entry of the customer into the store is detected, transmits order information including the product information and the display position information about the product indicated by the preference information to the terminal device based on the preference information. A first reception unit that receives order information indicating a product designated as a product to be ordered based on the order information from the terminal device. Comprising The terminal device A storage unit that stores the first face information and the preference information. A second transmission unit that transmits the first face information and the preference information to the server device. A second reception unit that receives the order information from the server device when the server device detects the entry of the customer into the store by collating the second face information with the first face information. A display unit that displays the received order information. An operation unit that receives a user operation for designating a product to be ordered based on the order information. A third transmission unit that transmits the order information indicating the product designated by the user operation to the server device. Comprising A sales system. (Appendix 12) The terminal device Stores first face information, which is the face information of a customer who is a customer of the store and uses the terminal device, and preference information indicating the preferences of the customer. Transmit the first face information and the preference information to the server device, When the server device detects the customer's entry into the store by comparing the second face information, which is the face information obtained by imaging the face of the person entering the store with an imaging device installed in the store, with the first face information, order information including product information indicating the products displayed in the showcase of the store and display position information indicating the display positions, which is generated for the products indicated by the preference information, is received from the server device, Display the received order information, Receive a user operation for specifying a product to be ordered based on the order information, Transmit order information indicating the product specified by the user operation to the server device. Ordering method. (Appendix 13) A computer, Stores, in association with each other, product information indicating the products displayed in the showcase and display position information indicating the display positions, as a result of performing image recognition processing on an image of the showcase captured by a first imaging device installed so as to be able to image the showcase of the store, Stores the first face information, which is the face information of the customer, and the preference information indicating the customer's preferences, transmitted from the terminal device used by the customer, Detects the customer's entry into the store by comparing the second face information, which is the face information obtained by imaging the face of the person entering the store with a second imaging device installed in the store, with the first face information, When the customer's entry into the store is detected, based on the preference information, transmits order information including the product information and the display position information for the products indicated by the preference information to the terminal device, Receives, from the terminal device, order information indicating the product specified as the product to be ordered based on the order information in the terminal device, Order receiving method. (Appendix 14) To a computer, Store customers, store the first face information, which is the face information of customers who use the computer, and the preference information indicating the preferences of the customers. Transmit the first face information and the preference information to the server device. When the server device detects the entry of the customer into the store by comparing the second face information, which is the face information obtained by imaging the face of the person entering the store with the imaging device installed in the store, with the first face information, order information including product information indicating the products displayed in the store showcase of the products indicated by the preference information and display position information indicating the display position is received from the server device. Display the received order information. Receive a user operation specifying a product to be ordered based on the order information. Transmit order information indicating the product specified by the user operation to the server device. A program for executing an order processing. (Appendix 15) On a computer, Link and store product information indicating products and display position information indicating display positions for the products displayed in the showcase, as a result of performing image recognition processing on an image of the showcase captured by a first imaging device installed so as to be able to image the store showcase. Store the first face information, which is the face information of the customer, and the preference information indicating the preferences of the customer, transmitted from the terminal device used by the customer. Detect the entry of the customer into the store by comparing the second face information, which is the face information obtained by imaging the face of the person entering the store with the second imaging device installed in the store, with the first face information. When the entry of the customer into the store is detected, based on the preference information, transmit order information including the product information and the display position information for the products indicated by the preference information to the terminal device. Receive order information indicating the product specified as the product to be ordered based on the order information from the terminal device. A program for executing an order receiving process.

[0154] Some or all of the elements (e.g., configurations and functions) described in Appendices 2 to 5 that are subordinate to the terminal device of Appendix 1, and some or all of the elements (e.g., configurations and functions) described in Appendices 7 to 10 that are subordinate to the server device of Appendix 6 may be subordinate to the sales system of Appendix 11 in the same subordinate relationship as Appendices 2 to 5 and 7 to 10. Some or all of the elements (e.g., configurations and functions) described in Appendices 2 to 5 that are subordinate to the terminal device of Appendix 1 may be subordinate to the ordering method of Appendix 12 and the program of Appendix 14 in the same subordinate relationship as Appendices 2 to 5. Some or all of the elements (e.g., configurations and functions) described in Appendices 7 to 10 that are subordinate to the server device of Appendix 6 may be subordinate to the order receiving method of Appendix 13 and the program of Appendix 15 in the same subordinate relationship as Appendices 7 to 10. Some or all of the elements described in any appendix may be applicable to various hardware, software, recording means for recording software, systems, and methods.

Explanation of Reference Signs

[0155] 1, 10 Terminal device 1a Storage unit 1b First transmission unit 1c First reception unit 1d Display unit 1e Operation unit 1f Second transmission unit 2 Server device 2a First storage unit 2b Second storage unit 2c Detection unit 2d First transmission unit 2e First reception unit 11 Control unit 12 Storage unit 12a First face information 12b Preference information 13 Communication unit 14 Operation unit 15 Display unit 16 Imaging unit 20 Store server device 21 Control unit 22 Storage unit 22a Display information 22b Customer information 23 Communication unit 30 First camera 40 Second camera 50 Settlement server device 60 Showcase 70 In-store display device 80 In-store printing device 90 POS terminal device 100 Sales system 200 Store 1000 Device 1001 Processor 1002 Memory 1003 Interface N Network

Claims

1. A terminal device, a storage unit that stores first face information, which is face information of a customer who is a customer of a store and uses the terminal device, and preference information indicating the preferences of the customer; a first transmission unit that transmits the first face information and the preference information to a server device; a first reception unit that receives, from the server device, order information including product information, which is information indicating products displayed in a showcase of the store and generated for products indicated by the preference information, and display position information indicating display positions, when entry of the customer into the store is detected by the server device by comparing second face information, which is face information obtained by imaging the face of a person entering the store with an imaging device installed in the store, with the first face information; a display unit that displays the received order information; an operation unit that receives a user operation for designating a product to be ordered based on the order information; a second transmission unit that transmits order information indicating the product designated by the user operation to the server device; A terminal device comprising the above.

2. The order information includes an image in which the display position information for the product indicated by the preference information is superimposed on an image of the showcase obtained by imaging the showcase. The terminal device according to Claim 1.

3. The first reception unit receives the order information from the server device when the product indicated by the preference information of the customer is displayed in the showcase. The terminal device according to Claim 1 or 2.

4. The display unit displays a user interface image including a menu of products sold in the store, and when the order information is received, superimposes and displays the order information on the user interface image. The terminal device according to Claim 1 or 2.

5. a first storage unit that stores, as a result of performing image recognition processing on an image of the showcase imaged by a first imaging device installed so as to be able to image the showcase of the store, product information, which is information indicating products displayed in the showcase, and display position information indicating display positions, in association with each other; a second storage unit that stores first face information, which is face information of the customer, and preference information indicating the preferences of the customer, transmitted from a terminal device used by the customer; A detection unit that detects the customer's entry into the store by comparing second face information, which is face information obtained by imaging the face of an entering customer with a second imaging device installed in the store, with the first face information; A first transmission unit that, when the customer's entry into the store is detected, transmits order information including the product information and the display position information about the product indicated by the preference information to the terminal device based on the preference information; A first reception unit that receives order information indicating a product designated as a product to be ordered based on the order information from the terminal device; A server device comprising the above.

6. A terminal device used by a customer and a server device; The server device: A first storage unit that stores in association product information, which is information indicating a product, and display position information indicating a display position about a product displayed in the showcase, as a result of performing image recognition processing on an image of the showcase imaged by a first imaging device installed so as to be able to image the store's showcase; A second storage unit that stores the first face information, which is the face information of the customer, and the preference information indicating the customer's preferences transmitted from the terminal device; A detection unit that detects the customer's entry into the store by comparing second face information, which is face information obtained by imaging the face of an entering customer with a second imaging device installed in the store, with the first face information; A first transmission unit that, when the customer's entry into the store is detected, transmits order information including the product information and the display position information about the product indicated by the preference information to the terminal device based on the preference information; A first reception unit that receives order information indicating a product designated as a product to be ordered based on the order information from the terminal device; Comprising the above; The terminal device: A storage unit that stores the first face information and the preference information; A second transmission unit that transmits the first face information and the preference information to the server device; A second reception unit that receives the order information from the server device when the server device detects the customer's entry into the store by comparing the second face information and the first face information; A display unit that displays the received order information; An operation unit that receives a user operation for designating a product to be ordered based on the order information; A third transmission unit that transmits the order information indicating the product designated by the user operation to the server device; Comprising the above. Sales system.

7. The terminal device stores first face information that is the face information of a customer of the store who uses the terminal device, and preference information indicating the preferences of the customer, transmits the first face information and the preference information to the server device, when the server device detects the entry of the customer into the store by collating second face information, which is face information obtained by imaging the face of an incoming person with an imaging device installed in the store, with the first face information, order information including product information, which is information indicating the products displayed in the showcase of the store and generated for the products indicated by the preference information, and display position information indicating the display positions, is received from the server device, the received order information is displayed, a user operation for designating a product to be ordered based on the order information is received, order information indicating the product designated by the user operation is transmitted to the server device. Ordering method.

8. The computer stores, as a result of performing image recognition processing on an image of the showcase imaged by a first imaging device installed so as to be able to image the showcase of the store, product information, which is information indicating the products displayed in the showcase, and display position information indicating the display positions, in association with each other, stores first face information that is the face information of the customer transmitted from a terminal device used by the customer, and preference information indicating the preferences of the customer, detects the entry of the customer into the store by collating second face information, which is face information obtained by imaging the face of an incoming person with a second imaging device installed in the store, with the first face information, when the entry of the customer into the store is detected, based on the preference information, order information including the product information and the display position information for the products indicated by the preference information is transmitted to the terminal device, order information indicating the product designated as the product to be ordered based on the order information in the terminal device is received from the terminal device. Order receiving method.

9. In the computer, first face information that is the face information of a customer of the store who uses the computer, and preference information indicating the preferences of the customer, are stored, the first face information and the preference information are transmitted to the server device, When the server device detects the customer's entry into the store by comparing the second face information, which is face information obtained by imaging the face of the person entering the store with an imaging device installed in the store, with the first face information, order information including product information indicating the products displayed in the store's showcase and display position information indicating the display positions, which is generated for the products indicated by the preference information, is received from the server device. The received order information is displayed. A user operation for designating a product to be ordered based on the order information is received. Order information indicating the product designated by the user operation is transmitted to the server device. A program for executing an order placement process.

10. In a computer, Product information, which is information indicating the products, and display position information indicating the display positions for the products displayed in the showcase are stored in association with each other as a result of performing image recognition processing on an image of the showcase captured by a first imaging device installed so as to be able to image the store's showcase. The first face information, which is the face information of the customer, and the preference information indicating the customer's preferences, which are transmitted from the terminal device used by the customer, are stored. The entry of the customer into the store is detected by comparing the second face information, which is face information obtained by imaging the face of the person entering the store with a second imaging device installed in the store, with the first face information. When the entry of the customer into the store is detected, order information including the product information and the display position information for the products indicated by the preference information is transmitted to the terminal device based on the preference information. Order information indicating the product designated as the product to be ordered based on the order information in the terminal device is received from the terminal device. A program for executing an order receiving process.

Images