Information providing device and control program thereof

Abstract

To enable information relating to a location where a person is present to be provided through a terminal used by the person, without tracking behaviors of the person.SOLUTION: An information providing device identifies, from an image photographed by photographing means, a person whose face is captured in the image. The information providing device identifies a location from information of the photographing means capturing the face of the person. The information providing device identifies a terminal used by the person. The information providing device provides information corresponding to the location of the terminal.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to an information providing device and a control program for causing a computer to function as the information providing device. [Background technology]

[0002] A system has been proposed that supports the purchasing behavior of customers by providing information related to a sales floor to customers in that sales floor. Most of the conventional systems of this type track customers moving around the sales floor in real time, and when the customer comes to a specific sales floor, provide information related to that sales floor to the terminal used by the customer. Known methods for tracking customers include those that use radio positioning technology such as GPS (Global Positioning System) and beacons.

[0003] However, when GPS is used, it is difficult to apply it indoors, and even if it can be applied, the positioning accuracy is poor. Even when beacons are used, it is necessary to place communication devices for beacons at high density in the store to improve the positioning accuracy. In this way, various problems that must be solved can occur when trying to track people's behavior. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 3931541 [Patent Document 2] JP 2010-009566 A [Patent Document 3] JP 2012-185576 A Summary of the Invention [Problem to be solved by the invention]

[0005] The problem that an embodiment of the present invention aims to solve is to provide an information providing device that can provide information related to a person's location via a terminal used by the person, without tracking the person's actions. [Means for solving the problem]

[0006] In one embodiment, the information providing device comprises: A first storage means; A person identification means; A coordinate information acquisition means; The terminal identification means and the providing means are provided. The first storage means stores, for each of a plurality of base cameras installed at a plurality of bases within the store where information to be provided to a purchaser is located, coordinate information that identifies a position within the store photographed by the base camera. The person identification means is Base Camera The system identifies the person whose face appears in the image captured by the camera. The coordinate information acquisition means acquires coordinate information stored in the first storage means for the base camera that captured the image in which the person is identified by the person identification means. The terminal identification means is Identified by a person identification method Identify the device used by the person. The terminal identified by the terminal identification means is provided to a purchaser associated with the coordinate information acquired by the coordinate information acquisition means. Provide information. [Brief description of the drawings]

[0007] [Figure 1] 1 is an overall configuration diagram of a store system established in a store to which an information providing device according to an embodiment is applied; [Diagram 2] FIG. 4 is a schematic diagram showing the data structure of a camera record. [Diagram 3] FIG. 4 is a schematic diagram showing a data structure of a purchaser record. [Figure 4] FIG. 4 is a schematic diagram showing the data structure of a provision information record. [Diagram 5] FIG. 2 is a block diagram showing a main circuit configuration of an information terminal. [Figure 6] FIG. 1 is a perspective view showing an example of a cart equipped with an information terminal. [Figure 7] FIG. 2 is a block diagram showing a main circuit configuration of an image processing server. [Figure 8] FIG. 2 is a schematic diagram showing a format of shooting data. [Figure 9] 4 is a flowchart showing the main steps of information processing executed by a processor of the image processing server according to a control program. [Figure 10] 4 is a flowchart showing the main steps of information processing executed by a processor of a purchaser's server according to a control program. [Figure 11] 11 is a flowchart specifically showing the procedure of the store entrance camera process shown in FIG. 10 . [Figure 12] 11 is a flowchart specifically illustrating the procedure of the cart camera process shown in FIG. 10 . [Figure 13] 11 is a flowchart specifically illustrating the procedure of the base camera processing shown in FIG. 10 . [Figure 14] 11 is a flowchart specifically illustrating the procedure of the accounting camera process shown in FIG. 10 . [Figure 15] 4 is a flowchart showing the main steps of information processing executed by a processor of a provision server according to a control program in the first embodiment. [Figure 16] 11 is a flowchart showing the main steps of information processing executed by a processor of a provision server according to a control program in the second embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of an information providing device that can provide information related to a location where a person is located via a terminal used by the person without tracking the person's actions will be described with reference to the drawings.

[0009] In this embodiment, when a person uses a shopping cart to shop at a retail store such as a supermarket, information related to the sales floor in which the person is located is provided via an information terminal provided in the shopping cart. In the following description, the person who makes a purchase is referred to as a "purchaser." The shopping cart is referred to as a "cart."

[0010] [First embodiment] Fig. 1 is an overall configuration diagram of a store system 1 established in a store to which the information provision device of this embodiment is applied. The store system 1 includes an information terminal 10, an access point 20, a payment machine 30, a camera interface 40, a store server 50, an image processing server 60, a purchaser server 70, and a provision server 80. The store system 1 also includes a network 2. The network 2 connects the access point 20, the payment machine 30, the camera interface 40, the store server 50, the image processing server 60, the purchaser server 70, and the provision server 80 so that they can communicate freely. The network 2 is, for example, a LAN (Local Area Network).

[0011] The information terminal 10 is a device that allows a purchaser to input data related to the registration of purchased items by himself / herself. An information terminal 10 is provided for each cart SC. The cart SC is an example of a moving object that transports purchased items by a purchaser who is a user of the cart SC.

[0012] The information terminal 10 is equipped with a wireless unit. The information terminal 10 performs wireless communication with an access point 20. The access point 20 relays communication between the information terminal 10 and each device connected to the network 2, such as the accounting machine 30, the store server 50, the image processing server 60, the purchaser server 70, or the provision server 80. Although only one access point 20 is shown in Fig. 1, there may be two or more access points 20 depending on the size of the store, etc.

[0013] The information terminal 10 is equipped with a camera CA1, which is one of the imaging means. The camera CA1 is for capturing a facial image of a purchaser who operates a cart SC on which the information terminal 10 is installed. Hereinafter, the camera CA1 functions as a moving body imaging means. Hereinafter, the camera CA1 will be referred to as a cart camera CA1.

[0014] The accounting machine 30 is a device that allows a store clerk or a customer to process the payment for purchased items. The accounting machine 30 is a well-known self-service POS terminal. The accounting machine 30 may be a well-known face-to-face POS terminal. There is no particular limit to the number of accounting machines 30.

[0015] The camera interface 40 receives image data captured by cameras provided as a separate imaging means from the cart camera CA1. The cameras include a store entrance camera CA2, a base camera CA3, and an accounting camera CA4.

[0016] The store entrance camera CA2 is for taking a facial image of a customer who enters the store. The store entrance camera CA2 is installed in a position where it can take a facial image of a customer. There are as many store entrance cameras CA2 as there are entrances.

[0017] The base camera CA3 is for taking a facial image of a purchaser who comes to a base in the sales floor. A base is a place where information to be provided to a purchaser is available. For example, when providing a purchaser who comes to a shelf where a sale item is displayed with information about the sale item, the floor surface in front of the shelf where the sale item is displayed is the base. For example, when providing a purchaser who comes to a shelf where ingredients for a recommended recipe are displayed with information introducing the recipe, the floor surface in front of the shelf where the ingredients are displayed is the base. The base camera CA3 is installed in a position where it can take a facial image of a purchaser who comes to the base. There are as many base cameras CA3 as there are bases.

[0018] The accounting camera CA4 is used to capture a facial image of a customer making a transaction at the payment machine 30. The accounting camera CA4 is installed in a position where it can capture a facial image of a customer making a transaction. There are as many accounting cameras CA4 as there are payment machines 30.

[0019] The store server 50 is a computer device specialized for supporting store operations. The store server 50 is equipped with a product database, a sales database, etc. These databases may be in a storage device built into the store server 50, or in a storage device external to the store server 50. The product database is a collection of product records that describe product data for each product sold at the store. The product data includes product code, price, product name, etc. The product code is an identification code for the product. The sales database is a collection of sales records that describe sales data for each item such as product, department, time period, date, week, month, etc. The sales data includes sales points, sales amount, etc.

[0020] The image processing server 60 is a computer device specialized in processing images captured by the imaging means, i.e., the cart camera CA1, the store entrance camera CA2, the base camera CA3, and the accounting camera CA4. The image processing server 60 includes a camera database 91. The camera database 91 may be stored in a storage device built into the image processing server 60, or in a storage device externally attached to the image processing server 60. The camera database 91 may be stored in a storage device of another server capable of communicating with the image processing server 60. The camera database 91 is a collection of camera records 91R created for each of the cameras CA1, CA2, CA3, and CA4.

[0021] 2 is a schematic diagram showing the data structure of the camera record 91R. As shown in the figure, the camera record 91R includes a camera ID, a camera category, and camera data. The camera ID is a unique code set for each camera CA1, CA2, CA3, and CA4 to individually identify each camera. Each camera CA1, CA2, CA3, and CA4 stores the camera ID set in the camera, and when outputting captured image data, outputs the image data together with the camera ID.

[0022] The camera category is information that indicates the type of camera identified by the camera ID. That is, the camera category is information for distinguishing whether the camera is a cart camera CA1, a store entrance camera CA2, a base camera CA3, or an accounting camera CA4.

[0023] The camera data is in-store coordinates or cart number. In the case of camera record 91R for cart camera CA1, the camera data is the cart number. The cart number is an identification number of the cart SC equipped with the information terminal 10 in which the cart camera CA1 is installed. Therefore, the cart number is information for identifying the information terminal 10 equipped in the cart SC identified by the cart number. In the case of camera record 91R for entrance camera CA2, base camera CA3, and accounting camera CA4, the camera data is in-store coordinates. The in-store coordinates are information for identifying the position in the store photographed by the camera. The in-store coordinates are expressed by regarding the floor of the store as an XY plane, setting an arbitrary position as the origin (X,Y)=(0,0), and expressing the difference from the origin in X and Y coordinates.

[0024] Returning to the explanation of Figure 1. The purchaser server 70 is a computer device specialized for managing information about purchasers. The purchaser server 70 is equipped with a purchaser database 92. The purchaser database 92 may be in a storage device built into the purchaser server 70, or in a storage device external to the purchaser server 70. The purchaser database 92 may be in a storage device of another server that can communicate with the purchaser server 70. The purchaser database 92 is a collection of purchaser records 92R in which information related to purchasers is described.

[0025] FIG. 3 is a schematic diagram showing the data configuration of the purchaser record 92R. As shown in the figure, the purchaser record 92R includes a purchaser ID, a facial image, purchaser attributes, store entry time, location information, cart number, and a checkout flag. The purchaser ID is a unique code assigned to each purchaser to identify each individual purchaser. The facial image is an image of the purchaser's face. The facial image is basically extracted from an image captured by the store entry camera CA2 and described in the purchaser record 92R. The facial image may also be extracted from an image captured by the cart camera CA1 or the base camera CA3 and described in the purchaser record 92R.

[0026] Purchaser attributes are information that indicates the purchaser's attributes such as gender and age. Entry time is the time when the purchaser entered the store. Location information is the in-store coordinates that identify the location within the store where the purchaser is located and the time at that time. Cart number is the identification number of the cart SC used by the purchaser. The transaction flag F is one-bit data that identifies whether or not the purchaser has made a transaction. In this embodiment, the transaction flag F is set to "0" before a transaction is made, and the transaction flag F is set to "1" after a transaction is made.

[0027] Returning to the explanation of Figure 1. The providing server 80 is a computer device specialized in managing information to be provided to purchasers who come to the base. The providing server 80 has a provided information database 93. The provided information database 93 may be in a storage device built into the providing server 80, or in a storage device external to the providing server 80. The provided information database 93 may be in a storage device of another server that can communicate with the providing server 80. The provided information database 93 is a collection of provided information records 93R in which information to be provided to purchasers is described.

[0028] Fig. 4 is a schematic diagram showing the data structure of the provided information record 93R. As shown in the figure, the provided information record 93R includes base coordinates, purchaser attributes, and display data. The base coordinates are information for identifying the location within the store that serves as the base providing the information managed in the provided information record 93R. The base coordinates match the in-store coordinates set in the camera record 91R that includes the camera ID of the base camera CA3. The purchaser attributes are attributes of the purchaser providing the information managed in the provided information record 93R.

[0029] The display data is an example of information provided to a purchaser who has attributes matching the purchaser attributes and who has come to the base identified by the base coordinates of the provided information record 93R. In this embodiment, information is provided to the purchaser by displaying the information on a display device of an information terminal 10 provided in a cart SC used by the purchaser. Therefore, the display data is, for example, text data. The display data may be image data of a still image or a video. Incidentally, if a speaker is provided on the information terminal 10, audio data may be applied in addition to or instead of the display data.

[0030] 5 is a block diagram showing the main circuit configuration of the information terminal 10. The information terminal 10 includes a processor 11, a main memory 12, an auxiliary storage device 13, a wireless unit 14, a touch panel 15, a scanner 16, a reader 17, a camera CA1, and a system transmission path 18. The system transmission path 18 includes an address bus, a data bus, a control signal line, and the like. The information terminal 10 connects the processor 11, the main memory 12, the auxiliary storage device 13, the wireless unit 14, the touch panel 15, the scanner 16, the reader 17, and the camera CA3 to the system transmission path 18. In the information terminal 10, the processor 11, the main memory 12, the auxiliary storage device 13, and the system transmission path 18 that connects them together constitute a computer.

[0031] The processor 11 corresponds to the central part of the computer. The processor 11 controls each part to realize various functions of the information terminal 10 according to an operating system or an application program. The processor 11 is, for example, a CPU (Central Processing Unit).

[0032] The main memory 12 corresponds to the main storage portion of the computer. The main memory 12 includes a nonvolatile memory area and a volatile memory area. The main memory 12 stores an operating system or an application program in the nonvolatile memory area. The main memory 12 may store data required for the processor 11 to execute processes for controlling each part in the nonvolatile or volatile memory area. The main memory 12 uses the volatile memory area as a work area where data is appropriately rewritten by the processor 11. The nonvolatile memory area is, for example, a ROM (Read Only Memory). The volatile memory area is, for example, a RAM (Random Access Memory).

[0033] The auxiliary storage device 13 corresponds to the auxiliary storage portion of the computer. For example, an EEPROM (Electric Erasable Programmable Read-Only Memory), an HDD (Hard Disc Drive), or an SSD (Solid State Drive) can be the auxiliary storage device 13. The auxiliary storage device 13 stores data used by the processor 11 in performing various processes, or data created by the processes in the processor 11. The auxiliary storage device 13 may also store the application programs. The auxiliary storage device 13 stores the identification number of the cart SC on which the information terminal 10 is provided, i.e., the cart number.

[0034] The wireless unit 14 wirelessly communicates data with the access point 20 in accordance with a wireless communication protocol.

[0035] The touch panel 15 is a device that functions as both an input device and a display device for the information terminal 10. The touch panel 15 detects a touch position on a displayed image, and outputs the touch position information to the processor 11.

[0036] The scanner 16 reads code symbols such as barcodes and two-dimensional data codes attached to products. Products are provided with code symbols that represent their product codes. The scanner 16 outputs data of the read code symbols to the processor 11. The scanner 16 may be of a type that reads code symbols by scanning with laser light, or of a type that reads code symbols from images captured by an imaging device.

[0037] Reader 17 reads data recorded on the recording medium and outputs the read data to processor 11. Reader 17 is a magnetic card reader when the recording medium is a magnetic card, and is an IC card reader when the recording medium is a contact IC card. In the case of a recording medium using RFID (Radio Frequency Identification) such as a contactless IC card or a smartphone, an RFID reader is used as reader 17.

[0038] In the information terminal 10 having the circuit components as described above, the processor 11, the main memory 12, the auxiliary storage device 13, the wireless unit 14, and the touch panel 15 are configured as a tablet terminal TM. The information terminal 10 is configured by electrically connecting the scanner 16, the reader 17, and the camera CA1 to this tablet terminal TM.

[0039] FIG. 6 is a perspective view showing an example of a cart SC equipped with an information terminal 10. The cart SC is equipped with a caster portion C1 for moving, a handle frame portion C2, and a basket receiving portion C3. The caster portion C1 has four wheels C11 for smooth movement on the floor surface. The caster portion C1 also has a receiving portion C12 for placing large luggage that does not fit into the basket SB. The handle frame portion C2 includes a pair of vertical frames C21, C21 erected on the rear wheel side of the caster portion C1, and a handle bar C22 connecting the upper ends of these vertical frames C21, C21. The basket receiving portion C3 is located forward from the midpoint of the handle frame portion C2. The cart SC can place a basket SB provided by the store on the basket receiving portion C3. The basket SB is for storing products.

[0040] The scanner 16 is located midway along the handle bar C22. The scanner 16 is attached to the handle bar C22 so that a reading window is located on the near side. The near side is the side on which a customer stands while holding the handle bar C22 and pushing the cart SC.

[0041] A pole C4 is attached to one of the vertical frames C21. The tip of the pole C4 is located above the handlebars C22. The tablet terminal TM is attached to the tip of the pole C4 with the screen of the touch panel 15 facing forward. The reader 17 is attached to the tablet terminal TM so that the card slit is located on the front side. In FIG. 5, the reader 17 is a magnetic card reader. The camera CA1 is attached to the middle of the pole C4 so as to capture an image including the face of a customer holding the handlebars C22.

[0042] The battery BT is attached across the vertical frames C21, C21 at the lower end side of the handle frame portion C2. The battery BT serves as a driving power source for the tablet terminal TM, the scanner 16, the reader 17, and the camera CA1.

[0043] 7 is a block diagram showing the main circuit configuration of the image processing server 60. The image processing server 60 includes a processor 61, a main memory 62, an auxiliary storage device 63, a clock 64, a communication interface 65, and a system transmission path 66. The system transmission path 66 includes an address bus, a data bus, a control signal line, and the like. The image processing server 60 connects the processor 61, the main memory 62, the auxiliary storage device 63, the clock 64, and the communication interface 65 to the system transmission path 66 directly or via a signal input / output circuit. The image processing server 60 constitutes a computer by the processor 61, the main memory 62, the auxiliary storage device 63, and the system transmission path 66 that connects them.

[0044] The processor 61 corresponds to the central part of the computer. The processor 61 controls each part to realize various functions of the image processing server 60 according to an operating system and application programs. The processor 61 is, for example, a CPU.

[0045] The main memory 62 corresponds to the main storage portion of the computer. The main memory 62 includes a nonvolatile memory area and a volatile memory area. The main memory 62 stores an operating system and application programs in the nonvolatile memory area. The main memory 62 may also store data required for the processor 61 to execute processes for controlling each part in the nonvolatile or volatile memory area. The main memory 62 uses the volatile memory area as a work area where data is appropriately rewritten by the processor 61. In the main memory 62, the nonvolatile memory area is, for example, a ROM, and the volatile memory area is, for example, a RAM.

[0046] The auxiliary storage device 63 corresponds to the auxiliary storage portion of the computer. For example, an EEPROM, a HDD, or an SSD is used as the auxiliary storage device 63. The auxiliary storage device 63 stores data used by the processor 61 when performing various processes and data generated by the processes in the processor 61. The auxiliary storage device 63 may also store the application programs described above.

[0047] The clock 64 functions as a time information source for the image processing server 60. The processor 61 keeps track of the current date and time based on the time information kept by the clock 64.

[0048] The communication interface 65 transmits and receives data to and from each unit connected via the network 2 in accordance with a communication protocol.

[0049] The main circuit configurations of the purchaser server 70 and the provision server 80 are also the same as those of the image processing server 60. Therefore, below, the components of the purchaser server 70 are represented as a processor 71, a main memory 72, an auxiliary storage device 73, a clock 74, a communication interface 75, and a system transmission path 76. Also, the components of the provision server 80 are represented as a processor 81, a main memory 82, an auxiliary storage device 83, a clock 84, a communication interface 85, and a system transmission path 86.

[0050] FIG. 8 is a schematic diagram showing the format of the photographed data D1 output from the image processing server 60 to the purchaser server 70. FIG. 9 is a flow chart showing the main steps of the information processing executed by the processor 61 of the image processing server 60 according to a control program. FIGS. 10 to 14 are flow charts showing the main steps of the information processing executed by the processor 71 of the purchaser server 70 according to a control program. FIG. 15 is a flow chart showing the main steps of the information processing executed by the processor 81 of the provision server 80 according to a control program. The main operations of the store system 1 will be explained below with reference to each figure. Note that the operations explained below are just an example. As long as similar results can be obtained, the processing procedures can be modified as appropriate.

[0051] First, the entrance camera CA2, the base camera CA3, and the accounting camera CA4 repeat the shooting operation at a fixed cycle from the opening time to the closing time, and output the captured image data to the camera interface 40 together with the camera ID of the camera itself. The camera ID and image data output to the camera interface 40 are sent to the image processing server 60 via the network 2. The cart camera CA1 is started when a user logs into the information terminal 10, and repeats the shooting operation at a fixed cycle. The shooting operation continues until the information terminal 10 logs out. The image data captured by the cart camera CA1 is wirelessly transmitted together with the camera ID of the camera itself via the wireless unit 14 under the control of the processor 11, and is received by the access point 20. The camera ID and image data received by the access point 20 are sent to the image processing server 60 via the network 2.

[0052] Here, the operation of the image processing server 60 will be described with reference to FIG. The processor 61 of the image processing server 60 waits for image data in Act 1. When the image data is sent via the network 2, the processor 61 judges YES in Act 1 and proceeds to Act 2. The processor 61 obtains the date and time kept by the clock 64 as the shooting time in Act 2. The processor 61 also analyzes the image data and extracts a person's face image in Act 3.

[0053] In Act 4, the processor 61 checks whether a facial image that can recognize a person's face with a certain recognition rate or higher has been extracted. If no facial image has been extracted, the processor 61 determines NO in Act 4 and proceeds to Act 5. The processor 61 discards the image data in Act 5. With this, the processor 61 ends the processing of the image data received in Act 1.

[0054] If the face image can be extracted, the processor 61 determines YES in Act 4 and proceeds to Act 6. The processor 61 acquires the camera ID received together with the image data in Act 6. The processor 61 searches the camera database 91 using the camera ID as search data in Act 7. Then, the processor 61 reads the camera record 91R including the search data as the camera ID in Act 8, and acquires the camera category from the camera record 91R.

[0055] The processor 61 checks whether the camera classification indicates the cart camera CA1 in Act 9. If the camera classification indicates the cart camera CA1, the processor 61 determines YES in Act 9 and proceeds to Act 10. The processor 61 obtains the camera data of the camera record 91R as the cart number in Act 10.

[0056] If the camera category does not indicate the cart camera CA1, that is, if the camera category indicates the store entrance camera CA2, the base camera CA3, or the checkout camera CA4, the processor 61 determines NO in Act 9 and proceeds to Act 11. In Act 11, the processor 61 obtains the camera data of the camera record 91R as in-store coordinates.

[0057] After completing the processing of Act 10 or Act 11, the processor 61 generates the photographed data D1 in the format shown in Fig. 8 in Act 12. That is, the processor 61 generates the photographed data D1 using the camera ID acquired in Act 6, the camera category acquired in Act 8, the photographed time acquired in Act 2, the cart number or in-store coordinates acquired in Act 10 or Act 11, and the face image extracted in Act 3. The processor 61 controls the communication interface 65 to output the photographed data D1 to the customer server 70 in Act 13. By this control, the photographed data D1 is transmitted from the communication interface 65 to the customer server 70 via the network 2.

[0058] The processor 61, which has controlled the output of the photographed data D1, checks whether another facial image has been extracted in Act 14. If multiple facial images have been extracted from the image data received in Act 1, the processor 61 determines YES in Act 14 and returns to Act 12. That is, the processor 61 generates photographed data D1 that differs from the photographed data D1 generated immediately before only in the facial image, and controls the communication interface 65 to output the photographed data D1 to the consumer server 70. In this way, if multiple facial images have been extracted from the image data, the processor 61 controls the communication interface 65 to generate photographed data D1 for the number of facial images and output the photographed data D1 to the consumer server 70.

[0059] When the processor 61 has output the same number of pieces of photographic data D1 as the number of face images extracted from the image data, the processor 61 ends the processing of that image data.

[0060] Next, the operation of the purchaser server 70 will be described with reference to FIGS. The processor 71 of the consumer server 70 waits for the photographed data D1 in Act 21 of Fig. 10. When the photographed data D1 is received via the network 2, the processor 71 judges YES in Act 21 and proceeds to Act 22. The processor 71 acquires the camera classification from the photographed data D1 in Act 22. Then, the processor 71 checks whether the camera classification indicates the store entry camera CA2 in Act 23. If the camera classification indicates the store entry camera CA2, the processor 71 judges YES in Act 23 and proceeds to Act 24. The processor 71 executes store entry camera processing in Act 24.

[0061] 11 is a flow chart specifically showing the procedure of the store entrance camera process. As described above, the store entrance camera CA2 captures the face image of the customer who enters the store. Therefore, the face image included in the photographed data D1 generated from the image captured by the store entrance camera CA2 is the face image of the customer who enters the store.

[0062] When the store entrance camera process starts, the processor 71 acquires a face image from the photographed data D1 in Act 31. Then, the processor 71 searches the customer database 92 using the face image as search data in Act 32. The processor 71 sequentially compares the face image as the search data with face images in customer records 92R in which the transaction flag is not set to "1" in Act 33. The processor 71 then searches for a customer record 92R that stores a face image with a similarity that can be recognized as the same person as the person in the face image as the search data. If the processor 71 detects a corresponding customer record 92R, it determines YES in Act 33 and ends the store entrance camera process.

[0063] If the corresponding purchaser record 92R cannot be found, the processor 71 determines NO in Act 33 and proceeds to Act 34. The processor 71 generates a new purchaser ID in Act 34. The processor 71 also determines attributes of the purchaser, such as gender and age, from the face image included in the photographed data D1 in Act 35. The processor 71 also obtains the photographed time of the photographed data D1 as the store entry time in Act 36. The processor 71 also generates location information from the photographed time of the photographed data D1 and the in-store coordinates in Act 37.

[0064] Then, the processor 71 creates a customer record 92R from the customer ID, customer attributes, store entry time, and location information obtained in the processes of Acts 34 to 37, and the facial image included in the photographed data D1 (Act 38). In this customer record, the cart number is not described. Also, the accounting flag is reset to "0". The processor 71 saves the customer record 92R in the customer database 92 (Act 39).

[0065] In this way, when a customer enters the store through the store entrance, the customer's facial image is captured by the store entrance camera CA2. When the customer's facial image is captured by the store entrance camera CA2, a customer record 92R is generated with a customer ID unique to that customer set and stored in the customer database 92. However, if a customer for whom a customer record 92R has been created leaves the store and re-enters, the determination is YES in the processing of Act 33 in Fig. 11, so there is no risk of duplicate customer records being created for the same customer.

[0066] Returning to the explanation of Figure 10. When the processor 71 finishes the store entrance camera process, it ends the process for the photographed data D1 received in Act 21.

[0067] If the camera section acquired from the photographed data D1 does not indicate the store entry camera CA2, the processor 71 judges NO in Act 23 and proceeds to Act 25. The processor 71 checks whether the camera section indicates the cart camera CA1 in Act 25. If the camera section indicates the cart camera CA1, the processor 71 judges YES in Act 25 and proceeds to Act 26. The processor 71 executes cart camera processing in Act 26.

[0068] 12 is a flow chart specifically showing the procedure of the cart camera process. As described above, the cart camera CA1 captures a facial image of a customer operating the cart SC. Therefore, the facial image included in the photographed data D1 generated from the image captured by the cart camera CA1 is the facial image of the customer operating the cart SC to make a purchase.

[0069] When the cart camera process starts, the processor 71 acquires a face image from the photographed data D1 in Act 41. Then, the processor 71 searches the customer database 92 using the face image as search data in Act 42. The processor 71 sequentially compares the face image as the search data with face images of customer records 92R whose accounting flags are not set to "1" in Act 43. The processor 71 searches for a customer record 92R that stores a face image with a similarity that can be recognized as the same person as the person in the face image as the search data. If the processor 71 detects a corresponding customer record 92R, it determines YES in Act 43 and proceeds to Act 44. The processor 71 reads the corresponding customer record 92R from the customer database 92 as the customer record 92R to be updated in Act 44.

[0070] The processor 71 checks whether or not a cart number is set in the purchaser record 92R in Act 45. If a cart number is set, the processor 71 determines YES in Act 45 and proceeds to Act 46. The processor 71 checks whether or not the cart number set in the purchaser record 92R matches the cart number included in the photographed data D1 in Act 46. If the two cart numbers match, the processor 71 determines YES in Act 46 and ends the cart camera process.

[0071] If the cart number is not set in the purchaser record 92R to be updated, the processor 71 judges NO in Act 45 and proceeds to Act 47. Also, if the cart number set in the purchaser record 92R does not match the cart number included in the photographed data D1, the processor 71 judges NO in Act 46 and proceeds to Act 47.

[0072] The processor 71 acquires the cart number included in the photographed data D1 in Act 47. Then, the processor 71 writes the cart number acquired in Act 47 in the customer record 92R to be updated in Act 48, and then overwrites and saves the customer record 92R in the customer database 92. The processor 71 then ends the cart camera process.

[0073] In this way, when a purchaser operates the cart SC, the purchaser's facial image is captured by the cart camera CA1. When the purchaser's facial image is captured by the cart camera CA1, the cart number of the cart SC operated by the purchaser is written in the purchaser record 92R in which the purchaser's unique purchaser ID is set.

[0074] If the cart number of the cart SC operated by the purchaser has already been written in the purchaser record 92R, the process in Act 46 of Fig. 12 will be judged as YES, and the processes in Act 47 and Act 48 will not be executed. However, if the purchaser starts to operate another cart SC, the process in Act 46 of Fig. 12 will be judged as NO, and the processes in Act 47 and Act 48 will be executed. Therefore, the cart number of the latest cart SC being operated by the purchaser identified by the purchaser ID of the purchaser record 92R is written in the purchaser record 92R.

[0075] Meanwhile, if the processor 71 cannot find the customer record 92R that stores a face image with a similarity that can be recognized as the same person as the face image as the search data in Act 43, the processor 71 proceeds to the process of Act 34 of the store entrance camera process shown in Fig. 11. The processor 71 then executes the processes of Act 34 to Act 39 in the same manner as described above. At this time, the processor 71 generates the location information from only the shooting time in Act 37, and also writes the cart number in the customer record 92R in Act 38.

[0076] Therefore, for a purchaser who is not photographed by the store entrance camera CA2, a purchaser record 92R with a purchaser ID unique to that purchaser is generated by operating the cart SC and is stored in the purchaser database 92.

[0077] Returning to the explanation of Figure 10. When the processor 71 finishes the cart camera process, it ends the process for the image capture data D1 received in Act 21.

[0078] If the camera classification acquired from the photographing data D1 does not indicate the store entrance camera CA2 or the cart camera CA1, the processor 71 judges NO in Act 25 and proceeds to Act 27. The processor 71 checks whether the camera classification indicates the base camera CA3 in Act 27. If the camera classification indicates the base camera CA3, the processor 71 judges YES in Act 27 and proceeds to Act 28. The processor 71 executes base camera processing in Act 28.

[0079] 13 is a flow chart specifically showing the procedure of the base camera processing. As described above, the base camera CA3 is for taking a facial image of a customer who comes to the base of the sales floor. Therefore, the facial image included in the photographed data D1 generated from the image taken by the base camera CA3 is the facial image of the customer who comes to the base of the sales floor.

[0080] When the base camera process starts, the processor 71 acquires a face image from the photographed data D1 in Act 51. Then, the processor 71 searches the customer database 92 using the face image as search data in Act 52. The processor 71 sequentially compares the face image as the search data with face images of customer records 92R whose accounting flags are not set to "1" in Act 53. The processor 71 searches for a customer record 92R that stores a face image with a similarity that can be recognized as the same person as the person in the face image as the search data. If the processor 71 detects a corresponding customer record 92R, it determines YES in Act 53 and proceeds to Act 54. The processor 71 reads the corresponding customer record 92R from the customer database 92 as the customer record 92R to be updated in Act 54.

[0081] The processor 71 generates location information from the shooting time and in-store coordinates included in the shooting data D1 in Act 55. Then, the processor 71 writes the location information generated in Act 55 in the customer record 92R to be updated in Act 56, and then overwrites and saves the customer record 92R in the customer database 92. At this time, if location information has already been written in the customer record 92R, the location information generated in Act 55 is written in place of the previously written location information.

[0082] The processor 71 checks whether or not a cart number is described in the customer record 92R to be updated in Act 57. If the cart number is not described, the processor 71 determines NO in Act 57 and ends the base camera process.

[0083] If a cart number is described in the customer record 92R to be updated, the processor 71 determines YES in Act 57 and proceeds to Act 58. The processor 71 controls the communication interface 75 to send a provision command to the provision server 80 in Act 58. By this control, the communication interface 75 sends the provision command to the provision server 80 via the network 2. The provision command includes the location information, customer attributes, and cart number data described in the customer record 92R to be updated. With this, the processor 71 ends the base camera processing.

[0084] When the customer moves to a base location in this manner, a facial image of the customer is captured by the base camera CA3 installed at the base. When the facial image of the customer is captured by the base camera CA3, location information associating the in-store coordinates for the base camera CA3 with the current date and time is written in a customer record 92R in which a customer ID unique to that customer is set. Then, a delivery command including the location information, the customer attributes and cart number written in the customer record 92R, and the like, is sent to the delivery server 80.

[0085] Meanwhile, if the processor 71 cannot detect the customer record 92R that stores a face image with a similarity that can be recognized as the same person as the face image as the search data in Act 53, the processor 71 proceeds to the process of Act 34 of the store entrance camera process shown in Fig. 11. Then, the processor 71 executes the processes of Act 34 to Act 39 in the same manner as described above.

[0086] Therefore, for a purchaser who is not photographed by the store entrance camera CA2 or the cart camera CA1, when the purchaser moves to a location within the store, a purchaser record 92R with a purchaser ID unique to that purchaser is generated and stored in the purchaser database 92.

[0087] Returning to the explanation of Figure 10. When the processor 71 finishes the base camera process, it ends the process for the captured image data D1 received in Act 21.

[0088] If the camera classification acquired from the photographed data D1 does not indicate the store entrance camera CA2, the cart camera CA1, or the base camera CA3, the processor 71 determines NO in Act 27 and proceeds to Act 29. The processor 71 executes accounting camera processing in Act 29.

[0089] 14 is a flow chart specifically illustrating the procedure of the accounting camera process. As described above, accounting camera CA4 is used to capture facial images of a customer performing a transaction at accounting machine 30. Therefore, the facial image included in captured data D1 generated from an image captured by accounting camera CA4 is the facial image of the customer performing a transaction at accounting machine 30.

[0090] When the accounting camera process begins, the processor 71 acquires a facial image from the captured data D1 in Act 61. Then, the processor 71 searches the customer database 92 using the facial image as search data in Act 62. The processor 71 sequentially compares the facial image as the search data with facial images of customer records 92R whose accounting flags are not set to "1" in Act 63. The processor 71 then searches for a customer record 92R that stores a facial image with a similarity that allows the person in the facial image as the search data to be recognized as the same person. If the processor 71 finds a corresponding customer record 92R, it determines YES in Act 63 and proceeds to Act 64. The processor 71 reads the corresponding customer record 92R from the customer database 92 as the customer record to be updated in Act 64.

[0091] In Act 65, the processor 71 sets the accounting flag of the customer record 92R to "1." Then, in Act 66, the processor 71 overwrites and saves the customer record 92R to be updated in the customer database 92. With this, the processor 71 ends the accounting camera process.

[0092] When a customer is photographed by the accounting camera CA4 in this way, the accounting flag F of the customer record 92R in which that customer's unique customer ID is set is updated from "0" to "1." Therefore, from then on, that customer record 92R is no longer searched for in the cart camera processing, base camera processing, and accounting camera processing described above.

[0093] If a customer record 92R that stores a face image with a similarity sufficient to identify the person as the same person as the face image in the search data cannot be found in Act 63, the processor 71 ends the checkout camera process.

[0094] Returning to the explanation of Figure 10. When the processor 71 finishes the accounting camera process, it ends the processing for the captured data D1 received in Act 21.

[0095] Next, the operation of the provision server 80 will be described with reference to FIG. When the processor 81 of the provision server 80 receives a provision command via the communication interface 85, it starts information processing according to the procedure shown in the flow chart of Fig. 15. That is, the processor 81 acquires the in-store coordinates of the location information included in the provision command in Act 71. The processor 81 also acquires the purchaser attributes included in the provision command in Act 72. The processor 81 further acquires the cart number included in the provision command in Act 73.

[0096] The processor 81 searches the provided information database 93 using the in-store coordinates and the customer attributes acquired from the provision command as search data in Act 74. The processor 81 determines whether or not there is a provided information record 93R that includes the in-store coordinates as the search data as base coordinates and matches the customer attributes in Act 75. If the corresponding provided information record 93R is not stored in the provided information database 93, the processor 81 determines NO in Act 75 and ends this process.

[0097] If the corresponding provision information record 93R is stored in the provision information database 93, the processor 81 determines YES in Act 75 and proceeds to Act 76. The processor 81 reads display data from the corresponding provision information record 93R in Act 76. Then, the processor 81 controls the communication interface 85 to transmit the display data to the information terminal 10 identified by the cart number acquired in the processing of Act 73 in Act 77. By this control, the communication interface 85 transmits communication data including the cart number and the display data. This communication data is transmitted to the access point 20 via the network 2 and wirelessly transmitted from the access point 20. With the above, the processor 81 ends the processing when receiving the provision command.

[0098] The communication data wirelessly transmitted from the access point 20 is received by the information terminal 10. The processor 11 of the information terminal 10 that receives the communication data via the wireless unit 14 checks whether the cart number included in the communication data matches the cart number stored in the auxiliary storage device 13. If they do not match, the processor 11 discards the communication data. If the cart numbers match, the processor 11 causes the touch panel 15 to display the display data included in the communication data.

[0099] In this way, when a purchaser moves to a base location, the presence or absence of information targeted at the attributes of the purchaser is determined among the information to be provided to the purchaser set for that base. If there is information targeted at the attributes of the purchaser, that information is displayed on the touch panel 15 of the information terminal 10 provided on the cart SC operated by the purchaser.

[0100] For example, when a customer comes before a shelf displaying special sale items, an image is displayed introducing the special sale items that are estimated to stimulate purchasing desire for the customer of that gender or age. For example, when a customer comes before a shelf displaying ingredients for a recommended recipe, an image is displayed introducing the recipe that is estimated to stimulate appetite for the customer of that gender or age.

[0101] Conversely, even if a customer comes before a shelf displaying sale items, if there are no sale items that are estimated to stimulate the customer's appetite for that customer's gender or age, no images will be displayed.Similarly, even if a customer comes before a shelf displaying ingredients for a recommended recipe, if there are no recipes that are estimated to stimulate the customer's appetite for that customer's gender or age, no images will be displayed.

[0102] Here, the purchaser server 70 functions as a person identification means that identifies a purchaser whose face is shown in an image captured by the store entrance camera CA2, cart camera CA1, or base camera CA3, by a computer mainly consisting of a processor 71 executing the processing of Act 34 to Act 39 of Figure 11.

[0103] In addition, the consumer server 70 functions as an attribute determining means for determining attributes such as the gender and age of the consumer by a computer mainly including the processor 71 executing the process of Act 35 of FIG.

[0104] In addition, the purchaser server 70 functions as an association means for associating the facial image of the purchaser operating the cart SC with the information terminal 10 provided on the cart SC from the image captured by the cart camera CA1, by a computer mainly consisting of the processor 71 executing the processing of Act 41 to Act 48 of Figure 12.

[0105] On the other hand, the provision server 80 functions as a location specifying means for specifying a location from information captured by the local camera CA3 capturing the face of the purchaser by a computer mainly including a processor 81 executing the process of Act 71 in FIG.

[0106] In addition, the provision server 80 functions as a terminal identification means for identifying the information terminal 10 used by the purchaser by a computer mainly including a processor 81 executing the process of Act 73 in Fig. 15. Specifically, the provision server 80 identifies the information terminal 10 associated with the face image of the purchaser identified by the person identification means.

[0107] The provision server 80 constitutes a provision means for providing information according to a location to the information terminal 10 by a computer mainly including the processor 81 executing the processes of Act 74 to Act 77 in Fig. 15. Furthermore, the provision server 80 functions as a provision means for providing information according to the attributes and location of a purchaser by a computer mainly including the processor 81 executing the processes of Act 74 to Act 77 including the process of Act 72.

[0108] Therefore, in this embodiment, the information providing device is made up of the purchaser server 70 that manages the purchaser database 92 and the providing server 80 that manages the provided information database 93. The information providing device may also be configured to include the image processing server 60 that manages the camera database 91. The information providing device may also be configured with one server instead of multiple servers.

[0109] As is clear from the above description, according to this embodiment, it is possible to provide information related to the location of a purchaser via the information terminal 10 used by the purchaser, without tracking the purchaser's behavior. Therefore, since there is no need to track the purchaser's behavior, it is possible to eliminate the use of radio positioning technology such as GPS and beacons.

[0110] Furthermore, according to this embodiment, it is possible to selectively provide information relating to the purchaser's attributes such as gender, age, etc., thereby further increasing the purchaser's desire to purchase and providing the purchaser with accurate information that satisfies the purchaser.

[0111] In addition, since the information is provided from the information terminal 10 installed in the cart SC used by the purchaser, the purchaser does not need to operate a terminal such as a smartphone to obtain the information. Therefore, it is possible to provide appropriate information without burdening the purchaser.

[0112] [Second embodiment] In the first embodiment, a case is exemplified in which a purchaser is considered as one person, and when the purchaser moves to a base, information related to the base is provided to the purchaser. However, purchasers also include purchasers who come to the store in groups, such as with children, couples, friends, etc. Therefore, a second embodiment is described with reference to FIG. 16 in which, for purchasers who come to the store in a group, when someone in the group moves to a base, information related to the base is displayed on the information terminal 10 of the cart SC used by the group.

[0113] Fig. 16 is a flow chart showing the main steps of information processing executed by the processor 81 of the provision server 80 upon receiving a provision command via the communication interface 85. In the first embodiment, this step is shown by the flow chart of Fig. 15. Therefore, steps common to the first embodiment are given the same reference numerals as in Fig. 15, and detailed explanations are omitted. Furthermore, the contents explained in the first embodiment using Figs. 1 to 14 are also common to the second embodiment. Therefore, Figs. 1 to 14 are used as they are, and explanations thereof are omitted.

[0114] When the processor 81 of the provision server 80 receives the provision command via the communication interface 85, the processor 81 starts information processing according to the procedure shown in the flowchart of Fig. 16. That is, in Act 71 to Act 73, the processor 81 acquires the in-store coordinates of the location information, the purchaser attributes, and the cart number from the provision command.

[0115] Next, the processor 81 controls the communication interface 85 to output a cart number inquiry command to the consumer server 70 in Act 81. By this control, the communication interface 85 transmits the cart number inquiry command to the consumer server 70 via the network 2. This inquiry command includes the cart number obtained from the provision command.

[0116] The processor 71 of the consumer server 70 that receives the query command searches the consumer database 92 using the cart number included in the query command as search data. The processor 71 then extracts all consumer records 92R that contain the cart number, which is the search data, from among the consumer records 92R whose accounting flags are not set to "1." The processor 71 controls the communication interface 75 to send the consumer record 92R extracted from the consumer database 92 to the provision server 80. Through this control, the communication interface 75 transmits the consumer record 92R extracted from the consumer database 92 to the provision server 80 via the network 2.

[0117] As described in the first embodiment, when a customer's face image is acquired from an image captured by the cart camera CA1, the cart number of the cart SC equipped with the cart camera CA1 is written in the customer record 92R in which the customer's customer ID is set. When a customer is a group, the members of the group usually share one cart SC. For this reason, there is a high possibility that multiple members of the group will be photographed by the cart camera CA1. When multiple members are photographed by one cart camera CA1, the same cart number will be written in the customer record 92R of each member. As a result, multiple customer records 92R will be sent from the customer server 70 in response to a cart number inquiry from the provision server 80.

[0118] Therefore, the processor 81 of the provision server 80 makes a cart number inquiry in Act 81, and then checks whether multiple purchaser records 92R have been received in Act 82. If multiple purchaser records 92R have been received, the processor 81 determines YES in Act 82 and proceeds to Act 83. The processor 81 collects purchaser attributes from the multiple purchaser records 92R in Act 83. Then, the processor 81 estimates group attributes from the multiple purchaser attributes in Act 84.

[0119] For example, if one of the two buyer attributes is male and the other is female and they are close in age, they are presumed to be a married couple or a couple. For example, if two buyer attributes are the same sex and close in age, they are presumed to be friends. For example, if one of the three buyer attributes is male and the other is female and they are close in age, and the remaining buyer attribute is a child, they are presumed to be a family.

[0120] After estimating the group attribute, the processor 81 searches the provided information database 93 using the in-store coordinates acquired from the provision command and the group attribute acquired in the processing of Act 84 as search data in Act 85. The provided information database 93 contains a provided information record 93R in which a group attribute is described instead of a purchaser attribute. The processor 81 determines whether or not there is a provided information record 93R that contains the in-store coordinates as search data as base coordinates and has a matching group attribute in Act 86. If the corresponding provided information record 93R is not stored in the provided information database 93, the processor 81 determines NO in Act 86 and ends this processing.

[0121] If the corresponding provision information record 93R is stored in the provision information database 93, the processor 81 determines YES in Act 86 and proceeds to Act 87. The processor 81 reads display data from the corresponding provision information record 93R in Act 87. Then, the processor 81 controls the communication interface 85 to transmit the display data to the information terminal 10 identified by the cart number acquired in the process of Act 73 in Act 88. By this control, the communication interface 85 transmits communication data including the cart number and the display data. This communication data is transmitted to the access point 20 via the network 2 and wirelessly transmitted from the access point 20.

[0122] If only one purchaser record 92R is received as a result of the inquiry about the cart number, the processor 81 determines NO in Act 82 and proceeds to Act 85. Then, the processor 81 executes the same processes as those in Act 74 to Act 77 described in the first embodiment in Act 85 to Act 88. This is the end of the process performed by the processor 81 when a provision command is received.

[0123] Here, the provision server 80 functions as an estimation means for estimating a group composition from the attributes of multiple purchasers when the facial images of multiple purchasers are associated with the same information terminal 10 by the association means, by a computer mainly consisting of a processor 81 executing the processing of Act 81 to Act 84 of Figure 15.

[0124] According to the second embodiment configured as described above, if the purchasers are a group, when someone in the group moves to a base, information related to the base can be displayed on the information terminal 10 of the cart SC used by the group. In that case, for example, if the group is a family, information suitable for families can be displayed. For example, if the group is a couple, information suitable for couples can be displayed.

[0125] Incidentally, the accounting flag of the purchaser record 92R is not set to "1" unless the purchaser identified in that purchaser record 92R is photographed by the accounting camera CA4. On the other hand, if a purchaser is a group, one person may complete the transaction while the other members do not approach the transaction machine 30. In such a case, the accounting flag of the purchaser record 92R of the member who did not approach the transaction machine 30 will not be set to "1" even if the transaction is completed.

[0126] Therefore, for example, if a certain amount of time has passed since the time of entering the store and the customer is presumed to have already made a payment, the payment flag is automatically set to "1" for the customer record 92R. By adopting such a configuration, it is possible to prevent the customer record 92R of a customer who has already made a payment from being subject to a search.

[0127] In this regard, a camera equivalent to accounting camera CA4 can be installed at the entrance of the store. Then, the accounting flag in customer record 92R for a customer whose face image is captured by this entrance camera can be set to "1." This is effective because when a customer leaves the store, the accounting flag in customer record 92R for that customer is set to "1."

[0128] The above describes an embodiment of an information providing device that can provide information related to a person's location via a terminal used by the person without tracking the person's actions, but such an embodiment is not limited to this.

[0129] In the above embodiment, the terminal used by the purchaser is the information terminal 10 provided in the cart SC. The terminal is not limited to the information terminal 10. For example, identification information of a communication terminal such as a smartphone used by the purchaser is registered in advance together with a facial image of the purchaser. The provision server 80 may then obtain the identification information registered for the facial image captured by the base camera CA3, and provide information according to the base to the communication terminal identified by the identification information. In this case, the cart camera CA1 and the store entrance camera CA2 may be omitted.

[0130] For example, in the second embodiment, the function of the estimation means is assumed to be possessed by the provision server 80. The estimation means may be possessed by the purchaser server 70. In this case, when the purchaser server 70 detects multiple purchaser records 92R from the purchaser database 92, it estimates the group attributes from the purchaser attributes of each purchaser record. The purchaser server 70 then notifies the provision server 80 of the group attributes.

[0131] In the above embodiment, the image processing server 60, the purchaser server 70, and the provider server 80 have been described as having a control program stored in advance in a main memory or an auxiliary storage device. In this regard, each server may be transferred without the control program being stored therein. In this case, the control program transferred separately from this server is written in a writable storage device provided in each server in response to an operation by a user or the like. The control program can be transferred by recording it on a removable recording medium or by communication via a network. The recording medium may be in any form, such as a CD-ROM or memory card, as long as it can store the program and is readable by the device.

[0132] Although several other embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope of the invention and the scope of the invention and its equivalents described in the claims. The invention as originally claimed in the present application is set forth below. [1] An information providing device comprising: a person identification means for identifying a person whose face appears in an image captured by a photographing means; a location identification means for identifying a location from information from the photographing means that captured the person's face; a terminal identification means for identifying a terminal used by the person; and a providing means for providing information corresponding to the location to the terminal. [2] The information providing device according to appendix [1], further comprising an attribute determination means for determining attributes of the person, wherein the providing means provides information according to the attributes of the person and the location. [3] The information providing device described in appendix [1] or [2], wherein the photographing means includes a mobile body photographing means for photographing a person operating a mobile body equipped with a terminal, and further includes an association means for associating a facial image of the person operating the mobile body with the terminal equipped on the mobile body from the image photographed by the mobile body photographing means, and the terminal identification means identifies the terminal associated with the facial image of the person identified by the person identification means. [4] The photographing means includes a mobile object photographing means for photographing a person operating a mobile object equipped with a terminal, and further includes an association means for associating a facial image of the person operating the mobile object with the terminal equipped on the mobile object from the image photographed by the mobile object photographing means, an attribute determination means for determining attributes of the person, and an estimation means for estimating a group composition from the attributes of the multiple people when the association means associates facial images of multiple people with the same terminal, and the providing means provides information corresponding to the group composition and the location. [5] The information providing device according to claim [4], wherein the terminal identification means identifies a terminal associated with a facial image of the person identified by the person identification means. [6] A control program for causing a computer of an information providing device to function as a person identification means for identifying a person whose face is captured in an image taken by a photographing means, a location identification means for identifying a location from information from the photographing means that captured the person's face, a terminal identification means for identifying a terminal used by the person, and a providing means for providing information corresponding to the location to the terminal. [Explanation of symbols]

[0133] 1...store system, 2...network, 10...information terminal, 20...access point, 30...accounting machine, 40...camera interface, 50...store server, 60...image processing server, 70...purchaser server, 80...providing server, 91...camera database, 92...purchaser database, 93...provided information database, CA1...cart camera, CA2...entry camera, CA3...base camera, CA4...accounting camera, SC...shopping cart.

Claims

1. A first storage means for storing, for each of a plurality of base cameras installed at a plurality of bases within a store where information to be provided to a purchaser is available, coordinate information that identifies a position within the store that is photographed by the base camera; a person identification means for identifying a person whose face is shown in an image captured by the base camera; a coordinate information acquisition means for acquiring the coordinate information stored in the first storage means for the base camera that captured the image in which a person is identified by the person identification means; a terminal identification means for identifying a terminal used by the person identified by the person identification means; a providing means for providing information to be provided to the purchaser, the information being associated with the coordinate information acquired by the coordinate information acquiring means, to the terminal identified by the terminal identifying means; An information providing device comprising:

2. A first face image acquisition means for acquiring a face image of a person from an image captured by an entrance camera that captures a person entering a store; Further comprising:

2. The information providing device according to claim 1, wherein the person identification means identifies a person whose face is shown in an image captured by the base camera when a facial image of the person is acquired by the first facial image acquisition means.

3. A second face image acquisition means for acquiring a face image of a person operating a mobile body equipped with the terminal from an image captured by a mobile body camera that captures the person; an association means for associating the face image acquired by the second face image acquisition means with the terminal provided on a mobile object operated by the person photographed by the mobile object camera; Further comprising:

3. The information providing device according to claim 1, wherein the terminal identification means identifies a terminal associated with a face image of the person identified by the person identification means.

4. A second storage means for storing, for each of the mobile cameras, information identifying the terminal provided on the mobile body operated by a person photographed by the mobile camera; Further comprising:

4. The information providing device according to claim 3, wherein the terminal identification means identifies the terminal based on information identifying the terminal stored in the second storage means for the mobile camera that photographed the person identified by the person identification means.

5. An information providing device as described in claim 4, wherein the first memory means and the second memory means are a collection of camera records created for each camera, the camera record of the base camera includes a camera classification identifying it as a base camera as well as coordinate information identifying the location within the store photographed by the base camera, and the camera record of the mobile camera includes a camera classification identifying it as a mobile camera as well as information identifying the terminal provided on a mobile body operated by a person photographed by the mobile camera.

6. The computer of the information providing device, a first storage means for storing, for each of a plurality of base cameras installed at a plurality of bases within a store where information to be provided to a purchaser is available, coordinate information for identifying a position within the store photographed by the base camera; a person identification means for identifying a person whose face is shown in an image captured by the base camera; a coordinate information acquisition means for acquiring the coordinate information stored in the first storage means for the base camera that captured the image in which a person is identified by the person identification means; A terminal identification means for identifying a terminal used by the person identified by the person identification means; and a providing means for providing information to be provided to the purchaser, the information being associated with the coordinate information acquired by the coordinate information acquiring means, to the terminal identified by the terminal identifying means; A control program that functions as a

Images