Store support device, system, method, and program

Abstract

A shop assistance device (1) comprises: an acquisition unit (11) that acquires an image obtained by photographing a first visitor by a camera installed in a commercial facility; a first determination unit (12) that refers to a first database that registers in advance biometric information for a member of a shop in the commercial facility to determine whether first biometric authentication for first biometric information extracted from the acquired image has succeeded or failed; an extraction unit (13) that extracts object information relating to a worn object from the region of the first visitor if the first biometric authentication is determined to have failed; a second determination unit (14) that refers to a second database associating a product with a shop selling the product to determine, on the basis of the object information, whether the object has a predetermined or higher level of relevance to the product in terms of luxuriousness, and determines that the first visitor is a candidate visitor to be served if the object is determined to have a predetermined or higher level of relevance; and a notification unit (15) that issues notification information based on the object information to a first terminal of a related person in the commercial facility if the first visitor is determined to be a candidate visitor to be served.

Description

Technical Field

[0001] The present invention relates to a store support device, system, method, and program, and more particularly to a store support device, system, method, and program for supporting a store by notifying the store of information about visitors to the store.

Background Art

[0002] In a store, so-called CRM (Customer Relationship Management) may be used to appropriately serve each customer. CRM is an information system that records and analyzes customer service information including purchase history for customers identified by name, membership number, etc.

[0003] Further, Patent Document 1 discloses a technique for analyzing a photographed image of a visitor to generate possession-related information and outputting customer identification information when it is determined that it is necessary to provide a service suitable for the customer based on the possession-related information.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Here, the above-described CRM provides services and the like to customers registered in advance in the customer database. However, it can also be said to be beneficial for the store to find visitors who are candidates for customer service other than registered customers and serve them appropriately.

[0006] An object of the present disclosure is to provide a store support device, system, method, and program for detecting candidates for customer service from among visitors and supporting the acquisition of new customers in a physical store in view of the above-described problems.

Means for Solving the Problem

[0007] The store support device according to the first aspect of the present disclosure includes: an acquisition means for acquiring an image of a first visitor photographed by a camera installed in a commercial facility; a first determination means for determining whether a first biometric authentication for first biometric information extracted from the area of the first visitor in the acquired image is successful or failed by referring to a first database in which biometric information is registered in advance as a member of a store in the commercial facility; an extraction means for extracting accessory information regarding accessories from the area of the first visitor when it is determined that the first biometric authentication has failed; a second determination means for determining whether there is a relevance of a predetermined level or higher between the accessory and the product based on the accessory information by referring to a second database associating the product with the store handling the product, and determining the first visitor as a candidate visitor for customer service when it is determined that there is such relevance; a notification means for notifying notification information based on the accessory information to a first terminal of a person concerned in the commercial facility when the first visitor is determined to be a candidate visitor for customer service; and is provided with.

[0008] The store support system according to the second aspect of the present disclosure includes: a camera installed in a commercial facility; a first terminal of a person concerned in the commercial facility; a first database in which biometric information is registered in advance as a member of a store in the commercial facility; a second database associating the product with the store handling the product; a store support device connected to the camera, the first terminal, the first database, and the second database; and is provided with. The store support device includes: an acquisition means for acquiring an image of a first visitor photographed by the camera; First determination means for determining whether the first biometric authentication for the first biometric information extracted from the area of the first visitor in the acquired image is successful or failed with reference to the first database; When it is determined that the first biometric authentication has failed, extraction means for extracting accessory information regarding accessories from the area of the first visitor; With reference to the second database, based on the accessory information, it is determined whether there is a high degree of relevance of a predetermined level or more between the accessory and the product, and when it is determined that there is such a relevance, the first visitor is determined as a customer candidate visitor Second determination means; When the first visitor is determined as a customer candidate visitor, notification means for notifying the first terminal of notification information based on the accessory information; It is provided with.

[0009] The store support method according to the third aspect of the present disclosure is A computer Acquires an image of a first visitor photographed by a camera installed in a commercial facility, With reference to a first database in which biometric information is registered in advance as a member of a store in the commercial facility, it is determined whether the first biometric authentication for the first biometric information extracted from the area of the first visitor in the acquired image is successful or failed. When it is determined that the first biometric authentication has failed, accessory information regarding accessories is extracted from the area of the first visitor, With reference to a second database associating a product with a store handling the product, based on the accessory information, it is determined whether there is a high degree of relevance of a predetermined level or more between the accessory and the product, and when it is determined that there is such a relevance, the first visitor is determined as a customer candidate visitor. When the first visitor is determined as a customer candidate visitor, notification information based on the accessory information is notified to the first terminal of a person concerned in the commercial facility.

[0010] The store support program stored in the non-temporary computer-readable medium according to the fourth aspect of the present disclosure is An acquisition process for acquiring an image of a first visitor captured by a camera installed in a commercial facility, A first determination process for determining whether a first biometric authentication for first biometric information extracted from the area of the first visitor in the acquired image is successful or failed by referring to a first database in which biometric information is registered in advance as a member of a store in the commercial facility, If it is determined that the first biometric authentication has failed, an extraction process for extracting accessory information regarding accessories from the area of the first visitor, By referring to a second database associating a product with a store handling the product, based on the accessory information, it is determined whether there is a relevance of a predetermined level or higher between the accessory and the product, and if it is determined that there is such a relevance, a second determination process for determining the first visitor as a candidate visitor for customer service, If the first visitor is determined to be a candidate visitor for customer service, a notification process for notifying notification information based on the accessory information to a first terminal of a person concerned in the commercial facility, To be executed by a computer.

Effect of the Invention

[0011] According to the present disclosure, it is possible to provide a store support device, a system, a method, and a program for detecting candidates for customer service from among visitors and supporting the acquisition of new customers in a physical store.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and redundant descriptions are omitted as necessary for clarity of explanation.

[0014] <Embodiment 1> FIG. 1 is a block diagram showing the configuration of the store support device 1 according to Embodiment 1. The store support device 1 is an information processing device for supporting sales promotion and the like in a predetermined store in a commercial facility including one or more stores. The store support device 1 is connected to a first database (not shown) and a second database (not shown).

[0015] The first database is a database system in which biometric information is registered in advance as a member of a store in a commercial facility. Here, the biometric information is feature information indicating physical characteristics extracted from an image obtained by photographing at least a part of a person's body. The biometric information is, for example, but not limited to, face feature information. For example, the first database may be a first storage device that stores the identification information of a member in association with the biometric information. Further, the first storage device may be built in the store support device 1 or may be in a database server connected to the store support device 1 via a communication network.

[0016] The second database is a database system that associates products with the stores that handle those products. Also, the stores include at least the stores within the above commercial facility. Therefore, the stores registered in the second database are at least operating within the above commercial facility, and may include stores that do not necessarily register the biometric information of members in the first database. Also, for example, the second database may be a second storage device that stores the association between products and stores. Also, the second storage device may be built into the store support device 1 or may be within a database server connected to the store support device 1 via a communication network.

[0017] The store support device 1 includes an acquisition unit 11, a first determination unit 12, an extraction unit 13, a second determination unit 14, and a notification unit 15. The acquisition unit 11 acquires an image of a first visitor captured by a camera (not shown) installed within the commercial facility. The first determination unit 12 refers to the first database and determines whether the first biometric authentication for the first biometric information extracted from the area of the first visitor in the acquired image is successful or failed. Note that the biometric information of members of the first store within the commercial facility is pre-registered in the first database. Therefore, when the first biometric authentication is successful, it indicates that the first visitor is a member of the first store, and when the first biometric authentication fails, it indicates that the first visitor is a non-member (unregistered person) of the first store. Therefore, it can also be said that the first determination unit 12 determines whether the first visitor is a member of a predetermined store by the first biometric authentication. Note that the first biometric information is biometric information extracted from the body area of the first visitor in the acquired image by image recognition or the like.

[0018] When it is determined that the first biometric authentication has failed, that is, when it is determined that the first visitor is not a member, the extraction unit 13 extracts accessory information regarding the accessories from the area of the first visitor in the image. For example, the extraction unit 13 estimates the accessories worn by the first visitor by image recognition or the like. Here, the accessory is an item worn by the first visitor and may also be called a possession. Accessories include, for example, a bag, ornaments (accessories, rings, necklaces, earrings, etc.), a wristwatch, glasses, sunglasses, shoes, clothes (coat, jacket, trousers, skirt, etc.), etc., but are not limited thereto. And the accessory information is information indicating the image of the area recognized as an accessory in the image, the type of the accessory, the brand name, the product name, the color, the pattern, the logo, the material, the shape, etc.

[0019] The second determination unit 14 refers to the second database and determines whether there is a high-degree relevance of a predetermined level or higher between the worn item information and the product. If it is determined that there is a relevance, the first visitor is determined as a candidate visitor for customer service. Here, the relevance between the worn item and the product may be, for example, the commonality of the attributes of the worn item and the product or a certain degree of similarity in product characteristics. For example, having a relevance between the worn item and the product may mean that the worn item of the first visitor to the store is the same or similar product or product type as the product handled in the store within the commercial facility. Or, having a relevance between the worn item and the product may mean that the worn item and the product are of the same or similar brand even if the product types are different. And having a high-degree relevance of a predetermined level or higher between the worn item and the product means that the worn item and the product are common or similar in that they are high-class products or brands of a certain level or higher. Here, the high-degree is an index value corresponding to the brand, price, etc. of the product, for example, a numerical value, a class or rank (such as a symbol), a level value, or information combining these. For example, having a high-degree relevance of a predetermined level or higher between the worn item and the product may mean that the worn item and the product are high-class clothes of a certain level or higher. Therefore, when there is a certain relevance between the worn item indicated by the worn item information and the product registered in the second database, it can be said that the first visitor is highly likely to be wearing a worn item that has a certain commonality or similarity in attributes with the product handled in the store within the commercial facility. Thus, it can be said that the first visitor is highly likely to be interested in the product relevant to the worn item. Also, at this point, the first visitor is determined not to be a member of the store within the commercial facility. Therefore, in these cases, the second determination unit 14 can determine that the first visitor is a candidate visitor for customer service.

[0020] When the notification unit 15 determines that the first visitor is a customer candidate visitor, it notifies the first terminal of a related person within the commercial facility of the notification information based on the wearable item information. Here, the related persons within the commercial facility are, for example, the store clerks of the stores within the commercial facility, or the staff of the entire commercial facility, etc. And the staff of the entire commercial facility refers to persons other than the store clerks of specific stores, such as the staff at the service counters of the commercial facility or the concierge of the entire commercial facility. And the first terminal is an information terminal worn or carried by a related person within the commercial facility, or a computer device connected to a screen visible to the related person.

[0021] Figure 2 is a flowchart showing the flow of the store support method according to Embodiment 1. First, the acquisition unit 11 acquires an image of the first visitor captured by a camera installed within the commercial facility (S11). Next, the first determination unit 12 refers to the first database and performs a first biometric authentication on the first biometric information extracted from the area of the first visitor in the image acquired in step S11 (S12). For example, the first determination unit 12 specifies the face area of the first visitor from the acquired image by image recognition or the like, and extracts face feature information as the first biometric information from the specified face area. Alternatively, the first determination unit 12 acquires the first biometric information extracted from the image by another configuration (not shown). And the first determination unit 12 collates the one or more biometric information registered in advance in the first database with the first biometric information, and calculates the degree of coincidence to perform the first biometric authentication.

[0022] Then, the first determination unit 12 determines whether the first biometric authentication is successful or failed (S13). Specifically, the first determination unit 12 collates the first biometric information with the biometric information registered in the first database to calculate a degree of match, and determines whether the degree of match is equal to or greater than a predetermined value. When it is determined that the degree of match is equal to or greater than the predetermined value, the first determination unit 12 determines that the first biometric authentication is successful. In that case, the processing of the store support method according to the first embodiment is terminated. On the other hand, when it is determined that the degree of match is less than the predetermined value, the first determination unit 12 determines that the first biometric authentication has failed. In other words, it can be said that the first determination unit 12 determines whether the first visitor is a member based on the authentication result of the first biometric authentication. When the authentication result indicates success, the first determination unit 12 determines that the first visitor is a member. On the other hand, when the authentication result indicates failure, the first determination unit 12 determines that the first visitor is a non-member, that is, an unregistered person in the first database.

[0023] Alternatively, the first determination unit 12 transmits an authentication request including the first biometric information to the first database, and receives an authentication result by the authentication process performed in the first database. In that case, the first determination unit 12 may determine whether the first biometric authentication is successful or failed, and whether the first visitor is a member based on the received authentication result.

[0024] When the first biometric authentication fails (NO in step S13), that is, when it is determined that the first visitor is not a member, the extraction unit 13 extracts accessory information regarding accessories from the area of the first visitor in the image (S14). Then, the second determination unit 14 refers to the second database and determines the presence or absence of relevance between the accessories and the products based on the accessory information (S15). When there is relevance between the accessories and the products, it is determined that the first visitor is a candidate visitor for customer service. On the other hand, when there is no relevance between the accessories and the products, it is determined that the first visitor is not a candidate visitor for customer service.

[0025] When it is determined in step S15 that there is a relationship between the worn item and the product, the notification unit 15 notifies the first terminal of the relevant person in the commercial facility of the notification information based on the worn item information (S16). Then, the process of the store support method according to the first embodiment ends. Note that when it is determined in step S15 that there is no relationship between the worn item and the product, the process of the store support method according to the first embodiment ends.

[0026] As described above, the store support device 1 according to the present embodiment can determine from the first biometric information extracted by analyzing the captured image by the camera installed in the commercial facility including one or more stores that the first visitor is a non-member of the store in the commercial facility by referring to the first database. Then, the store support device 1 extracts the worn item information of the first visitor who is a non-member from the captured image, and by referring to the second database, when there is a certain relationship between the worn item indicated by the worn item information and the product, it can be determined that the first visitor is a customer candidate visitor. Here, the certain relationship may be a high-class degree equal to or higher than a predetermined level as described above. And when the first visitor is a non-member of the store in the commercial facility and a customer candidate visitor, the store support device 1 notifies the terminal of the relevant person in the commercial facility to that effect. Therefore, due to the notification based on the worn item information of the customer candidate visitor, it becomes easier for the relevant person such as a store clerk to visually recognize that the first visitor has entered the store, and it becomes easier to provide appropriate customer service to the first visitor. Therefore, it is possible to appropriately detect candidates for customer service from among the visitors and support the acquisition of new customers in the actual store.

[0027] Note that the store support device 1 has a configuration (not shown) including a processor, a memory, and a storage device. Further, the storage device stores a computer program in which the process of the store support method according to the present embodiment is implemented. Then, the processor causes the memory to read the computer program and the like from the storage device and executes the computer program. Thereby, the processor realizes the functions of the acquisition unit 11, the first determination unit 12, the extraction unit 13, the second determination unit 14, and the notification unit 15.

[0028] Alternatively, each component of the store support device 1 may be implemented by dedicated hardware. Also, some or all of the components of each device may be implemented by general-purpose or dedicated circuitry, a processor, etc., or a combination thereof. These may be constituted by a single chip or by a plurality of chips connected via a bus. Some or all of the components of each device may be implemented by a combination of the circuitry etc. described above and a program. Also, as the processor, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), a quantum processor (quantum computer control chip), etc. can be used.

[0029] <Embodiment 2> This Embodiment 2 is a specific example of the above-described Embodiment 1. FIG. 3 is a block diagram showing the overall configuration of the store support system 1000 according to this Embodiment 2. The store support system 1000 is an information system for supporting each store and the commercial facility 100 in the commercial facility 100 including stores 110, ··· 1n0 (n is a natural number of 1 or more). Specifically, the store support system 1000 photographs a first visitor C1 with a camera 101 installed near the entrance and exit of the commercial facility 100, and determines whether the first visitor C1 is a customer candidate visitor to any of the stores such as the store 110 based on the analysis result of the photographed image. When the first visitor C1 is determined to be a customer candidate visitor, the store support system 1000 notifies the terminal of the store clerk handling the brand of the clothing information of the first visitor C1 of the physical characteristics and clothing information of the visitor.

[0030] The store support system 1000 includes a camera 101, a store employee terminal 111, ··· 1n1, a store support device 200, and a facility management server 300. Each of the camera 101, the store employee terminal 111, ··· 1n1, the store support device 200, and the facility management server 300 is communicably connected via a network N. Here, the network N is a wired or wireless communication line or communication network, such as an in-store LAN (Local Area Network), the Internet, a wireless communication line network, a mobile phone line network, etc. The network N is not limited by the type of communication protocol.

[0031] The camera 101 is installed near the entrance and exit of the entire commercial facility 100. The camera 101 captures a predetermined shooting range near the entrance and exit at a predetermined interval or when a person is detected by a motion sensor. The camera 101 transmits the captured image to the store support device 200 via the network N. For example, when the first customer C1 visits the commercial facility 100 and is near the entrance and exit, the first customer C1 is detected by the motion sensor. Therefore, the camera 101 captures an image including the first customer C1. Note that a plurality of cameras may be installed in the commercial facility 100. Also, the camera 101 may be installed at a location other than the entrance and exit of the entire commercial facility 100.

[0032] Stores 110~1n0 are tenant stores that occupy and operate in the commercial facility 100. For example, at least store employee S1 is in store 110. The store employee S1 provides customer service to customers who enter store 110. The store employee S1 wears the store employee terminal 111 for business use. The store employee terminal 111 is an example of a wearable terminal, such as a headset compatible with wireless communication. The store employee terminal 111 communicates with the store support device 200 and the facility management server 300 via the network N by means of a built-in computer and a wireless communication function. For example, the store employee terminal 111 receives notification information of a customer from the store support device 200, etc. via the network N, plays back a voice corresponding to the notification information, and outputs it from a speaker.

[0033] In addition, there is at least salesperson Sn in store 1n0. The salesperson Sn provides customer service to the customers who enter store 1n0. The salesperson Sn carries a salesperson terminal 1n1 for business use. The salesperson terminal 1n1 is an example of a portable terminal, such as a smartphone or a tablet terminal (corresponding to wireless communication). The salesperson terminal 1n1 communicates with the store support device 200 and the facility management server 300 via the network N through its built-in computer and wireless communication function. For example, the salesperson terminal 1n1 receives the notification information of the customer from the store support device 200 etc. via the network N, performs the incoming call notification operation according to the notification information (such as vibration operation or pop-up display of the application), and displays a message etc. according to the notification information on the screen.

[0034] Incidentally, the salespersons S1 to Sn are an example of the related persons in the above-mentioned commercial facility. Also, the salesperson terminals 111 to 1n1 are an example of the above-mentioned first terminal. Therefore, the salesperson terminal may be a computer and a display device installed in the store. Also, the related persons in the commercial facility 100 may be the staff at a place other than store 110 etc., for example, the information center of the commercial facility 100. In that case, the first terminal may be a computer and a display device installed in the information center.

[0035] The facility management server 300 is a computer server that manages various types of information related to the commercial facility 100. The facility management server 300 includes at least a member database (DB) 310 and a product DB 320. The facility management server 300 accepts access to the member DB 310 and the product DB 320 via the network N from the store support device 200 or the like, performs processing corresponding to the access on the member DB 310 or the like, and returns the processing result to the store support device 200 or the like via the network N. For example, when the facility management server 300 receives a face authentication request including face information from the store support device 200, it compares the received face information with the face information 312 in the member DB 310 and returns the comparison result (face authentication result) to the store support device 200. Here, the face authentication result includes information indicating success or failure of face authentication. When face authentication indicates success, the face authentication result includes member information that has succeeded in face authentication (information for uniquely identifying a user (member), such as at least a member ID). Further, when the facility management server 300 receives accessory information from the store support device 200, it compares the product information included in the accessory information with the product information 321 in the product DB 320. When the degree of coincidence between the product information 321 and the received product information is equal to or higher than a threshold value, the facility management server 300 returns a set of the product information 321 and the store information 322 to the store support device 200. Alternatively, when the degree of coincidence between the product information 321 including a high-class degree 3211 of a predetermined level or higher and the received product information is equal to or higher than a threshold value, the facility management server 300 may return a set of the product information 321 and the store information 322 to the store support device 200. Note that other configurations of the facility management server 300 can be realized by known techniques.

[0036] The member DB 310 is an example of the first database described above. The member DB 310 is a database that manages the member information 311, the face information 312, and the store information 313 in association with each other. The member information 311 is information for managing members registered as at least one customer of the store 110 or the like. Note that the member information 311 may be member information for the entire commercial facility 100. The member information 311 includes a member ID, personal information such as name, gender, date of birth, and a purchase history.

[0037] The face information 312 is an example of the above-described biometric information and is the face feature information of the member. For example, the face information 312 is a set of the positions of feature points and feature amounts extracted from the face image of the member. The store information 313 is information that can identify the store where the member is registered, and for example, is the identification information of the store, the store ID, etc. The store information 313 typically corresponds to any one of the stores 110 in the commercial facility 100, etc.

[0038] Note that the face information 312 does not necessarily have to be managed by the facility management server 300. For example, assume that there is an authentication device that manages the face information and the user ID in association with each other outside the facility management server 300, and the authentication device is connected to the network N. In that case, it is sufficient that the member ID in the member information 311 and the above user ID uniquely correspond to each other in the member DB 310. Also, assume that the member information 311 and the face information 312 are registered in the member DB 310 but have been pre-approved by the member (customer).

[0039] The product DB 320 is an example of the above-described second database. The product DB 320 is a database that manages the product information 321 and the store information 322 in association with each other. The product information 321 includes a product ID, a product name, a product type, a brand name, a price, a product image, a color, a pattern, a logo, a material, a shape, etc. The product information 321 further includes a high-class degree 3211. The high-class degree 3211 is an example of an index value corresponding to the brand, price, etc. of the product corresponding to the product information 321. The high-class degree 3211 is, for example, a numerical value, a class or a rank (symbol, etc.), a level value, or information combining these. Assume that the product DB 320 manages at least the product information 321 whose high-class degree 3211 is equal to or higher than a predetermined level.

[0040] The store information 322 is information that identifies any one of the stores 110 etc. within the commercial facility 100 that handles the products corresponding to the product information 321. That is, the store information 322 is information indicating the store that sells the product corresponding to the product information 321. Therefore, one piece of store information 322 may be associated with two or more pieces of product information 321. Also, two or more pieces of store information 322 may be associated with the same piece of product information 321. Further, the store information 322 is information corresponding to the above-described store information 313. The store information 322 includes, for example, a store ID, a store name, the location of the store, etc. The store information 322 further includes store employee terminal information 3221. The store employee terminal information 3221 is information regarding a first terminal that can be worn, carried, or visually recognized by an employee who is on duty at the store corresponding to the store information 322, and includes at least destination information that can be accessed via the network N.

[0041] The store support device 200 is an example of the above-described store support device 1. The store support device 200 is an information processing device connected via the network N to the camera 101, the store employee terminals 111 to 1n1, the member DB 310, and the product DB 320. When a first visitor C1 who has come to the commercial facility 100 is determined to be a candidate visitor for customer service, the store support device 200 notifies the store employee terminals 111 etc. such as the store 110 of the physical characteristics and the clothing information of the first visitor C1 regarding the clothing information of the first visitor C1. The store support device 200 may be redundant across a plurality of servers, or each functional block may be implemented on a plurality of computers.

[0042] FIG. 4 is a block diagram showing the configuration of the store support device 200 according to the second embodiment. The store support device 200 includes a storage unit 210, a memory 220, a communication unit 230, and a control unit 240. The storage unit 210 is an example of a storage device such as a hard disk or a flash memory. The storage unit 210 stores a program 211. The program 211 is a computer program in which the store support processing etc. according to the second embodiment is implemented.

[0043] The memory 220 is a volatile storage device such as a RAM (Random Access Memory), and is a storage area for temporarily holding information during the operation of the control unit 240. The communication unit 230 is a communication interface with the network N.

[0044] The control unit 240 is a processor, that is, a control device that controls each component of the store support device 200. The control unit 240 causes the memory 220 to read the program 211 from the storage unit 210 and execute the program 211. Thereby, the control unit 240 realizes the functions of the acquisition unit 241, the extraction unit 242, the determination unit 243, the specification unit 244, the generation unit 245, and the notification unit 246.

[0045] The acquisition unit 241 is an example of the acquisition unit 11 described above. The acquisition unit 241 acquires an image captured by the camera 101 of the first visitor C1.

[0046] The extraction unit 242 is an example of the extraction unit 13 described above. The extraction unit 242 extracts the face information of the first visitor C1 from the image acquired by the acquisition unit 241. Further, when the determination unit 243 determines that the first visitor C1 is a non-member, the extraction unit 242 extracts accessory information from the area of the first visitor C1 in the image. Specifically, the extraction unit 242 detects the areas of various accessories by image recognition for the area of the first visitor C1, analyzes the detected areas of each accessory, and extracts the type, brand name, product name, color, pattern, logo, material, shape, etc. of the accessory as accessory information. Note that a known image recognition technique may be used for the logic of extracting accessory information from the area in the image. Further, the extraction unit 242 may refer to a predetermined database or the product DB 320 to extract accessory information.

[0047] Further, the extraction unit 242 may further extract feature information regarding the body or appearance of the first visitor C1 from the area of the first visitor C1. Specifically, the extraction unit 242 extracts the feature information of the body or appearance of the first visitor C1 by performing image recognition on the area of the first visitor C1 in the image acquired by the acquisition unit 241. The feature information of the body or appearance is, for example, but not limited to, height, hair length, hairstyle, presence or absence of wearing glasses, shape of clothes, color of clothes, etc. Note that a known image recognition technique may be used for the logic of extracting the feature information of the body or appearance from the area in the image.

[0048] The determination unit 243 is an example of the above-described first determination unit 12 and second determination unit 14. The determination unit 243 refers to the member DB 310 and performs a first face authentication on the face information extracted by the extraction unit 242. For example, the determination unit 243 transmits a face authentication request including the face information to the facility management server 300 and receives a face authentication result from the facility management server 300. Then, the determination unit 243 determines whether or not the first visitor C1 has succeeded in the first face authentication according to the face authentication result. When the face authentication result indicates that the first face authentication has succeeded, the determination unit 243 determines that the first visitor C1 is a member. On the other hand, when the face authentication result indicates that the first face authentication has failed, the determination unit 243 determines that the first visitor C1 is a non-member.

[0049] Further, the determination unit 243 refers to the merchandise DB 320 and determines whether or not the first visitor C1 is a customer candidate visitor based on the accessory information extracted by the extraction unit 242. Specifically, the determination unit 243 determines the presence or absence of a certain relevance between the information indicated by the accessory information and the merchandise information 321 in the merchandise DB 320. Having a certain relevance may mean, for example, that the information indicated by the accessory information is included in the merchandise information 321 in the merchandise DB 320. When there is a certain relevance, for example, when the accessory information is included in the merchandise information 321, the determination unit 243 determines that the first visitor C1 is a customer candidate visitor.

[0050] When it is determined that the first visitor C1 has successfully passed the first face authentication, the specifying unit 244 refers to the member DB 310 and specifies member information 311 based on the face authentication result. Then, the specifying unit 244 specifies the store information 313 associated with the specified member information 311 as the notification destination. Then, the specifying unit 244 specifies the store clerk terminal information 3221 corresponding to the specified store information 313. Further, the specifying unit 244 specifies the face image that has successfully passed the face authentication based on the face authentication result. For example, the specifying unit 244 specifies the face area of the first visitor C1 in the image as the face image.

[0051] When it is determined that the first visitor C1 is a customer candidate visitor, the specifying unit 244 refers to the product DB 320 and specifies the store in the commercial facility 100 that is the notification destination based on the wearable product information. That is, when the specifying unit 244 determines that there is a certain relevance between the information indicated by the wearable product information and the product information 321 in the product DB 320, the specifying unit 244 specifies the store that handles the product determined to have the relevance. Specifically, the specifying unit 244 specifies the product information 321 corresponding to the wearable product information, and specifies the store information 322 associated with the specified product information 321 as the notification destination. Further, the specifying unit 244 specifies the store clerk terminal information 3221 included in the specified store information 322.

[0052] When it is determined that the first visitor C1 has successfully passed the first face authentication and is a member, the generation unit 245 generates notification information including the member information 311 specified by the specifying unit 244 and the specified face image. Further, when it is determined that the first visitor C1 is a customer candidate visitor, the generation unit 245 generates notification information including the wearable product information and the feature information of the body or appearance.

[0053] The notification unit 246 is an example of the above-described notification unit 15. When it is determined that the first visitor C1 has successfully passed the first face authentication and is a member, the notification unit 246 notifies the store clerk terminal of the notification information including the member information and the face image, with the specified store clerk terminal information as the destination.

[0054] Further, when the notification unit 246 determines that the first visitor C1 is a candidate customer for customer service, the notification unit 246 notifies the store staff terminal of the notification information based on the wearable item information, with the specified store staff terminal information as the destination. In particular, the notification unit 246 notifies the notification information using the terminal of the staff of the specified store as the first terminal. Further, the notification unit 246 includes the feature information of the body or appearance in the notification information and notifies the first terminal.

[0055] FIG. 5 is a flowchart showing the flow of the store support process according to the second embodiment. As a premise, it is assumed that the first visitor C1 enters the entrance of the commercial facility 100, and the camera 101 captures the shooting range including the first visitor C1 near the entrance. Then, the camera 101 transmits the captured image to the store support device 200 via the network N.

[0056] Accordingly, the store support device 200 acquires an image (including the first visitor C1) captured by the camera 101 from the camera 101 via the network N (S201). Next, the store support device 200 extracts the face information of the first visitor C1 from the acquired image (S202). Subsequently, the store support device 200 refers to the member DB 310 and performs face authentication on the face information (S203). Then, the store support device 200 determines whether the face authentication is successful or failed (S204). In other words, the store support device 200 determines whether the first visitor C1 is a member according to the face authentication result.

[0057] If the face authentication is successful in step S204 and it is determined that the first visitor C1 is a member, the store support device 200 identifies the member information, the store, and the staff terminal (S205). Specifically, the identification unit 244 of the store support device 200 identifies the member information 311 included in the face authentication result. Then, the identification unit 244 identifies, from the member DB 310, the store information 313 associated with the identified member information 311 as the notification destination. Then, the identification unit 244 identifies, from the product DB 320, the staff terminal information 3221 corresponding to the identified store information 313 (included in the store information 322). In addition, the identification unit 244 identifies the face area of the first visitor C1 in the image acquired in step S201 as a face image. That is, in this case, since the first visitor C1 has succeeded in face authentication, the first visitor C1 is a member of the identified store. Therefore, the identification unit 244 identifies the face image of the member who is the first visitor C1.

[0058] Then, the store support device 200 generates notification information including the identified member information and the identified face image (S206). Thereafter, the store support device 200 notifies the staff terminal of the generated notification information with the identified staff terminal information 3221 as the destination (S212). For example, when the staff terminal information 3221 indicates the staff terminal 111, the staff terminal 111 receives the notification information from the store support device 200, plays back the voice corresponding to the notification information, and outputs it from the speaker. As a result, the staff member S1 can recognize that the first visitor C1, who is a member of the store 110, has visited the commercial facility 100. Therefore, when the first visitor C1 enters the store 110, the staff member S1 can promptly provide appropriate customer service.

[0059] On the other hand, when face authentication fails in step S204 and the first visitor C1 is determined to be a non-member, the store support device 200 extracts accessory information from the area of the first visitor C1 in the image acquired in step S201 (S207). Then, the store support device 200 determines whether the degree of luxury of the accessories is equal to or higher than a predetermined level from the extracted accessory information (S208). That is, the store support device 200 determines whether there is a certain relevance between the information indicated by the extracted accessory information and the product information 321 in the product DB 320. Specifically, the determination unit 243 of the store support device 200 collates the accessory information with the product information 321 in the product DB 320 and calculates a degree of match. Then, the determination unit 243 determines whether the degree of match is equal to or higher than a predetermined value. If the degree of match is equal to or higher than the predetermined value, the determination unit 243 determines whether the degree of luxury 3211 included in the corresponding product information 321 is equal to or higher than the predetermined level. When it is determined in step S208 that the degree of luxury of the accessories is less than the predetermined level, the store support process ends.

[0060] On the other hand, when it is determined in step S208 that the degree of luxury of the accessories is equal to or higher than the predetermined level, that is, when it is determined that the first visitor C1 is a customer candidate visitor, the store support device 200 executes steps S209 and S210.

[0061] In that case, first, the store support device 200 identifies the store handling the brand of the accessories and the store clerk terminal (S209). For example, the identification unit 244 of the store support device 200 identifies the brand name of the product information 321 corresponding to the accessory information. Then, the identification unit 244 identifies the store information 322 associated with the product information 321 corresponding to the identified brand name as the notification destination. Then, the identification unit 244 identifies the store clerk terminal information 3221 included in the identified store information 322.

[0062] Also, independently of step S209, the store support device 200 extracts body / appearance feature information from the area of the first visitor C1 in the image acquired in step S201 (S210). Then, the store support device 200 generates notification information including the accessory information extracted in step S207 and the body or appearance feature information extracted in step S210 (S211).

[0063] After steps S209 and S211, the store support device 200 notifies the store staff terminal of the generated notification information, with the identified store staff terminal information 3221 as the destination (S212). For example, when the store staff terminal information 3221 indicates the store staff terminal 111, the store staff terminal 111 receives the notification information from the store support device 200, plays back the voice corresponding to the notification information, and outputs it from the speaker. For example, the store staff terminal 111 outputs voice messages such as the type of the worn item, the product name, and the fact that the customer candidate visitor has arrived at the store from the speaker. Or, when the store staff terminal information 3221 indicates the store staff terminal 1n1, the store staff terminal 1n1 receives the notification information from the store support device 200, performs an incoming call notification operation corresponding to the notification information, and displays a message or the like corresponding to the notification information on the screen. For example, the store staff terminal 1n1 may display the worn item information and the physical or appearance feature information of the first visitor C1 included in the notification information on the screen in the form of graphics or text. Thus, the store staff S1, etc. can recognize that the first visitor C1, who is a customer candidate visitor to the store 110, etc. where they work, has come to the commercial facility 100. Therefore, when the first visitor C1 enters their store 110, etc., the store staff S1, etc. can promptly provide appropriate customer service. In particular, since the first visitor C1 is wearing relatively expensive worn items, it can be said that there is a high probability that the purchasing power of the first visitor C1 is high. Therefore, the possibility that the store staff recommends high-value goods to the first visitor C1 for purchase increases. In other words, this embodiment can contribute to the improvement of sales promotion by the store staff as compared with the case where there is no prior information about the first visitor C1. That is, the store can easily realize customer service that effectively utilizes human resources. Also, the store staff can more easily recommend membership registration of their own store to the customer candidate visitors. Therefore, according to this embodiment, the probability that the first visitor C1 registers as a new customer of the store can be improved.

[0064] Furthermore, in this embodiment, since the extraction process of the worn item information is performed when the visitor is a non-member, the load and processing time of the image analysis process can be reduced when the visitor is a member.

[0065] <Embodiment 3> Embodiment 3 is an improved example of Embodiment 2 described above. Similar to Embodiment 2 described above, a high level is associated with each product in the second database according to Embodiment 3. Then, the specifying means according to Embodiment 3 refers to the second database and specifies, within a commercial facility, a store handling products determined to be relevant and other products whose high levels are within a certain range as the notification destination. Thereby, even when there is no store handling the same product or the same brand as the customer's wearing item in the commercial facility, if a customer candidate visitor is detected, the relevant store can grasp the visitor in advance. Or, even if there is a store handling the same product or the same brand as the customer's wearing item in the commercial facility, notification information can be similarly notified to stores handling similar brands separately. Therefore, this embodiment can contribute to improving sales promotion by store clerks even in stores handling similar brands.

[0066] Note that since the overall configuration of the store support system according to this embodiment is the same as that in FIG. 3 described above, duplicate explanations will be omitted as appropriate. Therefore, the following will mainly describe the differences from Embodiment 2 described above.

[0067] FIG. 6 is a flowchart showing the flow of the store support process according to Embodiment 3. Steps S201 to S203, S204, and S205, S206, and S212 when determined to be a member are the same processes as those in FIG. 5 described above. Also, steps S207, S208, S210, and S211 when determined to be a member in step S204 are the same processes as those in FIG. 5 described above. When it is determined in step S208 that the level of the wearing item is equal to or higher than a predetermined level, that is, when it is determined that the first visitor C1 is a customer candidate visitor, the store support device 200 executes S209a independently of step S210.

[0068] In that case, the store support device 200 identifies the store and the staff terminal that handle other products whose level of luxury of the worn item is within a certain range (S209a). For example, the identification unit 244 of the store support device 200 identifies the luxury level 3211 of the product information 321 corresponding to the worn item information. Then, the identification unit 244 identifies other product information from the product DB 320 whose identified luxury level and level are within a certain range. Then, the identification unit 244 identifies the store information 322 associated with the identified other product information from the product DB 320 as the notification destination. Then, the identification unit 244 identifies the staff terminal information 3221 included in the identified store information 322.

[0069] After steps S209a and S211, the store support device 200 notifies the staff terminal of the generated notification information with the identified staff terminal information 3221 as the destination (S212). As a result, the notification information is notified to the staff terminal that handles products or brands with the same level of luxury as the worn item of the first visitor C1. Therefore, the staff member wearing or carrying the staff terminal to which the notification information is notified can grasp in advance that the customer candidate has visited the store and the characteristic information of the worn item, body, or appearance. Thus, the same effects as those of the above-described Embodiment 2 can be achieved.

[0070] <Embodiment 4> Embodiment 4 is an improved example of Embodiment 2 or 3 described above. The first database according to Embodiment 4 includes information indicating whether a member is an excellent member. And the image captured by the camera includes a second visitor together with the first visitor. And the first determination means determines whether the second visitor is an excellent member when the second biometric authentication for the second biometric information extracted from the area of the second visitor in the image is successful based on the first database. And when the first visitor is determined to be a customer candidate visitor and the second visitor is determined to be an excellent member, the notification means includes an image of the face area of the second visitor in the notification information and notifies the first terminal. Thereby, when a registered excellent member (second visitor) comes to the store together with a customer candidate visitor (first visitor), it is possible to notify in advance the store where the excellent member is registered that the excellent member has come to the store and the accompanying person is the customer candidate visitor. In particular, the store clerk can grasp the face image of the excellent member and the characteristic information of the body or appearance and the information of the accessories of the customer candidate visitor via the screen of the store clerk terminal. Therefore, it can further contribute to the improvement of sales promotion by the store clerk. The probability that the first visitor C1 registers as a new customer of the store can be further improved.

[0071] FIG. 7 is a block diagram showing the overall configuration of the store support system 1000a according to Embodiment 4. The store support system 1000a shows a case where a second visitor C2 comes to the commercial facility 100 together with the first visitor C1 as compared with the above-described store support system 1000. For example, the first visitor C1 and the second visitor C2 may be a couple, parent and child, siblings, partner, friends, etc. Also, the store support system 1000a is different from the above-described store support system 1000 in that the store support device 200a and the member DB 310a are changed. In addition, since other configurations of the store support system according to the present embodiment are the same as those in FIG. 3 described above, duplicate explanations will be omitted as appropriate. Therefore, the following description will focus on the differences from Embodiment 2 described above.

[0072] The member DB 310a has the member type 314 associated with the member information 311, the face information 312, and the store information 313, as compared with the above-described member DB 310. The member type 314 is an example of information indicating whether a member is a premium member or not, and is information such as a premium member or a regular member.

[0073] The store support device 200a is an improvement of the above-described store support device 200. The determination unit 243 of the store support device 200a refers to the member DB 310a and determines whether the second visitor C2 is a premium member or not by performing a second face authentication on the face information extracted from the area of the second visitor C2 in the image. Further, when the first visitor C1 is determined to be a customer candidate visitor, the determination unit 243 may determine whether the second visitor C2 is a premium member or not by performing a second face authentication. That is, the order of the first face authentication and the second face authentication does not matter. Then, when it is determined that the second visitor C2 is a premium member, the notification unit 246 of the store support device 200a includes an image of the face area of the second visitor C2 in the notification information and notifies the store clerk terminal.

[0074] FIG. 8 is a flowchart showing the flow of the store support process according to the fourth embodiment. First, the first visitor C1 and the second visitor C2 are present near the entrance and exit of the commercial facility 100, and the first visitor C1 and the second visitor C2 are photographed by the camera 101. Then, the camera 101 transmits an image including the first visitor C1 and the second visitor C2 to the store support device 200a.

[0075] In response to this, it is determined in steps S201 to S203 and S204 of FIG. 5 described above that the first visitor C1 is a non-member, step S207 is executed, and it is determined in S208 that the degree of luxury of the accessories worn by the first visitor C1 is equal to or higher than a predetermined level. That is, it is determined that the first visitor C1 is a customer candidate visitor.

[0076] Thereafter, the store support device 200a determines whether a companion of the first visitor C1 is included in the image acquired in step S201 (S221). Specifically, the store support device 200a determines, by image recognition of the image, whether a person's area exists within a predetermined distance from the area of the first visitor C1. If it is determined in step S221 that no companion is included, steps S209 to S212 in FIG. 5 described above are executed.

[0077] However, hereinafter, as described above, a case where the second visitor C2 visits the commercial facility 100 together with the first visitor C1 will be described. Therefore, it is assumed that the area of the second visitor C2 exists within a predetermined distance from the area of the first visitor C1 in the image. Thus, the store support device 200a determines in step S221 that a companion of the first visitor C1 is included in the image.

[0078] Then, the store support device 200a extracts the face information of the second visitor C2 from the acquired image (S222). Subsequently, the store support device 200a refers to the member DB 310a and performs face authentication on the face information (S223). Then, the store support device 200a determines whether the face authentication has succeeded or failed (S224). In other words, the store support device 200a determines whether the second visitor C2 is a member according to the face authentication result. If it is determined in step S224 that the second visitor C2 is a member, the store support device 200a further determines whether the second visitor C2 who has succeeded in face authentication is an excellent member. Specifically, the store support device 200a refers to the member DB 310a and determines whether it is an excellent member from the face information 312 that has succeeded in face authentication or the member type 314 associated with the member information 311. If it is determined in step S224 that the second visitor C2 is a non-member, the store support device 200a executes steps S207 to S212 in FIG. 5 described above for the area of the first visitor C1 and the companion. Alternatively, if the second visitor C2 is determined not to be an excellent member, for example, a regular member, the store support device 200a may execute steps S207 to S212 in FIG. 5 described above for the area of the first visitor C1 and the companion.

[0079] On the other hand, when it is determined in step S224 that the second visitor C2 is a premium member, that is, a companion of the first visitor C1 who is a candidate for customer service is a premium member, the store support device 200a executes steps S226 and S210. In that case, first, the store support device 200a identifies the member information of the companion, the store, and the store staff terminal in the same manner as step S205 described above (S226). In addition, the identification unit 244 identifies the face area of the second visitor C2 in the image acquired in step S201 as a face image. Independently of step S226, the store support device 200a extracts body / appearance feature information from the area of the first visitor C1 in the image acquired in step S201 (S210).

[0080] After steps S226 and S210, the store support device 200a generates notification information including the wearing item information of the first visitor C1, the body or appearance feature information, and the member information and face image of the companion (second visitor C2) (S227). Note that the notification information does not necessarily have to include the face image of the premium member, and it is sufficient if it includes at least information that can identify the premium member.

[0081] Thereafter, the store support device 200a notifies the store staff terminal of the generated notification information with the identified store staff terminal information 3221 as the destination (S228). In response to this, the store staff who wears or carries the store staff terminal to which the notification information is notified can, in response to the notification, be aware in advance that the candidate for customer service has visited the store, and in addition to the wearing items, body or appearance feature information thereof, a premium member of their own store has visited the store. In particular, the store staff can grasp the member information such as the name, gender, and age group of the second visitor C2 who is a premium member via the store staff terminal. Furthermore, the store staff can grasp the face image of the premium member via the screen of the store staff terminal. Therefore, it becomes easier for the store staff to identify the candidate for customer service than when the first visitor C1 who is the candidate for customer service visits the store alone.

[0082] Furthermore, the determination order by face authentication of the first visitor C1 and the second visitor C2 may be either one first. Also, after the first visitor C1 is determined to be an excellent member, the store support device 200a may determine whether the second visitor C2, who is a companion, is a non-member and a customer service candidate visitor. And when the second visitor C2 is a customer service candidate visitor, the store support device 200a may execute steps S226, S210, S227, and S228. For example, in step S204, after the first visitor C1 is determined to be a member, the store support device 200a may further determine whether the first visitor C1 is an excellent member. And when the first visitor C1 is determined to be an excellent member, the store support device 200a may determine whether a companion is included in the image as in step S221. When a companion is included, it may be determined whether the companion area is a non-member and a customer service candidate visitor. Therefore, when either one of the first visitor and the second visitor included in the image is determined to be a customer service candidate visitor and the other is determined to be an excellent member, the notification unit 246 may include an image of the face area of the excellent member in the notification information and notify the first terminal.

[0083] This embodiment may be further improved as follows. That is, when it is determined that the second visitor is an excellent member, the store support device may notify the second terminal of the second visitor of the privilege information in the store for either the first visitor or the second visitor. In this case, for example, assume that the member information 311 in the member DB 310a includes the mobile terminal information of the excellent member. Then, in S226 of FIG. 8 described above, the store support device 200a further specifies the mobile terminal information of the second visitor C2 who is an excellent member from the member DB 310a. Then, the store support device 200a notifies the privilege information in the corresponding store to the specified mobile terminal information as the destination. Here, the privilege information may be, for example, the recommended products in the store for the first visitor C1 who is a candidate for customer service and their discount coupons. Or the privilege information may be the incentive information in the store for the second visitor C2 who is an excellent member. For example, the incentive information may be a member rank up, a discount coupon, etc. for the second visitor C2 introducing the store to the first visitor C1. Thereby, it is possible to promote the induction of the candidate for customer service visitors from excellent members to the store and to encourage the promotion of member registration. Therefore, the probability of member registration can be improved compared to the case where the candidate for customer service visitors come to the store alone.

[0084] <Embodiment 5> Embodiment 5 is a modification of Embodiments 2 to 4 described above. The camera according to Embodiment 5 is installed at the entrance and exit of a store that handles products of a certain level of luxury or higher within a commercial facility.

[0085] FIG. 9 is a block diagram showing the overall configuration of the store support system 1000b according to Embodiment 5. The store support system 1000b shows the case where the cameras in the commercial facility 100b are installed near the entrances and exits of each store as compared with the store support system 1000 described above. For example, the commercial facility 100b includes stores 110b to 1n0b. The commercial facility 100b has a camera 112 installed near the entrance and exit. Similarly, hereinafter, the store 1n0b has a camera 1n2 installed near the entrance and exit. Each of the cameras 112 to 1n2 is communicably connected to the network N.

[0086] Each of cameras 112 to 1n2, similar to the above-described camera 101, captures a predetermined imaging range near the entrance / exit at a predetermined interval or when a person is detected by a human presence sensor. Each of cameras 112 to 1n2 transmits the captured image to the store support device 200 via the network N. At this time, each camera may transmit the identification information of the camera and the identification information of the store to the store support device 200 together with the image.

[0087] For example, when the first visitor C1 visits the store 110b within the commercial facility 100 and is near the entrance / exit of the store 110b, the first visitor C1 is detected by the human presence sensor. Therefore, the camera 112 captures an image including the first visitor C1. Note that a plurality of cameras may be installed in each store. Also, in this embodiment as well as in FIG. 3 described above, the cameras may be installed at the entrances / exits of the entire commercial facility 100 or at locations outside the stores.

[0088] Also, the store support system 1000b is different from the above-described store support system 1000 in that the store support device 200b is changed. Note that since the other configurations of the store support system according to this embodiment are the same as those in FIG. 3 described above, duplicate explanations are omitted as appropriate. Therefore, the following description will focus on the differences from the above-described Embodiment 2.

[0089] The store support device 200b acquires the captured image from the camera 112 etc. via the network N, and determines whether a person in the image (for example, the first customer C1) is a customer candidate who has come to the store, in the same manner as the above-described store support device 200. When the store support device 200b determines that the person is a customer candidate who has come to the store, it extracts feature information on the body and appearance from the detected area of the first customer C1 and generates notification information, in the same manner as the above-described store support device 200. And when the store support device 200b according to the present embodiment determines that the person is a customer candidate who has come to the store, it identifies the store to which the camera that is the transmission source of the image belongs and the staff terminal of the store clerk who works at the store. For example, when the store support device 200b has acquired the image and the identification information of the store 110b from the camera 112, it identifies the store 110b as the notification destination from the identification information, and identifies the staff terminal information 3221 included in the store information 322 corresponding to the store 110b in the product DB 320. Then, the store support device 200b notifies the notification information to the staff terminal 111 with the identified staff terminal information 3221 as the destination. As a result, the store clerk S1 of the store 110b can easily recognize, via the staff terminal 111, that the first customer C1 who has come to the commercial facility 100b is a customer candidate who has come to the store. Therefore, the store clerk S1 can appropriately serve the first customer C1. That is, the store support device 200b can support sales promotion etc. of the store 110b etc.

[0090] <Other Embodiments>

[0091] In addition, the above-described biometric authentication extracts feature information indicating the physical characteristics of a person from a body image, collates the pre-registered feature information with the extracted feature information, and assumes that the authentication is successful when the degree of coincidence is equal to or higher than a threshold value. The biometric authentication may be, for example, face authentication. In that case, the biometric information becomes face feature information. In addition, the biometric information may use data (feature amounts) calculated from physical characteristics unique to an individual, such as face feature information, fingerprints, voiceprints, veins, retinas, iris of the pupil, and patterns on the palm of the hand.

[0092] In the above example, when the program is loaded into a computer, it includes a set of instructions (or software code) for causing the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may also be transmitted on a transient computer-readable medium or a communication medium. By way of example and not limitation, the transient computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0093] It should be noted that the present disclosure is not limited to the above embodiments, and can be appropriately changed without departing from the spirit. Also, the present disclosure may be implemented by appropriately combining each embodiment.

[0094] Some or all of the above embodiments may be described as follows, but are not limited thereto. (Appendix 1) An acquisition means for acquiring an image of a first visitor taken by a camera installed in a commercial facility; A first determination means for determining whether a first biometric authentication for first biometric information extracted from the area of the first visitor in the acquired image is successful or failed by referring to a first database in which biometric information is registered in advance as a member of a store in the commercial facility; An extraction means for extracting accessory information regarding accessories from the area of the first visitor when it is determined that the first biometric authentication has failed; With reference to a second database associating the merchandise with the store handling the merchandise, determine whether there is a high-level relevance of a predetermined level or more between the worn item and the merchandise based on the worn item information, and if it is determined that there is such a relevance, a second determination means for determining the first visitor as a candidate visitor to be served; When the first visitor is determined as a candidate visitor to be served, a notification means for notifying notification information based on the worn item information to a first terminal of a person concerned within the commercial facility; A store support device comprising: (Appendix 2) Further comprising a specifying means for specifying a store handling the merchandise determined to have the relevance with reference to the second database The notification means notifies the notification information using a terminal of a store clerk of the store as the first terminal The store support device according to Appendix 1. (Appendix 3) In the second database, the level of high-classness is associated with each merchandise, The specifying means refers to the second database and specifies, within the commercial facility, stores handling the merchandise determined to have the relevance and other merchandise whose high-classness levels are within a certain range The store support device according to Appendix 2. (Appendix 4) The extraction means further extracts feature information regarding the body or appearance of the first visitor from the area of the first visitor, The notification means includes the feature information in the notification information and notifies the first terminal The store support device according to any one of Appendices 1 to 3. (Appendix 5) The first database includes information indicating whether the member is an excellent member, The image includes a second visitor together with the first visitor, The first determination means determines whether the second visitor is the excellent member when a second biometric authentication for the second biometric information extracted from the area of the second visitor is successful based on the first database, When the notification means determines that the first visitor is a candidate visitor for customer service and the second visitor is a premium member, the notification means includes an image of the face area of the second visitor in the notification information and notifies the first terminal thereof. The store support device according to any one of Appendices 1 to 4. (Appendix 6) When the notification means determines that the second visitor is a premium member, the notification means notifies the second terminal of the second visitor of the privilege information in the store for either the first visitor or the second visitor. The store support device according to Appendix 5. (Appendix 7) The camera is installed at the entrance and exit of the store that handles the merchandise within the commercial facility. The store support device according to any one of Appendices 1 to 6. (Appendix 8) A camera installed within a commercial facility, A first terminal of a person related to the commercial facility, A first database in which biometric information is registered in advance as a member of the store within the commercial facility, A second database that associates merchandise with the store that handles the merchandise, A store support device connected to the camera, the first terminal, the first database, and the second database, and comprising The store support device An acquisition means for acquiring an image of a first visitor photographed by the camera, A first determination means for referring to the first database and determining whether the first biometric authentication for the first biometric information extracted from the area of the first visitor in the acquired image is successful or failed, When it is determined that the first biometric authentication has failed, an extraction means for extracting accessory information regarding accessories from the area of the first visitor, Referring to the second database, based on the wearing item information, determining whether there is a relevance of a certain level or higher between the wearing item and the product, and when it is determined that there is such a relevance, determining the first visitor as a candidate visitor for customer service; a second determination means When the first visitor is determined as a candidate visitor for customer service, a notification means for notifying the first terminal of the notification information based on the wearing item information A store support system comprising (Appendix 9) The store support device Further comprising a specifying means for specifying a store that handles a product determined to have such a relevance by referring to the second database The notification means notifies the notification information using the terminal of a store clerk in the store as the first terminal The store support system according to Appendix 8 (Appendix 10) A computer Obtaining an image of a first visitor captured by a camera installed in a commercial facility, Referring to a first database in which biometric information is pre-registered as a member of a store in the commercial facility, determining whether a first biometric authentication for the first biometric information extracted from the area of the first visitor in the obtained image is successful or failed, When it is determined that the first biometric authentication has failed, extracting wearing item information regarding a wearing item from the area of the first visitor, Referring to a second database associating a product with a store handling the product, determining whether there is a relevance of a certain level or higher between the wearing item and the product based on the wearing item information, and when it is determined that there is such a relevance, determining the first visitor as a candidate visitor for customer service, When the first visitor is determined as a candidate visitor for customer service, notifying the first terminal of a related person in the commercial facility of the notification information based on the wearing item information A store support method (Appendix 11) An acquisition process of acquiring an image of a first visitor captured by a camera installed in a commercial facility, Referring to a first database in which biometric information is pre-registered as a member of a store in the commercial facility, a first determination process is performed to determine whether the first biometric authentication for the first biometric information extracted from the area of the first visitor in the acquired image is successful or failed. When it is determined that the first biometric authentication has failed, an extraction process is performed to extract accessory information regarding accessories from the area of the first visitor. Referring to a second database that associates products with the stores that handle the products, based on the accessory information, it is determined whether there is a relevance of a certain level or higher between the accessories and the products. When it is determined that there is such a relevance, a second determination process is performed to determine the first visitor as a candidate visitor for customer service. When the first visitor is determined as a candidate visitor for customer service, a notification process is performed to notify the first terminal of a person related to the commercial facility of the notification information based on the accessory information. A non-transitory computer-readable medium storing a store support program for causing a computer to execute the above.

[0095] Although the present invention has been described with reference to the embodiments (and examples) above, the present invention is not limited to the above embodiments (and examples). Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

Explanation of Reference Numerals

[0096] 1 Store support device 11 Acquisition unit 12 First determination unit 13 Extraction unit 14 Second determination unit 15 Notification unit 1000 Store support system 1000a Store support system 1000b Store support system N Network 100 Commercial facility 100b Commercial facility 101 Camera C1 First visitor C2 Second customer 110 Store 110b Store 111 Store employee terminal 112 Camera S1 Store employee 1n0 Store 1n0b Store 1n1 Store employee terminal 1n2 Camera Sn Store employee 200 Store support device 200a Store support device 200b Store support device 210 Memory unit 211 Program 220 Memory 230 Communication unit 240 Control unit 241 Acquisition unit 242 Extraction unit 243 Judgment unit 244 Identification unit 245 Generation unit 246 Notification unit 300 Facility management server 310 Member DB 310a Member DB 311 Member information 312 Facial information 313 Store information 314 Member type 320 Product DB 321 Product information 3211 Degree of luxury 322 Store information 3221 Store employee terminal information

Claims

1. An acquisition means for acquiring an image of a first visitor photographed by a camera installed in a commercial facility; A first determination means for determining whether a first biometric authentication for first biometric information extracted from the area of the first visitor in the acquired image is successful or failed by referring to a first database in which biometric information is registered in advance as a member of a store in the commercial facility; An extraction means for extracting accessory information regarding accessories from the area of the first visitor when it is determined that the first biometric authentication has failed; A second determination means for determining whether there is a relevance of a predetermined level or higher between the accessory and the product based on the accessory information by referring to a second database in which products and stores handling the products are associated, and determining the first visitor as a candidate visitor for customer service when it is determined that there is such a relevance; A notification means for notifying notification information based on the accessory information to a first terminal of a person concerned in the commercial facility when the first visitor is determined as a candidate visitor for customer service; A store support device comprising the above.

2. Further comprising a specifying means for specifying a store handling a product determined to have the relevance by referring to the second database. The notification means notifies the notification information using the terminal of a store clerk of the store as the first terminal. The store support device according to Claim 1.

3. In the second database, the level of the high-class degree is associated with each product. The specifying means specifies, by referring to the second database, stores in the commercial facility that handle other products whose high-class degree levels are within a certain range and the product determined to have the relevance. The store support device according to Claim 2.

4. The extraction means further extracts feature information regarding the body or appearance of the first visitor from the area of the first visitor. The notification means includes the feature information in the notification information and notifies the first terminal. The store support device according to any one of Claims 1 to 3.

5. The first database includes information indicating whether the member is an excellent member. The image includes a second visitor together with the first visitor. When the second biometric authentication for the second biometric information extracted from the area of the second visitor is successful based on the first database, the first determination means determines whether the second visitor is the excellent member. When the notification means determines that the first visitor is a candidate visitor for customer service and the second visitor is determined to be an excellent member, the notification means includes an image of the face area of the second visitor in the notification information and notifies the first terminal. The store support device according to any one of claims 1 to 4.

6. When the notification means determines that the second visitor is an excellent member, the notification means notifies the second terminal of the second visitor of privilege information in the store for either the first visitor or the second visitor. The store support device according to claim 5.

7. The camera is installed at the entrance and exit of the store that handles the product within the commercial facility. The store support device according to any one of claims 1 to 6.

8. A camera installed within a commercial facility, A first terminal of a person related to the commercial facility, A first database in which biometric information is registered in advance as a member of a store within the commercial facility, A second database that associates a product with the store that handles the product, A store support device connected to the camera, the first terminal, the first database, and the second database, Comprising: The store support device, An acquisition means for acquiring an image of a first visitor photographed by the camera, A first determination means for referring to the first database and determining whether the first biometric authentication for the first biometric information extracted from the area of the first visitor in the acquired image is successful or failed, When it is determined that the first biometric authentication has failed, an extraction means for extracting accessory information related to accessories from the area of the first visitor, Referring to the second database, determining whether there is a relevance of a predetermined level or higher between the accessory and the product based on the accessory information, and when it is determined that there is such a relevance, a second determination means for determining the first visitor as a candidate visitor for customer service, A notification means for notifying the first terminal of notification information based on the accessory information when the first visitor is determined to be a candidate visitor for customer service. A store support system comprising:

9. A computer, Acquires an image of a first visitor photographed by a camera installed within a commercial facility, Refers to a first database in which biometric information is registered in advance as a member of a store within the commercial facility, and determines whether the first biometric authentication for the first biometric information extracted from the area of the first visitor in the acquired image is successful or failed. When it is determined that the first biometric authentication has failed, extract accessory information regarding accessories from the area of the first visitor, refer to a second database in which products and the stores handling the products are associated, and based on the accessory information, determine whether there is a high degree of relevance of a predetermined level or more between the accessory and the product. If it is determined that there is such a relevance, determine the first visitor as a candidate visitor for customer service, When the first visitor is determined to be a candidate visitor for customer service, notify the first terminal of the relevant person in the commercial facility of the notification information based on the accessory information, Store support method.

10. An acquisition process of acquiring an image of a first visitor captured by a camera installed in a commercial facility, a first determination process of determining whether the first biometric authentication for the first biometric information extracted from the area of the first visitor in the acquired image is successful or failed by referring to a first database in which biometric information is registered in advance as a member of a store in the commercial facility, an extraction process of extracting accessory information regarding accessories from the area of the first visitor when it is determined that the first biometric authentication has failed, a second determination process of referring to a second database in which products and the stores handling the products are associated, and based on the accessory information, determining whether there is a high degree of relevance of a predetermined level or more between the accessory and the product. If it is determined that there is such a relevance, determine the first visitor as a candidate visitor for customer service, a notification process of notifying the first terminal of the relevant person in the commercial facility of the notification information based on the accessory information when the first visitor is determined to be a candidate visitor for customer service, A store support program for causing a computer to execute.

Images