Processing device, processing method, and program

Abstract

The present invention provides a processing device (10) having: a customer information storage unit (11) for storing customer information that includes the customer attribute information and face information pertaining to a first customer whose customer attribute information and face information have been registered and the face information of a second customer whose face information has been registered without registering customer attribute information; a detection unit (12) for verifying the face information against an image photographed in a store and detecting the first and second customers within the image; a behavior monitoring unit (13) for identifying the in-store behavior of the first and second customers detected within the image and registering the in-store behavior in the customer information; and a second customer registration unit (14) for, when an unregistered person whose face information has not been registered is detected within the image, generating face information on the basis of the face image of the unregistered person included in the image and registering the face information in the customer information as the face information of the second customer.

Description

[Technical field]

[0001] The present invention relates to a processing device, a processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technology that uses facial recognition technology to identify customers who visit a store, registers the detection results of the identified customer's in-store behavior linked to that customer, and performs marketing analysis, promotion support, etc. based on the registered data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2008-257488 A Summary of the Invention [Problem to be solved by the invention]

[0004] From the viewpoint of data analysis, it is preferable to collect more information. However, in the case of the technology described in Patent Document 1, the information that can be collected is limited to the information of customers whose face information has been registered in advance. In this respect, the technology described in Patent Document 1 has room for improvement.

[0005] An object of the present invention is to make it possible to collect more customer information. [Means for solving the problem]

[0006] According to the present invention, a customer information storage means for storing customer information including customer attribute information and face information of a first customer who is a customer who has registered customer attribute information and face information, and face information of a second customer who has not registered the customer attribute information and has registered the face information; a detection means for detecting the first customer and the second customer in an image captured in a store by comparing the face information with the image; A behavior monitoring means for identifying the in-store behavior of the first customer and the second customer detected in the image and registering the behavior in the customer information; a second customer registration means for generating the face information based on a face image of the unregistered person included in the image when the unregistered person whose face information is not registered is detected in the image, and registering the face information in the customer information as the face information of the second customer; A processing apparatus is provided having:

[0007] Further, according to the present invention, The computer store customer information including the customer attribute information and the face information of a first customer who is a customer who has registered the customer attribute information and face information, and the face information of a second customer who is a customer who has not registered the customer attribute information but has registered the face information; a detection step of matching an image taken in a store with the face information and detecting the first customer and the second customer in the image; a behavior monitoring step of identifying the in-store behaviors of the first customer and the second customer detected in the image and registering the identified behaviors in the customer information; a second customer registration step of generating face information based on a face image of the unregistered person included in the image when the unregistered person whose face information is not registered is detected in the image, and registering the face information in the customer information as the face information of the second customer; A processing method for performing the above is provided.

[0008] Further, according to the present invention, Computer, a customer information storage means for storing customer information including customer attribute information and face information of a first customer who is a customer who has registered customer attribute information and face information, and the face information of a second customer who is a customer who has not registered the customer attribute information but has registered the face information; a detection means for comparing an image taken in the store with the face information and detecting the first customer and the second customer within the image; A behavior monitoring means for identifying the in-store behavior of the first customer and the second customer detected in the image and registering the behavior in the customer information; a second customer registration means for generating the face information based on a face image of the unregistered person included in the image when the unregistered person whose face information is not registered is detected in the image, and registering the face information in the customer information as the face information of the second customer; A program is provided to function as a Effect of the Invention

[0009] According to the present invention, it becomes possible to collect more customer information. [Brief description of the drawings]

[0010] [Figure 1] FIG. 2 is a diagram illustrating an example of a hardware configuration of a processing device according to the present embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional block diagram of the processing apparatus according to the present embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of information processed by the processing device of the present embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of information processed by the processing device of the present embodiment. [Diagram 5] 4 is a flowchart showing an example of a processing flow of the processing device of the present embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of a functional block diagram of the processing apparatus according to the present embodiment. [Figure 7] 4 is a flowchart showing an example of a processing flow of the processing device of the present embodiment. [Figure 8] FIG. 2 is a diagram illustrating an example of a functional block diagram of the processing apparatus according to the present embodiment. [Figure 9] FIG. 2 is a diagram illustrating an example of information processed by the processing device of the present embodiment. [Figure 10] FIG. 2 is a diagram illustrating an example of a functional block diagram of the processing apparatus according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are given the same reference numerals and the description will be omitted as appropriate.

[0012] <First embodiment> "overview" The processing device of this embodiment performs the following process using an image captured in a store. First, the processing device detects a customer whose face information has been registered in advance in the image using face recognition technology. Then, the processing device identifies the in-store behavior of the detected customer and registers it in association with that customer. For example, the face information of a customer who has registered as a member is registered in advance.

[0013] Furthermore, when an unregistered person whose facial information is not registered is detected in the image, the processing device generates facial information based on the facial image of the unregistered person included in the image, and registers the generated facial information by linking it to new customer identification information. This process realizes registration of facial information of a customer whose facial information has not been registered through membership registration or the like. Thereafter, the customer is detected in the image by processing using the facial recognition technology, and his / her in-store behavior is registered.

[0014] According to the processing device of this embodiment, it is possible to collect not only information on customers whose face information has been registered in advance through membership registration or the like, but also information on customers whose face information has not been registered in advance through membership registration or the like.

[0015] "Hardware Configuration" Next, an example of the hardware configuration of the processing device will be described.

[0016] 1 is a diagram showing an example of a hardware configuration of a processing device 10. Each functional unit of the processing device 10 is realized by any combination of hardware and software, centering on a CPU (Central Processing Unit) of any computer, memory, programs loaded into the memory, a storage unit such as a hard disk for storing the programs (which can store programs that are stored beforehand when the device is shipped, as well as programs downloaded from storage media such as CDs (Compact Discs) or servers on the Internet), and a network connection interface. Those skilled in the art will understand that there are various variations in the methods and devices for implementing the above.

[0017] As shown in FIG. 1, the processing device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The processing device 10 does not have to have the peripheral circuit 4A. The processing device 10 may be composed of multiple physically and / or logically separated devices, or may be composed of a single device that is physically and logically integrated. When composed of multiple physically and / or logically separated devices, each of the multiple devices can have the above hardware configuration.

[0018] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to transmit and receive data to each other. The processor 1A is an arithmetic processing device such as a CPU or a GPU (Graphics Processing Unit). The memory 2A is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The input / output interface 3A includes an interface for acquiring information from an input device, an external device, an external server, an external sensor, etc., and an interface for outputting information to an output device, an external device, an external server, etc. Examples of the input device include a keyboard, a mouse, a microphone, etc. Examples of the output device include a display, a speaker, a printer, a mailer, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0019] "Function Configuration" Next, a description will be given of the functional configuration of the processing device 10. The processing device 10 may be a store server installed corresponding to each store, or may be a center server installed corresponding to a center that manages multiple stores.

[0020] 2 shows an example of a functional block diagram of the processing device 10. As shown in the figure, the processing device 10 has a customer information storage unit 11, a detection unit 12, a behavior monitoring unit 13, and a second customer registration unit 14.

[0021] The customer information storage unit 11 stores customer information of a first customer and a second customer.

[0022] The first customer is a customer who has registered customer attribute information and face information. For example, the customer attribute information and face information of the first customer are registered in the customer information storage unit 11 by membership registration or the like. The customer information of the first customer includes at least customer attribute information and face information.

[0023] The customer attribute information is information about a customer, and may include the name, address, telephone number, age, email address, etc. Note that the customer attribute information may include other information, or may not include some of the information exemplified here.

[0024] The face information is information generated based on a face image, and may be a facial feature amount or the face image itself.

[0025] A customer who wishes to register as a member operates a terminal device and transmits his / her customer attribute information and facial image to the processing device 10 via a dedicated application or web page. If the received customer attribute information and facial image satisfy predetermined registration conditions, the processing device 10 registers the received customer attribute information and facial information generated based on the received facial image in the customer information storage unit 11 by linking them to newly issued customer identification information.

[0026] The terminal device operated by the customer is a smartphone, a tablet terminal, a mobile phone, a smart watch, a personal computer, etc. The registration conditions are "the same combination of name, address, and telephone number has not been registered", "the size of the face in the face image is equal to or larger than a standard", "the feature amount extracted from the face image satisfies a standard", etc.

[0027] A second customer is a customer who has not registered customer attribute information but has registered face information. The face information of the second customer is registered in the customer information storage unit 11 by a second customer registration unit 14 described later. The customer information of the second customer includes at least face information, but does not include customer attribute information.

[0028] An example of customer information of a first customer is shown in FIG. 3. The customer information shown in the figure includes customer identification information, name, address, telephone number, face image, face information, and in-store behavior history. The in-store behavior history will be described later.

[0029] An example of customer information of the second customer is shown in Fig. 4. The customer information shown in the figure includes customer identification information, a face image, face information, and a store behavior history. The store behavior history will be described later.

[0030] Comparing FIG. 3 and FIG. 4, the customer information of the first customer and the customer information of the second customer differ in whether or not they include customer attribute information (such as name, address, and telephone number).

[0031] The customer information of the first customer and the customer information of the second customer are common in that they both include a face image and face information, but differ in the means of acquiring them. As described above, the face image and face information of the first customer are generated based on a face image sent by the customer himself for membership registration, etc. And, certain conditions can be imposed on the face image of the first customer, such as "the size of the face in the face image is equal to or larger than a standard" and "the feature amount extracted from the face image meets a standard."

[0032] On the other hand, the face image and face information of the second customer are generated based on an image taken in the store (details will be described later). The face image of the second customer is not subject to the conditions imposed on the face image of the first customer described above.

[0033] Returning to FIG. 2, the detection unit 12 compares an image taken in a store (hereinafter referred to as a "store image") with face information stored in the customer information storage unit 11, and detects the first customer and the second customer in the store image. The face information compared with the store image includes both face information of the first customer and face information of the second customer. The comparison is realized by using any conventional face recognition technology.

[0034] The store image is generated by one or more cameras installed in the store. The cameras may capture moving images or still images. Some of the multiple cameras may capture moving images and other cameras may capture still images. The cameras may be installed in various locations, including, but not limited to, the entrance and exit of the store, the sales floor of the store, and the area where the checkout process is performed.

[0035] The behavior monitoring unit 13 identifies the in-store behavior of the first and second customers detected in the store image, and registers the identified behavior in customer information in association with the customers.

[0036] Identification of in-store behavior is realized by using any conventional technology. For example, the behavior monitoring unit 13 may analyze an in-store image to identify the product picked up by the customer. Alternatively, the behavior monitoring unit 13 may analyze an in-store image to identify the product put in the shopping cart by the customer. Alternatively, the behavior monitoring unit 13 may identify the product purchased by the customer by identifying the correspondence between an in-store image taken near a POS (point of sales) register and information registered in the POS register based on time information or the like. Alternatively, the behavior monitoring unit 13 may analyze an in-store image to identify the customer's movement trajectory in the store. Alternatively, the behavior monitoring unit 13 may analyze an in-store image to identify the position where the customer stopped in the store. Alternatively, the behavior monitoring unit 13 may analyze an in-store image to identify the product tried by the customer in the store (e.g., tasting, viewing, test run, etc.). Alternatively, the behavior monitoring unit 13 may analyze an in-store image taken near a POS register and an in-store image taken of the place where the products are packed into a take-out bag to identify whether or not the customer brought a bag to take the purchased products home. It should be noted that the in-store actions and the means for specifying the actions are merely examples and are not limited to these.

[0037] The behavior monitoring unit 13 registers the in-store behavior of each customer by linking it to each customer. The in-store behavior history shown in Fig. 3 and Fig. 4 is information indicating the in-store behavior of each customer that is linked to each customer and registered by the behavior monitoring unit 13. As shown in the figures, the behavior performed in the store may be registered by linking it to the date and time when the behavior was detected.

[0038] Returning to FIG. 2, when an unregistered person whose facial information is not registered is detected in a store image, the second customer registration unit 14 generates facial information based on the facial image of the unregistered person included in the store image. Then, the second customer registration unit 14 links the generated facial information to the newly issued customer identification information as facial information of the second customer, and registers it in the customer information stored in the customer information storage unit 11. The process of detecting an unregistered person from a store image can be realized according to conventional technology. For example, among people detected from a store image by conventional person detection technology, a person whose matching facial information is not stored in the customer information storage unit 11 can be detected as an unregistered person.

[0039] Next, an example of the flow of processing by the processing device 10 will be described with reference to the flowchart of FIG.

[0040] First, when the processing device 10 acquires a store image generated by a camera installed in the store (S10), it compares the store image with the facial information stored in the customer information storage unit 11 and performs a process of detecting the first customer and the second customer within the store image (S11).

[0041] When the first and second customers are detected in the store image (S1 2 "Detecting the first and second customers"), the processing device 10 identifies the in-store behavior of the customer, and registers information indicating the identified in-store behavior in customer information by linking it to the customer identification information of the customer (S13).

[0042] On the other hand, if an unregistered person whose face information is not registered is detected (S1 2 In step S14, the processing device 10 performs the process of S14 based on the face image of the unregistered person included in the store image. Then, the processing device 10 registers the generated face information as face information of the second customer in the customer information stored in the customer information storage unit 11 by linking the generated face information to the newly issued customer identification information.

[0043] Thereafter, the processing device 10 identifies the in-store behavior of the second customer whose face information has been newly registered, and registers the information indicating the identified in-store behavior in the customer information in association with the customer identification information of the second customer (S15).

[0044] When multiple people are detected in the store image, the processing device 10 can execute the processes of S12 to S15 for each detected person.

[0045] "Action and effect" When a customer whose face information has been registered in advance by membership registration or the like is detected in a store image, the processing device 10 of this embodiment can identify the customer's in-store behavior and register the behavior in association with the customer. Furthermore, when an unregistered person whose face information has not been registered is detected in a store image, the processing device 10 of this embodiment can generate face information of the unregistered person based on the store image and register the generated face information in association with newly issued customer identification information. Thereafter, the processing device 10 can detect the unregistered person in the store image using the face information, identify the in-store behavior of the unregistered person, and register the behavior in association with the customer identification information of the unregistered person.

[0046] According to the processing device 10 of the present embodiment, it is possible to collect not only information on customers whose face information has been registered in advance through membership registration etc., but also information on customers whose face information has not been registered in advance through membership registration etc. As a result, it becomes possible to collect information on more customers.

[0047] <Second embodiment> The processing device 10 of this embodiment continues to store the registration information of the second customer, specifically, the face information and the information indicating the in-store behavior, even after the visit when the registration was made. In other words, a customer who has not registered face information by membership registration or the like is not registered as a second customer every time he or she visits the store, but once registered as a second customer, the registration status as a second customer is maintained.

[0048] Therefore, when the second customer visits the store for the second time or later, the second customer will be detected in the store image as a "customer whose face information has been registered in advance." Also, if an entity with multiple stores shares customer registration information among the multiple stores, and a customer who was registered as a second customer when visiting one store subsequently visits another store, the customer will be detected in the store image as a "customer whose face information has been registered in advance."

[0049] According to the processing device 10 of the present embodiment, information on the in-store behavior of a customer who has not registered his / her face information in advance by membership registration or the like during each of his / her multiple visits to a store can be collectively stored as information on one customer. This will be described in detail below.

[0050] The customer information storage unit 11 continues to store the customer information of the second customer even after the customer visits the store where the customer information is registered. Therefore, when the second customer visits the store for the second or subsequent times, the customer information of the second customer is already stored in the customer information storage unit 11. As a result, when the second customer visits the store for the second or subsequent times, the second customer is not detected as an unregistered person.

[0051] The detection unit 12 detects the second customer from the store image by comparing the store image with the face information of the second customer using the customer information of the second customer stored in the above-mentioned customer information storage unit 11. As a result, the detection unit 12 can detect the second customer who visits the store for the second or subsequent times in the store image.

[0052] The behavior monitoring unit 13 registers information indicating the in-store behavior of the second customer identified during each of the multiple visits to the store, linked to the customer identification information of the second customer. As a result, the information indicating the in-store behavior of the second customer identified during each of the multiple visits to the store is registered in the customer information linked to each other (linked to the same customer identification information).

[0053] Other configurations of the processing apparatus 10 of this embodiment are similar to those of the first embodiment.

[0054] According to the processing device 10 of this embodiment, the same action and effect as in the first embodiment is realized. Also, according to the processing device 10 of this embodiment, information on each of multiple visits by a customer who has not registered face information in advance by membership registration or the like can be collectively stored as information on one customer.

[0055] <Third embodiment> The processing device 10 of this embodiment executes a process in which, when a customer who is registered as a second customer registers customer attribute information and face information by membership registration or the like and is then registered as a first customer, the information (information indicating in-store behavior, etc.) registered when the customer was registered as a second customer is passed on to the customer as is. This will be described in detail below.

[0056] An example of a functional block diagram of the processing device 10 of this embodiment is shown in Fig. 6. As shown in the figure, the processing device 10 of this embodiment differs from the first and second embodiments in that it includes a registration application receiving unit 18, a first customer registration unit 19, a judgment unit 20, and a handover unit 21.

[0057] The registration application receiving unit 18 acquires registration application information including customer attribute information and a facial image. For example, a customer who wishes to register as a member operates a terminal device and transmits registration application information including his / her customer attribute information and a facial image to the processing device 10 via a dedicated application or web page. The registration application receiving unit 18 acquires the registration application information transmitted from the terminal device in this manner. The terminal device operated by the customer is a smartphone, a tablet terminal, a mobile phone, a smart watch, a personal computer, etc.

[0058] The first customer registration unit 19 registers the applicant's customer attribute information included in the registration application information and facial information generated based on the facial image included in the registration application information as the customer attribute information and facial information of the first customer in the customer information, linking them to the newly issued customer identification information.

[0059] When the customer attribute information and face image included in the registration application information satisfy a predetermined registration condition, the first customer registration unit 19 may execute the above-mentioned "process of linking the customer attribute information and face information of the first customer to the newly issued customer identification information and registering them in the customer information." The registration conditions are "the same combination of name, address, and telephone number has not been registered," "the face size in the face image is equal to or larger than a standard," "the feature amount extracted from the face image satisfies a standard," etc.

[0060] The determination unit 20 compares the facial image included in the registration application information with the facial information of the second customer included in the customer information stored in the customer information storage unit 11, and determines whether the applicant who sent the registration application information is registered in the customer information as a second customer. If facial information of the second customer that matches the facial image included in the registration application information is registered, the determination unit 20 determines that the applicant who sent the registration application information is registered in the customer information as a second customer. On the other hand, if facial information of the second customer that matches the facial image included in the registration application information is not registered, the determination unit 20 determines that the applicant who sent the registration application information is not registered in the customer information as a second customer.

[0061] When an applicant who has sent the registration application information is registered in the customer information as a second customer, the handover unit 21 registers information indicating in-store behavior that was registered in association with the applicant when the applicant was registered as a second customer in the customer information as linked to the customer identification information of the applicant registered in the customer information as a first customer. Through this process, the information registered when the applicant was registered in the customer information as a second customer (information indicating in-store behavior, etc.) is passed on to the customer even after the customer is registered as a first customer.

[0062] Next, an example of the flow of processing by the processing device 10 will be described with reference to the flowchart of FIG.

[0063] When the processing device 10 acquires registration application information including customer attribute information and a facial image (S20), it registers the customer attribute information of the applicant included in the registration application information and the facial information generated based on the facial image included in the registration application information as the customer attribute information and facial information of the first customer in the customer information by linking them to newly issued customer identification information (S21). Note that, when the customer attribute information and facial image included in the registration application information satisfy a predetermined registration condition, the processing device 10 may execute the above-mentioned "process of linking them to newly issued customer identification information as the customer attribute information and facial information of the first customer in the customer information".

[0064] Next, the processing device 10 compares the facial image contained in the registration application information with the facial information of the second customer contained in the customer information stored in the customer information storage unit 11, and determines whether the applicant who sent the registration application information is registered in the customer information as the second customer (S22).

[0065] If face information of a second customer that matches the face image included in the registration application information is registered (Yes in S23), the processing device 10 determines that the applicant who sent the registration application information is registered in the customer information as a second customer. Then, the processing device 10 registers the registration information (information indicating in-store behavior, etc.) of the second customer that matches the face image included in the registration application information by linking it to the customer identification information of the first customer newly issued in S21 (S24). Note that the processing device 10 may delete the information of the second customer that matches the face image included in the registration application information (information indicating in-store behavior, face information, customer identification information as the second customer, etc.) from the customer information storage unit 11 in accordance with the processing of S24.

[0066] If the facial information of the second customer that matches the facial image included in the registration application information is not registered (No in S23), the processing device 10 determines that the applicant who sent the registration application information is not registered in the customer information as the second customer. In this case, the processing of S24 is not performed.

[0067] Other configurations of the processing apparatus 10 of this embodiment are similar to those of the first and second embodiments.

[0068] According to the processing device 10 of this embodiment, the same effects as those of the first and second embodiments are achieved. Furthermore, according to the processing device 10 of this embodiment, when a customer who was registered as a second customer is registered as a first customer, the information registered when the customer was registered as the second customer (information indicating in-store behavior, etc.) can be passed on to the customer. As a result, it becomes possible to collect more information on more customers.

[0069] <Fourth embodiment> The processing device 10 of this embodiment awards points to both the first customer and the second customer based on information indicating in-store behavior stored in the customer information storage unit 11. Then, the processing device 10 allows only the first customer to use the awarded points. The processing device 10 also notifies the second customer of the awarded points and notifies the second customer that the points will be available for use if the second customer completes a prescribed procedure, thereby encouraging the second customer to register as the first customer. This will be described in detail below.

[0070] 8 shows an example of a functional block diagram of the processing device 10 of this embodiment. As shown in the figure, the processing device 10 of this embodiment differs from the first to third embodiments in that it has a points management unit 15, a points usage management unit 16, and a guide unit 17. Although not shown in the figure, the processing device 10 of this embodiment may also have a registration application acceptance unit 18, a first customer registration unit 19, a judgment unit 20, and a handover unit 21.

[0071] The points management unit 15 manages points given to the first customer and the second customer based on the in-store actions registered in the customer information. Figure 9 shows an example of information managed by the points management unit 15. In the example shown, customer identification information and the given points are registered in association with each other.

[0072] The point management unit 15 gives points to each customer based on a predetermined criterion. For example, the point management unit 15 may give points based on the number of visits to the store, the time spent in the store, the number of times products were tried, the number of times shopping was done, the amount of payment, the number of times a customer stopped in front of a promotional product, the number of times a customer passed in front of a promotional product, the number of times a customer purchased a promotional product, the number of times a customer brought a bag to carry the purchased products home, and the like. The point management unit 15 can give more points the more the customer visits the store, the longer the time spent in the store, the more the customer tried products, the more the customer shopped, the more the amount of payment, the more the customer stopped in front of a promotional product, the more the customer passed in front of a promotional product, the more the customer purchased a promotional product, and the more the customer brought a bag to carry the purchased products home.

[0073] The point usage management unit 16 allows the first customer to use points. The point management unit 16 does not allow the second customer to use points. There are no particular limitations on how the points can be used. For example, the points may be converted into merchandise, or may be converted into electronic money, or may be used to pay at a store. Such use of points can be realized using any conventional technology. When points are used, the point usage management unit 16 requests the presentation of customer identification information. Based on the customer identification information, the point management unit 16 can confirm that the customer wishing to use points is the first customer.

[0074] The notification unit 17 notifies the second customer of the guidance information. The guidance information indicates the points granted to the second customer. The guidance information also indicates that the points can be used by completing a predetermined procedure, or indicates that there are points that can be used by completing a predetermined procedure.

[0075] The predetermined procedure includes at least one of registering customer attribute information, registering face information that satisfies a predetermined condition, and expressing an intention to consent to the store using the information indicating the in-store behavior associated with the second customer and registered. The predetermined procedure may also be membership registration, etc.

[0076] The guidance unit 17 can notify the second customer of guidance information through a device installed in the store. The guidance unit 17 identifies the second customer included in the customer image by comparing a customer image generated by a camera (imaging means) included in the device installed in the store with face information of the second customer included in the customer information. Then, the guidance unit 17 displays guidance information indicating points given to the identified second customer on a display device included in the device. Examples of the device installed in the store include, but are not limited to, a POS register and a digital signage. In addition, the guidance unit 17 may realize the presentation of guidance information for the second customer using multiple cameras (imaging means) installed in the store and multiple display devices installed in the store. Specifically, the guidance unit 17 identifies the position of the second customer by detecting the second customer from images generated by multiple cameras (surveillance cameras, etc.). Then, the guidance unit 17 displays the guidance information on a display device installed in the vicinity of the identified position of the second customer.

[0077] In addition, if the processing device 10 of this embodiment has the configuration of the third embodiment, when a second customer is registered as a first customer, the points awarded while the second customer was registered as the second customer may be carried over to that customer even after the customer is registered as the first customer.

[0078] The other configurations of the processing apparatus 10 of this embodiment are similar to those of the first to third embodiments.

[0079] According to the processing device 10 of this embodiment, the same action and effect as those of the first to third embodiments can be realized. In addition, according to the processing device 10 of this embodiment, points can be given to customers based on their behavior in the store. This can encourage customers to visit the store.

[0080] In addition, according to the processing device 10 of this embodiment, the face information of a customer who has registered face information in advance by membership registration or the like can be displayed on the screen. Customer's Furthermore, points can be awarded to customers who have not registered their face information in advance through membership registration, etc. This makes it possible to encourage customers who have not registered their face information in advance through membership registration, etc. to visit the store.

[0081] In addition, according to the processing device 10 of the present embodiment, the use of points by customers who have not registered their face information in advance through membership registration or the like can be restricted after completing a predetermined procedure. This makes it possible to encourage customers to complete the predetermined procedure.

[0082] <Fifth embodiment> The processing device 10 of this embodiment analyzes the in-store behavior of a first customer and a second customer registered as customer information, and performs marketing analysis, promotion support, etc. This will be described in detail below.

[0083] 10 shows an example of a functional block diagram of the processing device 10 of this embodiment. As shown in the figure, the processing device 10 of this embodiment differs from the first to fourth embodiments in that it has an analysis unit 22. Although not shown, the processing device 10 of this embodiment may have a registration application acceptance unit 18, a first customer registration unit 19, a judgment unit 20, and a handover unit 21. In addition, the processing device 10 of this embodiment may have a point management unit 15, a point usage management unit 16, and a guidance unit 17.

[0084] The analysis unit 22 analyzes the in-store behavior of the first customer and the second customer registered in the customer information.

[0085] For example, the analysis unit 22 may perform statistical processing of the in-store behavior of multiple customers collectively and calculate the tendency of in-store behavior of customers who visit the store. For example, it may calculate what kind of in-store behavior customers who visit the store perform at each date, time, and time period. Furthermore, the analysis unit 22 may perform statistical processing of the in-store behavior of each customer and calculate the tendency of the in-store behavior of each customer. Then, based on the calculation results, the analysis unit 22 may determine the store's strategy (optimization of orders, optimization of product display methods, etc.) and the action to be taken for each customer (product recommendation, advertisement presentation, etc.), etc.

[0086] For example, the analysis unit 22 can identify customers whose frequency of visits to the store is equal to or higher than a threshold as good customers. The analysis unit 22 can then perform statistical processing of the in-store behavior of good customers to identify trends in the timing of visits to the store (day of the week, time of day, etc.) and trends in the products they purchase. In this case, orders can be placed so that the products purchased by good customers are not out of stock when the good customers visit the store, thereby preventing missed sales opportunities.

[0087] In addition, sales promotion is expected by determining the products to be recommended to each customer and the advertisements to be presented to each customer based on the purchasing trends of the customers.

[0088] Here, a specific example of the analysis performed by the analysis unit 22 will be described. Note that the specific example is merely an example, and the present invention is not limited to this.

[0089] By analyzing each customer's in-store behavior, it is possible to calculate trends such as the following:

[0090] The probability that customer A visits store A on Tuesday is X% Customer A stopped in front of a bread shelf for more than t minutes but did not purchase the product. At that time, product A was not on display at store A. · There is a history of purchasing product A at other stores. The probability of purchasing product A when it is on display is Y%.

[0091] From the above, the analysis unit 22 can calculate the probability that customer A will purchase product A if product A is ordered at store A on Tuesday. In addition, by performing a similar analysis on other customers, the analysis unit 22 can calculate the probability that other customers will purchase product A if product A is ordered at store A on Tuesday. Then, based on these calculation results, the analysis unit 22 can calculate whether product A will be ordered at store A on Tuesday, the number of products, etc.

[0092] The analysis unit 22 can also calculate the tendency of the time period when product A is purchased. For example, it can calculate the Z % possibility that product A will be purchased around noon. The analysis unit 22 can also analyze data from a POS register and calculate the tendency of products to be purchased together with product A. For example, it can calculate the M % possibility that coffee will be purchased at the same time as product A. The analysis unit 22 can also calculate the tendency of customer behavior when product A is not on display, based on the in-store behavior of customers who tend to purchase product A when product A is not on display (when it is sold out or is not even sold in the store).

[0093] Then, the analysis unit 22 can calculate the timing of ordering the product A at the store A, the number of items, etc. based on the calculation result.

[0094] Other configurations of the processing apparatus 10 of this embodiment are similar to those of the first to fourth embodiments.

[0095] According to the processing device 10 of this embodiment, the same operational effects as those of the first to fourth embodiments are realized. In addition, according to the processing device 10 of this embodiment, it is possible to perform marketing analysis, promotion support, and the like based on the in-store behavior of each registered customer.

[0096] <Modification> In the above-described embodiment, the processing device 10 registers a customer as a first customer in response to an action from the customer (registration of customer attribute information and a face image). As a modified example, the processing device 10 may register a second customer as a first customer in response to acquisition of customer attribute information of the second customer registered as a second customer by another method.

[0097] For example, when customer attribute information of a customer is acquired by the POS register based on the presentation of a membership card or the like during a transaction, the processing device 10 acquires the acquired customer attribute information of the customer. The processing device 10 also acquires a facial image of the customer acquired (photographed) when reading the membership card during the transaction. The processing device 10 then collates the facial image with the facial information of the second customer stored in the customer information storage unit 11 to identify that the customer performing the transaction is the second customer. Next, the processing device 10 re-registers the customer as a first customer by linking the information registered in the customer information storage unit 11 associated with the identified second customer with the information read from the membership card. The facial information of the first customer stored in the customer information storage unit 11 may be the facial information when the customer was registered as the second customer, or a facial image photographed when the membership card is read may be registered.

[0098] In this specification, "acquisition" includes at least one of the following: "the device itself goes to retrieve data stored in another device or storage medium (active acquisition)" based on user input or based on program instructions, such as receiving data by making a request or inquiry to another device, or accessing another device or storage medium and reading it, and "the device itself inputs data output from another device (passive acquisition)" based on user input or based on program instructions, such as receiving data that is distributed (or transmitted, push notification, etc.), and selecting and acquiring data from among the received data or information, and "editing data (converting it to text, rearranging data, extracting some data, changing the file format, etc.) to generate new data and acquiring the new data."

[0099] A part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. 1. A customer information storage means for storing customer information including customer attribute information and face information of a first customer who is a customer who has registered customer attribute information and face information, and face information of a second customer who has not registered the customer attribute information and has registered the face information; a detection means for detecting the first customer and the second customer in an image captured in a store by comparing the face information with the image; A behavior monitoring means for identifying the in-store behavior of the first customer and the second customer detected in the image and registering the behavior in the customer information; a second customer registration means for generating the face information based on a face image of the unregistered person included in the image when the unregistered person whose face information is not registered is detected in the image, and registering the face information in the customer information as the face information of the second customer; A processing device having 2. The detection means compares the image with the facial information of the second customer, and detects the second customer who is visiting the store for the second or subsequent time in the image; The processing device according to claim 1, wherein the behavior monitoring means links the in-store behavior of the second customer identified during each of multiple visits to the store with each other and registers the linked in-store behavior in the customer information. 3. A point management means for managing points given to the first customer and the second customer based on the in-store behavior registered in the customer information; a point usage management means for permitting the first customer to use the points and not permitting the second customer to use the points; 3. The processing device according to claim 1 or 2, 4. The processing device described in 3, further comprising a guidance means for indicating the points granted to the second customer and for notifying the second customer of guidance information indicating that the points can be used by completing a specified procedure. 5. The said prescribed procedure is as follows: 5. The processing device according to claim 4, further comprising at least one of registering the customer attribute information, registering the face information that satisfies a predetermined condition, and expressing consent to the store using the registered in-store behavior. 6. The guide means is identifying the second customer included in the customer image by comparing a customer image generated by an imaging means installed in the store with the face information included in the customer information; 6. The processing device according to claim 4 or 5, further comprising a display device that displays the guidance information relating to the points granted to the identified second customer. 7. A registration application receiving means for acquiring registration application information including the customer attribute information and a face image; a first customer registration means for registering the customer attribute information of the applicant included in the registration application information and the face information generated based on a face image included in the registration application information as the customer attribute information and the face information of the first customer in the customer information; a determination means for comparing the facial image included in the registration application information with the facial information of the second customer included in the customer information and determining whether the applicant who transmitted the registration application information is registered in the customer information as the second customer; a transfer means for registering, when the applicant is registered in the customer information as the second customer, the in-store behavior registered in association with the applicant when the applicant was registered as the second customer in association with customer identification information of the applicant registered in the customer information as the first customer; 7. A processing device according to any one of 1 to 6, comprising: 8. A processing device according to any one of 1 to 7, further comprising an analysis means for analyzing the in-store behavior of the first customer and the second customer registered in the customer information. 9. The computer: store customer information including the customer attribute information and the face information of a first customer who is a customer who has registered the customer attribute information and face information, and the face information of a second customer who is a customer who has not registered the customer attribute information but has registered the face information; a detection step of matching an image taken in a store with the face information and detecting the first customer and the second customer in the image; a behavior monitoring step of identifying the in-store behaviors of the first customer and the second customer detected in the image and registering the identified behaviors in the customer information; a second customer registration step of generating face information based on a face image of the unregistered person included in the image when the unregistered person whose face information is not registered is detected in the image, and registering the face information in the customer information as the face information of the second customer; The processing method to be performed. 10. Computer a customer information storage means for storing customer information including customer attribute information and face information of a first customer who is a customer who has registered customer attribute information and face information, and the face information of a second customer who is a customer who has not registered the customer attribute information but has registered the face information; a detection means for comparing an image taken in the store with the face information and detecting the first customer and the second customer within the image; A behavior monitoring means for identifying the in-store behavior of the first customer and the second customer detected in the image and registering the behavior in the customer information; a second customer registration means for generating the face information based on a face image of the unregistered person included in the image when the unregistered person whose face information is not registered is detected in the image, and registering the face information in the customer information as the face information of the second customer; A program that functions as a [Explanation of symbols]

[0100] 10 Processing equipment 11 Customer information storage section 12 Detection section 13 Behavioral Surveillance Division 14 Second Customer Registration Section 15 Points Management Department 16 Point Usage Management Department 17 Information Department 18 Registration Application Reception Department 19 First Customer Registration Section 20 Judgment Department 21 Handover Department 22 Analysis Department 1A Processor 2A Memory 3A input / output I / F 4A Peripheral circuit 5A Bus

Claims

1. A customer information storage means for storing customer information including the customer attribute information and the face information of a first customer who has registered as a member and has registered customer attribute information and face information, and the face information of a second customer who has not registered as a member, has not registered the customer attribute information, and has registered the face information; a detection means for detecting the first customer and the second customer in an image captured in a store by comparing the face information with the image; A behavior monitoring means for identifying the in-store behavior of the first customer and the second customer detected in the image and registering the behavior in the customer information; a second customer registration means for generating face information based on a face image of the unregistered person included in the image when the unregistered person who has not registered as a member and whose face information is not registered is detected in the image, and registering the face information in the customer information as the face information of the second customer; A processing device having

2. The detection means compares the image with the face information of the second customer, and detects the second customer who visits the store for the second or subsequent time in the image; The processing device according to claim 1 , wherein the behavior monitoring means links the in-store behaviors of the second customer identified during each of a plurality of visits to the store with each other and registers them in the customer information.

3. a point management means for managing points given to the first customer and the second customer based on the in-store actions registered in the customer information; a point usage management means for permitting the first customer to use the points and not permitting the second customer to use the points; 3. The processing device according to claim 1, further comprising:

4. The processing device according to claim 3, further comprising a notification means for notifying the second customer of guidance information indicating the points granted to the second customer and indicating that the points can be used by completing a specified procedure.

5. The prescribed procedure is as follows: The processing device according to claim 4 , further comprising at least one of registering the customer attribute information, registering the face information that satisfies a predetermined condition, and expressing an intention to consent to the store using the registered in-store behavior.

6. The guiding means is identifying the second customer included in the customer image by comparing a customer image generated by an imaging means installed in the store with the face information included in the customer information; The processing device according to claim 4 or 5, wherein the guidance information relating to the points given to the identified second customer is displayed on a display device.

7. A registration application receiving means for acquiring registration application information including the customer attribute information and a facial image of an applicant; a first customer registration means for registering the customer attribute information of the applicant included in the registration application information and the face information generated based on a face image included in the registration application information as the customer attribute information and the face information of the first customer in the customer information; a determination means for comparing the face image included in the registration application information with the face information of the second customer included in the customer information and determining whether the applicant who transmitted the registration application information is registered in the customer information as the second customer; a transfer means for registering, when the applicant is registered in the customer information as the second customer, the in-store behavior registered in association with the applicant when the applicant was registered as the second customer in association with customer identification information of the applicant registered in the customer information as the first customer; The processing device according to any one of claims 1 to 6, comprising:

8. The processing device according to claim 1 , further comprising an analysis unit that analyzes the in-store behavior of the first customer and the second customer registered in the customer information.

9. The computer store customer information including the customer attribute information and the face information of a first customer who has registered as a member and has registered customer attribute information and face information, and the face information of a second customer who has not registered as a member, has not registered the customer attribute information, and has registered the face information; a detection step of matching an image taken in a store with the face information and detecting the first customer and the second customer in the image; a behavior monitoring step of identifying the in-store behaviors of the first customer and the second customer detected in the image and registering the identified behaviors in the customer information; a second customer registration step of generating face information based on a face image of the unregistered person included in the image when the unregistered person who has not registered as a member and whose face information is not registered is detected in the image, and registering the face information in the customer information as the face information of the second customer; The processing method to be performed.

10. Computer, a customer information storage means for storing customer information including the customer attribute information and the face information of a first customer who has registered as a member and has registered customer attribute information and face information, and the face information of a second customer who has not registered as a member, has not registered the customer attribute information, and has registered the face information; a detection means for comparing an image taken in the store with the face information and detecting the first customer and the second customer within the image; A behavior monitoring means for identifying the in-store behavior of the first customer and the second customer detected in the image and registering the behavior in the customer information; a second customer registration means for generating face information based on a face image of the unregistered person included in the image when the unregistered person who has not registered as a member and whose face information is not registered is detected in the image, and registering the face information in the customer information as the face information of the second customer; A program that functions as a

Images