Product recommendation device, product recommendation method, and program

Abstract

The product purchased by the customer may differ depending on the emotion of the customer when the customer visits the store.SOLUTION: A product recommendation device according to the present disclosure includes image acquisition means for acquiring a face image of a customer visiting a store, estimation means for estimating an emotion of the customer at the time of visiting the store from the face image, determination means for determining a recommendation product according to the emotion at the time of visiting the store on the basis of information regarding a relationship between a product included in a purchase history of the customer and the emotion estimated at the time of purchasing the product, and output means for outputting product information of the recommendation product.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a product recommendation device, a product recommendation method, and a program. [Background technology]

[0002] In some cases, stores may provide information about recommended products to customers depending on the situation of the customers who visit the store.

[0003] Patent Document 1 describes a vending machine that determines which product to recommend to a purchaser based on the condition of the purchaser. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-203952 Summary of the Invention [Problem to be solved by the invention]

[0005] The products a customer purchases may depend on how they feel when they visit the store.

[0006] One object of the present disclosure is to provide a product recommendation device or the like that can recommend products according to the emotions of a customer. [Means for solving the problem]

[0007] A product recommendation device according to one aspect of the present disclosure includes an image acquisition means for acquiring a facial image of a customer who visits a store, an estimation means for estimating the customer's emotions at the time of the store from the facial image, a determination means for determining recommended products according to the customer's emotions at the time of the store based on information relating to the relationship between products included in the customer's purchase history and the emotions estimated when the products were purchased, and an output means for outputting product information about the recommended products.

[0008] A product recommendation method according to one aspect of the present disclosure acquires a facial image of a customer who visits a store, estimates the customer's emotions at the time of the store from the facial image, determines recommended products according to the customer's emotions at the time of the store based on information relating to the relationship between products included in the customer's purchase history and the emotions estimated at the time of purchasing the products, and outputs product information about the recommended products.

[0009] A program in one aspect of the present disclosure causes a computer to perform a process of acquiring a facial image of a customer who visits a store, estimating the customer's emotions at the time of the store from the facial image, determining recommended products according to the customer's emotions at the time of the store based on information regarding the relationship between products included in the customer's purchase history and the emotions estimated when the products were purchased, and outputting product information for the recommended products.

[0010] Each program may be stored in a non-transitory computer-readable recording medium. [Effects of the Invention]

[0011] One example of the effect of the present disclosure is that it becomes possible to recommend products that correspond to the emotions of a customer. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a system including a product recommendation device. [Figure 2] FIG. 1 is an image diagram showing an example of a store to which a product recommendation device is applied. [Figure 3] FIG. 1 is a block diagram illustrating an example of a configuration of a product recommendation device. [Figure 4] 10 is an example of a display screen on which product information of recommended products is displayed. [Figure 5] 10 is an example of a display screen displaying product information of a recommended product and a product recommended in combination with the recommended product. [Figure 6] 10 is an example of a display screen displaying a map of the inside of a store. [Figure 7] 10 is a flowchart showing the operation of the product recommendation device. [Figure 8]FIG. 2 is a diagram illustrating an example of a hardware configuration of a product recommendation device. DETAILED DESCRIPTION OF THE INVENTION

[0013] Embodiments of the present disclosure will be described in detail with reference to the drawings.

[0014] [Embodiment] An example of the configuration of a system including a product recommendation device 10 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of a system including the product recommendation device 10. Referring to Fig. 1, the product recommendation device 10 is connected to a camera 90, a display device 91, and a database 92 via a wired or wireless network.

[0015] The camera 90 is an imaging device capable of capturing images including moving images. The camera 90 is installed in a position where it can capture images of customers. For example, the camera 90 is installed at the entrance of a store. The camera 90 installed at the entrance of a store can capture images of customers who visit the store. The camera 90 installed at the entrance of a store captures facial images of customers that are used for face authentication for entering the store.

[0016] Referring to FIG. 1, one camera 90 is connected to the product recommendation device 10, but the number of cameras 90 is not limited. In other words, there may be one or more cameras 90. When there are multiple cameras 90, there may be multiple cameras 90 installed at the entrance. Furthermore, the cameras 90 may be installed inside the store in addition to at the entrance.

[0017] The cameras 90 installed in the store can capture images of customers staying in the store. The cameras 90 may be installed in the required number and locations depending on the layout of the store, the way products are displayed, etc. The cameras 90 installed in the store may include, for example, security cameras.

[0018] The display device 91 is a device capable of displaying product information. The display device 91 is installed at the entrance of the store. A customer who is about to enter the store can view the display screen of the display device 91 installed at the entrance of the store. The display device 91 installed at the entrance of the store may be used for facial authentication for entering the store. That is, a customer who is about to enter the store can undergo facial authentication using the display device 91 installed at the entrance. For example, the display device 91 installed at the entrance can display a facial image captured by the above-mentioned camera 90. The customer can then check the facial image displayed on the display device 91. The display device 91 installed at the entrance may also be capable of displaying whether facial authentication was successful. The display content of the display device 91 installed at the entrance is not limited to this.

[0019] Referring to FIG. 1, one display device 91 is connected to the product recommendation device 10, but the number of display devices 91 is not limited. That is, there may be one or more display devices 91. When there are multiple display devices 91, there may be multiple display devices 91 installed at the entrance. Furthermore, the display devices 91 may be installed inside the store as well as at the entrance. Customers who enter the store can view the display screens of the display devices 91 installed inside the store. The display devices 91 may be installed, for example, on each product display shelf. The locations where the display devices 91 can be installed in the store are not limited to these. The required number of display devices 91 may be installed in the required locations depending on the layout of the store and the way the products are displayed.

[0020] The display device 91 is, for example, a display. A digital signage display can also be used as the display device 91. However, the display device 91 is not limited to these.

[0021] The database 92 stores information about customers. The information about customers includes, for example, information about the customer's facial image, purchase history, and emotional state. The information about the customer's facial image is used, for example, for facial authentication to enter a store. The information about the customer's facial image may also be used for facial authentication payment. Use of the information about the customer's facial image is not limited to these.

[0022] A customer's purchase history is information about products purchased by the customer at a store. The customer's purchase history may include, for example, information about coupons used by the customer. The customer's purchase history may also be stored in association with information about the customer's emotions when the products included in the purchase history were purchased. The information about the customer's emotions is information about the customer's emotions estimated by the estimation unit 102, which will be described later.

[0023] The database 92 may store customer attribute information. The customer attribute information may include, for example, at least one of the customer's gender, age, occupation, and family structure. The customer attribute information is not limited to these. The customer attribute information may be registered by the customer, for example. The customer attribute information may also be attribute information estimated by the estimation unit 102, which will be described later.

[0024] Here, an application example of a product recommendation system including the product recommendation device 10 will be described with reference to Fig. 2. Fig. 2 is an image diagram showing an example of a store to which the product recommendation system including the product recommendation device 10 is applied. With reference to Fig. 2, a flow of a customer using a store to which the product recommendation system including the product recommendation device 10 is applied will be described.

[0025] A customer who visits a store first undergoes facial authentication to enter the store. For facial authentication, for example, a display device 91 and a camera 90 installed at the entrance are used. The camera 90 installed at the entrance can capture a facial image of the customer. The facial image may include body parts other than the face. In other words, the facial image only needs to include at least the face of the customer. The facial image may be displayed on the display device 91. The customer can register a facial image in advance. Facial authentication is performed using the facial image registered by the customer in advance and the facial image captured by the camera 90 installed at the entrance. Publicly known technology is used as the facial authentication method.

[0026] At this time, the product recommendation device 10 estimates the customer's emotions from the customer's facial image. Then, the product recommendation device 10 outputs product information of recommended products according to the customer's emotions. The product information of the recommended products is displayed, for example, on a display device 91 installed at the entrance. Customers who visit the store can check the product information of the recommended products via the display device 91. Details of the product recommendation device 10 will be explained later.

[0027] Here, customers who visit a store and have not registered their facial images in advance may be able to register their facial images at the store entrance. For example, a facial image is captured by a camera 90 installed at the entrance. If a display device 91 installed at the entrance is a touch panel display, the customer may input the information required for registration using the display device 91. Alternatively, the customer may input the information required for registration using a communication terminal used by the customer. The input information is registered in association with the information of the facial image captured by the camera 90.

[0028] After completing facial authentication at the entrance, the customer enters the store. Inside the store, the customer can look around at the products on display. The customer can also select a product to purchase. At this time, the product recommendation device 10 may cause a camera 90 installed in the store to capture a facial image of the customer inside the store. The product recommendation device 10 may also estimate the customer's emotions from the facial image of the customer inside the store. The product recommendation device 10 may also identify the customer's position based on the facial image of the customer inside the store. Then, the product recommendation device 10 may output product information of the recommended products to a display device 91 close to the customer's position. In other words, product information of the recommended products recommended to the customer may be displayed on a display device 91 close to the customer. The customer can check the product information of the recommended products via the display device 91.

[0029] Once a customer has selected the products they wish to purchase, they make payment at the cash register. The customer registers the products they wish to purchase. Publicly known technology is used to register the products. The customer then pays for the registered products. Information about the products that the customer has paid for, i.e., purchased, is saved as a purchase history. The purchase history is saved in association with identification information that identifies the customer. The purchase history and identification information are saved, for example, in a database. The destination where the purchase history and identification information are saved is not limited to this.

[0030] Here, the customer may make payment by facial recognition payment. If the customer has registered a facial image and information regarding a payment method in advance, the customer can make payment by facial recognition payment. Information regarding the payment method is information necessary for the customer to make a payment. Information regarding the payment method is, for example, credit card information. Facial recognition payment is performed using publicly known technology. Information about products paid for by facial recognition payment is saved as a purchase history. At this time, the customer is identified by the facial image used for facial recognition payment. Therefore, the customer's identification information and purchase history are saved in association with each other.

[0031] The payment method is not limited to facial recognition payment. Any payment method is acceptable as long as it is possible to associate and store the customer's identification information with the customer's purchase history. In other words, any payment method that can identify the customer is acceptable. Furthermore, regardless of the payment method, payment may be made along with an operation that can identify the customer. An example of an operation that can identify the customer is reading a membership card. The membership card or barcode is associated with identification information. Operation that can identify the customer is not limited to these.

[0032] Then, the customer who has purchased the product leaves the store. Application examples of the product recommendation system including the product recommendation device 10 are not limited to these.

[0033] The configuration of the product recommendation device 10 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the configuration of the product recommendation device 10. The product recommendation device 10 includes, as basic functions, an image acquisition unit 101, an estimation unit 102, a determination unit 103, and an output unit 104. The product recommendation device 10 may further include an analysis unit 105.

[0034] The image acquisition unit 101 is one aspect of image acquisition means for acquiring a facial image of a customer who visits a store. The image acquisition unit 101 acquires a facial image of the customer captured by a camera 90 installed in the store. The facial image is an image that includes the customer's face. In other words, the facial image of the customer may include something other than the customer's face.

[0035] The image acquisition unit 101 acquires a facial image of a customer who visits the store, for example, from a camera 90 installed at the entrance. Customers who visit the store undergo facial authentication in order to enter the store. Therefore, the camera 90 captures an image of the customer's face for facial authentication. The image acquisition unit 101 can then acquire the facial image of the customer captured by the camera 90 for facial authentication. Here, customers who visit the store include customers whose facial images have been registered in advance and customers whose facial images have not been registered in advance. Unregistered customers whose facial images have not been registered in advance can register their facial images at the store entrance as described above.

[0036] The facial image acquired by the image acquisition unit 101 is used by the estimation unit 102, which will be described later, to estimate the emotion of the customer. The estimation unit 102 can estimate the emotion of the customer from the facial expression of the customer. However, when capturing a facial image for facial recognition, the customer may not have a natural expression. For example, the facial image of the customer may be captured with a serious or nervous expression. Therefore, it may be difficult for the estimation unit 102 to accurately estimate the emotion of the customer.

[0037] Therefore, the image acquisition unit 101 may acquire a face image of a customer at a timing different from the timing at which the face image used for face authentication is acquired. In other words, the image acquisition unit 101 may acquire a face image of a customer in addition to the face image of the customer for face authentication. Here, the face image of a customer acquired by the image acquisition unit 101 at a timing different from the timing at which the face image used for face authentication is acquired is called an additional face image.

[0038] For example, the image acquisition unit 101 may acquire a facial image of a customer at a timing other than the timing of acquiring a facial image used for facial authentication by the camera 90 installed at the entrance of the store. In addition to the image for facial authentication, the image acquisition unit 101 can acquire a facial image captured while the customer's face is within the range that the camera 90 installed at the entrance can capture.

[0039] The period during which the customer's face is included within the range that can be captured by the camera 90 installed at the entrance refers to the period during which the customer's face is included within the range that can be captured by the camera 90 installed at the entrance after facial authentication is completed. That is, the image acquisition unit 101 may acquire a facial image of the customer after facial authentication at the store entrance. Here, an example of the additional facial image is a facial image of the customer acquired after facial authentication at the store entrance. In the image for facial authentication, the customer may not have a natural expression. However, the customer may return to a natural expression after facial authentication is completed. Therefore, for example, when facial authentication is completed, the display device 91 installed at the entrance displays that facial authentication has been completed. After the display displays that facial authentication has been completed, the camera 90 installed at the entrance captures the customer's face. Then, the image acquisition unit 101 can acquire the facial image of the customer after facial authentication from the camera 90.

[0040] In addition, the period during which the customer's face is included within the range that can be captured by the camera 90 installed at the entrance refers to, for example, a period before facial authentication and during which the customer's face is included within the range that can be captured by the camera 90 installed at the entrance. In other words, the image acquisition unit 101 may acquire a facial image of the customer before facial authentication at the store entrance. Here, another example of the additional facial image is a facial image of the customer acquired before facial authentication at the store entrance. In the image for facial authentication, the customer may not have a natural expression. However, before facial authentication is performed, the customer may have a natural expression. Therefore, for example, before facial authentication is performed, when the customer's face is included within the range that can be captured by the camera 90 installed at the entrance, the camera 90 captures an image of the customer's face. Then, the image acquisition unit 101 can acquire a facial image of the customer before facial authentication from the camera 90.

[0041] Furthermore, the image acquisition unit 101 may acquire, for example, a facial image of the customer in the store. That is, an example of the additional facial image is a facial image of the customer in the store. Here, the facial image of the customer in the store is a facial image captured while the customer is staying in the store. The facial image of the customer in the store is captured, for example, by a camera 90 installed in the store. The camera 90 installed in the store may be a security camera. Furthermore, the camera 90 installed in the store may be a camera 90 installed on a product shelf. The camera 90 installed in the store is not limited to these.

[0042] As described above, the facial image of the customer in the store may be used for emotion estimation by the estimation unit 102. In addition, the facial image of the customer in the store may be used to identify the customer's position in the store. Identifying the customer's position in the store will be described later.

[0043] The estimation unit 102 is one aspect of estimation means that estimates the emotion of a customer when they visit a store from a facial image. The estimation unit 102 estimates the emotion of the customer using the facial image acquired by the image acquisition unit 101. The estimation unit 102 can analyze the facial expression of the customer captured in the facial image and estimate the emotion of the customer. Publicly known technology is used to estimate the emotion of the customer by analyzing the facial expression of the customer.

[0044] Customer emotions are the psychological state of a customer. For example, some examples of customer emotions are enjoyment, joy, a sense of accomplishment, and relaxation. Other examples of customer emotions are anxiety, depression, sadness, anger, and impatience. In other words, customer emotions represent the customer's mood. Customer emotions are not limited to these.

[0045] The estimation unit 102 estimates the emotion of the customer when he / she comes to the store. The emotion of the customer when he / she comes to the store is the emotion the customer is feeling when he / she enters the store. In other words, the emotion of the customer when he / she comes to the store indicates the mood in which the customer comes to the store.

[0046] The estimation unit 102 may estimate the emotion of a customer when he or she visits a store from a facial image used for facial authentication. The facial image used for facial authentication is captured by, for example, a camera 90 installed at the entrance. The image acquisition unit 101 then acquires the facial image used for facial authentication. The estimation unit 102 can analyze the facial expression of the customer captured in the facial image used for facial authentication and estimate the emotion of the customer.

[0047] The estimation unit 102 may further estimate the emotion of the customer when he / she visits the store from the additional facial image. As described above, when a facial image is captured for facial recognition, the customer may not have a natural facial expression. For example, the facial image of the customer may be captured with a serious or nervous expression. Therefore, it may be difficult for the estimation unit 102 to accurately estimate the emotion of the customer. Therefore, the estimation unit 102 may estimate the emotion of the customer when he / she visits the store based on the facial image used for facial recognition to enter the store and the additional facial image.

[0048] The additional facial image includes at least one of a facial image of the customer acquired after facial authentication at the store entrance, a facial image of the customer acquired before facial authentication at the store entrance, and a facial image of the customer inside the store. The additional facial image used by the estimation unit 102 may be one or more. The estimation unit 102 estimates the customer's emotion at the time of entering the store using the facial image used for facial authentication to enter the store and the additional facial image. This enables improvement in the accuracy of the estimated emotion.

[0049] Here, the customer's emotion when visiting the store may include not only the customer's emotion when entering the store but also the customer's emotion while staying in the store. For example, the image acquisition unit 101 acquires a facial image used for facial recognition and a facial image of the customer inside the store. The estimation unit 102 estimates the customer's emotion when entering the store from the facial image used for facial recognition. The estimation unit 102 may then estimate the customer's emotion while staying in the store from the facial image of the customer inside the store. If the customer's emotion when entering the store differs from the customer's emotion while staying in the store, the determination unit 103 (described later) may determine recommended products according to the customer's emotion while staying in the store. Furthermore, the determination unit 103 may determine recommended products according to the customer's emotion when entering the store and the customer's emotion while staying in the store. In other words, the determination unit 103 may determine recommended products according to multiple emotions.

[0050] The estimation unit 102 may further estimate customer attributes from a facial image of the customer. The customer attributes estimated by the estimation unit 102 include, for example, the gender, age, and occupation of the customer. The customer attributes are not limited to these. The estimation unit 102 can estimate customer attributes from a facial image used for face authentication. The estimation result of the customer attributes by the estimation unit 102 is stored in the database 92, for example, in association with the customer's identification information. Publicly known technology is used to estimate the customer attributes.

[0051] The estimation unit 102 may further estimate vital data of the customer from a facial image of the customer. Examples of vital data estimated by the estimation unit 102 include heart rate and body temperature. The vital data is not limited to these. The estimation unit 102 can estimate vital data of the customer from a facial image used for face authentication. The estimation result of the vital data of the customer by the estimation unit 102 is used, for example, to determine recommended products. Publicly known techniques are used to estimate the vital data of the customer.

[0052] The determination unit 103 is one aspect of a determination means for determining recommended products according to the customer's emotions at the time of visiting the store, based on information relating to the relationship between products included in the customer's purchase history and the emotions estimated at the time of purchasing the products. The determination unit 103 determines recommended products to be recommended to a customer based on products previously purchased by the customer who visited the store and the emotions estimated at the time of purchasing the products. At this time, the determination unit 103 determines recommended products according to the customer's emotions at the time of visiting the store. The recommended products are, for example, products displayed in the store. The recommended products recommended to the customer may be product categories. Product categories include, for example, rice balls, sweets, snacks, and snacks. Product categories may be, for example, new products or limited-time products. There may be one or more recommended products.

[0053] The determination unit 103 determines recommended products using information regarding the relationship between products included in the customer's purchase history and the emotion estimated at the time of purchasing the products. The information regarding the relationship between products included in the customer's purchase history and the emotion estimated at the time of purchasing the products is the result of analyzing the relationship between products included in the customer's purchase history and the emotion estimated at the time of purchasing the products. The emotion estimated at the time of purchasing the products is the emotion the customer had when they entered the store to purchase the products. In other words, the information regarding the relationship between products included in the customer's purchase history and the emotion estimated at the time of purchasing the products is the result of analyzing what emotions the customer had when they visited the store and what products they purchased. The analysis of the relationship between products included in the customer's purchase history and the emotion estimated at the time of purchasing the products will be described later.

[0054] The information relating to the relationship between the products included in the customer's purchase history and the emotion estimated at the time of purchasing the product is, for example, a table in which the customer's emotion is associated with a recommended product recommended for that emotion. For example, a happy emotion may be associated with a cream puff. Also, a frustrated emotion may be associated with a protein bar. Examples of associations between emotions and recommended products are not limited to these. The determination unit 103 can then refer to the table and determine, as the recommended product, a product associated with the same emotion as the emotion the customer was feeling when they visited the store.

[0055] In a table in which customer emotions are associated with recommended products recommended for each emotion, the emotions may be categorized. An example in which emotions are categorized into positive emotions and negative emotions will be described. In this case, sweets are associated with positive emotions, and energy drinks are associated with negative emotions. The determination unit 103 can then refer to the table and determine, as a recommended product, a product associated with an emotion in the category that corresponds to the emotion the customer felt when visiting the store.

[0056] The information regarding the relationship between the products included in the customer's purchase history and the emotion estimated when the product was purchased may be, for example, a trained model that has been trained on the relationship between the products included in the customer's purchase history and the emotion estimated when the product was purchased. The determination unit 103 inputs the customer's emotion when visiting the store into the trained model. The trained model then outputs recommended products according to the customer's emotion when visiting the store. Therefore, the determination unit 103 obtains recommended products as output by the trained model. In this way, the determination unit 103 can determine recommended products according to the emotion when visiting the store using the trained model that has been trained on the relationship between the products included in the customer's purchase history and the emotion estimated when the product was purchased.

[0057] Here, the product recommendation device 10 may include an analysis unit 105. The analysis unit 105 analyzes the relationship between the products included in the customer's purchase history and the emotion estimated at the time of purchasing the products. As a result of the analysis by the analysis unit 105, information regarding the relationship between the products included in the customer's purchase history and the emotion estimated at the time of purchasing the products is generated. For example, as a result of the analysis by the analysis unit 105, a table is generated in which the customer's emotion is associated with a recommended product that is recommended in the case of that emotion. Furthermore, as a result of the analysis by the analysis unit 105, the above-mentioned trained model is generated.

[0058] The generation of a trained model by the analysis unit 105 will be described. The analysis unit 105 trains a model on the relationship between products included in a customer's purchase history and emotions estimated when the products were purchased. For example, the analysis unit 105 trains the model on product information such as product category, product price, product size, and product features, and the emotions estimated when the products were purchased. A customer may purchase different products depending on, for example, their emotions. Therefore, the analysis unit 105 can train the model on the customer's emotions and the tendency of products the customer purchases when they are emotional. In this case, the analysis unit 105 can acquire, for example, the customer's purchase history stored in the database 92. Furthermore, the analysis unit 105 can acquire emotion information stored in the database 92 in association with the purchase history. Then, the determination unit 103 trains the model on the relationship between products included in the customer's purchase history and emotions estimated when the products were purchased.

[0059] Here, the purchase history and emotion information used for analysis by the analysis unit 105 may be information about a single customer. In other words, information regarding the relationship between products included in the customer's purchase history and the emotion estimated at the time of purchasing the products may be generated for each customer. For example, even if the emotion is the same, the products purchased by each customer may differ. Therefore, the analysis unit 105 analyzes, for each customer, the relationship between products included in the customer's purchase history and the emotion estimated at the time of purchasing the products. As a result, the determination unit 103 can determine, for each customer, recommended products according to the emotion at the time of visiting the store.

[0060] Returning to the description of the determination unit 103, the determination unit 103 may determine recommended products according to the attributes and emotions at the time of visiting the store, based on information regarding the relationship between the products included in the customer's purchase history, the emotions estimated when the products were purchased, and the customer's attributes. In other words, the determination unit 103 can further use attributes to determine recommended products. The information regarding the relationship between the products included in the customer's purchase history, the emotions estimated when the products were purchased, and the customer's attributes is the result of analyzing the relationship between the products included in the customer's purchase history, the emotions estimated when the products were purchased, and the customer's attributes. In other words, the information regarding the relationship between the products included in the customer's purchase history, the emotions estimated when the products were purchased, and the customer's attributes is the result of analyzing what kind of customers with what attributes purchased what kind of products when they visited the store with what kind of emotions.

[0061] The information regarding the relationship between the products included in the customer's purchase history, the emotion estimated at the time of purchasing the product, and the customer's attributes is, for example, a table in which the customer's attributes and emotions are associated with the recommended products recommended for the emotion. The table may be generated by the analysis unit 105 described above. For example, chocolate is associated with a happy emotion for a female customer in her twenties. Also, coffee is associated with a tired emotion for an office worker. Examples of associations between attributes and emotions and recommended products are not limited to these. The determination unit 103 can then refer to the table and determine, as recommended products, products associated with the attributes and emotions of the customer who visited the store.

[0062] In a table in which customer attributes and emotions are associated with recommended products, emotions may be categorized. An example in which emotions are categorized into positive and negative emotions will be described. For example, when a man in his twenties has positive emotions, beer is associated with them. The determination unit 103 can then refer to the table and determine, as recommended products, products associated with the attributes and emotions of the customer who visited the store in the category that corresponds to the customer's emotions.

[0063] The information regarding the relationship between the products included in the customer's purchase history, the emotion estimated when the product was purchased, and the customer's attributes may be, for example, a trained model that has learned the relationship between the products included in the customer's purchase history, the emotion estimated when the product was purchased, and the customer's attributes. The determination unit 103 inputs the customer's emotion at the time of visiting the store and the customer's attributes into the trained model. The trained model then outputs recommended products according to the customer's emotion at the time of visiting the store and the customer's attributes. Therefore, the determination unit 103 obtains recommended products as output by the trained model. In this way, the determination unit 103 can determine recommended products according to the customer's emotion at the time of visiting the store and the customer's attributes using the trained model that has learned the relationship between the products included in the customer's purchase history, the emotion estimated when the product was purchased, and the customer's attributes.

[0064] The analysis unit 105 may generate a trained model that learns the relationship between products included in a customer's purchase history, emotions estimated when the products were purchased, and customer attributes. The analysis unit 105 trains the model on the relationship between products included in a customer's purchase history, emotions estimated when the products were purchased, and customer attributes. For example, a customer may purchase different products depending on their emotions. Furthermore, the tendency of products to be purchased may differ depending on the customer's attributes. Therefore, the analysis unit 105 can train the model on customer attributes and emotions, and the tendency of products purchased by customers with the corresponding attributes when they are emotional. The analysis unit 105 can acquire customer attribute information and purchase history stored in, for example, the database 92. Here, the customer attribute information may be attribute information registered by the customer or attribute information estimated by the estimation unit 102. The analysis unit 105 can then acquire emotion information linked to the purchase history and stored in the database 92. The determination unit 103 trains the model on the relationship between products included in a customer's purchase history, emotions estimated when the products were purchased, and customer attributes.

[0065] Here, the purchase history, emotion information, and attribute information used for analysis by the analysis unit 105 may be information on multiple customers. For example, even if the emotion is the same, the products purchased may differ depending on the customer's attributes. Therefore, the analysis unit 105 analyzes the relationship between the products included in each customer's purchase history, the emotion estimated when the product was purchased, and the attributes of each customer for multiple customers. As a result, the determination unit 103 can determine recommended products according to the customer's attributes and emotion at the time of visiting the store.

[0066] By having the determination unit 103 determine recommended products using attribute information, it becomes possible to determine recommended products even for customers who have no purchase history at the store. In other words, for customers who have no purchase history, recommended products can be determined based on the purchase history and emotional information of customers with similar attributes.

[0067] The method of determining recommended products by the determination unit 103 is not limited to these. Furthermore, the determination unit 103 may determine recommended products using different methods depending on the customer.

[0068] The determining unit 103 may further determine a product to be recommended in a set with the recommended product. For example, if the recommended product is rice balls, the determining unit 103 may determine that a plastic bottle of tea is the product to be recommended in a set with the rice balls.

[0069] The products to be recommended in a set with the recommended product may be predetermined by the store, for example. For example, information on the products to be recommended in a set is stored in the database 92. The determination unit 103 determines the recommended products. Then, the determination unit 103 refers to the information on the products to be recommended in a set. Then, the determination unit 103 can determine the products that are set with the recommended product as the products to be recommended in a set with the recommended product.

[0070] The product to be recommended as a set with the recommended product may be, for example, a product that the customer has previously purchased as a set with the recommended product. The determination unit 103 refers to the purchase history of the customer. Then, the determination unit 103 may determine that the product that has a history of being purchased as a set with the recommended product is the product to be recommended as a set with the recommended product.

[0071] The products recommended in combination with the recommended product are not limited to these. Furthermore, the method by which the determination unit 103 determines the products to be recommended in combination with the recommended product is not limited to these.

[0072] The output unit 104 is one aspect of output means for outputting product information of recommended products. An example of product information of recommended products is an advertisement for the recommended products. In other words, the product information of recommended products includes the name and price of the recommended products. The product information of recommended products may include an image of the recommended products and a catchphrase for the recommended products. The product information of recommended products may also include a coupon related to the recommended products. Another example of product information of recommended products may include a map showing the display location of the recommended products. The product information of recommended products is not limited to these. The product information of recommended products may be any information that the store wants to provide to customers regarding the recommended products.

[0073] The content of the product information of recommended products output by the output unit 104 may be predetermined for each product. Furthermore, the content of the product information of recommended products output by the output unit 104 may be predetermined for each store. For example, in the case of a store with a large sales floor, it may be predetermined that a map of the display positions of recommended products is output as product information. Furthermore, the content of the product information of recommended products output by the output unit 104 may be predetermined for each time period in which the product information of recommended products is output. For example, when product information is output between 11:00 and 13:00, it may be predetermined that a coupon related to the recommended product is output as product information.

[0074] The content of the product information of the recommended products output by the output unit 104 may be determined depending on the customer. For example, the output unit 104 may output a coupon related to the recommended products as product information to a customer who frequently visits the store. Furthermore, the output unit 104 may output a map of the display locations of the recommended products as product information to a customer who has not visited the store for a predetermined period of time.

[0075] The content of the product information of the recommended product output by the output unit 104 may be determined depending on the recommended product. For example, if the recommended product is a new product, a catchphrase indicating that it is a new product and a coupon related to the recommended product may be output as product information. Furthermore, if the display position of the recommended product has been changed, a map of the display position of the recommended product may be output as product information.

[0076] Furthermore, when the determination unit 103 determines a product to be recommended in a set with the recommended product, the output unit 104 may output product information of the recommended product and the product recommended in a set with the recommended product.

[0077] Here, the content of the product information of recommended products output by the output unit 104 may be determined by the determination unit 103. In other words, the determination unit 103 may determine the content to be output by the output unit 104 as the product information of recommended products. The method of determining the product information of recommended products to be output by the output unit 104 is not limited to these. Furthermore, examples of the product information of recommended products to be output by the output unit 104 are not limited to these.

[0078] The output destination to which the output unit 104 outputs the product information of the recommended products is, for example, the display device 91. The output unit 104 outputs the product information of the recommended products to the display device 91. The display device 91 can then display the product information of the recommended products received from the output unit 104 on a display screen. The customer can then view the display device 91 and recognize the product information of the recommended products.

[0079] As described above, one example of the display device 91 is a display device 91 installed at an entrance. Another example of the display device 91 is a display device 91 installed inside a store. The output unit 104 may output product information to one display device 91. Alternatively, the output unit 104 may output product information of recommended products to multiple display devices 91. Here, the display device 91 to which the output unit 104 outputs product information of recommended products may be determined by the determination unit 103. In other words, the determination unit 103 may determine the display device 91 to which the product information of recommended products is to be output. The determination unit 103 can determine where to output the product information.

[0080] The determination unit 103 may determine the display device 91 that outputs the product information according to the customer's location. That is, the output unit 104 may output the product information to the display device 91 according to the customer's location. For example, the output unit 104 outputs the product information of the recommended product to a display device 91 within a predetermined range from the customer's location in the store. The predetermined range is a range within which the display screen of the display device 91 is visible from the customer's location. An example of a display device 91 within a predetermined range from the customer's location is the display device 91 that is closest to the customer's location. The display device 91 within a predetermined range from the customer's location is not limited to this.

[0081] The determination of the display device 91 by the determination unit 103 will be described. First, a camera 90 installed in the store captures a facial image of a customer. Then, the image acquisition unit 101 acquires the facial image. The customer's location is identified from the position of the camera 90 that captured the facial image of the customer. The determination unit 103 can determine the display device 91 that outputs product information based on the customer's location. Here, the customer's location may be identified, for example, by an identification unit (not shown). The method of identifying the customer's location is not limited to this. The identification unit may identify the customer whose facial image was captured. The customer is identified, for example, by facial recognition. The method of identifying the customer is not limited to this. For example, when multiple customers are staying in the store, the customer whose facial image was captured is identified using the facial image. Then, the output unit 104 can output product information of products recommended to the identified customer to a display device 91 within a predetermined range from the customer's location. For example, the output unit 104 can output product information to a display device 91 close to the customer's location.

[0082] The determination unit 103 may determine the display device 91 to output the product information according to the position of the recommended product. That is, the output unit 104 may output the product information to the display device 91 according to the display position of the recommended product. The determination unit 103 determines the recommended product. Then, the output unit 104 can output the product information to the display device 91 installed on the shelf on which the recommended product is displayed.

[0083] The determination unit 103 may determine the display device 91 to output product information on, depending on the position of the recommended product. For example, the determination unit 103 may determine that the product information of the recommended product is output to a display device 91 installed within a predetermined range from the display position of the recommended product. The predetermined range is, for example, a range around the display position of the recommended product. The predetermined range is not limited to this. The predetermined range may be any range close to the display position of the recommended product.

[0084] The determination unit 103 may determine the display device 91 to output product information on, depending on the location of the customer and the location of the recommended product. The determination unit 103 may determine to output product information of the recommended product to a display device 91 installed within a predetermined range from the location of the customer identified by the facial image of the customer in the store and within a predetermined range from the display location of the recommended product. For example, when the customer is near the display location of the recommended product, the output unit 104 may output product information to a display device 91 installed on the display shelf of the recommended product. The location of the customer is identified by the method described above.

[0085] The method of determining the display device 91 to which the output unit 104 outputs the product information is not limited to these. Also, examples of the display device 91 to which the output unit 104 outputs the product information are not limited to these.

[0086] Here, an example of the display screen of the display device 91 will be described with reference to the drawings. The output unit 104 outputs product information to the display device 91. The display device 91 then displays the received product information on the display screen. Figures 4 to 6 show examples of the display screen of the display device 91.

[0087] FIG. 4 is an example of a display screen displaying product information for a recommended product. FIG. 4 is an example of a display screen when a refresh drink has been selected as the recommended product. Referring to FIG. 4, the display screen displays an image, name, and price of the recommended product as product information. In addition, in the example of FIG. 4, a catchphrase for the recommended product is displayed as product information. Furthermore, coupon information related to the recommended product is displayed as product information. Customers can obtain product information for the recommended products by checking a display screen such as the example shown in FIG. 4.

[0088] In the example of Figure 4, a "Use" button is displayed on the display screen. When a customer wants to use a coupon, they press the "Use" button. By pressing the "Use" button, the customer can use the coupon. When a customer presses the "Use" button, the customer's information is linked to the coupon. Then, when the customer registers a recommended product, the coupon content is reflected. In other words, the customer can receive a discount on the recommended product.

[0089] The indication of intent to use does not have to be a button. After pressing a button, the coupon can be used by scanning a QR code (registered trademark) or barcode. The coupon can also be printed. The method of providing the coupon to the customer is not limited to these. It is sufficient that the coupon that the customer has indicated their intent to use can be used.

[0090] Fig. 5 is an example of a display screen displaying product information for a recommended product and a product recommended as a set with the recommended product. In the example of Fig. 5, the recommended product is a sandwich, and the product recommended as a set is a refreshing drink. In the example of Fig. 5, the product information for the recommended product includes an image, name, and price of the recommended product. Also in Fig. 5, the product information for the product recommended as a set includes an image, name, and price of the product. Also, the product information for the product recommended as a set includes an image, name, and price of the product.

[0091] In the example of FIG. 5, a "Map" button is displayed below the products recommended as a set. When the customer presses the "Map" button, the output unit 104 may transition the display screen of the display device 91. As a result of the transition of the display screen, product information different from the product information displayed on the display screen before the transition is displayed. In the example shown in FIG. 5, when the "Map" button is pressed, a map of the products is displayed as product information. In the example shown in FIG. 5, a map of the products recommended as a set is displayed. The displayed map may represent the locations of the recommended products. The displayed map may also represent the locations of the products recommended as a set. The transition of the display screen may be performed by an operation other than pressing a button. The display screen may also transition after a predetermined time has elapsed. The method of displaying product information is not limited to these.

[0092] FIG. 6 is an example of a display screen displaying a map of the inside of a store. In the example shown in FIG. 6, a map showing the display location of a recommended product, a refreshing drink, is displayed. Also, in the example shown in FIG. 6, a route from the customer's current location to the display location of the recommended product is displayed on the map. FIG. 6 is an example of a display screen displayed on a display device 91 installed at the entrance of the store. Therefore, a route from the entrance of the store to the display location of the recommended product is displayed. The method of displaying the map and the method of displaying the route are not limited to these.

[0093] Another example of the route display screen will be described. An example will be described in which the output unit 104 outputs product information for the recommended product and the product recommended as a set. In this case, it is assumed that the output unit 104 outputs the product information to a display device 91 installed on the display shelf of the recommended product. At this time, the customer is near the display position of the recommended product. Therefore, the output unit 104 may display the route from the display position of the recommended product, i.e., from the vicinity of the customer's position, to the display position of the product recommended as a set.

[0094] An example of a route displayed when multiple recommended products are determined by the determination unit 103 will be described below. In this case, for example, a route passing through the display positions of multiple recommended products may be displayed. A route passing through the display positions of multiple recommended products may be a route that passes through the display positions of the recommended products in order of proximity from the position of the display device 91 where the route is displayed. Furthermore, a route passing through the display positions of multiple recommended products may be a route according to the characteristics of the recommended products. For example, if the recommended products include a frozen product, a route that arrives last at the display position of the frozen product may be displayed. Furthermore, for example, if the recommended products include a product that is heavier than a predetermined weight, a route that arrives last at the display position of the product may be displayed. The route is not limited to these. The order in which multiple recommended products are passed through may be determined by the determination unit 103, for example.

[0095] The output unit 104 may further output trends in products purchased at the store. For example, there may be a bias in the emotions of customers using the store depending on the store. As a result, the products purchased may differ depending on the store. For example, sweets may be frequently purchased by customers who are feeling happy. Also, cup ramen may be sold in large quantities by customers who are feeling impatient. Therefore, the analysis unit 105 analyzes the trends in products purchased at a store based on information about products purchased at a store and the customer's emotions estimated at the time of purchasing the products. Then, the output unit 104 may output the trends in products purchased at the store. The output unit 104 outputs the trends, which are the analysis results, to, for example, a store terminal (not shown). The store terminal is a terminal used by a store clerk. The store terminal may be, for example, a PC, tablet, or POS terminal located in the store. Examples of the store terminal are not limited to these. The store terminal may be, for example, a terminal installed at a base that manages multiple stores. The store terminal may also be a device having other functions, such as inventory management. In other words, the store terminal may be a terminal that can be used by the person who manages the store.

[0096] The analysis unit 105 acquires information about products purchased at a store and estimated customer emotions when the products were purchased. The information about products purchased at a store and the customer emotions when the products were purchased are stored, for example, in a database 92. The analysis unit 105 then analyzes trends in the products purchased for each store. The trends represent the sales of the products at the store. The analysis by the analysis unit 105 uses publicly known techniques. Using customer emotions in the analysis makes it possible to explain the reasons for the sales of products. For example, an analysis result may be obtained that energy drinks are frequently purchased by customers who feel tired at a certain store. This analysis result allows the store to understand why energy drinks are selling well at the store. As a result, the store can use the analysis result as a reference for the products it carries. For example, the store may increase the variety of energy drinks or increase the number of products it carries that target customers who feel tired.

[0097] Trends in product purchases may be analyzed for each time period. The customer demographics visiting a store may differ depending on the time period. Therefore, the analysis unit 105 acquires information on purchased products and estimated customer emotions at the time of purchasing the products for each time period. The analysis unit 105 then analyzes trends in product purchases for each time period. Here, a time period is a predetermined time span. For example, a time period is a time span divided into one-hour intervals. Examples of time periods are not limited to this. A time period may be any time span suitable for analysis.

[0098] Trends in purchased products may be analyzed for each store location. Store locations are classified by the surrounding environment of the store. For example, the emotions of customers visiting a store may differ depending on the surrounding environment of the store. As a specific example, in a store located in an office district, many customers may feel tired or frustrated, whereas in a store located in a residential area, many customers may feel happy. Therefore, the analysis unit 105 acquires information on purchased products and estimated customer emotions when purchasing the products for multiple stores. Then, the analysis unit 105 analyzes trends in purchased products for each store location.

[0099] With reference to FIG. 7, the operation of the product recommendation device 10, which includes an image acquisition unit 101, an estimation unit 102, a determination unit 103, and an output unit 104, will be described. FIG. 7 is a flowchart showing the operation of the product recommendation device 10. In step S101, the image acquisition unit 101 acquires a facial image of a customer who visits a store. In step S102, the estimation unit 102 estimates the customer's emotion at the time of visiting the store from the facial image acquired by the image acquisition unit 101. In step S103, the determination unit 103 determines recommended products according to the emotion at the time of visiting the store, based on information regarding the relationship between products included in the customer's purchase history and the emotion estimated when the product was purchased. In step S104, the output unit 104 outputs product information of the recommended products. Then, the product recommendation device 10 ends its operation.

[0100] When the product recommendation device 10 includes the analysis unit 105, the analysis unit 105 analyzes in advance the relationship between the products included in the customer's purchase history and the emotion estimated at the time of purchasing the products. Then, in step S103 of the flowchart shown in Fig. 7, the determination unit 103 can determine the recommended products based on the analysis result of the relationship between the products included in the customer's purchase history and the emotion estimated at the time of purchasing the products.

[0101] In this embodiment, in the product recommendation device 10, the estimation unit 102 estimates the emotion of a customer when the customer visits the store from a facial image of the customer. Then, the determination unit 103 determines recommended products according to the emotion at the time of the store based on information regarding the relationship between the product included in the customer's purchase history and the emotion estimated when the product was purchased. The products purchased by a customer when visiting the store may differ depending on the emotion of the customer when visiting the store. Furthermore, the products purchased when each customer is feeling a certain emotion may differ from customer to customer. In other words, even when each customer is feeling the same emotion, the products purchased may differ from customer to customer. Therefore, the determination unit 103 can determine recommended products based on information regarding the relationship between the product previously purchased by the customer when visiting the store and the emotion estimated when the product was purchased. Furthermore, the determination unit 103 determines recommended products according to the emotion of the customer when visiting the store. Therefore, the product recommendation device 10 can recommend products according to the emotion of the customer.

[0102] When the product recommendation device 10 recommends to a customer a recommended product according to the customer's emotion, the customer may purchase the recommended product. As a result, the store may see an increase in sales. Furthermore, when the recommended product is recommended, the customer may be able to learn about products that match their emotion. For example, a customer may not be aware of their own emotion. Furthermore, a customer may not be aware of what product they purchase depending on their emotion. Therefore, the product recommendation device 10 determines a recommended product using products that the customer has purchased in the past and the emotion estimated at the time of purchasing the product, thereby making it possible to recommend products that better match the customer's emotion. As a result, it becomes possible to improve customer satisfaction with the store.

[0103] In the product recommendation device 10 according to this embodiment, the facial image is a facial image used for facial authentication to enter a store. In some stores, facial authentication is performed on customers to enter the store. The image acquisition unit 101 can acquire a facial image captured for facial authentication. Then, the determination unit 103 can determine recommended products using the customer's emotions estimated from the facial image. In other words, the product recommendation device 10 does not need to acquire a new facial image to determine recommended products. Furthermore, customers can be saved the trouble of capturing a facial image separately from facial authentication.

[0104] The image acquisition unit 101 acquires an additional facial image of the customer at a timing different from the timing at which the facial image used for facial authentication was acquired. Then, the estimation unit 102 further estimates the customer's emotions at the time of visiting the store from the additional facial image. When capturing a facial image for facial authentication, the customer may not have a natural facial expression. For example, the customer may have a formal or nervous facial expression when the facial image is captured. Therefore, it may be difficult for the estimation unit 102 to accurately estimate the customer's emotions. Therefore, the image acquisition unit 101 acquires an additional facial image of the customer at a timing different from the timing at which the facial image used for facial authentication was acquired. Then, the estimation unit 102 estimates the customer's emotions at the time of visiting the store from the additional facial image, thereby enabling the estimation unit 102 to estimate the customer's emotions from the customer's natural facial expression. In other words, the accuracy of emotion estimation by the estimation unit 102 can be improved.

[0105] Here, the additional facial image is, for example, an image acquired after facial authentication at the entrance of a store. A customer may return to a natural expression after facial authentication is complete. Therefore, the estimation unit 102 can estimate the emotion from the customer's natural expression. Also, the additional facial image is, for example, an image acquired before facial authentication at the entrance of a store. Furthermore, the additional facial image may be a facial image of a customer inside a store. By the estimation unit 102 estimating the emotion using the above-mentioned additional facial image, it becomes possible to improve the accuracy of emotion estimation.

[0106] Furthermore, when the image acquisition unit 101 acquires a facial image of a customer in the store, the output unit 104 outputs product information about the recommended products to a display device 91 within a predetermined range from the customer's location in the store, as determined by the facial image of the customer. The display device 91 within the predetermined range from the customer's location is, for example, a display device 91 that is close to the customer's location. That is, the output unit 104 outputs product information about the recommended products to a display device 91 that is close to the customer's location, as determined by the facial image of the customer in the store. When a customer is browsing products in the store, outputting product information to a display device 91 within a predetermined range from the customer's location may increase the likelihood that the customer will view the product information. As a result, the customer may view the product information about the recommended products and purchase the recommended products. This may result in increased sales for the store. Furthermore, when product information about the recommended products is displayed on a predetermined display device 91, the customer may need to move around the store to view the product information. However, outputting the product information to a display device 91 close to the customer's location eliminates the need for the customer to move around the store to view the product information about the recommended products.

[0107] In this embodiment, the output unit 104 of the product recommendation device 10 outputs product information of the recommended products to a display device 91 installed within a predetermined range from the display position of the recommended products. For example, outputting the product information of the recommended products to a display device 91 installed near the display position of the recommended products can increase the likelihood that customers will purchase the recommended products. Even if a customer sees the product information of the recommended products displayed on the display device 91, if the display position of the recommended products is far away, they may not purchase the recommended products. On the other hand, if the recommended products are displayed nearby, customers who see the product information of the recommended products may be more likely to purchase the recommended products. Therefore, outputting the product information of the recommended products to a display device 91 installed near the display position of the recommended products can increase store sales. Furthermore, customers who intend to purchase the recommended products can avoid the hassle of traveling to the display position of the recommended products.

[0108] In this embodiment, the product recommendation device 10 has a determination unit 103 that determines recommended products according to the emotions and attributes of a customer at the time of the store visit, based on information regarding the relationship between products included in the purchase history of the customer who previously visited the store, the emotions estimated at the time of the product purchase, and the attributes of the customer who previously visited the store. The determination unit 103 further determines recommended products using the customer's attributes, so that the determination unit 103 can determine recommended products even for a customer visiting the store for the first time. A customer visiting a store for the first time may not have a purchase history with the store. However, by determining recommended products using the customer's attributes, it becomes possible to determine recommended products according to the emotions and attributes of the customer, even for a customer visiting the store for the first time.

[0109] At this time, the estimation unit 102 estimates the attributes of the customer. For example, a customer visiting a store for the first time may not have registered their attribute information in advance. However, by having the estimation unit 102 estimate the attributes of the customer from a facial image of the customer, the determination unit 103 can determine recommended products for the customer even if their attribute information has not been registered in advance. Furthermore, the customer can be spared the trouble of registering their attribute information. In other words, convenience for the customer can be improved.

[0110] In this embodiment, the product recommendation device 10 has a determination unit 103 that determines products to be recommended in a set with the recommended product. Then, the output unit 104 outputs product information about the recommended product and the products recommended in a set with the recommended product. When the determination unit 103 determines not only the recommended product but also the products recommended in the set, a customer may purchase both the recommended product and the products recommended in the set. As a result, the store may see an increase in sales.

[0111] [Hardware configuration example] 8 is a diagram illustrating an example of the hardware configuration of the product recommendation device 20 in the present disclosure. The product recommendation device 20 is realized by a computer. The product recommendation device 20 is an example of the product recommendation device 10 realized by a computer.

[0112] The product recommendation device 20 includes a processor 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a storage device 204 such as a hard disk for storing programs, an input / output interface 205 for inputting and outputting data, and a communication interface 206 for network connection. Each component is connected via a bus 207.

[0113] The processor 201 runs an operating system to control the entire computer. Examples of the processor 201 include a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit). The processor 201 loads a program stored in, for example, a ROM 202 or a storage device 204. The processor 201 then executes each process coded in the program. The processor 201 may execute the processes or instructions in the illustrated flowchart based on the program.

[0114] The ROM 202 stores application programs, programs according to the embodiments, etc. The RAM 203 is used as a work area for the processor 201.

[0115] Examples of the storage device 204 include semiconductor memory such as flash memory, a hard disk drive (HDD), etc. The storage device 204 stores, for example, an operating system (OS) program, application programs, and programs according to each embodiment.

[0116] The input / output interface 205 is connected to peripheral devices (not shown) via a wired network or a wireless network.

[0117] The communication interface 206 is connected to a communication network (not shown), such as a LAN (Local Network) or a WAN (Wide Area Network), via a wireless or wired network. The communication network may be configured by a plurality of communication networks. This allows the computer to be connected to an external device via the communication network. The product recommendation device 20 may have components other than those shown in FIG. 8. For example, the product recommendation device 20 may have a drive device or the like. For example, the processor 201 may be attached to a drive device or the like and read out programs and data stored in a non-transitory tangible recording medium into the RAM 203.

[0118] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, the configurations in the respective embodiments can be combined with each other without departing from the scope of the present disclosure.

[0119] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0120] (Appendix 1) image acquisition means for acquiring a facial image of a customer who visits the store; an estimation means for estimating the emotion of the customer when he or she visited the store from the facial image; a determining means for determining a recommended product according to the emotion at the time of the customer's visit based on information relating to a relationship between a product included in the customer's purchase history and the emotion estimated at the time of the purchase of the product; an output means for outputting product information of the recommended products; A product recommendation device comprising:

[0121] (Appendix 2) The face image is a face image used for face authentication for entering the store. 1. A product recommendation device as described in Appendix 1.

[0122] (Appendix 3) the image acquisition means acquires an additional face image of the customer at a timing different from the timing at which the face image used for the face authentication is acquired; The estimation means further estimates the emotion of the customer at the time of visiting the store from the additional face image. Item recommendation device as described in Appendix 2.

[0123] (Appendix 4) The additional facial image is an image acquired after the facial authentication at the entrance of the store. Item recommendation device as described in Appendix 3.

[0124] (Appendix 5) the image acquisition means acquires a facial image of the customer in the store; The output means outputs product information of the recommended products to a display device within a predetermined range from the position of the customer identified by the facial image of the customer in the store. 1. A product recommendation device as described in Appendix 1.

[0125] (Appendix 6) The output means outputs product information of the recommended products to the display devices installed within a predetermined range from the display positions of the recommended products. Item recommendation device as described in Appendix 5.

[0126] (Appendix 7) The estimation means estimates attributes of the customer, The determining means determines the recommended products according to the emotions at the time of the store visit and the attributes of the customer based on information regarding the relationship between the products included in the purchase history of the customer who visited the store in the past, the emotions estimated when the products were purchased, and the attributes of the customer who visited the store in the past. 1. A product recommendation device as described in Appendix 1.

[0127] (Appendix 8) The determining means determines a product to be recommended in combination with the recommended product; The output means outputs product information of the recommended product and a product recommended as a set with the recommended product. 1. A product recommendation device as described in Appendix 1.

[0128] (Appendix 9) Acquires facial images of customers who visit the store, Estimating the emotion of the customer at the time of visiting the store from the facial image; determining recommended products according to the customer's emotions at the time of the store visit based on information relating to the relationship between the products included in the customer's purchase history and the emotions estimated at the time of the purchase of the products; outputting product information of the recommended products; Product recommendation methods.

[0129] (Appendix 10) Acquires facial images of customers who visit the store, Estimating the emotion of the customer at the time of visiting the store from the facial image; determining recommended products according to the customer's emotions at the time of the store visit based on information relating to the relationship between the products included in the customer's purchase history and the emotions estimated at the time of the purchase of the products; outputting product information of the recommended products; A product recommendation program that causes a computer to execute processing.

[0130] (Appendix 11) Acquires facial images of customers who visit the store, Estimating the emotion of the customer at the time of visiting the store from the facial image; determining recommended products according to the customer's emotions at the time of the store visit based on information relating to the relationship between the products included in the customer's purchase history and the emotions estimated at the time of the purchase of the products; outputting product information of the recommended products; A recording medium that stores a program that causes a computer to execute a process.

[0131] Some or all of the configurations described in Supplementary Notes 2-8, which are dependent on Supplementary Note 1, may also be dependent on Supplementary Notes 9-11 in the same dependent relationship as Supplementary Note 2-8. Not limited to Supplementary Notes 1 and 9-11, some or all of the configurations described as Supplements may also be dependent on various hardware, software, various recording devices for recording software, or systems, within the scope of each of the above-mentioned embodiments. [Explanation of symbols]

[0132] 10, 20 Product recommendation device 101 Image acquisition unit 102 Estimation part 103 Decision Section 104 Output section 105 Analysis Department 201 processor 202 ROM 203 RAM 204 Storage device 205 Input / Output Interface 206 Communication Interface 207 Bus 90 Camera 91 Display device 92 databases

Claims

1. image acquisition means for acquiring a facial image of a customer who visits the store; an estimation means for estimating the emotion of the customer when he or she visited the store from the facial image; a determining means for determining a recommended product according to the emotion at the time of the customer's visit based on information relating to a relationship between a product included in the customer's purchase history and the emotion estimated at the time of the purchase of the product; an output means for outputting product information of the recommended products; A product recommendation device comprising:

2. The face image is a face image used for face authentication for entering the store. The product recommendation device according to claim 1 .

3. the image acquisition means acquires an additional face image of the customer at a timing different from the timing at which the face image used for the face authentication is acquired; The estimation means further estimates the emotion of the customer at the time of visiting the store from the additional face image. The product recommendation device according to claim 2 .

4. The additional facial image is an image acquired after the facial authentication at the entrance of the store. The product recommendation device according to claim 3 .

5. the image acquisition means acquires a facial image of the customer in the store; The output means outputs product information of the recommended products to a display device within a predetermined range from the position of the customer identified by the facial image of the customer in the store. The product recommendation device according to claim 1 .

6. The output means outputs product information of the recommended products to the display devices installed within a predetermined range from the display positions of the recommended products. The product recommendation device according to claim 5 .

7. The estimation means estimates attributes of the customer, The determining means determines the recommended products according to the emotions at the time of the store visit and the attributes of the customer based on information regarding the relationship between the products included in the purchase history of the customer who visited the store in the past, the emotions estimated when the products were purchased, and the attributes of the customer who visited the store in the past. The product recommendation device according to claim 1 .

8. The determining means determines a product to be recommended in combination with the recommended product; The output means outputs product information of the recommended product and a product recommended as a set with the recommended product. The product recommendation device according to claim 1 .

9. Acquires facial images of customers who visit the store, Estimating the emotion of the customer at the time of visiting the store from the facial image; determining recommended products according to the customer's emotions at the time of the store visit based on information relating to the relationship between the products included in the customer's purchase history and the emotions estimated at the time of the purchase of the products; outputting product information of the recommended products; Product recommendation methods.

10. Acquires facial images of customers who visit the store, Estimating the emotion of the customer at the time of visiting the store from the facial image; determining recommended products according to the customer's emotions at the time of the store visit based on information relating to the relationship between the products included in the customer's purchase history and the emotions estimated at the time of the purchase of the products; outputting product information of the recommended products; A product recommendation program that causes a computer to execute processing.

Images