Information processing device, information processing method, and program

Abstract

This information processing device (10) comprises: an authentication control unit (13) which, when a user is positioned at the front of mirror, acquires a collation result between face feature information extracted from a captured image of at least the face of the user, and pre-registered face feature information about a plurality of people; a selection unit (14) which when the collation result indicates that the face authentication is successful, selects, as a recommendation commodity, at least one commodity from among a plurality of commodities on the basis of pre-registered user information related to the user; and a display control unit (15) which outputs the display information related to the recommendation commodity to a display device provided in association with the mirror.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing system, an information processing method, and a non-transitory computer-readable medium, and more particularly to an information processing device, an information processing system, an information processing method, and a non-transitory computer-readable medium that provide recommendation information to a user. [Background technology]

[0002] Methods for recommending cosmetics suited to a user have been proposed. For example, Patent Document 1 discloses a makeup support device that, when identity authentication of a first user is successful, selects similar users based on the preferences of the first user and displays, on a display device, cosmetic information related to the cosmetics used by the similar users in their makeup. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 029963 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a need to improve sales promotion effectiveness by making it possible to recommend cosmetics that suit users without requiring the user to specifically log in with the intention of purchasing cosmetics. This applies not only to cosmetics but also to other products.

[0005] In view of the above-mentioned problems, an object of the present disclosure is to provide an information processing device, an information processing system, an information processing method, and a non-transitory computer-readable medium that can improve sales promotion effectiveness. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present disclosure includes: an authentication control means for acquiring a matching result between facial feature information extracted from a captured image of at least the face of the user and facial feature information of a plurality of people registered in advance, when the user is positioned in front of the mirror; a selection means for selecting at least one product from a plurality of products as a recommended product based on pre-registered user information related to the user when the matching result indicates that face authentication has been successful; a display control means for outputting display information related to the recommended products to a display device provided in correspondence with the mirror; Equipped with.

[0007] An information processing system according to one aspect of the present disclosure includes: an information processing device; Facial recognition device Equipped with The information processing device includes: an authentication control means for acquiring, when a user is positioned in front of a display device having a mirror attached on the user side, from the face authentication device, a matching result between facial feature information extracted from a captured image of at least the user's face and facial feature information of a plurality of people registered in advance; a selection means for selecting at least one product from a plurality of products as a recommended product based on pre-registered user information related to the user when the matching result indicates that face authentication has been successful; a display control means for outputting display information related to the recommended products to the display device; It has.

[0008] An information processing method according to one aspect of the present disclosure includes: When a user is positioned in front of a mirror, a matching result is obtained between facial feature information extracted from a captured image in which at least the face of the user is captured and facial feature information of a plurality of people registered in advance; If the matching result indicates that face authentication has been successful, select at least one product from a plurality of products as a recommended product based on user information related to the user that has been registered in advance; Display information relating to the recommended products is output to a display device provided in association with the mirror.

[0009] According to one aspect of the present disclosure, there is provided a non-transitory computer-readable medium, comprising: a step of acquiring a matching result between facial feature information extracted from a captured image of at least the face of the user and facial feature information of a plurality of people registered in advance, when the user is positioned in front of the mirror; If the matching result indicates that face authentication has been successful, selecting at least one product from a plurality of products as a recommended product based on user information related to the user that has been registered in advance; outputting display information related to the recommended products to a display device provided in correspondence with the mirror; A program for causing a computer to execute the above is stored. [Effects of the Invention]

[0010] The present disclosure makes it possible to provide an information processing device, an information processing system, an information processing method, and a non-transitory computer-readable medium that can recommend products without requiring special intention from the user, thereby improving sales promotion effectiveness. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first embodiment. [Figure 2] 1 is a flowchart showing the flow of an information processing method according to the first embodiment. [Figure 3] FIG. 10 is a block diagram showing the overall configuration of an information processing system according to a second embodiment. [Figure 4] FIG. 10 is a block diagram showing the configuration of a mirror signage according to a second embodiment. [Figure 5] FIG. 10 is a block diagram showing the configuration of a face authentication device according to a second embodiment. [Figure 6] 10 is a flowchart showing the flow of face information registration processing according to the second embodiment. [Figure 7] 10 is a flowchart showing the flow of face authentication processing according to the second embodiment. [Figure 8] FIG. 10 is a block diagram showing the configuration of a user terminal according to the second embodiment. [Figure 9] FIG. 10 is a block diagram showing the configuration of a server according to a second embodiment. [Figure 10] FIG. 10 is a sequence diagram showing an example of the flow of a user registration process according to the second embodiment. [Figure 11] FIG. 10 is a sequence diagram showing an example of the flow of information provision processing according to the second embodiment. [Figure 12] FIG. 10 is a diagram showing an example of a display on a display device of a mirror signage according to a second embodiment. [Figure 13] FIG. 10 is a diagram showing an example of a display on a display device of a mirror signage according to a second embodiment. [Figure 14] FIG. 10 is a diagram showing an example of a display on a display device of a mirror signage according to a second embodiment. [Figure 15] FIG. 10 is a block diagram showing the configuration of a server according to a third embodiment. [Figure 16] FIG. 11 is a sequence diagram showing a modified example of the flow of the information provision process according to the third embodiment. [Figure 17] FIG. 11 is a diagram showing an example of a captured image according to the third embodiment. [Figure 18] FIG. 11 is a diagram showing an example of a display on a display device of a mirror signage according to a third embodiment. [Figure 19] FIG. 11 is a diagram showing an example of a captured image according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and for clarity of explanation, duplicate explanations will be omitted as necessary.

[0013] <Issues of the embodiment> Here, the problems of the embodiment will be explained again. In department stores and other cosmetics stores, sales staff recommend cosmetics that suit customers. However, due to the spread of infectious diseases, an increasing number of customers are avoiding face-to-face services, and a decline in cosmetics sales at physical stores has become a problem. Because cosmetics are deeply related to each customer's preferences and skin type, if recommendations cannot be made in-store, it is difficult to effectively follow up on customers.

[0014] Therefore, as in the above-mentioned Patent Document 1, a method has been proposed for recommending cosmetics suited to a user. However, if a special intention or operation by the user is required, such as the user having to intentionally log in for the purpose of purchasing cosmetics or requiring complicated input operations, the promotional effect is limited. The need for special operations by the user is also a factor that limits the promotional effect. Therefore, it is desirable to increase the promotional effect by making it possible to recommend cosmetics without requiring special intention or operation by the user.

[0015] The above-mentioned problem is particularly prominent in cosmetics, but is similar to other products. The present embodiment has been made to solve such a problem.

[0016] <Terminology> In the following, the following terms are defined as follows: The "user ID" is information that identifies a user. The "user ID" may be a user name, a user's phone number, a credit card number, or any other identification number. "User information" is information related to a user. "User information" includes user attribute information. "User information" may also include a user's usage history and user input information. A "product" is any product sold in a physical or online store. A "product" is preferably a cosmetic product, clothing, hat, accessory, bag, shoe, or other product that can be worn by a user. However, a "product" may also be a product that cannot be worn by a user. "Usage history" indicates at least one of the following: a user has visited a physical store that sells a product; an user has ordered or purchased a product at a physical store or online; and an user has requested delivery of a sample product.

[0017] <Embodiment 1> Next, a first embodiment of the present disclosure will be described. FIG. 1 is a block diagram showing a configuration of an information processing device 10 according to the first embodiment. The information processing device 10 is a computer device that recommends products to a user positioned in front of a mirror. Positioning in front of a mirror may mean standing in front of the mirror or sitting in front of the mirror. For example, a user stands in front of a mirror to groom themselves.

[0018] The information processing device 10 is connected to a network (not shown). The network may be wired or wireless. A display device (not shown) associated with a mirror may be connected to the network. The information processing device 10 includes an authentication control unit 13, a selection unit 14, and a display control unit 15.

[0019] The authentication control unit 13 is also called authentication control means. When a user is positioned in front of a mirror, the authentication control unit 13 obtains a comparison result between facial feature information extracted from a captured image of at least the user's face and facial feature information of multiple people registered in advance. In other words, the authentication control unit 13 obtains a face authentication result of the user positioned in front of the mirror. As a result, the authentication control unit 13 identifies the user. For example, the authentication control unit 13 identifies a user ID.

[0020] The selection unit 14 is also called a selection means. When the matching result acquired by the authentication control unit 13 indicates that face authentication has been successful, the selection unit 14 selects at least one product from multiple products as a recommended product based on user information. The user information is associated with the user and registered in advance in a user database (DB) (not shown). Information on multiple products is also registered in advance in a product database (DB) (not shown).

[0021] The display control unit 15 is also called a display control means. The display control unit 15 outputs display information related to the recommended product selected by the selection unit 14 to a display device provided in association with the mirror described above. The display information may be product information about the recommended product, information indicating where to purchase the recommended product, or information for receiving delivery or an order for a sample of the recommended product. The display information may also be a simulation image of what would happen if the user used the recommended product. "Output" may mean transmitting, or transmitting and displaying.

[0022] 2 is a flowchart showing the flow of the information processing method according to the first embodiment. First, the authentication control unit 13 of the information processing device 10 determines whether or not the user is located in front of a mirror (S10). For example, the authentication control unit 13 may determine that the user is located in front of the mirror when a human region of a predetermined size or larger is detected from an image captured by a camera (not shown) capturing the scenery around the mirror. Alternatively, the authentication control unit 13 may determine that the user is located in front of the mirror when a human region of a predetermined size or larger is detected from the image for a predetermined period of time or longer. Alternatively, the authentication control unit 13 may determine that the user is located in front of the mirror when a predetermined input operation is performed or when a human is detected by a human sensor.

[0023] If it is determined that the user is located in front of the mirror (Yes in S10), the authentication control unit 13 acquires a matching result between facial feature information extracted from the captured image of the user and facial feature information of multiple people registered in advance (S11). For example, the authentication control unit 13 may acquire the matching result (i.e., the facial authentication result) by performing facial authentication using the captured image of the user. Alternatively, for example, the authentication control unit 13 may acquire the matching result by transmitting the captured image of the user or the facial feature information extracted from the captured image to a face authentication device (not shown) connected via a network and receiving the matching result from the face authentication device.

[0024] Next, the authentication control unit 13 determines whether or not the face authentication has been successful (S12). Successful face authentication means that the face feature information of the user matches that of any registered user. A match may mean that the degree of match between the face feature information is equal to or greater than a predetermined value.

[0025] If the face authentication is not successful (No in S12), the information processing device 10 ends the process.

[0026] On the other hand, if the face authentication is successful (Yes in S12), the selection unit 14 selects recommended products based on pre-registered user information (S13). As an example, the selection unit 14 analyzes the user's preferences or inclinations based on the user information and selects, as recommended products, products having product attributes that match the preferences or inclinations from among a plurality of predetermined products. As another example, the selection unit 14 analyzes the user's preferences or inclinations based on the user information from among a plurality of predetermined products and selects, as recommended products, products from manufacturers or brands that match the preferences or inclinations. As another example, the selection unit 14 selects, as recommended products, products having product attributes that match the user's skin type or skin color included in the user information from among a plurality of predetermined products.

[0027] Next, the display control unit 15 outputs display information related to the recommended product to the display device (S14).

[0028] As described above, according to the first embodiment, the information processing device 10 performs face authentication to verify the identity of the user when the user uses the mirror, and provides the user with personalized information about recommended products. The user does not need to have any special intention or operation, such as intentionally logging in, to obtain information about recommended products. This can enhance the effectiveness of sales promotion. Furthermore, the user does not need to perform any special operation, such as inputting information each time. This can also enhance the effectiveness of sales promotion.

[0029] <Embodiment 2> Next, a second embodiment of the present disclosure will be described. Fig. 3 is a block diagram showing the overall configuration of an information processing system 1000 according to the second embodiment. The information processing system 1000 is a computer system that recommends products to a user standing in front of mirror signage including a mirror. In the second embodiment, the "products" are cosmetics, as an example.

[0030] The information processing system 1000 includes a face authentication device 100, an information processing device (hereinafter referred to as a server) 200, a user terminal 300, and a mirror signage 400. The devices and terminals are connected to each other via a network N. The network N is a wired or wireless communication line.

[0031] The user terminal 300 is an information terminal used by a user. The user terminal 300 transmits a user registration request to the server 200. As a result, the user's facial feature information is registered and a user ID is issued. The user terminal 300 also transmits user information to the server 200, causing the server 200 to register the user information in association with the user ID.

[0032] The mirror signage 400 is a display device with a mirror. The mirror signage 400 displays information while functioning as a mirror. The mirror signage 400 may be installed at a bathroom sink in a user's home. The mirror signage 400 may also be installed in a place used by an unspecified number of people, such as a shared space in a company, a convenience store, a bathroom sink in a public restroom (especially a women's restroom), inside an elevator, or in a waiting area. The mirror signage 400 may also be installed in a corner of a cosmetics counter in a department store. The mirror signage 400 is further provided with a camera (not shown) that captures at least the face of a user U positioned in front of the mirror. The mirror signage 400 then transmits the captured image to the server 200 via the network N. As a result, the server 200 starts an information provision process if the captured image includes a person area.

[0033] The server 200 is an example of the information processing device 10 described above. The server 200 receives a captured image from the mirror signage 400, and performs information provision processing if it determines that a person is included in the captured image. In the information provision processing, the server 200 determines recommended products based on user information of the user U identified by face authentication based on the captured image, and generates display information related to the recommended products. The server 200 then transmits the display information to the mirror signage 400. The mirror signage 400 receives the received display information from the server 200 and displays the display information.

[0034] The face authentication device 100 is a computer device that stores facial feature information of multiple people. The face authentication device 100 also has a face authentication function that, in response to a face authentication request received from an external device, matches the facial image or facial feature information included in the request with the facial feature information of each user. In the second embodiment, the face authentication device 100 registers the facial feature information of a user when the user is registered. The face authentication device 100 then acquires a captured image of the user standing in front of the mirror signage 400 from the mirror signage 400 via the server 200, and performs face authentication using the facial area in the captured image. The face authentication device 100 then returns the matching result (face authentication result) to the server 200.

[0035] 4 is a block diagram showing the configuration of a mirror signage 400 according to the second embodiment. The mirror signage 400 includes a half mirror 410 as an example of a mirror, a display device 420, and a camera 440.

[0036] The half mirror 410 is a half mirror. However, instead of this, the half mirror 410 may be another type of beam splitter. The half mirror 410 functions as a mirror because it reflects at least a portion of the light toward the user U positioned in front. This allows the user U to see his or her own appearance. Furthermore, the half mirror 410 transmits at least a portion of the light from the display device 420 provided on the rear side, so the user U can view the display information displayed on the display device 420.

[0037] The camera 440 is connected to the display device 420. For example, the camera 440 is provided above the half mirror 410 and is installed so as to capture an image of the scenery in front of the half mirror 410. When the user U is standing in front of the half mirror 410, the captured image includes at least the body region of the user U, including the face. The camera 440 supplies the captured image to the display device 420.

[0038] The display device 420 is a device including a display unit such as a liquid crystal display or an organic electroluminescence (EL) display. Alternatively, the display device 420 may be a tablet terminal having a display unit. The display device 420 is provided on the rear side of the half mirror 410. The display device 420 displays display information received from the server 200 via the network N. The display device 420 may rotate a captured image of the user U positioned in front of the half mirror 410 from the camera 440 left and right, thereby causing the mirror signage 400 to function as a mirror. The display device 420 may be disposed over the entire area of ​​the half mirror 410 or in an upper area of ​​the half mirror 410. For example, the display device 420 may be disposed more than 100 cm above the bottom of the half mirror 410, so that display information is displayed on approximately the upper half of the body. Alternatively, the display device 420 may be disposed more than 180 cm above the bottom of the half mirror 410, so that display information is displayed near the head.

[0039] In addition to the display unit, the display device 420 may include a communication unit (not shown) that is a communication interface with the network N, and a control unit (not shown) that controls the hardware that the display device 420 has.

[0040] FIG. 5 is a block diagram showing the configuration of a face authentication device 100 according to the second embodiment. The face authentication device 100 includes a face information database (DB) 110, a face detection unit 120, a feature point extraction unit 130, a registration unit 140, and an authentication unit 150. The face information DB 110 stores a user ID 111 and facial feature information 112 corresponding to the user ID in association with each other. The facial feature information 112 is a collection of feature points extracted from a face image and is an example of face information. The face authentication device 100 may delete the facial feature information 112 from the facial feature DB 110 in response to a request from a registered user of the facial feature information 112. Alternatively, the face authentication device 100 may delete the facial feature information 112 after a certain period of time has elapsed since the registration of the facial feature information 112.

[0041] The face detection unit 120 detects a face area included in a registration image for registering face information and supplies the detected face area to the feature point extraction unit 130. The feature point extraction unit 130 extracts feature points from the face area detected by the face detection unit 120 and supplies the facial feature information to the registration unit 140. The feature point extraction unit 130 also extracts feature points included in a captured image received from the server 200 and supplies the facial feature information to the authentication unit 150.

[0042] The registration unit 140 issues a new user ID 111 when registering facial feature information. The registration unit 140 associates the issued user ID 111 with facial feature information 112 extracted from the registered image and registers them in the facial information DB 110. The authentication unit 150 performs facial authentication using the facial feature information 112. Specifically, the authentication unit 150 compares the facial feature information extracted from the captured image with the facial feature information 112 in the facial information DB 110. The authentication unit 150 returns a match of the facial feature information to the server 200. The match of the facial feature information corresponds to the success or failure of authentication. The match of the facial feature information (match present) refers to a case where the degree of match is equal to or greater than a predetermined value.

[0043] FIG. 6 is a flowchart showing the flow of face information registration processing according to the second embodiment. First, the face authentication device 100 acquires a registration image of the user U included in a face registration request (S21). For example, the face authentication device 100 receives the face registration request from the server 200, which has received the user registration request from the user terminal 300, via the network N. Note that the face authentication device 100 may also receive the face registration request directly from the user terminal 300. Next, the face detection unit 120 detects a face area included in the registration image (S22). Next, the feature point extraction unit 130 extracts feature points from the face area detected in step S22 and provides facial feature information to the registration unit 140 (S23). Finally, the registration unit 140 issues a user ID 111 and associates the user ID 111 with facial feature information 112, and registers the user ID 111 in the face information DB 110 (S24). The face authentication device 100 may receive the face feature information 112 from the face registration request source, and register it in the face information DB 110 in association with the user ID 111.

[0044] FIG. 7 is a flowchart showing the flow of face authentication processing according to the second embodiment. First, the feature point extraction unit 130 acquires facial feature information for authentication (S31). For example, the face authentication device 100 receives a face authentication request from the server 200 via the network N and extracts facial feature information from the captured image included in the face authentication request as in steps S21 to S23. Alternatively, the face authentication device 100 may receive the facial feature information from the server 200. Next, the authentication unit 150 compares the acquired facial feature information with the facial feature information 112 in the face information DB 110 (S32). If the facial feature information matches, that is, if the degree of match between the facial feature information is equal to or greater than a predetermined value (Yes in S33), the authentication unit 150 identifies the user ID 111 of the user whose facial feature information matches (S34). Then, the authentication unit 150 returns to the server 200 the fact that face authentication was successful and the identified user ID 111 as the face authentication result (S35). If there is no matching facial feature information (No in S33), the authentication unit 150 returns a message to the server 200 indicating that the facial authentication has failed as a facial authentication result (S36).

[0045] 8 is a block diagram showing the configuration of a user terminal 300 according to embodiment 2. The user terminal 300 includes a camera 310, a storage unit 320, a communication unit 330, a display unit 340, an input unit 350, and a control unit 360. The camera 310 is an imaging device that takes pictures under the control of the control unit 360. The memory unit 320 is a storage device that stores programs for realizing each function of the user terminal 300. The communication unit 330 is a communication interface with the network N. The display unit 340 is a display device. The input unit 350 is an input device that accepts input from the user. The display unit 340 and the input unit 350 may be configured integrally, such as a touch panel. The control unit 360 controls the hardware that the user terminal 300 has.

[0046] 9 is a block diagram showing the configuration of the server 200 according to the second embodiment. The server 200 includes a storage unit 210, a memory 220, a communication unit 230, and a control unit 240. The storage unit 210 is a storage device such as a hard disk or a flash memory. The storage unit 210 stores a program 211, a user DB 212, and a product DB 213. The program 211 is a computer program that implements the processing of the information processing method according to the second embodiment.

[0047] The user DB 212 stores information related to users. Specifically, the user DB 212 stores user information 2122 in association with a user ID 2121. The user ID 2121 is a user ID issued by the face authentication device 100 when face information is registered. In the second embodiment, the user information 2122 includes attribute information of the user. The attribute information may include age, address, sex, occupation, and annual income. The user information 2122 may also include registration information related to the user's skin type or skin color or the user's usage history. The user information 2122 may also include personal information such as the user's credit card number.

[0048] The product DB 213 stores information about products. Specifically, the product DB 213 stores product information 2132 in association with a product ID 2131. The product ID 2131 is information for identifying a product. The product ID 2131 may be a product name or a product model number. The product information 2132 is information about a product, and may include, for example, manufacturer information, product attributes, and product images. The manufacturer information may be the manufacturer ID of the manufacturer that produces and sells the product, or the ID of the manufacturer's group company. The manufacturer ID and group ID may be the name or identification number. The product attributes are attribute information about the product. For example, the product attributes include the type of product (for example, whether it is skin care or makeup, or whether it is foundation, blush, or lipstick), price, the area to which the product is to be applied (for example, skin, cheek, or lip), or the skin type or skin color suitable for the product (for example, for dry skin or for fair skin). Furthermore, for example, the product attributes may include color information of the product (for example, color names such as light beige or ochre, or RGB information) and texture information (for example, matte or glossy).

[0049] The product information 2132 may also include information on where to order a sample of the product, or where to purchase the product. The ordering information and purchasing information may include, for example, the URL (Uniform Resource Locator) of an online store that accepts orders or purchases.

[0050] The memory 220 is a volatile storage device such as a RAM (Random Access Memory), and is a storage area for temporarily storing information during operation of the control unit 240. The communication unit 230 is a communication interface with the network N.

[0051] The control unit 240 is a processor, i.e., a control device, that controls each component of the server 200. The control unit 240 loads the program 211 from the storage unit 210 into the memory 220 and executes the program 211. As a result, the control unit 240 realizes the functions of a registration unit 241, an image acquisition unit 242, an authentication control unit 243, a selection unit 244, a display control unit 245, and a response processing unit 246.

[0052] The registration unit 241 is also called a registration means. When the registration unit 241 receives a registration image from the user terminal 300, it transmits a face registration request to the face authentication device 100. Then, when the face authentication device 100 registers face information and issues a user ID, the registration unit 241 registers the user ID in the user DB 212. Furthermore, when the registration unit 241 receives a user registration request from the user terminal 300, it registers the user information of the user in the user DB 212 in association with the user ID of the user using the user terminal 300.

[0053] The image acquisition unit 242 is also called an image acquisition means. When the image acquisition unit 242 receives a captured image from the mirror signage 400, the image acquisition unit 242 supplies the captured image to the authentication control unit 243.

[0054] The authentication control unit 243 is an example of the above-mentioned authentication control unit 13. In the second embodiment, when the authentication control unit 243 detects a human region of a predetermined size or larger from a captured image, the authentication control unit 243 starts the following authentication control process. In the authentication control process, the authentication control unit 243 controls face authentication for the face region of user U included in the captured image to identify the user. That is, the authentication control unit 243 causes the face authentication device 100 to perform face authentication for the captured image acquired from the mirror signage 400. For example, the authentication control unit 243 transmits a face authentication request including the acquired captured image to the face authentication device 100 via the network N. The authentication control unit 243 may extract the face region of user U from the captured image and include the extracted image in the face authentication request. The authentication control unit 243 may also extract facial feature information from the facial region and include the facial feature information in the face authentication request. The authentication control unit 243 then receives the face authentication result from the face authentication device 100. As a result, the authentication control unit 243 identifies the user ID of the user.

[0055] The selection unit 244 is an example of the selection unit 14 described above. The selection unit 244 refers to the user DB 212 and extracts, from the user DB 212, the user information associated with the user ID identified by the authentication control unit 243. Then, based on the user information, the selection unit 244 selects recommended products from products of various manufacturers stored in the product DB 213. For example, the selection unit 244 analyzes the user's preferences or inclinations based on attribute information included in the user information. If the user information includes a usage history, the usage history may be included as a basis for the analysis.

[0056] The selection unit 244 then refers to the product DB 213 and selects, as recommended products, products having product attributes that match the analysis results from among products from various manufacturers stored in the product DB 213. If the user information includes a usage history, the selection basis may include a purchase cycle according to the product type and the purchase history of products of that product type. For example, if user U purchased a blush three months ago and the purchase cycle for blush is three months, products of the blush product type may be preferentially selected as recommended products. Furthermore, if the user information includes registered information related to skin type or skin color, the registered information may be included as the selection basis. For example, the selection unit 244 may preferentially select, as recommended products, products having product attributes that match the user's skin type or skin color.

[0057] The selection unit 244 then generates display information based on the product information of the selected recommended product. For example, the selection unit 244 generates the display information based on the product name, manufacturer name, and price information of the selected recommended product, and a product image. Furthermore, for example, if the product information 2132 includes ordering or purchasing information, the selection unit 244 may generate a display image indicating the ordering or purchasing source as the display information. Furthermore, the selection unit 244 may access a website for ordering or purchasing the recommended product, acquire information about the recommended product, and generate the display information based on this information.

[0058] The display control unit 245 is an example of the above-mentioned display control unit 15. The display control unit 245 transmits the display information generated by the selection unit 244 to the mirror signage 400, and causes the mirror signage 400 to display it.

[0059] Response processing unit 246 is also called a response processing means. When a response is received from user U as a result of mirror signage 400 displaying display information, response processing unit 246 executes processing according to the response. For example, the processing according to the response is processing for delivery of a sample of a recommended product using the user's personal information (credit card number and address), ordering of a recommended product, payment processing, or delivery processing.

[0060] FIG. 10 is a sequence diagram showing an example of the flow of user registration processing according to the second embodiment. First, the user terminal 300 photographs the user U (S500) and transmits a user registration request including a registration image generated by photographing the user U to the server 200 (S501). Then, the registration unit 241 of the server 200 incorporates the registration image included in the received user registration request into a face registration request and transmits the request to the face authentication device 100 (S502). Then, the face authentication device 100 registers face information (facial feature information) of the user U based on the registration image included in the received face registration request (S503). Then, the face authentication device 100 notifies the server 200 of the issued user ID (S504). Furthermore, the user terminal 300 accepts input of user information from the user and transmits the user information to the server 200 (S505). The transmitted user information includes user attribute information. The registration unit 241 of the server 200 associates the notified user ID and user information with each other and registers them in the user DB 212 (S506).

[0061] FIG. 11 is a sequence diagram illustrating an example of the flow of information provision processing according to the second embodiment. First, the mirror signage 400 transmits a captured image of the area in front of the half mirror 410 captured by the camera 440 to the server 200 (S510). The image acquisition unit 242 of the server 200 then acquires the captured image. Next, the authentication control unit 243 of the server 200 executes a person detection process to determine whether the captured image includes a person area of ​​a predetermined size or larger (S511). Here, it is assumed that the person area is included. The authentication control unit 243 of the server 200 then transmits a face authentication request for the face area of ​​user U in the captured image to the face authentication device 100 (S512). At this time, the authentication control unit 243 may include the captured image in the face authentication request, or may include an image in which the person area is cropped from the captured image. The face authentication device 100 then performs face authentication on the face area of ​​user U in the face image included in the received face authentication request (S513). Here, it is assumed that a user ID has been successfully authenticated. The face authentication device 100 transmits the face authentication result, including a notification that the face authentication was successful and the user ID, to the server 200 (S514). The authentication control unit 243 of the server 200 identifies the user U based on the user ID included in the face authentication result.

[0062] The selection unit 244 of the server 200 extracts user information associated with the user ID from the user DB 212 (S515). Next, the selection unit 244 selects recommended products from the products stored in the product DB 213 based on the user information (S516). Next, the selection unit 244 generates display information for the recommended products based on at least a part of the product information of the recommended products (S517).

[0063] Next, the display control unit 245 of the server 200 transmits the display information to the mirror signage 400 (S518). As a result, the display device 420 of the mirror signage 400 displays the display information (S519).

[0064] FIG. 12 is a diagram showing an example of the display of the display device 420 of the mirror signage 400 according to the second embodiment. Display information C1 and display information C2 are displayed on the display unit of the display device 420. The display information C1 includes the manufacturer name, product type, and product image of the recommended product. The display information C2 includes a QR code (registered trademark) indicating the URL of a website for applying for sample delivery. The user U can read the QR code using the user terminal 300 and request sample delivery. The request for sample delivery can be accepted by the response processing unit 246. Upon receiving the request, the response processing unit 246 may perform sample delivery processing using address information associated with the user ID in the user DB 212. Therefore, the user does not need to enter an address to request sample delivery.

[0065] The display device 420 may display the photographed image of the user U captured by the camera 440 in a horizontally reversed manner. However, since the figure of the user U is reflected on the mirror signage 400 by the half mirror 410, the display device 420 does not need to display the photographed image of the user U.

[0066] As described above, according to the second embodiment, the server 200 performs face authentication to verify the user's identity as the user uses the mirror, and provides the user with personalized information about recommended products. The user does not need to make any special intentions or operations, such as intentionally logging in, to obtain information about recommended products. Furthermore, the server 200 selects recommended products based on pre-registered user information (or pre-collected user information if the user information is a usage history), so the user does not need to perform any special operations, such as inputting information on the spot. Therefore, the user can easily obtain information about products that suit them while grooming themselves or during a short waiting time. This can enhance the effectiveness of sales promotion.

[0067] The second embodiment can be modified as follows.

[0068] For example, if the product is a cosmetic product, the display control unit 245 may cause the display device 420 of the mirror signage 400 to output display information in a display mode according to the position of a part of the user U's face. One example of the display mode is the display position. The selection unit 244 may determine the display position based on the type of the recommended product (e.g., foundation, lipstick, or cheek) stored in the product DB 213 or information on the part of the face to which the product is to be applied, and the position of the part of the face of the user U included in the captured image.

[0069] 13 is a diagram showing an example of a display on the display device 420 of the mirror signage 400 according to the second embodiment. For example, if the type of recommended product is blush, the display position is a position corresponding to the cheek of the user U included in the captured image. The display device 420 receives the display information and the information on the display position from the server 200 and displays the display information C3 in association with the cheek of the user U on the mirror signage 400 based on the information on the display position. In FIG. 13, the display information C3 includes the manufacturer name, product type, and product image of the recommended product.

[0070] The display information may be a simulation image of what would happen if the user U used the recommended product. In this case, style data for generating the simulation image is stored in association with the product ID in the product DB 213. For example, the style data includes color information or texture information of the product.

[0071] To generate a simulation image, the selection unit 244 reads style data of the recommended products from the product DB 213. Then, based on the photographed image of the user U and the style data, the selection unit 244 generates a simulation image in which the recommended products are applied to the positions of parts of the face of the user U included in the photographed image. Then, the display control unit 245 transmits the simulation image as display information to the mirror signage 400, and causes it to be displayed.

[0072] FIG. 14 is a diagram illustrating an example of a display on the display device 420 of the mirror signage 400 according to the second embodiment. The display device 420 displays display information C4 including a manufacturer name and a product type, display information C5 including a QR code indicating the URL of a purchase website, and a simulation image C6. The simulation image C6 may or may not include a photographed image of the user U. For example, the simulation image C6 may be an image of the cheek when blush is used. In this case, the simulation image C6 of the cheek is displayed on the image of the user reflected on the half mirror 410.

[0073] The user U can read the QR code using the user terminal 300 to place an order and make a payment. The order and payment can be performed by the response processing unit 246. The response processing unit 246 may perform the order and payment process using a credit card number associated with the user ID in the user DB 212. The response processing unit 246 may also perform the product delivery process using address information associated with the user ID. Therefore, the user does not need to input a credit card number or address for the order, payment, and delivery.

[0074] In addition to the above-described display information, an advertisement for a product provider may be displayed on the display device 420. For example, the display control unit 245 of the server 200 may transmit advertising data for the product provider to the mirror signage 400 and cause the advertising data to be displayed on the display device 420.

[0075] <Embodiment 3> Next, a third embodiment of the present disclosure will be described. The third embodiment is characterized in that the server detects an operation from a user and executes a process according to the operation.

[0076] 15 is a block diagram showing the configuration of the server 200a according to the third embodiment. The server 200a has basically the same configuration and functions as the server 200. However, the server 200a differs from the server 200 in that the server 200a includes a storage unit 210a and a control unit 240a instead of the storage unit 210 and the control unit 240.

[0077] The storage unit 210a stores a program 211a instead of the program 211, and a user DB 212a instead of the user DB 212. The program 211a is a computer program in which the processing of the information processing method according to the third embodiment is implemented.

[0078] The user DB 212a differs from the user DB 212 in that it also stores an operation history 2123 and benefit information 2124 in association with a user ID 2121. The operation history 2123 is a history of operations performed by a user in response to a display. The benefit information 2124 is information related to benefits acquired by a user. The benefit information 2124 may be, for example, information related to awarded points, coupons, or an increase in the limit on the number of deliveries of a sample. The benefit information 2124 may also include expiration date information for the benefit.

[0079] The control unit 240a differs from the control unit 240 in that the control unit 240a includes a display control unit 245a and a response processing unit 246a instead of the display control unit 245 and the response processing unit 246.

[0080] The response processing unit 246a detects an operation by a user and executes processing according to the operation, in addition to the functions of the response processing unit 246. In order to execute such processing, the response processing unit 246a may include, for example, an operation detection unit 247, an operation recording unit 248, and a bonus granting unit 249.

[0081] The operation detection unit 247 is also called an operation detection means. The operation detection unit 247 detects an operation by the user U from a captured image of the user U when the mirror signage 400 displays display information. For example, the operation detection unit 247 may recognize that a selection operation has been performed when it detects from the captured image that the user U has held their hand over or pointed their finger at an operation area included in the display information. Furthermore, when the operation detection unit 247 detects a predetermined gesture (for example, a finger gesture such as moving a finger from right to left) from the captured image, it may accept execution of a process defined corresponding to the gesture.

[0082] The response processing unit 246a then executes processing in response to the operation. For example, in response to an operation requesting the display of a simulation image, the response processing unit 246a generates a simulation image of what the user would look like if they used the recommended product, and supplies the simulation image to the display control unit 245a. The method for generating the simulation image may be the same as the method described in the second embodiment. Depending on the operation, the response processing unit 246a may perform processing similar to the processing received from the user terminal 300 that reads the QR code, as described above. For example, in response to an operation requesting delivery of a sample, or ordering, payment, or delivery of a recommended product, the processing in response to the operation is delivery processing of the sample of the recommended product using the user's personal information (credit card number and address), ordering processing, payment processing, or delivery processing of the recommended product.

[0083] The operation recording unit 248 is also called an operation recording means. The operation recording unit 248 records the operation history in the user DB 212a in association with the user ID. For example, the operation recording unit 248 records the operation date and the operation type as the operation history in the user DB 212a. At this time, the operation recording unit 248 may calculate the total number of operations or the number of operations or frequency of operations for each operation type, and record the calculated number of operations in the operation history. The operation recording unit 248 may also record manufacturer information of recommended products related to the operations in the operation history.

[0084] The reward granting unit 249 is also called a reward granting means. The reward granting unit 249 grants a reward to the user U according to the operation history. For example, the reward granting unit 249 reads the operation history stored in the user DB 212a at a predetermined timing. The reward granting unit 249 then grants a larger reward to the user U the greater the total number of operations, the number of operations of a predetermined operation type (e.g., the number of trials or the number of purchases), or the number of manufacturers of products tried or purchased. Here, "trying" may mean requesting the display of a simulation image or requesting delivery of a sample product. By granting a reward, the user U can be encouraged to try the simulation or the delivery of a sample product. Here, the greater the number of simulations or the delivery of sample products, the more opportunities there are to try the actual product, which increases the user's desire to purchase and increases the likelihood of a purchase. Therefore, granting a reward can ultimately encourage the user U to purchase a product.

[0085] The display control unit 245a transmits the results of processing according to the operation to the mirror signage 400, and causes the display device 420 to display the results.

[0086] 16 is a sequence diagram showing a modified example of the flow of the information provision process according to the third embodiment. First, the information processing system 1000 executes the same processes as those in S510 to S518. As a result, display information related to the recommended product is displayed on the display device 420 of the mirror signage 400 (S519).

[0087] Next, the camera 440 of the mirror signage 400 captures an image of the user U while the display information is being displayed (S530) and transmits the captured image to the server 200a. The operation detection unit 247 of the response processing unit 246a of the server 200a detects an operation by the user U from the captured image (S532). Next, the response processing unit 246a executes processing according to the operation (S533). Next, the operation recording unit 248 of the response processing unit 246a associates the operation history with the user ID and records it in the user DB 212a (S534). Then, the benefit granting unit 249 of the response processing unit 246a grants the user a benefit according to the operation history 2123 of the user DB 212a, and records the benefit information in the user DB 212a associates it with the user ID (S535).

[0088] The bonus granting process shown in S535 may be performed every time the operation history is updated, or may be performed when a predetermined amount of updated operation history has been accumulated. Alternatively, the bonus granting process may be performed periodically.

[0089] Here, a case will be described in which the user U desires to display a simulation image (so-called virtual trial) after the display information is displayed in S519. In this case, the server 200a acquires the photographed image 500 shown in FIG. FIG. 17 is a diagram illustrating an example of a captured image 500 according to the third embodiment. Currently, the display device 420 of the mirror signage 400 displays display information C6 including the manufacturer name and product type of the recommended product, and display information C7, which is an operation area related to the recommended product. In FIG. 17, the display information C7 is an operation area for requesting a virtual trial, but is not limited thereto. For example, the display information C7 may be an operation area for requesting viewing of detailed product information of the recommended product, or an operation area for requesting access to the website or e-commerce site of the recommended product. While FIG. 17 illustrates currently displayed display images C6 to C7 for illustrative purposes, these are not included in the captured image 500.

[0090] The operation detection unit 247 of the response processing unit 246a of the server 200a detects an operation based on the position of the user's hand H or finger F in the captured image 500 and the display position of the display information C7 displayed on the display device 420. For example, the operation detection unit 247 may determine that a selection operation of the display information C7 has been performed when the distance between the position of the user's hand H or finger F in the captured image 500 and the display position of the display information C7 displayed on the display device 420 is less than a predetermined threshold.

[0091] In this example, when the operation detection unit 247 detects a selection operation, the response processing unit 246a executes a process for generating a simulation image as a process corresponding to the selection operation. Then, the display control unit 245a transmits the simulation image to the mirror signage 400 and causes it to be displayed on the display device 420. As a result, the display device 420 displays the same image as that shown in FIG. 14 above.

[0092] As described above, according to the third embodiment, the server 200a detects simple movements of the user U as operations through image analysis. Therefore, if the user shows interest in a recommended product, the user can easily operate the server 200a to order a sample or the recommended product itself, or to make a payment. Moreover, the operation is completed without touching, which is hygienic. Note that even if the mirror signage 400 is a touch panel and can detect touch operations, it is effective in that it can be easily operated.

[0093] The third embodiment can be modified as follows. For example, if the mirror signage 400 is installed in a corner of a cosmetics counter in a department store, the response processing unit 246a may generate guidance data indicating the sales area for the recommended product that is the target of the operation, as processing in response to the operation of the user U. The display control unit 245 may then transmit the guidance data to the mirror signage 400 and display it on the display device 420. This allows the user U to try out multiple products from multiple stores in advance, decide on a product, and head directly to the sales area, even if they want to avoid being served by a store clerk or coming into contact with other people. This effectively encourages purchases.

[0094] In the above description, the reward granting unit 249 grants a reward to the user U according to the operation history, for example, a reward according to the number of "simulations" or "free sample deliveries." In this case, the display control unit 245 may transmit the current number of "simulations" or "free sample deliveries" to the mirror signage 400 and display it on the display device 420. FIG. 18 is a diagram showing an example of a display on the display device 420 of the mirror signage 400 according to the third embodiment. Display information C8 is displayed on the display device 420 in FIG. 18. Display information C8 indicates the current number of trials. Display information C8 may also indicate information on how many trials are required before a special benefit is awarded. In this figure, the special benefit is a coupon.

[0095] Furthermore, the reward granting unit 249 may grant a reward to the user U according to the face recognition history. For example, the reward granting unit 249 may grant a larger reward to the user U as the number of face recognitions (i.e., the number of logins) per predetermined period increases.

[0096] The reward granting unit 249 may also grant a reward to the provider of the product according to the operation history. The provider may be, for example, a manufacturer or a retailer. In this case, the operation recording unit 248 of the server 200a may count the number of user operations related to the recommended product for each provider of the recommended product, and the reward granting unit 249 may grant a reward to each provider of the recommended product according to the number of operations on the provider. As an example, the reward granting unit 249 may grant a larger reward to the provider the more operations on the provider. The reward for the provider may be a reduction in the fee for registering the product of the provider. Furthermore, if the display device 420 can display advertisements from the provider, the reward for the provider may be an increase in the number of advertisements from the provider displayed or a reduction in advertising fees.

[0097] It may also be possible to switch the recommended control mode according to the usage scene of the mirror or the mood and situation of the user U. For example, in response to the user U selecting a control mode by operation, the server 200a may switch the recommended control mode to the selected control mode.

[0098] 19 is a diagram showing an example of a captured image 500 according to the third embodiment. For example, an operation area I-1 for selecting a business mode and an operation area I-2 for selecting a casual mode are currently displayed on the display device 420 of the mirror signage 400. Note that, although the currently displayed operation areas I-1 and I-2 are shown in FIG. 17 for the sake of explanation, they are not included in the captured image 500.

[0099] The operation detection unit 247 of the response processing unit 246a of the server 200a detects a selection operation based on the position of the user's hand H or finger F in the captured image 500 and the display positions of each operation area I-1, I-2 displayed on the display device 420. The selection unit 244 of the server 200 then selects recommended products based on the user information and the type of selected control mode. In this example, the user U selects the business mode. Therefore, for example, the selection unit 244 may select, as recommended products, from the products stored in the product DB 213, products that have product attributes that match the user's preferences, inclinations, skin type, skin color, etc. obtained from the user information and that are suitable for business use.

[0100] This allows for recommendations that match the mirror usage scenario and the user U's mood and situation. This further increases the user U's satisfaction, thereby improving sales promotion effectiveness. The control mode setting does not require the user U to perform a selection operation, and may be preset, for example, according to the installation location of the mirror signage 400 (display device 420) or the date and time of facial authentication. For example, if the mirror signage 400 is installed in an office building, the selection unit 244 may preferentially select business-oriented products as recommended products, whereas if the mirror signage 400 is installed in a commercial facility, the selection unit 244 may preferentially select casual-style products as recommended products. For another example, if facial authentication is performed during the daytime on a weekday, the selection unit 244 may preferentially select business-oriented products as recommended products, whereas if facial authentication is performed on a holiday, the selection unit 244 may preferentially select casual-style products as recommended products. This also contributes to improving the user U's satisfaction and the sales promotion effectiveness.

[0101] Although the above-described embodiments have been described as hardware configurations, the present disclosure is not limited to such configurations. Any processing in the present disclosure can also be realized by causing a processor to execute a computer program.

[0102] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0103] The present disclosure is not limited to the above-described embodiment and may be modified as appropriate without departing from the spirit and scope of the present disclosure. For example, in the above-described embodiment, the selection unit 244 selects recommended products having product attributes corresponding to preferences, inclinations, or skin type or color estimated from user information. However, the selection unit 244 is not limited to this, and may estimate a personal color based on user information and select recommended products having product attributes that match the personal color. This configuration enables recommendations that reflect the user U's personal color, making it possible to give the user U the impression that the recommended products are special. As a result, the sales promotion effect is improved.

[0104] Furthermore, for example, the selection unit 244 may add the appearance characteristics of the user U on that day, such as complexion, clothing, or hairstyle, which can be ascertained from the captured image for face authentication, to the basis for selecting recommended products. That is, if face authentication is successful, the selection unit 244 selects recommended products based on information on preferences, inclination characteristics, skin type, or skin color estimated based on user information and appearance characteristics estimated based on the captured image for face authentication. The selection unit 244 may also diagnose the above-mentioned personal color based on appearance characteristics. This configuration enables recommendations that take into account the appearance of the day, making it possible to give the user U the impression that the recommended products are special. As a result, the effectiveness of sales promotion is improved.

[0105] Furthermore, in the above-described embodiment, the face authentication device 100 has a face authentication function, but instead of or in addition to the face authentication device 100, the server 200, 200a may have a face authentication function.

[0106] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) an authentication control means for acquiring a matching result between facial feature information extracted from a captured image of at least the face of the user and facial feature information of a plurality of people registered in advance, when the user is positioned in front of the mirror; a selection means for selecting at least one product from a plurality of products as a recommended product based on pre-registered user information related to the user when the matching result indicates that face authentication has been successful; a display control means for outputting display information related to the recommended products to a display device provided in correspondence with the mirror; An information processing device comprising: (Appendix 2) the mirror is a half mirror, The display device is provided on the rear side of the mirror. 2. The information processing device according to claim 1. (Appendix 3) a response processing means for detecting an operation by the user and executing a process in response to the operation; The display control means outputs a result of processing according to the operation to the display device. 3. The information processing device according to claim 1 or 2. (Appendix 4) The process according to the operation is a sample delivery process of the recommended product using the user's pre-registered personal information, an order process of the recommended product, a payment process, or a delivery process. 4. The information processing device according to claim 3. (Appendix 5) the processing in response to the operation is processing for generating a simulation image of a case in which the user uses the recommended product; The display control means outputs the simulation image to the display device. 5. The information processing device according to claim 3 or 4. (Appendix 6) The response processing means detects the operation based on the position of the user's hand or finger in the captured image and the display position of an operation area related to the recommended product displayed on the display device. 6. The information processing device according to any one of Supplementary Notes 3 to 5. (Appendix 7) A bonus granting means is provided for granting a bonus to the user according to the operation history. 7. The information processing device according to any one of Supplementary Notes 3 to 6. (Appendix 8) an operation recording means for counting the number of operations performed by the user related to the recommended product for each provider of the recommended product; a reward granting means for granting a reward to each provider of the recommended products according to the number of operations on the provider; 7. The information processing device according to claim 3, further comprising: (Appendix 9) The display control means outputs the display information to the display device in a display mode determined based on the type of the recommended product and the position of the facial part of the user included in the photographed image. 9. An information processing device according to any one of appendices 1 to 8. (Appendix 10) The selection means selects at least one product from a plurality of products as the recommended product based on the user information and the type of control mode. 10. An information processing device according to any one of appendices 1 to 9. (Appendix 11) If the face authentication is successful, the selection means selects the recommended product based on the user information and the appearance characteristics estimated based on the captured image. 11. The information processing device according to any one of Supplementary Notes 1 to 10. (Appendix 12) an information processing device; Facial recognition device Equipped with The information processing device includes: an authentication control means for acquiring, when a user is positioned in front of a mirror, from the face authentication device, a matching result between facial feature information extracted from a captured image of at least the user's face and facial feature information of a plurality of people registered in advance; a selection means for selecting at least one product from a plurality of products as a recommended product based on pre-registered user information related to the user when the matching result indicates that face authentication has been successful; a display control means for outputting display information related to the recommended products to a display device provided in correspondence with the mirror; have Information processing system. (Appendix 13) The display device further includes: 13. The information processing system of claim 12. (Appendix 14) When a user is positioned in front of a mirror, a matching result is obtained between facial feature information extracted from a captured image in which at least the face of the user is captured and facial feature information of a plurality of people registered in advance; If the matching result indicates that face authentication has been successful, select at least one product from a plurality of products as a recommended product based on user information related to the user that has been registered in advance; Display information related to the recommended product is output to a display device provided in correspondence with the mirror. Information processing methods. (Appendix 15) a step of acquiring a matching result between facial feature information extracted from a captured image of at least the face of the user and facial feature information of a plurality of people registered in advance, when the user is positioned in front of the mirror; If the matching result indicates that face authentication has been successful, selecting at least one product from a plurality of products as a recommended product based on user information related to the user that has been registered in advance; outputting display information related to the recommended products to a display device provided in correspondence with the mirror; A non-transitory computer-readable medium storing a program for causing a computer to execute the above. [Explanation of symbols]

[0107] 10. Information processing equipment 13,243 Authentication control section 14,244 Selection 15,245,245a Display control unit 100 Facial Recognition Device 110 Face Information DB 111 User ID 112 Facial Feature Information 120 Face detection unit 130 Feature point extraction unit 140 Registration Department 150 Authentication Department 200, 200a Information processing device (server) 210 Storage section 211 Program 212,212a User DB 2121 User ID 2122 User Information 2123 Operation History 2124 Bonus Information 213 Product DB 2131 Product ID 2132 Product information 220 memory 230 Communications Department 240,240a Control unit 241 Registration Department 242 Image acquisition unit 246,246a Response processing section 247 Operation detection unit 248 Operation Recording Section 249 Benefits Granting Department 300 User Terminals 310 Camera 320 Storage section 330 Communications Department 340 Display section 350 Input section 360 Control Unit 400 Mirror Signage 410 Half Mirror 420 Display device 430 Control device 431 Storage section 432 Communications Department 433 Control Unit 440 Camera 500 images 1000 Information Processing Systems U User

Claims

1. an authentication control means for acquiring a matching result between facial feature information extracted from a captured image of at least the face of the user and facial feature information of a plurality of people registered in advance, when the user is positioned in front of the mirror; a determination means for determining output information based on appearance characteristics indicating the state of the user estimated from the captured image and pre-registered user information related to the user when the matching result indicates that face authentication has been successful; a display control means for outputting the output information to a display device provided in correspondence with the mirror; a selection means for selecting at least one product from a plurality of products as a recommended product in accordance with a date and time when the facial feature information was compared with facial feature information of a plurality of people registered in advance, based on the appearance features and the user information; The display control means outputs display information related to the recommended product to a display device provided in correspondence with the mirror. Information processing device.

2. an operation recording means for counting the number of operations performed by the user related to the recommended product for each provider of the recommended product; a reward granting means for granting a reward to each provider of the recommended products according to the number of operations of the provider; Further equipped The information processing device according to claim 1 .

3. the mirror is a half mirror, The display device is provided on the rear side of the mirror.

3. The information processing device according to claim 1.

4. a response processing means for detecting an operation by the user and executing a process in response to the operation; The display control means outputs a result of processing according to the operation to the display device. The information processing device according to claim 2 .

5. The processing in response to the operation is a sample delivery process of the recommended product using the user's pre-registered personal information, an order process of the recommended product, a payment process, or a delivery process. The information processing device according to claim 4 .

6. the processing in response to the operation is processing for generating a simulation image of a case in which the user uses the recommended product; The display control means outputs the simulation image to the display device.

6. The information processing device according to claim 4 or 5.

7. The response processing means detects the operation based on the position of the user's hand or finger in the captured image and the display position of an operation area related to the recommended product displayed on the display device. The information processing device according to claim 4 .

8. A bonus granting means is provided for granting a bonus to the user according to the operation history. The information processing device according to claim 4 .

9. When a user is positioned in front of a mirror, a matching result is obtained between facial feature information extracted from a captured image in which at least the face of the user is captured and facial feature information of a plurality of people registered in advance; If the matching result indicates that face authentication has been successful, determining output information based on appearance features indicating the state of the user estimated from the captured image and pre-registered user information related to the user; outputting the output information to a display device provided in association with the mirror; selecting at least one product from a plurality of products as a recommended product in accordance with a date and time when the facial feature information was compared with facial feature information of a plurality of people registered in advance, based on the appearance characteristics and the user information; outputting display information related to the recommended products to a display device provided in correspondence with the mirror; Information processing methods.

10. a step of acquiring a matching result between facial feature information extracted from a captured image of at least the face of the user and facial feature information of a plurality of people registered in advance, when the user is positioned in front of the mirror; a step of determining output information based on appearance characteristics indicating the state of the user estimated from the captured image and pre-registered user information related to the user when the matching result indicates that face authentication has been successful; outputting the output information to a display device provided in association with the mirror; a step of selecting at least one product from a plurality of products as a recommended product in accordance with a date and time when the facial feature information was compared with facial feature information of a plurality of people registered in advance, based on the appearance features and the user information; a step of outputting display information related to the recommended product to a display device provided in correspondence with the mirror; A program that causes a computer to execute the following.

Images